JSON Lines: one value per line for logs, datasets and streamed responses

What it is

The JSON Lines site (cited) states three requirements: UTF-8 encoding without a byte order mark; each line is a valid JSON value, so a blank line is an error; and the line terminator is \n (CRLF is tolerated because surrounding whitespace is ignored when a value is parsed). A terminator after the last value is recommended so that files concatenate cleanly. Conventions are the .jsonl extension, gzip or bzip2 for compression, and a media type application/jsonl that is not yet standardised. The IETF relative, JSON text sequences (RFC 7464, cited), puts the ASCII record separator byte (0x1E) before each text and registers application/json-seq; its parsing rules are written so that a truncated element can be skipped and the rest of the sequence recovered, and it has no end-of-sequence marker.

Why it matters

A JSON array is one value: the reader must either hold the whole text or use an incremental parser, and a truncated array is simply invalid. With one value per line every record is parsed on its own, so a writer can append without rewriting the file, a crash costs at most the last line, a large file can be split by line for parallel processing, and shell tools (grep, head, wc -l, jq -c) work directly. The same property makes the format suitable for streamed HTTP responses and for messages between processes.

How to apply

• Serialise records compactly, never pretty-printed: JSON escapes control characters inside strings, so a compact value contains no raw line break.
• Give every line the same shape (an object with stable keys) and version the shape in the file name or a leading header record when it must change.
• For a streamed response, flush after each line, and put a failure into a final line with an explicit "error" member; the client cannot rely on a closing bracket to know the stream ended cleanly, so add an explicit end record if completeness matters.
• Choose application/json-seq when the consumer must recover from corruption in the middle of a stream; choose JSON Lines when compatibility with text tools matters more.
• Compress whole files with a stream compressor; the format stays line-addressable after decompression.

Pitfalls

A single very long line still has to fit in memory. Tools that split on a lone CR or on U+2028 disagree with the specification. Counting "line 1" in an editor and "value 1" in a program (the site suggests the latter) gives off-by-one error reports. Validating the whole file as one JSON document fails by design; validate per line, and treat a final partial line as truncation rather than corruption.