Locale and date pitfalls in coreutils: LC_ALL, character ranges and GNU versus BSD date

What it is

The environment variables LANG, the LC_* categories and LC_ALL select a locale, and the coreutils consult it. The sort manual states that comparisons use the LC_COLLATE collating sequence, that LC_CTYPE decides which characters are blanks, and that LC_NUMERIC sets the thousands separator and decimal point for -n; its footnote recommends setting LC_ALL to C when a non-POSIX locale produces unexpected order, and notes that setting only LC_COLLATE is ineffective if LC_ALL is also set. The date manual documents that its options other than -u are GNU extensions to POSIX: -d datestr (--date) parses "almost any common format" but the string must be in locale-independent form, -u behaves as if TZ were UTC0, and -I prints ISO 8601. BSD date (FreeBSD, macOS) does not parse a date string with -d; the FreeBSD manual page documents -v [+|-]val[ymwdHMS] to adjust a date without setting the clock, -j to suppress setting, -f input_fmt to parse, and -r seconds to print an epoch value.

Why it matters

A script that passes on one machine and fails in CI, or sorts differently for a colleague, often differs only in locale: a language locale collates differently from byte order, which the sort manual's footnote warns about. GNU date -d arithmetic fails on BSD date, and BSD -v is unknown to GNU date. Output parsed by another program (date without a format, ls -l, printf '%.2f') changes shape with the locale.

How to apply

• Begin scripts with export LC_ALL=C for byte order and ASCII semantics, or LC_ALL=C.UTF-8 when multibyte text must stay intact; set TZ=UTC when times are compared or logged.
• Never parse locale-dependent default output; request a format: date -u +%Y-%m-%dT%H:%M:%SZ, date +%s for epoch seconds, stat or find -printf with explicit fields instead of ls.
• For date arithmetic, branch on the implementation: GNU date -d '2 days ago' +%F, BSD date -v-2d +%F; for anything beyond one offset use a language with a real date library.
• Write [[:lower:]] and [[:digit:]] instead of [a-z] and [0-9]: the grep manual (cited in the grep article) states that range expressions are unspecified outside the C locale.
• Convert epoch values explicitly: GNU date -d @1700000000, BSD date -r 1700000000.

Pitfalls

LC_ALL overrides every other category, including the user's message language; scope it to the command (LC_ALL=C sort ...) rather than the whole session. In the C locale each byte is a character, so tools that count or cut characters mishandle multibyte text; use C.UTF-8 for those. date -d accepts many spellings, which makes wrong input parse as a wrong date rather than fail; use --debug to see what was understood.