主题: unicode
-
JavaScript 文本偏移:区分 UTF-16 码元、码点和可见字符
在每个编辑器或 API 边界处注明偏移量的单位,并在切分文本前有意识地进行转换。
-
Normalize Unicode before comparing identifiers
Choose a normalization policy for human-readable identifiers without modifying opaque tokens or conflating visually similar characters.
-
不可见与重排文本:代码和提示中的双向控制符、标签字符及形似字符
Unicode 允许文本包含审查者看不见的字符,或改变显示顺序的字符。双向控制符可能使源码的视觉含义与编译器处理的含义不同;标签字符可隐藏模型仍会收到的指令,形似字符则模仿标识符。检测需要扫描码点,而不是查看渲染结果。
-
SMS pitfalls: GSM-7 versus UCS-2 encoding and message segments
An SMS carries 140 bytes: 160 GSM-7 characters or 70 UCS-2 characters. One character outside GSM-7 (a curly quote, an emoji) switches the whole message to UCS-2; longer messages are split into segments of 153 or 67 characters that providers typically bill separately. Count segments before sending and normalise punctuation.
-
Collations in PostgreSQL: libc, ICU and the builtin provider, and why a library upgrade can corrupt an index
A collation decides how text sorts and compares; PostgreSQL takes it from a provider (libc, ICU, or the builtin C and C.UTF-8 code-point orderings), fixes the default per database at creation, and allows overrides per column or expression. Language collations from libc or ICU change when the library changes, which invalidates B-tree indexes on text, so choose the provider deliberately and reindex after operating-system or ICU upgrades.
-
电子邮件的 MIME 结构:multipart/alternative、纯文本部分和编码
文本加 HTML 的邮件使用 multipart/alternative,各部分从最简单到最丰富排序(text/plain 在前,text/html 在后);含非 ASCII 字节的正文使用 quoted-printable 或 base64,标头使用 RFC 2047 编码词。应从与 HTML 相同的数据生成文本部分,转义变量,并遵守行长度限制。
机器可读: JSON