テーマ: search
-
ripgrepのデフォルトフィルタリングは、grep -rと比べてエージェントのコード検索を短縮する
仮説: ripgrepのデフォルトの無視ルール(gitignore対象、隠しファイル、バイナリファイルをスキップ)を使ってリポジトリを検索するコーディングエージェントは、除外設定のない`grep -r`を使うエージェントに比べて、タスクあたりの検索呼び出し回数が少なく、読み込む無関係な出力も少なくて済む。ビルド出力や依存関係の中でのヒットが発生しないためである。測定結果は報告しない。
-
PostgreSQLでのtsvectorを使った全文検索
PostgreSQLは、言語設定を使ってテキストを正規化された語彙素(lexeme)からなる`tsvector`に変換し、それを`tsquery`と照合し、`ts_rank`でスコアを付け、GINでインデックスを作成する。ステミングとストップワードの処理は標準で行われるが、タイプミスや同義語には標準では対応しない。
-
grep and ripgrep for code search: recursion, filters and exit codes
GNU grep searches recursively with -r, narrows with --include and --exclude-dir and exits 0, 1 or 2 for match, no match or error; ripgrep does the same by default while skipping git-ignored, hidden and binary files. Know what each tool skips before trusting an empty result.
-
Document search over a corpus walk-through: indexing pipeline, permissions and reindexing
A design walk-through for search over documents held in a system of record: a derived, rebuildable index fed by change events, ACL keys indexed as fields so filtering happens before ranking, versioned indexes switched by alias, a reconciler that finds drift, and a list of what to defer.
-
Tries for prefix lookups: autocomplete and longest-prefix matching
A trie stores strings with one node per common prefix, so lookup costs the key length regardless of how many keys exist, all keys with a prefix form one subtree, and the longest stored prefix of a query is found in one walk; use it for autocomplete and routing, and compare against a sorted array first.
-
Embeddings as a data type: fixed-length vectors, a distance function and what a column of them needs
An embedding is a fixed-length float vector produced by a specific model, meaningful only under the distance that model was trained for and only next to vectors from the same model version; storing it requires the dimension, the model identifier and the distance to be recorded, and querying it means nearest-neighbour search, exact or approximate.
-
Retrieval basics for LLM applications: chunking, passage identifiers and citing what was retrieved
Retrieval-augmented generation feeds retrieved passages to the model; the decisions that matter are how documents are split, what context each chunk carries, and how the answer points back to a specific passage so that a reader can check it.
機械可読: JSON