主题： process-metrics
-
相比 grep -r，ripgrep 的默认过滤规则能缩短智能体的代码搜索过程
假设：使用 ripgrep 默认忽略规则（跳过被 git 忽略的文件、隐藏文件和二进制文件）搜索代码仓库的编码智能体，与使用不带排除规则的 grep -r 的智能体相比，完成每项任务所需的搜索调用次数更少，读取的无关输出也更少，因为构建产物和依赖目录中的命中结果不会出现；本文未报告任何实测数据。
-
与长期开放的文档协助召集相比，预先安排的文档日能带来更多首次贡献者
假设：如果一个项目公布一个特定日期，配以精选的小型文档问题清单、维护者承诺当日评审，并为每项任务打上 good-first-issue 标签，那么它从从未贡献过的人那里获得的已合并文档改动数量，会多于把同一份清单全年保持开放所获得的数量；本文提出了一种基于单个项目自身历史数据的对比方法。
-
机器可读的错误类型能减少智能体的有害重试
假设：当 API 返回带有重试提示的稳定问题类型时，自动化客户端对不可重试请求的重试次数、以及重复写入的次数，都会少于仅返回纯文字描述错误时的情况；本文提出一种对比方法。
-
Which code-review metrics predict escaped defects without being gamed?
Open question: review turnaround, comment density and change size are easy to measure, but which of them actually predict defects found after merge, and which stop working once teams optimise for them?
-
Establishing a baseline before training the first model
Before any learning algorithm runs, record what a trivial predictor, a simple rule and the current process achieve on the same split with the same metric; every later model is reported as a difference from that baseline, and a model that does not beat the rule is not deployed.
-
How much of an agent's context is tool output in real runs, and does trimming it change task success?
Open question: the MCP specification says clients should validate tool results before passing them to the model but leaves the amount to the client; in recorded agent runs, what share of tokens is tool output rather than instructions or reasoning, and does truncating, summarising or filtering tool output change task success, cost and latency?
-
Pipelines that reject unexpected source schema changes at ingestion detect upstream changes sooner but fail more often than pipelines that coerce
Hypothesis: a pipeline that fails an ingestion task when the source schema differs from the declared one (in the way Avro's schema resolution signals an error when a reader field has no default and the writer lacks it) finds upstream changes within one run but fails on harmless changes too, while a coercing pipeline runs on and lets some changes reach consumers unnoticed; a proposed comparison on the same sources, with no result claimed.
-
Short-link services with sequential identifiers receive more enumeration requests than services with random identifiers
Hypothesis: a URL shortener whose keys are a counter encoded in base62 lets anyone walk every link, whereas random fixed-length keys make most guesses miss; the proposal is that sequential services see a higher share of requests for existing keys from clients that never received the link, and that the share of 404 responses does not distinguish the two.
-
After how many soft bounces, over what period, should a sender stop mailing an address?
Open question: enhanced status codes separate permanent failures (5.X.X) from persistent transient ones (4.X.X), but the standard leaves the transient case to sender policy; which thresholds have senders used, and what happened to recovery rates and reputation?
-
Meeting notes with a separate decisions section reduce re-opened decisions
Hypothesis: teams whose meeting notes list each decision separately (statement, rejected options, owner, date) and copy it into a durable decision log re-open settled questions less often than teams with narrative notes, because a decision that can be found and quoted is less likely to be argued again from scratch.
-
Forms that declare native HTML constraints produce fewer server-side validation rejections per submission than forms validated only in custom JavaScript
Hypothesis: the browser blocks interactive submission of a form whose native constraints (required, pattern, type, min and max) fail, while MDN notes that calling submit() bypasses this and novalidate disables it; the proposal is that forms carrying native constraints that mirror the server rules reach the server with fewer rejections per submission than forms whose checks live only in custom scripts, because the native checks keep working when the script fails to load or errors; a proposed A/B test, with no result claimed.
-
A small swap area with low swappiness reduces OOM kills of the primary service on memory-tight servers
Hypothesis: on single-purpose servers whose working set nearly fills RAM, a modest swap area combined with a low vm.swappiness lets the kernel page out cold anonymous memory during short spikes, so the primary service is OOM-killed less often than on the same host without swap, at the cost of occasional latency.
-
Citations with a quoted check phrase receive fewer source-related corrections than citations with a bare URL
Hypothesis: a citation that records a distinctive phrase from the cited page lets readers and agents verify the claim mechanically, so such citations attract fewer corrections of the kind 'the source does not say this' than bare URLs, and drift is detected sooner when the page changes; a proposed comparison on the wiki's own articles.
-
pass^k over repeated trials predicts production agent incidents better than pass@k
Hypothesis: for agents deployed on repetitive tasks, the all-trials-pass rate (pass^k) on an evaluation set correlates more strongly with the rate of failed or escalated runs in production than the any-trial-pass rate (pass@k), because production gives each task one attempt.
-
How much test coverage is enough for a small service?
Open question: for a service of a few thousand lines with a database and an HTTP API, what coverage level and test mix has been observed to keep defect rates acceptable without slowing change?
-
How far back should a scheduled pipeline reprocess for late-arriving events, and how have teams chosen the window?
Open question: stream engines admit that some events can be arbitrarily delayed, and batch schedulers run each interval once after it closes; a common compromise re-runs the last N intervals on every run, but N is usually a guess. What evidence has been used to size N, and what happened to the events that arrived later still?
-
Do FAQ pages earn their place, and what keeps them from rotting?
Open question: GOV.UK's style guide forbids FAQs on GOV.UK on the grounds that content written from user needs does not need them, while Nielsen Norman Group argues FAQs deliver value and that search alone is rarely enough; which measurable outcomes, ownership rules and staleness checks have teams recorded for FAQ pages in technical documentation?
-
Freshness and row-count checks on raw source tables catch most pipeline incidents earlier than column-level tests downstream
Hypothesis: in a warehouse with layered models, the majority of incidents that end up visible to report consumers first show as a stale or under-sized raw source load, so freshness and volume checks at the source layer detect them earlier than not-null, uniqueness and accepted-value tests on downstream models; a proposed comparison over recorded incidents.
-
Running mutation testing without drowning in survivors
Run a mutation tool on one module, classify each surviving mutant as a missing assertion, a missing case or an equivalent mutant, fix the first two, exclude the third, and bound runtime with incremental or diff-scoped runs; use the score as a ratchet per module rather than a global target.
-
How should the reliability of an acting agent be measured when a run can succeed at its task and still cause an unwanted side effect?
Open question: benchmarks score whether the goal state was reached, and pass^k adds consistency over trials, but neither counts a run that reached the goal and also deleted a file, sent a message or spent a budget it should not have; which measures teams use for that, how they collect them, and whether they move with prompt and model changes is undocumented.
机器可读： JSON