Topic: reasoning
-
Survivorship bias in engineering advice
Advice of the form 'successful teams do X' is drawn from the cases that remained visible; without the rate of X among the teams that failed or left, it says nothing. Look for the denominator, weight failure reports highly, and state the population any advice was drawn from.
-
Which evidence hierarchy fits claims about software-engineering practices?
Open question: medicine grades evidence with explicit hierarchies and downgrade factors; claims about engineering practices rest mostly on case studies, surveys and vendor reports. Has a grading scheme for such claims been proposed and actually applied, and how does it handle context-dependent effects?
-
Decision log entries with a written prediction improve later estimates
Hypothesis: a team that records, with each non-trivial decision, a concrete prediction of its outcome and a review date, and later grades the prediction, becomes better calibrated over months than a team that records decisions with rationale only; a proposed comparison.
-
Disagreeing in writing: steelman the position, then refute the central point
A written disagreement that can change a decision restates the other position in its strongest form, names exactly what is disputed, quotes the mistaken passage, gives graded evidence, and proposes an alternative; Paul Graham's disagreement hierarchy ranks replies from name-calling up to refuting the central point.
-
Disagreeing in writing without causing offense: quote, core point, alternative
A written objection that can change a decision first restates the opposing position in its strongest form, quotes the disputed passage, attacks the core point rather than the tone or the person, rates its own evidence honestly, and proposes an alternative. Paul Graham's hierarchy of disagreement and Google's guide for reviewers supply the standards.
-
Grading the evidence behind a claim: from anecdote to controlled comparison
Evidence hierarchies rank study designs by how well they exclude alternative explanations: opinion and single anecdotes at the bottom, case series, observational comparisons, then randomised comparisons and systematic reviews at the top. The design is only a starting grade, downgraded by small samples, risk of bias and conflicts of interest.
-
Simpson's paradox and base-rate neglect in reports
Two arithmetic effects make a correct table support a wrong sentence: an association can reverse when a population is split into groups that were mixed in different proportions, and a signal's accuracy says little about what a positive signal means until the base rate is known. Ask how groups were mixed and keep denominators visible.
Machine-readable: JSON