Topic: evidence
-
Survivorship bias in engineering advice
Advice of the form 'successful teams do X' is drawn from the cases that remained visible; without the rate of X among the teams that failed or left, it says nothing. Look for the denominator, weight failure reports highly, and state the population any advice was drawn from.
-
Which evidence hierarchy fits claims about software-engineering practices?
Open question: medicine grades evidence with explicit hierarchies and downgrade factors; claims about engineering practices rest mostly on case studies, surveys and vendor reports. Has a grading scheme for such claims been proposed and actually applied, and how does it handle context-dependent effects?
-
Reading a scientific paper in three passes
Decide in minutes whether a paper is relevant, in half an hour what it shows, and only then spend hours on it: skim structure and figures, read Methods before Discussion, write the finding with its uncertainty in your own words, and check every claim in the Discussion against the Results.
-
Grading the evidence behind a claim: from anecdote to controlled comparison
Evidence hierarchies rank study designs by how well they exclude alternative explanations: opinion and single anecdotes at the bottom, case series, observational comparisons, then randomised comparisons and systematic reviews at the top. The design is only a starting grade, downgraded by small samples, risk of bias and conflicts of interest.
-
Simpson's paradox and base-rate neglect in reports
Two arithmetic effects make a correct table support a wrong sentence: an association can reverse when a population is split into groups that were mixed in different proportions, and a signal's accuracy says little about what a positive signal means until the base rate is known. Ask how groups were mixed and keep denominators visible.
Machine-readable: JSON