Measuring what you learned with before-and-after self-tests, and what such a comparison cannot show

Goal

Replace the feeling of having learned something with a number that has a stated basis: how many of a fixed set of questions could be answered before a study period and how many of a comparable set after it, at a delay, and with the reasons the number may be wrong written next to it.

Prerequisites

The material and its scope defined in advance; a way to write questions with unambiguous answers (an answer key written before the tests); a calendar for the delayed post-test; and the willingness to score against the key without adjustment.

Steps

1. Write two sets of questions covering the same scope, alternating which set each question goes into, so that the sets are as similar as possible. Write the key with each question. Do not study while writing.
2. Take set A blind, with a time limit and the source closed; score it with the key and record the score, the date and the time taken.
3. Study for the planned period, keeping a log of hours and method.
4. After a delay decided in advance (a week, not the same evening), take set B under the same conditions and score it.
5. Record the difference and, next to it, the caveats that apply: the pre-test itself was a retrieval event (the cited review reports that retrieval practice improves learning, so answering set A is part of the treatment); sets A and B may differ in difficulty; the delay and conditions may not match; nothing was compared with not studying.
6. Optionally repeat set A at the delay as well, to see whether the pre-test questions specifically improved more than the parallel set; a large gap suggests the pre-test taught the answers.
7. Keep the questions, keys and scores; a repeat of the whole cycle on a later topic makes the numbers comparable with each other, not with anyone else's.

Expected result

Two scores with dates, conditions and a written list of the reasons the difference over- or understates learning, plus a question bank that can feed a retrieval-practice schedule.

Limits and test basis

A single learner without a control condition cannot attribute the gain to the study method rather than to the pre-test, to time, or to easier questions. The IES guide recommends delayed judgments of learning and quizzes to find what needs further study, which is what this protocol delivers; it does not turn the score into a measurement of a method. No numbers are claimed here.