There is one question that guides the reading of any audit report · how much the signatory actually saw. The answer is usually a sample · a selected fraction of the system over a limited period. The conclusions must be read on that basis.
This site’s manifesto states it as a principle · the sample is no excuse. Sampling does not authorise looking at little · it requires selection based on judgement, risk and documented justification. This criterion takes that principle to its operational consequence and names the boundary · the limits of the sample. Criterion 03 established the first epistemic boundary — scope limits the territory to which the certificate attests. The sample is the second · it limits the view within the territory. And both fail in the same way · a perfect scope covered with a convenient sample produces the same false assurance as a narrowed scope.
Every sample is a theory
Choosing a sample means asserting, in writing or by omission, where the system may fail. It comprises three decisions, even if no one writes them down · which units to examine — direction —, how many — depth — and when — timing. A serious sampling design can be shown and defended · these units, for these risks, at these times, with this justification. Where those decisions are not written down, they were still made. Convenience made them, and convenience also designs samples · it just does not sign the design.
Three sampling pathologies
Pathology 01 · the convenience sample. The usual site, the familiar process, the record the auditee has already prepared. An announced visit may be necessary to ensure access and availability; it can also make the portion observed predictable. The corrective is not to feign surprise, but to combine prepared evidence with historical records, independent selection and, where the programme allows, unannounced checks.
Pathology 02 · the inherited sample. The same sampling design, cycle after cycle. The audited system learns which units are examined and keeps them impeccable — this is one of the mechanisms behind the clean cycles described in Criterion 11. A repeated sample ceases to be a theory of risk and becomes shared choreography · the auditee rehearses their part, the auditor performs theirs, and no one asks again where the system may fail.
Pathology 03 · the mirage of one hundred per cent. An algorithm can review every recorded transaction and still fail to observe missing fields, rules no one programmed or events that occur outside the system. I call this the shifting of the limit · coverage increases within the dataset and the limit moves towards its design. Deciding which population to examine and what could not enter the data remains the auditor’s job.
The honesty of the report
The second half of the criterion lives in the report. A serious signature asserts exactly what the sample permits · these units, this period, this degree of confidence — and declares what was left unexamined. Declared limitations are part of the argument · a piece that omits them asserts more than it can support. The trained reader therefore reverses the reading order · before the conclusions, they look for the scope and limitations section. If it is generic — the same prose that would serve for any assignment — the conclusions inherit that generality.
The criterion test
Take the last audit report you received — internal or external — and put three questions to it.
Question one · how large was the population and how much was examined? Expected answer · both numbers appear in the report or in accessible working papers. Where neither appears, the report asks to be believed rather than read.
Question two · what criterion guided the selection of the sample? Expected answer · a written criterion that names risk — new processes, staff turnover, previous findings, complaints, changes of supplier — with its documented justification. The criterion that was not written down still existed · it was convenience.
Question three · what does the report declare about what was left unexamined? Expected answer · a limitations section specific to this assignment, not a template paragraph. And the in-person check · ask the signatory to name what they left out of the sample and why. Those who designed it with judgement answer from memory, because they made the decision. Those who inherited the design consult the papers.
The threshold is the one set by the manifesto · does the sample respond to risk or to convenience? A serious report answers that question in writing before anyone asks it.
The entire system was never on the table. A fraction was, chosen by someone, for reasons that were written down or left unsaid. A signature that counts states which fraction was there, why that one and what it permits one to assert. That declaration — more than the size of the sample — distinguishes the auditor who chose to look from the one who merely went through.