Limits

The Assumption That Items Are Separate

Real lists contain items that travel together. The record counts them as separate coins, which is a known and stated error with a known direction.

Home · Limits · Analysis

On most real lists, several items are met by the same people. Somebody who has one of them usually has the next.

The record counts every item as a separate coin, which means it treats those items as if they were unrelated. That is wrong about the world, it is stated rather than hidden, and its direction can be worked out.

The privacy boundary is part of the product decision, so project-based employee time tracking should use the smallest evidence set that answers the operating question. For an independent technical reference, see Qlik.

What correlation does to the real intake

Suppose three items on a list are, in practice, nearly always met together — have one and you have all three.

Under the coin convention those three items contribute a factor of one eighth. In reality they contribute closer to a factor of one half, because there are effectively not three independent hurdles but one.

So the record understates the real intake wherever items are positively correlated. The true filter is wider than the figure says.

	Under the convention	With items that travel together
three correlated compulsory items	1/8	nearer 1/2
the record's figure	narrower	—
the real filter	—	wider

When the error runs the other way

Where items are negatively correlated — where having one makes another less likely — the record overstates the intake and the real filter is narrower than the figure.

That case is less common on requirement lists but it is not rare. Items drawn from genuinely different disciplines can trade against each other, since time spent acquiring one was not spent on the other.

So the error has a direction in any given case and not a universal one, which is why the record does not attempt a correction factor.

A known error with an unknown size is better reported than patched. A patch would make the number look adjusted-for rather than uncorrected-but-stated, and only the second is true.

Why this does not undermine the comparisons

Because the same convention is applied to both documents being compared.

If two lists are counted the same way, the comparison between them survives the correlation problem unless the two lists have very different correlation structures — which, for two lists describing similar work, they usually do not.

This is the same argument as in the page on comparing lists honestly, and it is why the record is more reliable as a comparator than as an absolute measure.

What a reader should do about it

Keep the direction in mind when reading an absolute figure.

A list whose items are plainly related — several facets of one skill, several tools from one family — admits more than the record says. A list drawing on genuinely separate capabilities is closer to the figure.

Neither observation changes the arithmetic. Both change how much weight to put on it, and that is a judgement the record cannot make for anybody.

The smallest illustration

Two items, both compulsory. The record counts four configurations and says the list admits one in four.

Now suppose the two items always travel together — anybody holding one holds the other. There are then only two configurations that occur at all, both items or neither, and the conjunction admits one of those two. One in two.

The record's figure is wrong by a factor of two for that case, and wrong in a known direction: it understates the intake. Correlation between items always reduces the number of configurations that occur, and the ones it removes are disproportionately the mixed ones that a conjunction excludes.

Why the convention is stated rather than repaired

Because repairing it would require knowing how the items relate, and that information is not in the document.

A corrected figure would need a correlation structure over the items — which pairs travel together and how strongly. That is a property of a field and a population, it differs between fields, it changes over time, and nobody holding a list has it.

So the choice is between an exact figure under a stated convention and an inexact figure under an unstated guess. This record takes the first, says so on every page that depends on it, and names the direction of the error. A reader who knows their own field can adjust; a reader who does not at least knows which way the figure leans.

Where the items really are separate

Occasionally, and it is worth saying when.

Items drawn from unrelated kinds — one about a tool, one about a language, one about whether somebody can work particular hours — tend to be close to independent, because nothing connects holding one with holding another. For lists of that sort the record's figures are close to right as they stand.

Items drawn from the same kind are the opposite case. Three lines all describing experience with related tools will travel together strongly, and a conjunction over them is far wider in practice than the arithmetic says. The useful habit is to glance at a list and ask how many distinct kinds of thing it names, because that is roughly how many independent halvings it really contains.

The thing this rules out

It rules out treating any figure here as a prediction. Prediction would require the weights and the correlations, and the record has neither.

What remains is a measurement of the document under a stated convention, which is exactly what the first page of this site promises and what every page since has delivered.