The odds of a coin vary over this square. Six thousand sites each flip a coin about twenty times. Each dot is one site’s share of heads.
Each coin also has an effect of its own on its log-odds, with the standard deviation set by the slider above. Half the sites are held out: the estimator never sees them, and they are used to score the fit at the end.
For a binomial, the variance is determined by the mean. If you know a coin’s odds, you know how much its share of heads should vary, so any extra variation can be measured.
Against the fitted odds, … of sites fall outside the 95% lines. With no coin effects it would be about 5%.
The odds themselves have to be estimated, and an error in the mean also shows up as extra variance. A site’s error has two parts: the fitted surface can be off from the true surface, and the two axes may not capture everything that determines a coin’s odds.
Both parts add to the variance in the same way, growing with the square of the error. Since this page generated the coins, both parts are known. Even the true surface is off by the part the axes don’t capture, which here is the coin effect.
So there are two kinds of overdispersion. Coherent: neighbouring sites share it, and the surface should change there. Incoherent: each site has its own, and no surface can fit it.
An error in the surface is shared by neighbours, since they sit under the same part of it. The part the axes miss is specific to each site; anything in it that varied along the axes would belong to the surface. Against a flat surface the mean squared residual is … and the large residuals cluster; against the final fit it is … and they don’t. The mesh is refined for the coherent part. The incoherent part is the coin variance, which the gate uses as the baseline level of disagreement between sites.
The fit starts from a coarse mesh of right triangles with two free parameters, a level and a tilt. A triangle can only be split at the midpoint of its longest edge. If that would leave a vertex in the middle of a neighbour’s edge, the neighbour is split too.
The fit starts from an 8 × 8 grid of rectangles, with the corners and the two midlines free. A rectangle can be split along either midline, so both directions are offered as candidates and the gate picks. A vertex in the middle of a longer edge takes its height from that edge, so the surface stays continuous.
Each segment drawn here has a candidate vertex at its midpoint.
A vertex that isn’t free sits at the average of its two parents, the endpoints of the segment it splits. A free vertex can differ from that average by its surplus.
Admitting a candidate means allowing its surplus to be nonzero. A prior shrinks each surplus toward zero, more strongly for deeper vertices, so fine corrections need more evidence than coarse ones.
Each round scores every candidate against the current fit: would a nonzero surplus there improve it? The score is a weighted sum of the residuals of the sites under the candidate’s tent.
Blue means the data pull the surface up there, orange means down; larger dots are larger scores. Round 0 scored … candidates. Over the whole run, some more had too few sites under them to be scored.
The raw score is biased, and three corrections are applied. First, each residual is centred: shrinkage and the discreteness of the counts give it a nonzero mean even when the surface is right.
Second, the part of the candidate’s tent that the existing coefficients could absorb is removed (a first-order refit), so a candidate isn’t credited for a slope its parents could fit. Third, the bias from the prior’s shrinkage is removed. Dividing by the standard deviation gives the candidate’s z.
The gate looks at all of a round’s scores together. If no candidate mattered, they would follow a single bell curve, the null.
Lindsey’s method fits the density of the scores and estimates the null from its centre, so a round whose scores are shifted or spread out is compared with its own null rather than N(0, 1). Here the null has centre … and spread …, and … of candidates are estimated to be noise.
From the null, each candidate gets a local false discovery rate: the probability it is noise given its score. Sorted best first, the gate takes the longest run whose average stays at or below 10%.
In round 0 that is … candidates. A cutoff of |z| > 1.96 would have taken ….
The selected candidates are admitted one at a time, best first. Each is re-scored against the surface as updated by the earlier admissions, and admitted only if it still improves the fit.
…
The next round scores the new candidates against the new fit. As the signal is used up, the scores look more like noise and the null gets closer to N(0, 1).
After three rounds in a row with no admissions, the gate stops. This run took … rounds.
Two variances are refitted between rounds. The coin variance is the incoherent part, the spread around the surface. As the mesh picks up the coherent part, it falls toward the true variance of the coin effects.
The depth variance controls how large a surplus at each depth can be. It is estimated by EM from the admitted surpluses, averaging their squares plus their posterior variances by depth. Including the posterior variance keeps a depth’s variance from getting stuck at its floor.
The final surface is flat where the data show nothing and refined where they show a pattern.
On the held-out half: …. Changing any setting above refits and redraws every step.