Local case-control sampling

Results. (i) Under misspecification the limit of CC depends on the sampling ratio and generally differs from $\theta^*$. (ii) Consistency. If $\mathbb E\|X\|\lt\infty$, the classes are not separable by a hyperplane, and the pilot $\lambda_n\to\theta^*$ in probability, possibly depending on the data, then $\hat\theta_n\to\theta^*$ in probability. (iii) Distribution. If $\mathbb E\|X\|^2\lt\infty$ and the pilot is independent of the data with $\sqrt n(\lambda_n-\theta^*)\Rightarrow N(0,V)$, then $$\sqrt n(\hat\theta_n-\theta^*)\Rightarrow N\big(0,\ H^{-1}(CVC'+\bar a^{-1}J)H^{-1}\big),$$ where $H$ and $J$ are the Hessian and score covariance of the logistic loss under the subsampled law, $C$ is the cross-derivative of the score under that law in the pilot, and $\bar a=\mathbb E|Y-p^*(X)|$ is the acceptance rate. (iv) Efficiency. If the model is correct and the pilot consistent and independent, $\sqrt n(\hat\theta_n-\theta_0)\Rightarrow N(0,\,2\Sigma_{\rm full})$, with $\Sigma_{\rm full}$ the full-sample MLE's covariance, from a subsample of expected size $n\bar a$; with the factor $c\gt1$, the covariance is at most $(1+1/c)\Sigma_{\rm full}$, for a subsample about $(1+c)/2$ times larger under strong imbalance.

Label-dependent sampling shifts the log-odds by the log acceptance ratio. Bayes on the accepted points gives $$\operatorname{logit}P(Y=1\mid X=x,Z=1)=f(x)+\log\frac{a(x,1)}{a(x,0)} .$$ For CC the shift is a constant $b$, which the intercept correction removes. For LCC it is $-\tilde\theta'x$, so the subsample's log-odds are close to 0 everywhere when the pilot is good. If the model is correct, both corrections are exact; the methods differ only under misspecification.

Under misspecification, shifting the logit changes what the fit matches. The population score $\mathbb E[X(p(X)-p_\theta(X))]=0$ makes the best linear fit get probabilities right on average, so it fits most closely where $p$ changes with $f$, near $p=1/2$, and nearly ignores regions where $p\approx0$. A constant shift $b$ changes which regions sit near 1/2, so the CC limit moves with $b$. In the oatmeal example (family history in 10% of people, oatmeal in 50% independently; true log-odds $-5,-10$ without history and $-4,-1$ with it, for no oatmeal and oatmeal), the population oatmeal coefficient is 1.4, because only the high-risk column has probabilities that matter. Equal case-control sampling raises every cell by the same amount, the no-history column starts to count, and the coefficient becomes $-0.83$: the sign flips (we reproduced 1.39 and $-0.83$ on the four cells). Weighted CC restores consistency at a cost in variance.

With a perfect pilot, the subsample's score is half the population score. Take pilot $\theta^*$ and subsample parameter 0, where every fitted probability is 1/2. A positive is kept with probability $1-p^*(x)$ and a negative with probability $p^*(x)$, so $$\mathbb E\big[X(Y-\tfrac12)\,a_{\theta^*}(X,Y)\big]=\tfrac12\,\mathbb E\big[X\big(p(X)(1-p^*(X))-(1-p(X))p^*(X)\big)\big]=\tfrac12\,\mathbb E\big[X(Y-p^*(X))\big]=0 .$$ Hence $\theta^*$ is the LCC target whatever the true $f$: sampling stands in for the weights the full-sample score would have applied. CC is LCC with an intercept-only pilot, which is not $\theta^*$ under misspecification, hence its inconsistency.

A consistent pilot suffices for consistency, through coupling and convexity. The population version of the LCC risk is minimized at $\theta^*$ by Step 3, so what remains is convergence when the pilot $\lambda_n$ depends on the data. Drive the accept-reject decisions for every pilot by the same uniforms $u_i$: decisions under $\lambda_n$ and under its limit differ only when $u_i$ falls between two acceptance probabilities at most $\|x_i\|\,\|\lambda_n-\lambda_\infty\|$ apart, and Hoeffding conditional on the data and the pilot controls the few flipped points. Bounded gradients and a finite net make the convergence uniform on compacts, strict convexity keeps the minimizer near $\theta^*$, and nonseparability makes each sampled risk coercive.

With an independent pilot, the covariance is a sandwich formula plus a term for the pilot's error. Two errors add. Given the pilot, $\hat\theta$ is an M-estimator on about $n\bar a$ points, so $\sqrt n(\hat\theta-\bar\theta(\lambda_n))$ is asymptotically $N(0,\bar a^{-1}H^{-1}JH^{-1})$, where $\bar\theta(\lambda)$ is the limit for a fixed pilot. That limit moves with the pilot, and the implicit function theorem on its score equation gives $$\bar\theta(\lambda)=\theta^*+H^{-1}C(\lambda-\theta^*)+o(\|\lambda-\theta^*\|).$$ An independent pilot makes the two pieces independent, which gives (iii). Under correct specification $\bar\theta(\lambda)\equiv\theta_0$ for every pilot, so $C=0$, and to first order the pilot's error only changes which points are kept. All the experiments use a data-dependent pilot, which is outside this theorem.

The factor two follows from counting Fisher information. With a correct model and pilot $\theta_0$, each kept point is a fair coin with Fisher information $\tfrac14xx'$, and a point at $x$ is kept with probability $p(1-\tilde p)+(1-p)\tilde p=2p(1-p)$, so the expected information per original point is $$2p(x)(1-p(x))\cdot\tfrac14xx'=\tfrac12\,p(x)(1-p(x))\,xx',$$ exactly half the full sample's, from $n\bar a$ points with $\bar a=2\mathbb E[p(1-p)]\le2P(Y=1)$. For $c\gt1$ the same count gives $H=\tfrac c2I_{\rm full}$ and $J\preceq\tfrac{c(c+1)}4I_{\rm full}$, hence the bound $(1+1/c)\Sigma_{\rm full}$; $c=5$ costs 20% extra variance for a subsample about three times larger. A small check of ours agrees ($P(Y=1)\approx2.6\%$, $n=10^5$, independent pilot): about 4,350 kept points gave variance ratios to the full MLE of 2.3, 2.2 and 1.9, against 1.5, 3.4 and 3.6 for 1:1 case-control on about 5,170.

In the experiments LCC beats both case-control variants. In a misspecified simulation at 1% positives, with 2,000 points per method, LCC had squared bias 0.0049 against 0.023 (weighted CC) and 0.15 (CC), and variance 0.025 against 0.16 and 0.043. In a correctly specified one with strong separation, $\bar a\approx0.005$ and LCC is roughly 100 times more efficient than uniform subsampling. On web spam (60% spam, so marginally balanced but conditionally imbalanced), the predicted variance ratios to the full-sample fit, about 2 for LCC and 5 for CC, held approximately, slightly optimistic.