Projects  /  Decision Mesh

Decision Mesh

A regression tree cuts the plane into boxes and predicts a constant in each, so its fit is a staircase. A decision mesh cuts the plane into pieces that share their corners and predicts across each piece from those corner heights, so its fit is continuous. Below, four fits grow on the same noisy data, each given the same number of free parameters.

How each mesh cuts

Freeform triangles. Every edge carries a candidate vertex at its midpoint; activating it splits the one or two triangles on that edge through their opposite corners. Any edge can be cut, so the mesh can turn to follow a feature in any direction, at the price of thin triangles, which the aspect limit keeps in check.

Right triangles. Every triangle is right isosceles and is only ever cut through the midpoint of its hypotenuse, which gives two smaller right isosceles triangles. If the triangle across that hypotenuse would be left with a vertex hanging in its side, it is cut first (the completion step), so the mesh never has hanging vertices and its triangles never get thin, however long it grows. It cannot turn to follow a feature; it can only get finer where the feature is.

Rectangles. Every cell can be cut across either axis, so the refinement can follow anisotropy that lines up with the axes; inside a cell the fit is bilinear, which carries the interaction of the two coordinates that a flat triangle cannot. A vertex left hanging in the middle of a coarser cell's edge takes that edge's endpoints as its parents, which keeps the surface continuous with no constraints to store.

In the right triangles and the rectangles, a vertex's height is its parents' average plus a surplus of its own, and a new vertex costs a parameter only once the rule frees its surplus. In all three, each candidate knows the least-squares change it would make with everything else held fixed and how much squared error that would save; each step takes the largest saving. The tree is ordinary best-first CART. All four are grown to the same number of free parameters: one height or surplus per free vertex, one mean per tree leaf.

after each refit
Ready Starting when the meshes come into view.
Freeform triangles
Right triangles
Rectangles
Regression tree
Truth the surface under the noise
△ freeform triangles only

Error against the truth

Root-mean-square distance from the true surface over the square, against the number of free parameters; the dashed line is the noise σ. On all six surfaces the right triangles get closest to the truth. Best RMSE over a run (freeform / right / rectangles / tree): cliff 0.161 / 0.130 / 0.170 / 0.217, hills 0.127 / 0.091 / 0.123 / 0.152, saddle 0.099 / 0.075 / 0.120 / 0.229, disk 0.329 / 0.277 / 0.284 / 0.303, ripples 0.578 / 0.464 / 0.485 / 0.495, waves 0.206 / 0.162 / 0.164 / 0.232. Their lead is largest early, at fifty parameters. Probably because their pieces cannot go thin, and a cut costs no parameter until the rule frees its surplus, so resolution goes where the error is without paying for completion. Late in a run every mesh starts fitting noise and the curves turn up. A minimum of 3 points per piece is the default (0 is the published freeform algorithm), and "refresh fits" refits the freeform candidates around a vertex whose height has just moved, which the published algorithm leaves stale.

The freeform engine is a line-for-line JavaScript port of my Decision Mesh, matching the Python's squared error to 1e-12 over the first 400 steps. The right-triangle and rectangular engines are written for this page: they are the geometries of triangular-decision-mesh (newest-vertex bisection with completion) and rectangular-decision-mesh (dyadic bisection, two parents per vertex, bilinear cells), grown by the same greedy rule as the freeform triangles, so the panels differ only in their geometry. Data, cuts and pictures are all computed on this page.

None of these four has a stopping rule: they grow until the page runs out of budget. The estimator I actually use chooses its cuts with an empirical-null false-discovery gate and stops by itself; it runs here too, compiled from its C++, and How the gate decides walks through one of its fits step by step.

Also on this site: the noise-state game explorer, which solves the games of my dissertation in your browser.