# Wide-Angle Distortion Compensation — Design **Date:** 2026-06-11 **Status:** Approved, pending implementation plan ## Problem The overlay aligns the projected G-code to the live CNC camera feed using a single **homography** (8-DoF perspective transform, `src/geometry/homography.ts`). A homography maps straight lines to straight lines. The wide-angle camera introduces **barrel (radial) distortion**: straight machine paths near the bed edges bow outward in the image. No homography can represent that, so even with perfect corner calibration the overlay drifts in the middle of each edge and toward the corners. This is a **model** limitation, not a calibration-points problem. ## Constraints / context - Camera is **permanently fixed** relative to the bed → calibration is one-time and durable. - Correction strategy: **bend the overlay** to match the raw (curved) video. The camera image is left untouched (no per-frame image rectification). - Calibration budget: **~12–16 points** jogged to known machine X/Y across the bed, including near the corners (where distortion is worst). See *Calibration target* below. - Keep the existing overlay pipeline shape: `machine-mm → image-point` for drawing, and `image-point → machine-mm` for click-to-set-origin / drag-to-rotate. ## Chosen approach: bivariate polynomial warp Replace the homography with a **bivariate polynomial warp** (`PolyWarp`) that maps machine coordinates directly to normalized image coordinates. The cubic terms (`x³, x²y, xy², y³`) are what capture the symmetric radial barrel distortion (see the degree correction below); the lower-order terms absorb the bed's perspective trapezoid. Because the map is **linear in its coefficients**, it is fit with the same least-squares machinery the homography already used. Considered and rejected for now: - **Physical radial lens model + homography (Brown–Conrady):** physically correct and reusable, leaves the door open to rectifying the video later, but needs a nonlinear (Gauss–Newton) solver — more code than the problem currently warrants. - **Thin-plate spline / mesh warp:** highest accuracy but wants many control points (20–40) and more code; overkill for the ~8–12-point budget. Fallback path: with sparse calibration (<10 points) drop to degree 2; if residuals stay high after degree 3, graduate to the radial model. The polynomial is the smallest change that actually fixes the observed drift. > **Correction (validated during implementation):** barrel distortion multiplies the > coordinate by `(1 + k·r²)`, so as a function of machine coordinates it is **degree 3** > (`dx·r²` expands to `dx³ + dx·dy²`), not degree 2. A degree-2 polynomial therefore does > *not* beat a homography on real radial distortion — only degree 3 does. Hence degree 3 is > the default below, not an escalation. ### The model `PolyWarp` stores coefficients for **two independently-fit maps** over the same calibration point pairs: - **Forward** `machine(x,y) → image(u,v)` — draws the overlay. - **Inverse** `image(u,v) → machine(x,y)` — feeds click-to-set-origin and drag-to-rotate. A polynomial has no closed-form inverse, so rather than invert numerically we fit the reverse direction directly from the same points. The inverse map only serves UI clicks, so its few-pixel / sub-mm accuracy is sufficient; forward and inverse need not be exact mutual inverses. Default **degree 3** (10 coefficients per axis, needs ≥10 points) — the degree that actually compensates barrel distortion; the 16-point box+# target supports it comfortably. **Degree 2** (6 coefficients per axis, needs ≥6 points) is a manual fallback for sparse calibration. The calibration UI exposes a degree selector defaulting to 3. Image coordinates remain **normalized [0,1]** camera-frame fractions, as today — keeps calibration independent of display/canvas size. ## Components | File | Change | |------|--------| | `src/geometry/linalg.ts` | **New.** Extract `solveLinear` (Gaussian elimination w/ partial pivoting) from `homography.ts` so both the warp fit and any future model share it. | | `src/geometry/polywarp.ts` | **New.** `PolyWarp` type; `estimatePolyWarp(machine, image, degree)` — builds the polynomial design matrix, least-squares-fits forward + inverse; `applyPolyWarp(w, p)`, `applyPolyWarpInverse(w, p)`. | | `src/geometry/transform.ts` | `projectSegments(...)` and `imageToMachine(...)` take a `PolyWarp` instead of `Mat3`. | | `src/app/state.ts` | `homography: Mat3 \| null` → `warp: PolyWarp \| null`; `setHomography` → `setWarp`. | | `src/main.ts` | Wire `setWarp` / `getWarp`; load `cfg.calibration.warp`. | | `src/app/calibration-ui.ts` | Call `estimatePolyWarp`; keep per-point residual (px) readout as the degree-sufficiency signal; update emitted JSON. | | `src/app/alignment-ui.ts` | `getWarp(): PolyWarp \| null`. | | `src/types.ts` | `Calibration.homography: Mat3` → `Calibration.warp: PolyWarp`. | | `src/config.ts` | Validate `warp` (degree + forward/inverse coefficient arrays) instead of a 9-element homography. | | `src/geometry/homography.ts` | **Retired** (fully replaced). | ## Data flow ``` G-code (mm) → applyAlignment (per-job rotate/translate, mm) → applyPolyWarp forward (mm → normalized image) → scale to canvas px → draw overlay video click (px) → normalize [0,1] → applyPolyWarpInverse (normalized image → mm) → set work origin / rotation ``` ## Calibration target (practical procedure) A **box + "#"** drawn on the bed: lines at the thirds of each axis, i.e. `x ∈ {0, ⅓, ⅔, 1}` and `y ∈ {0, ⅓, ⅔, 1}`. Their 16 intersections are the calibration points — **12 on the perimeter** (4 corners + 2 per side) plus **4 interior crossings** at the centre of the "#". Click all 16 in the feed. The 4 interior points matter: a polynomial constrained only on the boundary can wiggle in the interior, so the centre crossings pin the fit where edge points can't. **Capture by jog-and-mark, not ruler-and-guess:** jog the spindle to each target coordinate and mark at the tip, so the commanded machine coordinate is exact and the mark sits exactly under the tip. Then click each mark in the video. For the 2440×1220 bed the 16 targets are every combination of: ``` X: 0 813.3 1626.7 2440 Y: 0 406.7 813.3 1220 ``` Interior four: (813.3, 406.7), (1626.7, 406.7), (813.3, 813.3), (1626.7, 813.3). This 16-point spread also clears the degree-3 minimum, so escalating from degree 2 to 3 needs no extra capture. ## Migration The committed default calibration in `dist/config.json` is a homography and **will not carry over** — a homography is not a polynomial. Recalibrate once with ~8–12 spread points (including corners). The `Calibration` object keeps `imagePoints` / `machinePoints` and replaces `homography` with `warp`. One-time effort; the fixed camera makes it durable. ## Error handling - `estimatePolyWarp` throws on fewer points than the degree requires, or a singular system (e.g. collinear / degenerate point layout) — surfaced in the calibration panel like the current "Need at least 4 points" message, adjusted to the degree's minimum. - `config.ts` validation rejects malformed `warp` (returns `null` calibration), same pattern as today's homography validation. - Per-point residual (px) shown after Compute so a bad fit (mis-click, too-low degree) is visible immediately. ## Testing - **`src/geometry/polywarp.test.ts`** (new): - Round-trip: `applyPolyWarpInverse(w, applyPolyWarp(w, p)) ≈ p` within tolerance. - Fit a **synthetic barrel-distorted grid** (known radial coefficient) and assert sub-pixel residuals — the case a homography provably fails; regression guard for the whole feature. - Too-few-points and degenerate-layout throw. - **`src/geometry/linalg.test.ts`** (new or moved): `solveLinear` correctness, singular throws. - Update `transform.test.ts`, `calibration-ui.test.ts`, `state.test.ts` for `PolyWarp`. - Retire `homography.test.ts`. ## Out of scope (YAGNI) - Per-frame video rectification (we bend the overlay instead). - Physical lens model / nonlinear solver. - Thin-plate spline / many-point calibration. - Backward-compat reading of legacy homography configs (one-time recalibration instead).