GCodeOverlay/docs/superpowers/specs/2026-06-11-wide-angle-distortion-design.md

Wide-Angle Distortion Compensation — Design

Date: 2026-06-11 Status: Approved, pending implementation plan

Problem

The overlay aligns the projected G-code to the live CNC camera feed using a single homography (8-DoF perspective transform, src/geometry/homography.ts). A homography maps straight lines to straight lines. The wide-angle camera introduces barrel (radial) distortion: straight machine paths near the bed edges bow outward in the image. No homography can represent that, so even with perfect corner calibration the overlay drifts in the middle of each edge and toward the corners.

This is a model limitation, not a calibration-points problem.

Constraints / context

• Camera is permanently fixed relative to the bed → calibration is one-time and durable.
• Correction strategy: bend the overlay to match the raw (curved) video. The camera image is left untouched (no per-frame image rectification).
• Calibration budget: ~12–16 points jogged to known machine X/Y across the bed, including near the corners (where distortion is worst). See Calibration target below.
• Keep the existing overlay pipeline shape: machine-mm → image-point for drawing, and image-point → machine-mm for click-to-set-origin / drag-to-rotate.

Chosen approach: bivariate polynomial warp

Replace the homography with a bivariate polynomial warp (PolyWarp) that maps machine coordinates directly to normalized image coordinates. The quadratic terms (x², xy, y²) capture both the bed's perspective trapezoid and the leading r² term of barrel distortion. Because the map is linear in its coefficients, it is fit with the same least-squares machinery the homography already used.

Considered and rejected for now:

• Physical radial lens model + homography (Brown–Conrady): physically correct and reusable, leaves the door open to rectifying the video later, but needs a nonlinear (Gauss–Newton) solver — more code than the problem currently warrants.
• Thin-plate spline / mesh warp: highest accuracy but wants many control points (20–40) and more code; overkill for the ~8–12-point budget.

Escalation path: if degree-2 residuals stay high, bump to degree 3 (needs ≥12 points), or graduate to the radial model. The polynomial is the smallest change that actually fixes the observed drift.

The model

PolyWarp stores coefficients for two independently-fit maps over the same calibration point pairs:

• Forward machine(x,y) → image(u,v) — draws the overlay.
• Inverse image(u,v) → machine(x,y) — feeds click-to-set-origin and drag-to-rotate.

A polynomial has no closed-form inverse, so rather than invert numerically we fit the reverse direction directly from the same points. The inverse map only serves UI clicks, so its few-pixel / sub-mm accuracy is sufficient; forward and inverse need not be exact mutual inverses.

Default degree 2 (6 coefficients per axis, needs ≥6 points). Degree 3 (10 coefficients per axis, needs ≥10 points; recommend ≥12 to avoid overfit) is a manual escalation when residuals warrant it.

Image coordinates remain normalized [0,1] camera-frame fractions, as today — keeps calibration independent of display/canvas size.

Components

File	Change
src/geometry/linalg.ts	New. Extract solveLinear (Gaussian elimination w/ partial pivoting) from homography.ts so both the warp fit and any future model share it.
src/geometry/polywarp.ts	New. PolyWarp type; estimatePolyWarp(machine, image, degree) — builds the polynomial design matrix, least-squares-fits forward + inverse; applyPolyWarp(w, p), applyPolyWarpInverse(w, p).
src/geometry/transform.ts	projectSegments(...) and imageToMachine(...) take a PolyWarp instead of Mat3.
src/app/state.ts	homography: Mat3 | null → warp: PolyWarp | null; setHomography → setWarp.
src/main.ts	Wire setWarp / getWarp; load cfg.calibration.warp.
src/app/calibration-ui.ts	Call estimatePolyWarp; keep per-point residual (px) readout as the degree-sufficiency signal; update emitted JSON.
src/app/alignment-ui.ts	getWarp(): PolyWarp | null.
src/types.ts	Calibration.homography: Mat3 → Calibration.warp: PolyWarp.
src/config.ts	Validate warp (degree + forward/inverse coefficient arrays) instead of a 9-element homography.
src/geometry/homography.ts	Retired (fully replaced).

Data flow

G-code (mm)
  → applyAlignment (per-job rotate/translate, mm)
  → applyPolyWarp forward (mm → normalized image)
  → scale to canvas px → draw overlay

video click (px)
  → normalize [0,1]
  → applyPolyWarpInverse (normalized image → mm)
  → set work origin / rotation

Calibration target (practical procedure)

A box + "#" drawn on the bed: lines at the thirds of each axis, i.e. x ∈ {0, ⅓, ⅔, 1} and y ∈ {0, ⅓, ⅔, 1}. Their 16 intersections are the calibration points — 12 on the perimeter (4 corners + 2 per side) plus 4 interior crossings at the centre of the "#". Click all 16 in the feed.

The 4 interior points matter: a polynomial constrained only on the boundary can wiggle in the interior, so the centre crossings pin the fit where edge points can't.

Capture by jog-and-mark, not ruler-and-guess: jog the spindle to each target coordinate and mark at the tip, so the commanded machine coordinate is exact and the mark sits exactly under the tip. Then click each mark in the video.

For the 2440×1220 bed the 16 targets are every combination of:

X: 0    813.3   1626.7   2440
Y: 0    406.7    813.3    1220

Interior four: (813.3, 406.7), (1626.7, 406.7), (813.3, 813.3), (1626.7, 813.3).

This 16-point spread also clears the degree-3 minimum, so escalating from degree 2 to 3 needs no extra capture.

Migration

The committed default calibration in dist/config.json is a homography and will not carry over — a homography is not a polynomial. Recalibrate once with ~8–12 spread points (including corners). The Calibration object keeps imagePoints / machinePoints and replaces homography with warp. One-time effort; the fixed camera makes it durable.

Error handling

• estimatePolyWarp throws on fewer points than the degree requires, or a singular system (e.g. collinear / degenerate point layout) — surfaced in the calibration panel like the current "Need at least 4 points" message, adjusted to the degree's minimum.
• config.ts validation rejects malformed warp (returns null calibration), same pattern as today's homography validation.
• Per-point residual (px) shown after Compute so a bad fit (mis-click, too-low degree) is visible immediately.

Testing

• src/geometry/polywarp.test.ts (new):
  • Round-trip: applyPolyWarpInverse(w, applyPolyWarp(w, p)) ≈ p within tolerance.
  • Fit a synthetic barrel-distorted grid (known radial coefficient) and assert sub-pixel residuals — the case a homography provably fails; regression guard for the whole feature.
  • Too-few-points and degenerate-layout throw.
• src/geometry/linalg.test.ts (new or moved): solveLinear correctness, singular throws.
• Update transform.test.ts, calibration-ui.test.ts, state.test.ts for PolyWarp.
• Retire homography.test.ts.

Out of scope (YAGNI)

• Per-frame video rectification (we bend the overlay instead).
• Physical lens model / nonlinear solver.
• Thin-plate spline / many-point calibration.
• Backward-compat reading of legacy homography configs (one-time recalibration instead).