GCodeOverlay/docs/superpowers/specs/2026-06-11-wide-angle-distortion-design.md

# Wide-Angle Distortion Compensation — Design

**Date:** 2026-06-11
**Status:** Approved, pending implementation plan

## Problem

The overlay aligns the projected G-code to the live CNC camera feed using a single
**homography** (8-DoF perspective transform, `src/geometry/homography.ts`). A homography
maps straight lines to straight lines. The wide-angle camera introduces **barrel (radial)
distortion**: straight machine paths near the bed edges bow outward in the image. No
homography can represent that, so even with perfect corner calibration the overlay drifts
in the middle of each edge and toward the corners.

This is a **model** limitation, not a calibration-points problem.

## Constraints / context

- Camera is **permanently fixed** relative to the bed → calibration is one-time and durable.
- Correction strategy: **bend the overlay** to match the raw (curved) video. The camera
  image is left untouched (no per-frame image rectification).
- Calibration budget: **~8–12 points** jogged to known machine X/Y across the bed,
  including near the corners (where distortion is worst).
- Keep the existing overlay pipeline shape: `machine-mm → image-point` for drawing, and
  `image-point → machine-mm` for click-to-set-origin / drag-to-rotate.

## Chosen approach: bivariate polynomial warp

Replace the homography with a **bivariate polynomial warp** (`PolyWarp`) that maps machine
coordinates directly to normalized image coordinates. The quadratic terms (`x², xy, y²`)
capture both the bed's perspective trapezoid and the leading `r²` term of barrel
distortion. Because the map is **linear in its coefficients**, it is fit with the same
least-squares machinery the homography already used.

Considered and rejected for now:
- **Physical radial lens model + homography (Brown–Conrady):** physically correct and
  reusable, leaves the door open to rectifying the video later, but needs a nonlinear
  (Gauss–Newton) solver — more code than the problem currently warrants.
- **Thin-plate spline / mesh warp:** highest accuracy but wants many control points
  (20–40) and more code; overkill for the ~8–12-point budget.

Escalation path: if degree-2 residuals stay high, bump to degree 3 (needs ≥12 points), or
graduate to the radial model. The polynomial is the smallest change that actually fixes the
observed drift.

### The model

`PolyWarp` stores coefficients for **two independently-fit maps** over the same calibration
point pairs:

- **Forward** `machine(x,y) → image(u,v)` — draws the overlay.
- **Inverse** `image(u,v) → machine(x,y)` — feeds click-to-set-origin and drag-to-rotate.

A polynomial has no closed-form inverse, so rather than invert numerically we fit the
reverse direction directly from the same points. The inverse map only serves UI clicks, so
its few-pixel / sub-mm accuracy is sufficient; forward and inverse need not be exact mutual
inverses.

Default **degree 2** (6 coefficients per axis, needs ≥6 points). **Degree 3** (10
coefficients per axis, needs ≥10 points; recommend ≥12 to avoid overfit) is a manual
escalation when residuals warrant it.

Image coordinates remain **normalized [0,1]** camera-frame fractions, as today — keeps
calibration independent of display/canvas size.

## Components

| File | Change |
|------|--------|
| `src/geometry/linalg.ts` | **New.** Extract `solveLinear` (Gaussian elimination w/ partial pivoting) from `homography.ts` so both the warp fit and any future model share it. |
| `src/geometry/polywarp.ts` | **New.** `PolyWarp` type; `estimatePolyWarp(machine, image, degree)` — builds the polynomial design matrix, least-squares-fits forward + inverse; `applyPolyWarp(w, p)`, `applyPolyWarpInverse(w, p)`. |
| `src/geometry/transform.ts` | `projectSegments(...)` and `imageToMachine(...)` take a `PolyWarp` instead of `Mat3`. |
| `src/app/state.ts` | `homography: Mat3 \| null` → `warp: PolyWarp \| null`; `setHomography` → `setWarp`. |
| `src/main.ts` | Wire `setWarp` / `getWarp`; load `cfg.calibration.warp`. |
| `src/app/calibration-ui.ts` | Call `estimatePolyWarp`; keep per-point residual (px) readout as the degree-sufficiency signal; update emitted JSON. |
| `src/app/alignment-ui.ts` | `getWarp(): PolyWarp \| null`. |
| `src/types.ts` | `Calibration.homography: Mat3` → `Calibration.warp: PolyWarp`. |
| `src/config.ts` | Validate `warp` (degree + forward/inverse coefficient arrays) instead of a 9-element homography. |
| `src/geometry/homography.ts` | **Retired** (fully replaced). |

## Data flow

```
G-code (mm)
  → applyAlignment (per-job rotate/translate, mm)
  → applyPolyWarp forward (mm → normalized image)
  → scale to canvas px → draw overlay

video click (px)
  → normalize [0,1]
  → applyPolyWarpInverse (normalized image → mm)
  → set work origin / rotation
```

## Migration

The committed default calibration in `dist/config.json` is a homography and **will not
carry over** — a homography is not a polynomial. Recalibrate once with ~8–12 spread points
(including corners). The `Calibration` object keeps `imagePoints` / `machinePoints` and
replaces `homography` with `warp`. One-time effort; the fixed camera makes it durable.

## Error handling

- `estimatePolyWarp` throws on fewer points than the degree requires, or a singular system
  (e.g. collinear / degenerate point layout) — surfaced in the calibration panel like the
  current "Need at least 4 points" message, adjusted to the degree's minimum.
- `config.ts` validation rejects malformed `warp` (returns `null` calibration), same
  pattern as today's homography validation.
- Per-point residual (px) shown after Compute so a bad fit (mis-click, too-low degree) is
  visible immediately.

## Testing

- **`src/geometry/polywarp.test.ts`** (new):
  - Round-trip: `applyPolyWarpInverse(w, applyPolyWarp(w, p)) ≈ p` within tolerance.
  - Fit a **synthetic barrel-distorted grid** (known radial coefficient) and assert
    sub-pixel residuals — the case a homography provably fails; regression guard for the
    whole feature.
  - Too-few-points and degenerate-layout throw.
- **`src/geometry/linalg.test.ts`** (new or moved): `solveLinear` correctness, singular
  throws.
- Update `transform.test.ts`, `calibration-ui.test.ts`, `state.test.ts` for `PolyWarp`.
- Retire `homography.test.ts`.

## Out of scope (YAGNI)

- Per-frame video rectification (we bend the overlay instead).
- Physical lens model / nonlinear solver.
- Thin-plate spline / many-point calibration.
- Backward-compat reading of legacy homography configs (one-time recalibration instead).