diff --git a/docs/superpowers/specs/2026-06-11-wide-angle-distortion-design.md b/docs/superpowers/specs/2026-06-11-wide-angle-distortion-design.md new file mode 100644 index 0000000..a6b3fed --- /dev/null +++ b/docs/superpowers/specs/2026-06-11-wide-angle-distortion-design.md @@ -0,0 +1,130 @@ +# Wide-Angle Distortion Compensation — Design + +**Date:** 2026-06-11 +**Status:** Approved, pending implementation plan + +## Problem + +The overlay aligns the projected G-code to the live CNC camera feed using a single +**homography** (8-DoF perspective transform, `src/geometry/homography.ts`). A homography +maps straight lines to straight lines. The wide-angle camera introduces **barrel (radial) +distortion**: straight machine paths near the bed edges bow outward in the image. No +homography can represent that, so even with perfect corner calibration the overlay drifts +in the middle of each edge and toward the corners. + +This is a **model** limitation, not a calibration-points problem. + +## Constraints / context + +- Camera is **permanently fixed** relative to the bed → calibration is one-time and durable. +- Correction strategy: **bend the overlay** to match the raw (curved) video. The camera + image is left untouched (no per-frame image rectification). +- Calibration budget: **~8–12 points** jogged to known machine X/Y across the bed, + including near the corners (where distortion is worst). +- Keep the existing overlay pipeline shape: `machine-mm → image-point` for drawing, and + `image-point → machine-mm` for click-to-set-origin / drag-to-rotate. + +## Chosen approach: bivariate polynomial warp + +Replace the homography with a **bivariate polynomial warp** (`PolyWarp`) that maps machine +coordinates directly to normalized image coordinates. The quadratic terms (`x², xy, y²`) +capture both the bed's perspective trapezoid and the leading `r²` term of barrel +distortion. Because the map is **linear in its coefficients**, it is fit with the same +least-squares machinery the homography already used. + +Considered and rejected for now: +- **Physical radial lens model + homography (Brown–Conrady):** physically correct and + reusable, leaves the door open to rectifying the video later, but needs a nonlinear + (Gauss–Newton) solver — more code than the problem currently warrants. +- **Thin-plate spline / mesh warp:** highest accuracy but wants many control points + (20–40) and more code; overkill for the ~8–12-point budget. + +Escalation path: if degree-2 residuals stay high, bump to degree 3 (needs ≥12 points), or +graduate to the radial model. The polynomial is the smallest change that actually fixes the +observed drift. + +### The model + +`PolyWarp` stores coefficients for **two independently-fit maps** over the same calibration +point pairs: + +- **Forward** `machine(x,y) → image(u,v)` — draws the overlay. +- **Inverse** `image(u,v) → machine(x,y)` — feeds click-to-set-origin and drag-to-rotate. + +A polynomial has no closed-form inverse, so rather than invert numerically we fit the +reverse direction directly from the same points. The inverse map only serves UI clicks, so +its few-pixel / sub-mm accuracy is sufficient; forward and inverse need not be exact mutual +inverses. + +Default **degree 2** (6 coefficients per axis, needs ≥6 points). **Degree 3** (10 +coefficients per axis, needs ≥10 points; recommend ≥12 to avoid overfit) is a manual +escalation when residuals warrant it. + +Image coordinates remain **normalized [0,1]** camera-frame fractions, as today — keeps +calibration independent of display/canvas size. + +## Components + +| File | Change | +|------|--------| +| `src/geometry/linalg.ts` | **New.** Extract `solveLinear` (Gaussian elimination w/ partial pivoting) from `homography.ts` so both the warp fit and any future model share it. | +| `src/geometry/polywarp.ts` | **New.** `PolyWarp` type; `estimatePolyWarp(machine, image, degree)` — builds the polynomial design matrix, least-squares-fits forward + inverse; `applyPolyWarp(w, p)`, `applyPolyWarpInverse(w, p)`. | +| `src/geometry/transform.ts` | `projectSegments(...)` and `imageToMachine(...)` take a `PolyWarp` instead of `Mat3`. | +| `src/app/state.ts` | `homography: Mat3 \| null` → `warp: PolyWarp \| null`; `setHomography` → `setWarp`. | +| `src/main.ts` | Wire `setWarp` / `getWarp`; load `cfg.calibration.warp`. | +| `src/app/calibration-ui.ts` | Call `estimatePolyWarp`; keep per-point residual (px) readout as the degree-sufficiency signal; update emitted JSON. | +| `src/app/alignment-ui.ts` | `getWarp(): PolyWarp \| null`. | +| `src/types.ts` | `Calibration.homography: Mat3` → `Calibration.warp: PolyWarp`. | +| `src/config.ts` | Validate `warp` (degree + forward/inverse coefficient arrays) instead of a 9-element homography. | +| `src/geometry/homography.ts` | **Retired** (fully replaced). | + +## Data flow + +``` +G-code (mm) + → applyAlignment (per-job rotate/translate, mm) + → applyPolyWarp forward (mm → normalized image) + → scale to canvas px → draw overlay + +video click (px) + → normalize [0,1] + → applyPolyWarpInverse (normalized image → mm) + → set work origin / rotation +``` + +## Migration + +The committed default calibration in `dist/config.json` is a homography and **will not +carry over** — a homography is not a polynomial. Recalibrate once with ~8–12 spread points +(including corners). The `Calibration` object keeps `imagePoints` / `machinePoints` and +replaces `homography` with `warp`. One-time effort; the fixed camera makes it durable. + +## Error handling + +- `estimatePolyWarp` throws on fewer points than the degree requires, or a singular system + (e.g. collinear / degenerate point layout) — surfaced in the calibration panel like the + current "Need at least 4 points" message, adjusted to the degree's minimum. +- `config.ts` validation rejects malformed `warp` (returns `null` calibration), same + pattern as today's homography validation. +- Per-point residual (px) shown after Compute so a bad fit (mis-click, too-low degree) is + visible immediately. + +## Testing + +- **`src/geometry/polywarp.test.ts`** (new): + - Round-trip: `applyPolyWarpInverse(w, applyPolyWarp(w, p)) ≈ p` within tolerance. + - Fit a **synthetic barrel-distorted grid** (known radial coefficient) and assert + sub-pixel residuals — the case a homography provably fails; regression guard for the + whole feature. + - Too-few-points and degenerate-layout throw. +- **`src/geometry/linalg.test.ts`** (new or moved): `solveLinear` correctness, singular + throws. +- Update `transform.test.ts`, `calibration-ui.test.ts`, `state.test.ts` for `PolyWarp`. +- Retire `homography.test.ts`. + +## Out of scope (YAGNI) + +- Per-frame video rectification (we bend the overlay instead). +- Physical lens model / nonlinear solver. +- Thin-plate spline / many-point calibration. +- Backward-compat reading of legacy homography configs (one-time recalibration instead).