164 lines
8.1 KiB
Markdown
164 lines
8.1 KiB
Markdown
# Wide-Angle Distortion Compensation — Design
|
||
|
||
**Date:** 2026-06-11
|
||
**Status:** Approved, pending implementation plan
|
||
|
||
## Problem
|
||
|
||
The overlay aligns the projected G-code to the live CNC camera feed using a single
|
||
**homography** (8-DoF perspective transform, `src/geometry/homography.ts`). A homography
|
||
maps straight lines to straight lines. The wide-angle camera introduces **barrel (radial)
|
||
distortion**: straight machine paths near the bed edges bow outward in the image. No
|
||
homography can represent that, so even with perfect corner calibration the overlay drifts
|
||
in the middle of each edge and toward the corners.
|
||
|
||
This is a **model** limitation, not a calibration-points problem.
|
||
|
||
## Constraints / context
|
||
|
||
- Camera is **permanently fixed** relative to the bed → calibration is one-time and durable.
|
||
- Correction strategy: **bend the overlay** to match the raw (curved) video. The camera
|
||
image is left untouched (no per-frame image rectification).
|
||
- Calibration budget: **~12–16 points** jogged to known machine X/Y across the bed,
|
||
including near the corners (where distortion is worst). See *Calibration target* below.
|
||
- Keep the existing overlay pipeline shape: `machine-mm → image-point` for drawing, and
|
||
`image-point → machine-mm` for click-to-set-origin / drag-to-rotate.
|
||
|
||
## Chosen approach: bivariate polynomial warp
|
||
|
||
Replace the homography with a **bivariate polynomial warp** (`PolyWarp`) that maps machine
|
||
coordinates directly to normalized image coordinates. The cubic terms (`x³, x²y, xy², y³`)
|
||
are what capture the symmetric radial barrel distortion (see the degree correction below);
|
||
the lower-order terms absorb the bed's perspective trapezoid. Because the map is **linear in
|
||
its coefficients**, it is fit with the same least-squares machinery the homography already
|
||
used.
|
||
|
||
Considered and rejected for now:
|
||
- **Physical radial lens model + homography (Brown–Conrady):** physically correct and
|
||
reusable, leaves the door open to rectifying the video later, but needs a nonlinear
|
||
(Gauss–Newton) solver — more code than the problem currently warrants.
|
||
- **Thin-plate spline / mesh warp:** highest accuracy but wants many control points
|
||
(20–40) and more code; overkill for the ~8–12-point budget.
|
||
|
||
Fallback path: with sparse calibration (<10 points) drop to degree 2; if residuals stay
|
||
high after degree 3, graduate to the radial model. The polynomial is the smallest change
|
||
that actually fixes the observed drift.
|
||
|
||
> **Correction (validated during implementation):** barrel distortion multiplies the
|
||
> coordinate by `(1 + k·r²)`, so as a function of machine coordinates it is **degree 3**
|
||
> (`dx·r²` expands to `dx³ + dx·dy²`), not degree 2. A degree-2 polynomial therefore does
|
||
> *not* beat a homography on real radial distortion — only degree 3 does. Hence degree 3 is
|
||
> the default below, not an escalation.
|
||
|
||
### The model
|
||
|
||
`PolyWarp` stores coefficients for **two independently-fit maps** over the same calibration
|
||
point pairs:
|
||
|
||
- **Forward** `machine(x,y) → image(u,v)` — draws the overlay.
|
||
- **Inverse** `image(u,v) → machine(x,y)` — feeds click-to-set-origin and drag-to-rotate.
|
||
|
||
A polynomial has no closed-form inverse, so rather than invert numerically we fit the
|
||
reverse direction directly from the same points. The inverse map only serves UI clicks, so
|
||
its few-pixel / sub-mm accuracy is sufficient; forward and inverse need not be exact mutual
|
||
inverses.
|
||
|
||
Default **degree 3** (10 coefficients per axis, needs ≥10 points) — the degree that
|
||
actually compensates barrel distortion; the 16-point box+# target supports it comfortably.
|
||
**Degree 2** (6 coefficients per axis, needs ≥6 points) is a manual fallback for sparse
|
||
calibration. The calibration UI exposes a degree selector defaulting to 3.
|
||
|
||
Image coordinates remain **normalized [0,1]** camera-frame fractions, as today — keeps
|
||
calibration independent of display/canvas size.
|
||
|
||
## Components
|
||
|
||
| File | Change |
|
||
|------|--------|
|
||
| `src/geometry/linalg.ts` | **New.** Extract `solveLinear` (Gaussian elimination w/ partial pivoting) from `homography.ts` so both the warp fit and any future model share it. |
|
||
| `src/geometry/polywarp.ts` | **New.** `PolyWarp` type; `estimatePolyWarp(machine, image, degree)` — builds the polynomial design matrix, least-squares-fits forward + inverse; `applyPolyWarp(w, p)`, `applyPolyWarpInverse(w, p)`. |
|
||
| `src/geometry/transform.ts` | `projectSegments(...)` and `imageToMachine(...)` take a `PolyWarp` instead of `Mat3`. |
|
||
| `src/app/state.ts` | `homography: Mat3 \| null` → `warp: PolyWarp \| null`; `setHomography` → `setWarp`. |
|
||
| `src/main.ts` | Wire `setWarp` / `getWarp`; load `cfg.calibration.warp`. |
|
||
| `src/app/calibration-ui.ts` | Call `estimatePolyWarp`; keep per-point residual (px) readout as the degree-sufficiency signal; update emitted JSON. |
|
||
| `src/app/alignment-ui.ts` | `getWarp(): PolyWarp \| null`. |
|
||
| `src/types.ts` | `Calibration.homography: Mat3` → `Calibration.warp: PolyWarp`. |
|
||
| `src/config.ts` | Validate `warp` (degree + forward/inverse coefficient arrays) instead of a 9-element homography. |
|
||
| `src/geometry/homography.ts` | **Retired** (fully replaced). |
|
||
|
||
## Data flow
|
||
|
||
```
|
||
G-code (mm)
|
||
→ applyAlignment (per-job rotate/translate, mm)
|
||
→ applyPolyWarp forward (mm → normalized image)
|
||
→ scale to canvas px → draw overlay
|
||
|
||
video click (px)
|
||
→ normalize [0,1]
|
||
→ applyPolyWarpInverse (normalized image → mm)
|
||
→ set work origin / rotation
|
||
```
|
||
|
||
## Calibration target (practical procedure)
|
||
|
||
A **box + "#"** drawn on the bed: lines at the thirds of each axis, i.e.
|
||
`x ∈ {0, ⅓, ⅔, 1}` and `y ∈ {0, ⅓, ⅔, 1}`. Their 16 intersections are the calibration
|
||
points — **12 on the perimeter** (4 corners + 2 per side) plus **4 interior crossings** at
|
||
the centre of the "#". Click all 16 in the feed.
|
||
|
||
The 4 interior points matter: a polynomial constrained only on the boundary can wiggle in
|
||
the interior, so the centre crossings pin the fit where edge points can't.
|
||
|
||
**Capture by jog-and-mark, not ruler-and-guess:** jog the spindle to each target
|
||
coordinate and mark at the tip, so the commanded machine coordinate is exact and the mark
|
||
sits exactly under the tip. Then click each mark in the video.
|
||
|
||
For the 2440×1220 bed the 16 targets are every combination of:
|
||
|
||
```
|
||
X: 0 813.3 1626.7 2440
|
||
Y: 0 406.7 813.3 1220
|
||
```
|
||
|
||
Interior four: (813.3, 406.7), (1626.7, 406.7), (813.3, 813.3), (1626.7, 813.3).
|
||
|
||
This 16-point spread also clears the degree-3 minimum, so escalating from degree 2 to 3
|
||
needs no extra capture.
|
||
|
||
## Migration
|
||
|
||
The committed default calibration in `dist/config.json` is a homography and **will not
|
||
carry over** — a homography is not a polynomial. Recalibrate once with ~8–12 spread points
|
||
(including corners). The `Calibration` object keeps `imagePoints` / `machinePoints` and
|
||
replaces `homography` with `warp`. One-time effort; the fixed camera makes it durable.
|
||
|
||
## Error handling
|
||
|
||
- `estimatePolyWarp` throws on fewer points than the degree requires, or a singular system
|
||
(e.g. collinear / degenerate point layout) — surfaced in the calibration panel like the
|
||
current "Need at least 4 points" message, adjusted to the degree's minimum.
|
||
- `config.ts` validation rejects malformed `warp` (returns `null` calibration), same
|
||
pattern as today's homography validation.
|
||
- Per-point residual (px) shown after Compute so a bad fit (mis-click, too-low degree) is
|
||
visible immediately.
|
||
|
||
## Testing
|
||
|
||
- **`src/geometry/polywarp.test.ts`** (new):
|
||
- Round-trip: `applyPolyWarpInverse(w, applyPolyWarp(w, p)) ≈ p` within tolerance.
|
||
- Fit a **synthetic barrel-distorted grid** (known radial coefficient) and assert
|
||
sub-pixel residuals — the case a homography provably fails; regression guard for the
|
||
whole feature.
|
||
- Too-few-points and degenerate-layout throw.
|
||
- **`src/geometry/linalg.test.ts`** (new or moved): `solveLinear` correctness, singular
|
||
throws.
|
||
- Update `transform.test.ts`, `calibration-ui.test.ts`, `state.test.ts` for `PolyWarp`.
|
||
- Retire `homography.test.ts`.
|
||
|
||
## Out of scope (YAGNI)
|
||
|
||
- Per-frame video rectification (we bend the overlay instead).
|
||
- Physical lens model / nonlinear solver.
|
||
- Thin-plate spline / many-point calibration.
|
||
- Backward-compat reading of legacy homography configs (one-time recalibration instead).
|