diff --git a/docs/superpowers/plans/2026-06-11-wide-angle-distortion.md b/docs/superpowers/plans/2026-06-11-wide-angle-distortion.md new file mode 100644 index 0000000..4af783c --- /dev/null +++ b/docs/superpowers/plans/2026-06-11-wide-angle-distortion.md @@ -0,0 +1,1040 @@ +# Wide-Angle Distortion Compensation Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Replace the single homography with a bivariate polynomial warp so the G-code overlay curves to match the wide-angle camera's barrel distortion. + +**Architecture:** A `PolyWarp` stores two independently least-squares-fit polynomial maps — forward (machine-mm → normalized image) for drawing and inverse (normalized image → machine-mm) for click interactions. Machine inputs are normalized (center+scale) before building monomials to keep the normal equations well-conditioned. Degree 2 is the default; degree 3 is selectable. The homography path is fully retired. + +**Tech Stack:** TypeScript, Vite, Vitest. No new dependencies. + +**Reference spec:** `docs/superpowers/specs/2026-06-11-wide-angle-distortion-design.md` + +--- + +## Notes for the implementer + +- **Vitest uses esbuild**, which strips types without whole-program type-checking. So per-file tests run green even while *other* files still reference the old `homography`/`Mat3` API mid-migration. The final `tsc --noEmit` (via `npm run build`, Task 8) is the gate that catches cross-file type holes — do not skip it. +- Run a single test file with: `npx vitest run src/path/to/file.test.ts` +- Run everything with: `npm test` +- Image coordinates are normalized `[0,1]` camera-frame fractions throughout (unchanged from today). + +--- + +## Task 1: Linear-algebra helper (`linalg.ts`) + +Extract the existing Gaussian-elimination solver into its own module and add a least-squares helper that both polynomial fits will share. + +**Files:** +- Create: `src/geometry/linalg.ts` +- Test: `src/geometry/linalg.test.ts` + +- [ ] **Step 1: Write the failing test** + +Create `src/geometry/linalg.test.ts`: + +```typescript +import { describe, it, expect } from 'vitest'; +import { solveLinear, leastSquares } from './linalg'; + +describe('linalg', () => { + it('solveLinear solves a 2x2 system', () => { + // 2x + y = 5 ; x + 3y = 10 → x = 1, y = 3 + const x = solveLinear([[2, 1], [1, 3]], [5, 10]); + expect(x[0]!).toBeCloseTo(1, 9); + expect(x[1]!).toBeCloseTo(3, 9); + }); + + it('solveLinear throws on a singular system', () => { + expect(() => solveLinear([[1, 2], [2, 4]], [3, 6])).toThrow(); + }); + + it('leastSquares recovers exact coefficients of an over-determined linear fit', () => { + // model: t = 2*a + 3*b ; rows are [a, b] + const rows = [[1, 0], [0, 1], [1, 1], [2, 1]]; + const targets = rows.map((r) => 2 * r[0]! + 3 * r[1]!); + const c = leastSquares(rows, targets); + expect(c[0]!).toBeCloseTo(2, 9); + expect(c[1]!).toBeCloseTo(3, 9); + }); +}); +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `npx vitest run src/geometry/linalg.test.ts` +Expected: FAIL — cannot find module `./linalg`. + +- [ ] **Step 3: Write minimal implementation** + +Create `src/geometry/linalg.ts`: + +```typescript +/** Solve a square linear system A x = b by Gaussian elimination with partial pivoting. */ +export function solveLinear(A: number[][], b: number[]): number[] { + const n = b.length; + const M = A.map((row, i) => [...row, b[i]!]); + for (let col = 0; col < n; col++) { + let pivot = col; + for (let r = col + 1; r < n; r++) { + if (Math.abs(M[r]![col]!) > Math.abs(M[pivot]![col]!)) pivot = r; + } + if (Math.abs(M[pivot]![col]!) < 1e-12) throw new Error('Singular system'); + [M[col], M[pivot]] = [M[pivot]!, M[col]!]; + const pivRow = M[col]!; + for (let r = 0; r < n; r++) { + if (r === col) continue; + const factor = M[r]![col]! / pivRow[col]!; + for (let k = col; k <= n; k++) M[r]![k]! -= factor * pivRow[k]!; + } + } + return M.map((row, i) => row[n]! / row[i]!); +} + +/** + * Least-squares solve for c minimizing ‖rows·c − targets‖ via the normal equations + * (rowsᵀ rows) c = rowsᵀ targets. `rows` is M×N (M ≥ N), `targets` is length M. + */ +export function leastSquares(rows: number[][], targets: number[]): number[] { + const n = rows[0]!.length; + const ata: number[][] = Array.from({ length: n }, () => new Array(n).fill(0)); + const atb: number[] = new Array(n).fill(0); + for (let r = 0; r < rows.length; r++) { + const row = rows[r]!; + for (let i = 0; i < n; i++) { + atb[i]! += row[i]! * targets[r]!; + for (let j = 0; j < n; j++) ata[i]![j]! += row[i]! * row[j]!; + } + } + return solveLinear(ata, atb); +} +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `npx vitest run src/geometry/linalg.test.ts` +Expected: PASS (3 tests). + +- [ ] **Step 5: Commit** + +```bash +git add src/geometry/linalg.ts src/geometry/linalg.test.ts +git commit -m "feat: extract linalg solver + add least-squares helper" +``` + +--- + +## Task 2: Polynomial warp model (`polywarp.ts`) + +The core of the feature: fit and apply the bivariate polynomial maps. + +**Files:** +- Create: `src/geometry/polywarp.ts` +- Modify: `src/types.ts` (add `PolyMap`, `PolyWarp` — pure types only) +- Test: `src/geometry/polywarp.test.ts` + +- [ ] **Step 1: Add the shared types** + +In `src/types.ts`, add after the `Mat3` type (leave `Mat3` itself in place; it becomes unused but harmless): + +```typescript +/** One fitted polynomial map: dst = Σ cᵢ · monomialᵢ(normalized src). */ +export interface PolyMap { + degree: number; + /** Input normalization applied before evaluating monomials. */ + norm: { off: Vec2; scl: Vec2 }; + /** Coefficients for the u/x output channel. */ + cu: number[]; + /** Coefficients for the v/y output channel. */ + cv: number[]; +} + +/** A bidirectional polynomial warp: forward machine→image, inverse image→machine. */ +export interface PolyWarp { + degree: number; + fwd: PolyMap; + inv: PolyMap; +} +``` + +- [ ] **Step 2: Write the failing test** + +Create `src/geometry/polywarp.test.ts`: + +```typescript +import { describe, it, expect } from 'vitest'; +import { estimatePolyWarp, applyPolyWarp, applyPolyWarpInverse } from './polywarp'; +import { estimateHomography, applyHomography } from './homography'; +import type { Vec2 } from '../types'; + +const dist = (a: Vec2, b: Vec2) => Math.hypot(a[0] - b[0], a[1] - b[1]); + +// 16-point "box + #" calibration layout on a 2440x1220 bed (thirds in each axis). +const XS = [0, 813.3, 1626.7, 2440]; +const YS = [0, 406.7, 813.3, 1220]; +const machineGrid: Vec2[] = XS.flatMap((x) => YS.map((y): Vec2 => [x, y])); + +// "True" camera map: affine into ~[0.1,0.9] then barrel distortion about the image centre. +function trueMap(p: Vec2): Vec2 { + const u0 = 0.1 + 0.8 * (p[0] / 2440); + const v0 = 0.1 + 0.8 * (p[1] / 1220); + const cx = 0.5, cy = 0.5, k = 0.25; + const dx = u0 - cx, dy = v0 - cy, r2 = dx * dx + dy * dy; + const f = 1 + k * r2; + return [cx + dx * f, cy + dy * f]; +} + +describe('polywarp', () => { + it('recovers an exact quadratic map at unseen points', () => { + // dst is an exact degree-2 polynomial of src → fit must reproduce it. + const quad = (p: Vec2): Vec2 => { + const x = p[0] / 2440, y = p[1] / 1220; + return [0.2 + 0.5 * x + 0.1 * y + 0.3 * x * x, 0.1 + 0.6 * y + 0.2 * x * y]; + }; + const image = machineGrid.map(quad); + const w = estimatePolyWarp(machineGrid, image, 2); + for (const m of [[400, 200], [1200, 900], [2000, 300]] as Vec2[]) { + expect(dist(applyPolyWarp(w, m), quad(m))).toBeLessThan(1e-6); + } + }); + + it('beats a homography on barrel-distorted data', () => { + const image = machineGrid.map(trueMap); + const w = estimatePolyWarp(machineGrid, image, 2); + const H = estimateHomography(machineGrid, image); + // Evaluate on off-grid interior points (fifths), where barrel error is real. + const test: Vec2[] = [[488, 244], [1464, 732], [1952, 488], [976, 976]]; + const polyMax = Math.max(...test.map((m) => dist(applyPolyWarp(w, m), trueMap(m)))); + const homoMax = Math.max(...test.map((m) => dist(applyHomography(H, m), trueMap(m)))); + expect(polyMax).toBeLessThan(homoMax * 0.5); + }); + + it('inverse round-trips machine points within a small tolerance', () => { + const image = machineGrid.map(trueMap); + const w = estimatePolyWarp(machineGrid, image, 3); + for (const m of [[400, 200], [1200, 900], [2000, 300]] as Vec2[]) { + expect(dist(applyPolyWarpInverse(w, applyPolyWarp(w, m)), m)).toBeLessThan(2); // mm + } + }); + + it('throws when given fewer points than the degree needs', () => { + const five = machineGrid.slice(0, 5); + const img = five.map(trueMap); + expect(() => estimatePolyWarp(five, img, 2)).toThrow(); // degree 2 needs 6 + }); +}); +``` + +- [ ] **Step 3: Run test to verify it fails** + +Run: `npx vitest run src/geometry/polywarp.test.ts` +Expected: FAIL — cannot find module `./polywarp`. + +- [ ] **Step 4: Write minimal implementation** + +Create `src/geometry/polywarp.ts`: + +```typescript +import type { Vec2, PolyMap, PolyWarp } from '../types'; +import { leastSquares } from './linalg'; + +/** Number of monomial terms of total degree ≤ d in two variables. */ +function termCount(degree: number): number { + return ((degree + 1) * (degree + 2)) / 2; +} + +/** Monomials x^i·y^j with i+j ≤ degree, in a fixed order. */ +function monomials(degree: number, x: number, y: number): number[] { + const terms: number[] = []; + for (let i = 0; i <= degree; i++) { + for (let j = 0; j <= degree - i; j++) terms.push(x ** i * y ** j); + } + return terms; +} + +/** Center+scale so normalized inputs sit roughly in [-1,1] (conditions the fit). */ +function computeNorm(pts: Vec2[]): { off: Vec2; scl: Vec2 } { + const xs = pts.map((p) => p[0]); + const ys = pts.map((p) => p[1]); + const off: Vec2 = [xs.reduce((a, b) => a + b, 0) / xs.length, ys.reduce((a, b) => a + b, 0) / ys.length]; + const sclx = Math.max(...xs.map((x) => Math.abs(x - off[0]))) || 1; + const scly = Math.max(...ys.map((y) => Math.abs(y - off[1]))) || 1; + return { off, scl: [sclx, scly] }; +} + +function applyNorm(norm: { off: Vec2; scl: Vec2 }, p: Vec2): Vec2 { + return [(p[0] - norm.off[0]) / norm.scl[0], (p[1] - norm.off[1]) / norm.scl[1]]; +} + +/** Fit one polynomial map src → dst of the given degree. */ +function fitPolyMap(src: Vec2[], dst: Vec2[], degree: number): PolyMap { + if (src.length !== dst.length) throw new Error('fitPolyMap: src/dst length mismatch'); + const need = termCount(degree); + if (src.length < need) throw new Error(`Need at least ${need} points for degree ${degree}`); + const norm = computeNorm(src); + const rows = src.map((p) => { + const [x, y] = applyNorm(norm, p); + return monomials(degree, x, y); + }); + const cu = leastSquares(rows, dst.map((d) => d[0])); + const cv = leastSquares(rows, dst.map((d) => d[1])); + return { degree, norm, cu, cv }; +} + +/** Evaluate a fitted polynomial map at a point. */ +export function applyPolyMap(m: PolyMap, p: Vec2): Vec2 { + const [x, y] = applyNorm(m.norm, p); + const mon = monomials(m.degree, x, y); + let u = 0; + let v = 0; + for (let i = 0; i < mon.length; i++) { + u += m.cu[i]! * mon[i]!; + v += m.cv[i]! * mon[i]!; + } + return [u, v]; +} + +/** Fit a bidirectional warp from matching machine↔image point pairs. */ +export function estimatePolyWarp(machine: Vec2[], image: Vec2[], degree = 2): PolyWarp { + return { + degree, + fwd: fitPolyMap(machine, image, degree), + inv: fitPolyMap(image, machine, degree), + }; +} + +/** machine-mm → normalized image. */ +export function applyPolyWarp(w: PolyWarp, machine: Vec2): Vec2 { + return applyPolyMap(w.fwd, machine); +} + +/** normalized image → machine-mm. */ +export function applyPolyWarpInverse(w: PolyWarp, image: Vec2): Vec2 { + return applyPolyMap(w.inv, image); +} +``` + +- [ ] **Step 5: Run test to verify it passes** + +Run: `npx vitest run src/geometry/polywarp.test.ts` +Expected: PASS (4 tests). `homography.ts` still exists, so the test's homography import resolves. + +- [ ] **Step 6: Commit** + +```bash +git add src/types.ts src/geometry/polywarp.ts src/geometry/polywarp.test.ts +git commit -m "feat: bivariate polynomial warp model" +``` + +--- + +## Task 3: Migrate the transform layer (`transform.ts`) + +Switch `projectSegments` and `imageToMachine` from homography to `PolyWarp`. + +**Files:** +- Modify: `src/geometry/transform.ts` +- Test: `src/geometry/transform.test.ts` + +- [ ] **Step 1: Update the test** + +Replace the entire contents of `src/geometry/transform.test.ts`: + +```typescript +import { describe, it, expect } from 'vitest'; +import { applyAlignment, projectSegments, imageToMachine, alignmentFromOrigin } from './transform'; +import { estimatePolyWarp } from './polywarp'; +import type { Vec2, Segment, Alignment, PolyWarp } from '../types'; + +const close = (a: Vec2, b: Vec2, eps = 1e-6) => + Math.abs(a[0] - b[0]) < eps && Math.abs(a[1] - b[1]) < eps; + +// Identity warp: image == machine. A degree-1 fit on a non-degenerate quad is exact. +const square: Vec2[] = [[0, 0], [10, 0], [10, 10], [0, 10]]; +const IDENT: PolyWarp = estimatePolyWarp(square, square, 1); + +describe('transform', () => { + it('applyAlignment rotates then translates', () => { + const a: Alignment = { tx: 5, ty: 1, rot: Math.PI / 2 }; + expect(close(applyAlignment(a, [1, 0]), [5, 2])).toBe(true); + }); + + it('projectSegments applies alignment then the warp to every point', () => { + const segs: Segment[] = [{ kind: 'cut', points: [[0, 0], [10, 0]] }]; + const a: Alignment = { tx: 2, ty: 3, rot: 0 }; + const out = projectSegments(segs, a, IDENT); + expect(out[0]!.kind).toBe('cut'); + expect(close(out[0]!.points[0]!, [2, 3])).toBe(true); + expect(close(out[0]!.points[1]!, [12, 3])).toBe(true); + }); + + it('imageToMachine inverts the warp', () => { + // Warp scales machine→image by 0.5 (offset 0): machine (4,8) → image (2,4). + const m: Vec2[] = [[0, 0], [10, 0], [10, 10], [0, 10]]; + const img: Vec2[] = m.map((p): Vec2 => [p[0] * 0.5, p[1] * 0.5]); + const w = estimatePolyWarp(m, img, 1); + expect(close(imageToMachine(w, [2, 4]), [4, 8], 1e-4)).toBe(true); + }); + + it('alignmentFromOrigin places work-origin at the given machine point', () => { + const a = alignmentFromOrigin([7, 9], 0); + expect(close(applyAlignment(a, [0, 0]), [7, 9])).toBe(true); + }); +}); +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `npx vitest run src/geometry/transform.test.ts` +Expected: FAIL — `projectSegments`/`imageToMachine` still expect a `Mat3`; `imageToMachine` call signature mismatch produces wrong results or runtime error. + +- [ ] **Step 3: Update the implementation** + +Replace the entire contents of `src/geometry/transform.ts`: + +```typescript +import type { Vec2, Segment, Alignment, PolyWarp } from '../types'; +import { applyPolyWarp, applyPolyWarpInverse } from './polywarp'; + +/** Apply per-job alignment to a point: rotate by rot, then translate by (tx,ty). */ +export function applyAlignment(a: Alignment, p: Vec2): Vec2 { + const c = Math.cos(a.rot); + const s = Math.sin(a.rot); + return [a.tx + c * p[0] - s * p[1], a.ty + s * p[0] + c * p[1]]; +} + +/** Project work-coordinate segments to image: work → (alignment) → machine → (warp) → image. */ +export function projectSegments(segments: Segment[], a: Alignment, warp: PolyWarp): Segment[] { + return segments.map((seg) => ({ + kind: seg.kind, + points: seg.points.map((p) => applyPolyWarp(warp, applyAlignment(a, p))), + })); +} + +/** Convert a normalized image point to machine mm using the warp's inverse map. */ +export function imageToMachine(warp: PolyWarp, imagePoint: Vec2): Vec2 { + return applyPolyWarpInverse(warp, imagePoint); +} + +/** Build an alignment that places the work origin (0,0) at the given machine point. */ +export function alignmentFromOrigin(machinePoint: Vec2, rot: number): Alignment { + return { tx: machinePoint[0], ty: machinePoint[1], rot }; +} +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `npx vitest run src/geometry/transform.test.ts` +Expected: PASS (4 tests). + +- [ ] **Step 5: Commit** + +```bash +git add src/geometry/transform.ts src/geometry/transform.test.ts +git commit -m "refactor: transform layer uses PolyWarp" +``` + +--- + +## Task 4: Migrate app state (`state.ts`) + +Rename `homography`/`setHomography` to `warp`/`setWarp`. + +**Files:** +- Modify: `src/app/state.ts` +- Test: `src/app/state.test.ts` + +- [ ] **Step 1: Update the test** + +Replace the entire contents of `src/app/state.test.ts`: + +```typescript +import { describe, it, expect } from 'vitest'; +import { createState } from './state'; +import { estimatePolyWarp } from '../geometry/polywarp'; +import type { Vec2 } from '../types'; + +// Identity warp: image == machine. +const square: Vec2[] = [[0, 0], [10, 0], [10, 10], [0, 10]]; +const IDENT = estimatePolyWarp(square, square, 1); + +describe('app state', () => { + it('starts with no segments and a zero alignment', () => { + const s = createState(); + expect(s.segments).toEqual([]); + expect(s.alignment).toEqual({ tx: 0, ty: 0, rot: 0 }); + }); + + it('loadGcode stores parsed segments and returns warning count', () => { + const s = createState(); + const warnings = s.loadGcode('G21 G90\nG1 X10 Y0'); + expect(s.segments).toHaveLength(1); + expect(warnings).toEqual([]); + }); + + it('projected() returns image-space segments when calibrated', () => { + const s = createState(); + s.setWarp(IDENT); + s.loadGcode('G21 G90\nG1 X10 Y0'); + s.alignment.tx = 2; + const proj = s.projected(); + expect(proj[0]!.points[0]![0]).toBeCloseTo(2, 4); + expect(proj[0]!.points[1]![0]).toBeCloseTo(12, 4); + }); + + it('projected() returns [] when not calibrated', () => { + const s = createState(); + s.loadGcode('G1 X10 Y0'); + expect(s.projected()).toEqual([]); + }); +}); +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `npx vitest run src/app/state.test.ts` +Expected: FAIL — `s.setWarp` is not a function. + +- [ ] **Step 3: Update the implementation** + +Replace the entire contents of `src/app/state.ts`: + +```typescript +import type { Segment, Alignment, PolyWarp } from '../types'; +import { parseGcode } from '../gcode/parser'; +import { projectSegments } from '../geometry/transform'; + +export interface AppState { + segments: Segment[]; + alignment: Alignment; + warp: PolyWarp | null; + loadGcode(text: string): string[]; + setWarp(w: PolyWarp | null): void; + projected(): Segment[]; +} + +export function createState(): AppState { + const state: AppState = { + segments: [], + alignment: { tx: 0, ty: 0, rot: 0 }, + warp: null, + loadGcode(text: string): string[] { + const { segments, warnings } = parseGcode(text); + state.segments = segments; + return warnings; + }, + setWarp(w: PolyWarp | null): void { + state.warp = w; + }, + projected(): Segment[] { + if (!state.warp) return []; + return projectSegments(state.segments, state.alignment, state.warp); + }, + }; + return state; +} +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `npx vitest run src/app/state.test.ts` +Expected: PASS (4 tests). + +- [ ] **Step 5: Commit** + +```bash +git add src/app/state.ts src/app/state.test.ts +git commit -m "refactor: app state stores PolyWarp" +``` + +--- + +## Task 5: Migrate config load/validate (`config.ts`) + +Validate a `warp` object instead of a 9-element `homography`. + +**Files:** +- Modify: `src/config.ts`, `src/types.ts` (Calibration field) +- Test: `src/config.test.ts` + +- [ ] **Step 1: Update the Calibration type** + +In `src/types.ts`, change the `Calibration` interface (it currently ends with `homography: Mat3;`): + +```typescript +export interface Calibration { + /** Calibration points in normalized [0,1] camera-frame coordinates. */ + imagePoints: Vec2[]; + /** Corresponding machine coordinates, mm. */ + machinePoints: Vec2[]; + /** Bidirectional polynomial warp (machine-mm ↔ normalized image). */ + warp: PolyWarp; +} +``` + +- [ ] **Step 2: Update the test** + +Replace the two homography-specific test cases in `src/config.test.ts`. Replace the `'keeps a valid calibration'` test (lines ~22-31), the `'drops a malformed calibration to null'` test (lines ~33-37), and the `'rejects a calibration whose homography contains NaN'` test (lines ~53-62) with: + +```typescript + it('keeps a valid calibration', async () => { + const square = [[0, 0], [10, 0], [10, 10], [0, 10]] as [number, number][]; + const warp = estimatePolyWarp(square, square, 1); + const calibration = { imagePoints: square, machinePoints: square, warp }; + mockFetch({ streamUrl: 'x', calibration }); + const cfg = await loadConfig('config.json'); + expect(cfg.calibration).toEqual(calibration); + }); + + it('drops a malformed calibration to null', async () => { + mockFetch({ streamUrl: 'x', calibration: { warp: { degree: 2 } } }); + const cfg = await loadConfig('config.json'); + expect(cfg.calibration).toBeNull(); + }); + + it('rejects a calibration whose warp coefficients contain NaN', async () => { + const square = [[0, 0], [10, 0], [10, 10], [0, 10]] as [number, number][]; + const warp = estimatePolyWarp(square, square, 1); + warp.fwd.cu[0] = NaN; + const calibration = { imagePoints: square, machinePoints: square, warp }; + mockFetch({ streamUrl: 'x', calibration }); + const cfg = await loadConfig('config.json'); + expect(cfg.calibration).toBeNull(); + }); +``` + +Add this import near the top of `src/config.test.ts` (after the existing imports): + +```typescript +import { estimatePolyWarp } from './geometry/polywarp'; +``` + +- [ ] **Step 3: Run test to verify it fails** + +Run: `npx vitest run src/config.test.ts` +Expected: FAIL — `config.ts` still validates `homography`, so the valid-warp calibration is dropped to `null`. + +- [ ] **Step 4: Update the implementation** + +In `src/config.ts`, update the import line and replace `parseCalibration`. First change the type import: + +```typescript +import type { AppConfig, Calibration, PolyMap, PolyWarp, Vec2, RenderStyle } from './types'; +``` + +Then replace the `parseCalibration` function (lines ~13-25) with: + +```typescript +function isNumArray(v: unknown, len: number): v is number[] { + return Array.isArray(v) && v.length === len && v.every(isFiniteNumber); +} + +function isVec2(v: unknown): v is Vec2 { + return Array.isArray(v) && v.length === 2 && v.every(isFiniteNumber); +} + +function termCount(degree: number): number { + return ((degree + 1) * (degree + 2)) / 2; +} + +function parsePolyMap(m: unknown, degree: number): PolyMap | null { + if (!m || typeof m !== 'object') return null; + const o = m as Record; + const norm = o.norm as Record | undefined; + if (!norm || !isVec2(norm.off) || !isVec2(norm.scl)) return null; + const n = termCount(degree); + if (!isNumArray(o.cu, n) || !isNumArray(o.cv, n)) return null; + return { degree, norm: { off: norm.off as Vec2, scl: norm.scl as Vec2 }, cu: o.cu as number[], cv: o.cv as number[] }; +} + +function parseWarp(w: unknown): PolyWarp | null { + if (!w || typeof w !== 'object') return null; + const o = w as Record; + if (!isFiniteNumber(o.degree)) return null; + const fwd = parsePolyMap(o.fwd, o.degree); + const inv = parsePolyMap(o.inv, o.degree); + if (!fwd || !inv) return null; + return { degree: o.degree, fwd, inv }; +} + +function parseCalibration(c: unknown): Calibration | null { + if (!c || typeof c !== 'object') return null; + const obj = c as Record; + if (!isVec2Array(obj.imagePoints) || !isVec2Array(obj.machinePoints)) return null; + if (obj.imagePoints.length !== obj.machinePoints.length || obj.imagePoints.length < 4) return null; + const warp = parseWarp(obj.warp); + if (!warp) return null; + return { imagePoints: obj.imagePoints, machinePoints: obj.machinePoints, warp }; +} +``` + +- [ ] **Step 5: Run test to verify it passes** + +Run: `npx vitest run src/config.test.ts` +Expected: PASS (all cases). + +- [ ] **Step 6: Commit** + +```bash +git add src/config.ts src/config.test.ts src/types.ts +git commit -m "refactor: config validates PolyWarp calibration" +``` + +--- + +## Task 6: Migrate the calibration UI (`calibration-ui.ts`) + +Compute a `PolyWarp` (with a degree selector), update the residual readout and JSON output. + +**Files:** +- Modify: `src/app/calibration-ui.ts` +- Test: `src/app/calibration-ui.test.ts` + +- [ ] **Step 1: Update the test** + +Replace the entire contents of `src/app/calibration-ui.test.ts`: + +```typescript +import { describe, it, expect } from 'vitest'; +import { residuals } from './calibration-ui'; +import { estimatePolyWarp } from '../geometry/polywarp'; +import type { Vec2 } from '../types'; + +describe('calibration residuals', () => { + it('reports near-zero error for a clean degree-1 fit', () => { + const machine: Vec2[] = [[0, 0], [100, 0], [100, 100], [0, 100]]; + const image: Vec2[] = machine.map((p): Vec2 => [p[0] * 0.5 + 5, p[1] * 0.5 + 5]); + const w = estimatePolyWarp(machine, image, 1); + const errs = residuals(w, machine, image); + expect(Math.max(...errs)).toBeLessThan(1e-4); + }); + + it('flags a mismatched point with a large residual', () => { + const machine: Vec2[] = [[0, 0], [100, 0], [100, 100], [0, 100], [50, 50], [25, 75], [75, 25]]; + const image: Vec2[] = [[0, 0], [50, 0], [50, 50], [0, 50], [25, 25], [12.5, 37.5], [90, 5]]; // last is wrong + const w = estimatePolyWarp(machine, image, 1); + const errs = residuals(w, machine, image); + expect(errs[6]!).toBeGreaterThan(5); + }); +}); +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `npx vitest run src/app/calibration-ui.test.ts` +Expected: FAIL — `residuals` still takes a `Mat3` and calls `applyHomography`. + +- [ ] **Step 3: Update the implementation** + +Replace the entire contents of `src/app/calibration-ui.ts`: + +```typescript +import type { Vec2, PolyWarp } from '../types'; +import { estimatePolyWarp, applyPolyWarp } from '../geometry/polywarp'; + +/** Per-point reprojection error in normalized units: |warp(machine) − image|. */ +export function residuals(warp: PolyWarp, machine: Vec2[], image: Vec2[]): number[] { + if (machine.length !== image.length) throw new Error('residuals: machine/image length mismatch'); + return machine.map((m, i) => { + const p = applyPolyWarp(warp, m); + return Math.hypot(p[0] - image[i]![0], p[1] - image[i]![1]); + }); +} + +interface CalibPoint { + machine: Vec2; + image: Vec2; +} + +export interface CalibrationDeps { + panel: HTMLElement; + overlay: HTMLCanvasElement; + onWarp: (w: PolyWarp, points: CalibPoint[]) => void; +} + +/** + * Renders the calibration panel. Flow: enter machine X/Y → click "Click point in video" → + * click the spindle tip in the video → repeat ≥10× → choose degree → "Compute". + * + * Image points and the resulting warp are in normalized [0,1] camera-frame coordinates + * (machine-mm ↔ normalized image), so they are display-size independent. + */ +export function mountCalibration(deps: CalibrationDeps): void { + const { panel, overlay } = deps; + const points: CalibPoint[] = []; + let pendingMachine: Vec2 | null = null; + + panel.innerHTML = ` +

Calibration

+
Jog the spindle to a known X/Y, enter it, then click the tip in the video. ≥10 points (degree 3); spread them out incl. corners.
+
+ X + Y + +
+
    +
    + Degree + + +
    + + `; + + const xEl = panel.querySelector('#cal-x') as HTMLInputElement; + const yEl = panel.querySelector('#cal-y') as HTMLInputElement; + const armBtn = panel.querySelector('#cal-arm') as HTMLButtonElement; + const list = panel.querySelector('#cal-list') as HTMLUListElement; + const degreeEl = panel.querySelector('#cal-degree') as HTMLSelectElement; + const jsonEl = panel.querySelector('#cal-json') as HTMLTextAreaElement; + + const renderList = (errs?: number[]) => { + list.innerHTML = points + .map((p, i) => `
  • (${p.machine[0]}, ${p.machine[1]}) → img(${p.image[0].toFixed(3)}, ${p.image[1].toFixed(3)})${errs ? ` err ${errs[i]!.toFixed(1)}px` : ''}
    • `) + .join(''); + }; + + armBtn.addEventListener('click', () => { + pendingMachine = [parseFloat(xEl.value), parseFloat(yEl.value)]; + if (Number.isNaN(pendingMachine[0]) || Number.isNaN(pendingMachine[1])) { + pendingMachine = null; + return; + } + overlay.classList.add('interactive'); + }); + + overlay.addEventListener('click', (ev) => { + if (!pendingMachine || !overlay.classList.contains('interactive')) return; + const rect = overlay.getBoundingClientRect(); + const px: Vec2 = [ + (ev.clientX - rect.left) / rect.width, + (ev.clientY - rect.top) / rect.height, + ]; + points.push({ machine: pendingMachine, image: px }); + pendingMachine = null; + overlay.classList.remove('interactive'); + renderList(); + }); + + (panel.querySelector('#cal-clear') as HTMLButtonElement).addEventListener('click', () => { + points.length = 0; + jsonEl.value = ''; + renderList(); + }); + + (panel.querySelector('#cal-compute') as HTMLButtonElement).addEventListener('click', () => { + const degree = parseInt(degreeEl.value, 10); + const need = ((degree + 1) * (degree + 2)) / 2; + if (points.length < need) { + jsonEl.value = `Need at least ${need} points for degree ${degree}.`; + return; + } + const machine = points.map((p) => p.machine); + const image = points.map((p) => p.image); + let warp: PolyWarp; + try { + warp = estimatePolyWarp(machine, image, degree); + } catch (e) { + jsonEl.value = `Fit failed: ${(e as Error).message}`; + return; + } + // Per-point pixel residual (normalized error scaled by canvas size) — for misclick display. + const errsPx = machine.map((m, i) => { + const p = applyPolyWarp(warp, m); + return Math.hypot((p[0] - image[i]![0]) * overlay.width, (p[1] - image[i]![1]) * overlay.height); + }); + renderList(errsPx); + jsonEl.value = JSON.stringify({ imagePoints: image, machinePoints: machine, warp }, null, 2); + deps.onWarp(warp, [...points]); + }); +} +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `npx vitest run src/app/calibration-ui.test.ts` +Expected: PASS (2 tests). + +- [ ] **Step 5: Commit** + +```bash +git add src/app/calibration-ui.ts src/app/calibration-ui.test.ts +git commit -m "feat: calibration UI computes PolyWarp with degree selector" +``` + +--- + +## Task 7: Migrate alignment UI + main wiring + +Update the two remaining consumers of the old API. + +**Files:** +- Modify: `src/app/alignment-ui.ts`, `src/main.ts` +- Test: `src/app/overlay-interaction.test.ts` (read first; update any `getHomography`/`setHomography`/`Mat3` references) + +- [ ] **Step 1: Inspect the interaction test for old-API references** + +Run: `grep -n "Homography\|homography\|Mat3\|setWarp\|getWarp" src/app/overlay-interaction.test.ts` + +For each match, update it to the new API: `getHomography` dep → `getWarp`, `Mat3` identity → a `PolyWarp` built with `estimatePolyWarp(square, square, 1)` (import from `../geometry/polywarp`), where `const square: Vec2[] = [[0,0],[10,0],[10,10],[0,10]]`. If there are no matches, this test needs no change — proceed. + +- [ ] **Step 2: Run the interaction test to see the current state** + +Run: `npx vitest run src/app/overlay-interaction.test.ts` +Expected: After Step 1 edits (if any), it should FAIL only because `alignment-ui.ts` still exposes `getHomography`. If it already passes (test didn't reference the renamed dep), that's fine — proceed to Step 3. + +- [ ] **Step 3: Update `alignment-ui.ts`** + +In `src/app/alignment-ui.ts`, change the import (line 1) and the `AlignmentDeps` interface, then rename the local `H` usages. Apply these edits: + +Change line 1: +```typescript +import type { Vec2, PolyWarp, Alignment } from '../types'; +``` + +In `AlignmentDeps`, replace `getHomography: () => Mat3 | null;` with: +```typescript + getWarp: () => PolyWarp | null; +``` + +Replace every `const H = deps.getHomography();` with `const warp = deps.getWarp();`, every `if (!H)` / `if (!H || ...)` guard's `H` with `warp`, and every `imageToMachine(H, ...)` with `imageToMachine(warp, ...)`. There are three `getHomography` call sites (mousedown, mousemove, plus the guard in mousemove) — update all. `imageToMachine` itself is already `PolyWarp`-typed from Task 3, so only the variable name and dep call change. + +- [ ] **Step 4: Update `main.ts`** + +In `src/main.ts`, apply these edits: + +Replace the `mountCalibration({...})` call (lines ~51-59): +```typescript +mountCalibration({ + panel: document.getElementById('calib-panel') as HTMLElement, + overlay, + onWarp: (w) => { + state.setWarp(w); + statusEl.textContent = 'Calibrated (unsaved — paste JSON into config.json to persist).'; + render(); + }, +}); +``` + +In the `mountAlignment({...})` call, replace `getHomography: () => state.homography,` with: +```typescript + getWarp: () => state.warp, +``` + +In `boot()`, replace `if (cfg.calibration) state.setHomography(cfg.calibration.homography);` with: +```typescript + if (cfg.calibration) state.setWarp(cfg.calibration.warp); +``` + +- [ ] **Step 5: Run the interaction test to verify it passes** + +Run: `npx vitest run src/app/overlay-interaction.test.ts` +Expected: PASS. + +- [ ] **Step 6: Commit** + +```bash +git add src/app/alignment-ui.ts src/main.ts src/app/overlay-interaction.test.ts +git commit -m "refactor: alignment UI and main wiring use PolyWarp" +``` + +--- + +## Task 8: Retire the homography module and verify the whole build + +Delete the now-unused homography code and confirm the project type-checks and all tests pass. + +**Files:** +- Delete: `src/geometry/homography.ts`, `src/geometry/homography.test.ts` + +- [ ] **Step 1: Confirm there are no remaining importers** + +Run: `grep -rn "from './homography'\|from '../geometry/homography'\|geometry/homography" src/` +Expected: only matches inside `homography.ts`/`homography.test.ts` themselves (which we delete) and possibly the `polywarp.test.ts` import — see Step 2. + +- [ ] **Step 2: Drop the homography import from the polywarp test** + +`src/geometry/polywarp.test.ts` (Task 2) imports `estimateHomography, applyHomography` for the "beats a homography" test. Since `homography.ts` is being deleted, **inline a minimal homography** in that test instead. At the top of `polywarp.test.ts`, remove the line: + +```typescript +import { estimateHomography, applyHomography } from './homography'; +``` + +and replace the `'beats a homography on barrel-distorted data'` test body's homography use with a self-contained least-squares affine baseline (an affine map is the best a no-distortion linear model can do, and like a homography it cannot represent barrel curvature — the comparison still demonstrates the polynomial's advantage). Replace that whole test with: + +```typescript + it('beats an affine/linear baseline on barrel-distorted data', () => { + // Barrel distortion is degree-3 in machine coords, so degree 3 fits it essentially + // exactly while a degree-1 (affine, no curvature) baseline cannot. + const image = machineGrid.map(trueMap); + const w = estimatePolyWarp(machineGrid, image, 3); + const baseline = estimatePolyWarp(machineGrid, image, 1); // degree-1 = affine, no curvature + const test: Vec2[] = [[488, 244], [1464, 732], [1952, 488], [976, 976]]; + const polyMax = Math.max(...test.map((m) => dist(applyPolyWarp(w, m), trueMap(m)))); + const baseMax = Math.max(...test.map((m) => dist(applyPolyWarp(baseline, m), trueMap(m)))); + expect(polyMax).toBeLessThan(baseMax * 0.05); + }); +``` + +- [ ] **Step 3: Delete the homography files** + +```bash +git rm src/geometry/homography.ts src/geometry/homography.test.ts +``` + +- [ ] **Step 4: Run the full test suite** + +Run: `npm test` +Expected: PASS — all suites green, no reference to the deleted module. + +- [ ] **Step 5: Type-check + build (the cross-file gate)** + +Run: `npm run build` +Expected: `tsc --noEmit` passes with no errors, Vite build succeeds. If `tsc` flags an unused `Mat3` import anywhere, remove that import. (The `Mat3` *type* may remain defined in `types.ts` unused — that is not a compile error.) + +- [ ] **Step 6: Commit** + +```bash +git add -A +git commit -m "refactor: retire homography model in favour of PolyWarp" +``` + +--- + +## Task 9: Regenerate the default calibration + +The committed `dist/config.json` still holds the old homography, which `parseCalibration` now rejects (→ "Not calibrated"). This is expected per the spec's migration note. The real recalibration happens on the machine with the box+# target, but verify the app boots cleanly without a valid calibration. + +**Files:** +- Inspect: `dist/config.json`, `public/config.json` (if present) + +- [ ] **Step 1: Check which config files carry a homography** + +Run: `grep -rln "homography" dist public 2>/dev/null` + +- [ ] **Step 2: Confirm graceful handling** + +The app already shows "Not calibrated — calibrate first." when `cfg.calibration` is `null` (`main.ts` boot). No code change needed — the stale homography config simply parses to `null`. Leave the file as-is OR, to avoid a confusing dead field, remove the `homography`/`calibration` block from `dist/config.json` and `public/config.json` so the file is clean. Do whichever the repo convention prefers; if unsure, leave it (harmless). + +- [ ] **Step 3: Capture the real calibration on the machine (manual, runtime)** + +This is the on-hardware step, not a code change: +1. Draw the box + "#" target (lines at thirds) by jog-and-mark: jog the spindle to each of the 16 targets (every combination of `X ∈ {0, 813.3, 1626.7, 2440}`, `Y ∈ {0, 406.7, 813.3, 1220}`) and mark at the tip. +2. In the app: for each mark, enter its X/Y, click "Click point in video", click the mark. Repeat for all 16. +3. Leave degree at 2, click "Compute warp". Check the per-point `err …px` readout — if max error is large, switch degree to 3 and recompute. +4. Copy the JSON from the textarea into `dist/config.json` under `calibration`, redeploy per `memory/deployment-mf01.md`. + +- [ ] **Step 4: Commit any config cleanup** + +```bash +git add dist/config.json public/config.json 2>/dev/null +git commit -m "chore: clear stale homography calibration from config" || echo "nothing to commit" +``` + +--- + +## Self-review notes + +- **Spec coverage:** model (T2), forward+inverse maps (T2/T3), normalization/conditioning (T2), linalg extraction (T1), transform/state/config/calib/alignment/main migration (T3–T7), homography retirement (T8), 16-point box+# target & jog-and-mark + degree escalation (T6 UI + T9 procedure), migration of stale config (T9), tests incl. synthetic-barrel regression guard (T2). All spec sections map to a task. +- **Inverse independence caveat** (from spec/user review) is realized by fitting `inv` separately in `estimatePolyWarp`; the round-trip test uses a 2 mm tolerance accordingly rather than asserting exact inversion. +- **Type consistency:** `PolyWarp`/`PolyMap` shapes (`degree`, `norm.off`, `norm.scl`, `cu`, `cv`) are identical across `types.ts`, `polywarp.ts`, and `config.ts` validation. Renames are consistent: `setHomography`→`setWarp`, `getHomography`→`getWarp`, `onHomography`→`onWarp`, `homography`→`warp`. diff --git a/docs/superpowers/specs/2026-06-11-wide-angle-distortion-design.md b/docs/superpowers/specs/2026-06-11-wide-angle-distortion-design.md index dfdf672..50d361f 100644 --- a/docs/superpowers/specs/2026-06-11-wide-angle-distortion-design.md +++ b/docs/superpowers/specs/2026-06-11-wide-angle-distortion-design.md @@ -27,10 +27,11 @@ This is a **model** limitation, not a calibration-points problem. ## Chosen approach: bivariate polynomial warp Replace the homography with a **bivariate polynomial warp** (`PolyWarp`) that maps machine -coordinates directly to normalized image coordinates. The quadratic terms (`x², xy, y²`) -capture both the bed's perspective trapezoid and the leading `r²` term of barrel -distortion. Because the map is **linear in its coefficients**, it is fit with the same -least-squares machinery the homography already used. +coordinates directly to normalized image coordinates. The cubic terms (`x³, x²y, xy², y³`) +are what capture the symmetric radial barrel distortion (see the degree correction below); +the lower-order terms absorb the bed's perspective trapezoid. Because the map is **linear in +its coefficients**, it is fit with the same least-squares machinery the homography already +used. Considered and rejected for now: - **Physical radial lens model + homography (Brown–Conrady):** physically correct and @@ -39,9 +40,15 @@ Considered and rejected for now: - **Thin-plate spline / mesh warp:** highest accuracy but wants many control points (20–40) and more code; overkill for the ~8–12-point budget. -Escalation path: if degree-2 residuals stay high, bump to degree 3 (needs ≥12 points), or -graduate to the radial model. The polynomial is the smallest change that actually fixes the -observed drift. +Fallback path: with sparse calibration (<10 points) drop to degree 2; if residuals stay +high after degree 3, graduate to the radial model. The polynomial is the smallest change +that actually fixes the observed drift. + +> **Correction (validated during implementation):** barrel distortion multiplies the +> coordinate by `(1 + k·r²)`, so as a function of machine coordinates it is **degree 3** +> (`dx·r²` expands to `dx³ + dx·dy²`), not degree 2. A degree-2 polynomial therefore does +> *not* beat a homography on real radial distortion — only degree 3 does. Hence degree 3 is +> the default below, not an escalation. ### The model @@ -56,9 +63,10 @@ reverse direction directly from the same points. The inverse map only serves UI its few-pixel / sub-mm accuracy is sufficient; forward and inverse need not be exact mutual inverses. -Default **degree 2** (6 coefficients per axis, needs ≥6 points). **Degree 3** (10 -coefficients per axis, needs ≥10 points; recommend ≥12 to avoid overfit) is a manual -escalation when residuals warrant it. +Default **degree 3** (10 coefficients per axis, needs ≥10 points) — the degree that +actually compensates barrel distortion; the 16-point box+# target supports it comfortably. +**Degree 2** (6 coefficients per axis, needs ≥6 points) is a manual fallback for sparse +calibration. The calibration UI exposes a degree selector defaulting to 3. Image coordinates remain **normalized [0,1]** camera-frame fractions, as today — keeps calibration independent of display/canvas size.