Compare commits
2 commits
29b4d8fea5
...
2194a26970
| Author | SHA1 | Date | |
|---|---|---|---|
|
2194a26970 | ||
|
|
962f07ca6b |
6 changed files with 247 additions and 143 deletions
12
CLAUDE.md
12
CLAUDE.md
|
|
@ -24,11 +24,13 @@ Reference + thin runbooks. It does **not** hold authoritative IPs/topology/secre
|
|||
run. For live switch/infra, follow that repo's idempotency + lockout-safety
|
||||
rules (run device plays twice; enable VLAN-filtering last; detached
|
||||
self-reverting jobs for mgmt changes).
|
||||
2. **Access path for Claude (on fisi): Netbird, on-demand only.** Bring the
|
||||
overlay up for the task, `netbird down` immediately after. Prefer the
|
||||
VPS-bastion path when it suffices (no tunnel on fisi at all). **Isolation is
|
||||
a hard requirement** — nothing from the makerspace should be able to reach
|
||||
fisi/the homelab. See [access.md](access.md) §C.
|
||||
2. **Access path for Claude (on ubongo): the VPS bastion.** Single ProxyJump
|
||||
via `makerfloss.eu:7576` onto the `wg1` overlay addresses; via mamba (boma
|
||||
wg hub) when it's on-site and the LAN is needed raw. ubongo has no Netbird
|
||||
agent and must not join makerspace overlays. **Isolation is a hard
|
||||
requirement** — nothing from the makerspace should be able to reach the
|
||||
homelab. See [access.md](access.md) §C/§E. (Updated 2026-07-17; fisi is
|
||||
decommissioned.)
|
||||
3. **Reference, don't duplicate.** When you need a fact, link to the source-repo
|
||||
file. If you cache a value here, note it can drift.
|
||||
4. **Log real work** in [incidents/](incidents/) — symptom, root cause, the
|
||||
|
|
|
|||
188
access.md
188
access.md
|
|
@ -3,10 +3,14 @@
|
|||
How to get a shell on the hosts we troubleshoot, from each place you might be
|
||||
working. **Read the section that matches your situation.**
|
||||
|
||||
Rewritten 2026-07-17 for the ubongo era (fisi decommissioned, boma ADR-036).
|
||||
Live-verified facts are dated; everything else is best-known and can drift.
|
||||
|
||||
There are two actors:
|
||||
- **You (sjat)** — at the makerspace with **mamba** (laptop), or elsewhere.
|
||||
- **Claude** — lives on **fisi** (homelab) and must *tunnel in* to help at the
|
||||
makerspace.
|
||||
- **Claude** — lives on **ubongo** (homelab control node, `10.20.10.151`) and
|
||||
must *tunnel in* to help at the makerspace. fisi is gone; anything that
|
||||
says fisi is stale.
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -14,147 +18,123 @@ There are two actors:
|
|||
|
||||
| Host | Address | SSH | Where it sits | Notes |
|
||||
|------|---------|-----|---------------|-------|
|
||||
| **makerfloss VPS** | `88.99.32.236` / `makerfloss.eu` | `:7576` | Public (Hetzner) | Public entrypoint + bastion. Forgejo on `:7577`. WG `wg1` hub `10.13.0.1`. Netbird control plane `nb.makerfloss.eu`. |
|
||||
| **makerfloss1** | `172.17.3.51` | `:7576` | Makerspace LAN | Local Docker/dev host. `wg1` peer `10.13.0.2` (full-tunnel). |
|
||||
| **mf04** | `10.0.0.184` | `:7576` | Makerspace LAN | Testmachine. `wg1` peer `10.13.0.3` (split-tunnel). Netbird peer. |
|
||||
| **crs310-maker** (switch) | mgmt `192.168.88.1` | `:22` | Makerspace, port `ether8` only | Mgmt VLAN 99, isolated. Data VLAN 30 = `10.2.30.0/24`. See [runbooks/switch-crs310.md](runbooks/switch-crs310.md). |
|
||||
| **fisi** (home) | `10.20.10.17` | `:7576` | Homelab | Where Claude runs. Behind OPNsense; baobab WG hub is **kuku** (`10.8.0.1`, UDP 51194). |
|
||||
| **mamba** (laptop) | LAN-dependent | `:7576` | Roams | Baobab WG peer `10.8.0.4`; makerfloss `wg1` roaming peer `10.13.0.5`; Netbird peer. |
|
||||
| **makerfloss VPS** | `88.99.32.236` / `makerfloss.eu` | `:7576` | Public (Hetzner) | Public entrypoint + bastion. Forgejo on `:7577`. `wg1` hub `10.13.0.1` (UDP `51820`). Netbird control plane `nb.makerfloss.eu` (**legacy — kept for now**, see §D). |
|
||||
| **srv07** (ex-mf04) | wg1 **`10.13.0.3`** (canonical); rack LAN `10.0.0.183` (DHCP) | `:7576` | Makerspace rack (rack01, shf02/1) | Testmachine, Debian 13. Renamed mf04 → srv07 on 2026-07-17 (matches sticker). wg unit `wg-quick@mf-srv07` auto-dials the VPS on boot. |
|
||||
| **mf01** (docs: srv02) | wg1 `10.13.0.8`; rack LAN `10.0.0.223` (DHCP) | `:7576` | Makerspace rack | Docker/dev host, publishes `*.mf01.makerfloss.eu` via the VPS. OS hostname still `mf01`. |
|
||||
| **makerfloss1** | recorded `172.17.3.51` | `:7576` | OrangeMakers LAN? | **Did not answer 2026-07-17** (LAN or wg1 `.2`). Status unknown — verify on-site before relying on it. |
|
||||
| **flossfw** | wg1 `10.13.0.9`; likely the rack gateway `10.0.0.1` | `:22` | Makerspace rack | TaPPaaS FLOSSFirewall (FreeBSD). wg1 peer down 2026-07-17. See `runbooks/` for peer-key swap on operator onboarding. |
|
||||
| **crs310-maker** (switch) | mgmt `192.168.88.1` | `:22` | Makerspace rack (sw01) | Mgmt VLAN reachable only from the mgmt port (sw01 p8, patched out via pp01:3, currently non-active). Rack re-cabling means the switch runbook may be stale — verify on-site. |
|
||||
| **mamba** (laptop) | LAN-dependent | `:7576` | Roams | boma wg spoke **`10.99.0.10`** (how Claude reaches it anywhere); makerfloss `wg1` roaming peer `10.13.0.5` (profile dormant, hub still has the peer); Netbird peer (legacy). |
|
||||
|
||||
> The makerspace addressing is **not fully pinned** — see the open question in
|
||||
> [network-map.md](network-map.md). Confirm the host's current IP before relying
|
||||
> on the table.
|
||||
The rack LAN is `10.0.0.0/24` (gw `.1`). Also seen there 2026-07-17: a trio of
|
||||
fresh Debian 13 boxes at `.10/.11/.12` (ssh `:22`, unidentified — likely
|
||||
TaPPaaS cluster nodes) and unidentified ssh hosts at `.158/.165/.209`.
|
||||
|
||||
---
|
||||
|
||||
## A. You're at the makerspace with mamba (on the LAN)
|
||||
## A. You're at the makerspace with mamba
|
||||
|
||||
Easiest case — mamba is directly on the makerspace network.
|
||||
|
||||
- **The switch (mgmt):** plug mamba into **`ether8`** (mgmt VLAN 99). You get a
|
||||
DHCP lease in `192.168.88.10–254`; switch is `192.168.88.1` (SSH, and web UI
|
||||
`http://192.168.88.1`). A pinned NetworkManager profile `crs310-bench` /
|
||||
static `192.168.88.2` is documented in the Mikrotik repo for this.
|
||||
- **The switch (data ports `ether2–7`):** you land on data VLAN 30
|
||||
(`10.2.30.0/24`, gateway `.1`) — normal user network, *no* switch mgmt here.
|
||||
- **makerfloss1 / mf04:** SSH directly to their LAN IPs (`172.17.3.51`,
|
||||
`10.0.0.184`) on `:7576`.
|
||||
|
||||
For Ansible against the switch from mamba, run from the `MakerFLOSS_Mikrotik`
|
||||
repo with mamba on `ether8`.
|
||||
|
||||
---
|
||||
|
||||
## B. Claude tunneling in from fisi (no laptop at the makerspace)
|
||||
|
||||
fisi has **no direct route** to the makerspace LAN. Pick one path. Ordered by
|
||||
preference given the **isolation requirement** (keep makerspace and homelab
|
||||
apart — see the rule box below).
|
||||
|
||||
### B1. Netbird overlay — *on-demand only* (primary)
|
||||
|
||||
**fisi is enrolled** (2026-06-09) as peer `fisi-61-26.netbird.selfhosted` =
|
||||
`100.99.61.26` on the self-hosted control plane `nb.makerfloss.eu`. The overlay
|
||||
is the `100.99.0.0/16` net; mf04 is `100.99.133.190`. The Netbird service on
|
||||
fisi is kept **stopped + disabled** — no standing tunnel.
|
||||
|
||||
Per-session use (no setup key needed — enrollment is cached):
|
||||
**Plug in the cable.** Wired, mamba lands on the rack LAN (`10.0.0.0/24`).
|
||||
The wifi (`172.17.3.0/24`, OrangeMakers house LAN) does **not** route to the
|
||||
rack LAN (verified 2026-07-17) — wifi-only is not enough.
|
||||
|
||||
```bash
|
||||
sudo systemctl start netbird # service is disabled by default
|
||||
sudo netbird up # reconnect to the overlay
|
||||
sudo netbird status # confirm Management/Signal Connected, peers up
|
||||
ssh -p 7576 sjat@100.99.133.190 # e.g. mf04 directly over the overlay
|
||||
|
||||
# TEAR DOWN when done — leave nothing standing:
|
||||
sudo netbird down
|
||||
sudo systemctl stop netbird
|
||||
ssh -p 7576 sjat@10.0.0.183 # srv07 by its current DHCP lease
|
||||
ssh -p 7576 sjat@10.0.0.223 # mf01
|
||||
```
|
||||
|
||||
If a peer shows but you can't reach it (`Peers count: 0/0` / 0 connected), the
|
||||
**Netbird access policy** isn't linking fisi's group to that peer's group —
|
||||
fix in the `nb.makerfloss.eu` dashboard (default policy is "all peers reach
|
||||
all"). Re-enrollment (lost config) needs a fresh setup key from the dashboard:
|
||||
`sudo netbird up --setup-key <KEY> --management-url https://nb.makerfloss.eu`.
|
||||
The leases are DHCP and can drift; the wg1 addresses (§B) are the stable
|
||||
names. If a box doesn't answer, check the physical layer first — srv07 was
|
||||
dark for weeks because its patch-panel run lost its rear patch (see
|
||||
`incidents/2026-07-17-srv07-access.md`).
|
||||
|
||||
> **Why on-demand:** sjat's explicit constraint — nothing from the makerspace
|
||||
> should be able to bleed into fisi/the homelab. Netbird stays **stopped +
|
||||
> disabled** on fisi; it is not a standing tunnel. See §C.
|
||||
## B. You're roaming (anywhere with internet)
|
||||
|
||||
### B2. Via the makerfloss VPS bastion (no tunnel on fisi)
|
||||
|
||||
The cleanest isolation-preserving path: fisi only ever talks to the **public
|
||||
VPS**, which hops onward over its own `wg1` tunnel. fisi itself joins no
|
||||
makerspace network.
|
||||
Single hop via the VPS bastion — works with your normal key, nothing to
|
||||
bring up first:
|
||||
|
||||
```bash
|
||||
# Land on the VPS:
|
||||
ssh -p 7576 sjat@makerfloss.eu
|
||||
|
||||
# From the VPS, reach wg1 peers on the makerfloss plane:
|
||||
ssh -p 7576 sjat@10.13.0.2 # makerfloss1
|
||||
ssh -p 7576 sjat@10.13.0.3 # mf04
|
||||
ssh -J sjat@makerfloss.eu:7576 -p 7576 sjat@10.13.0.3 # srv07
|
||||
ssh -J sjat@makerfloss.eu:7576 -p 7576 sjat@10.13.0.8 # mf01
|
||||
```
|
||||
|
||||
Or, if/when the per-machine SSH DNAT from the VPN design is in place, jump
|
||||
straight through the VPS with a single hop (ports `:7578`/`:7579` → testmachine
|
||||
`:7576`). Confirm these DNAT rules exist before relying on them.
|
||||
This rides each box's own always-on `wg1` tunnel to the VPS, so it survives
|
||||
makerspace LAN renumbering. Verified end-to-end 2026-07-17.
|
||||
|
||||
Alternative: bring up mamba's own `wg1` roaming profile (`10.13.0.5`, config
|
||||
hand-deployed on mamba) and reach `10.13.0.x` directly without the jump. The
|
||||
hub still carries the peer; the profile has simply not been up lately.
|
||||
|
||||
## C. Claude tunneling in from ubongo
|
||||
|
||||
Ordered by preference under the **isolation rule** (§E). ubongo has **no
|
||||
Netbird agent** (removed estate-wide, boma ADR-036) — the old fisi/Netbird
|
||||
path is gone.
|
||||
|
||||
### C1. VPS bastion (primary)
|
||||
|
||||
Same as §B: ubongo only ever talks to the public VPS, which hops onward over
|
||||
`wg1`. No makerspace network ever exists on ubongo.
|
||||
|
||||
```bash
|
||||
ssh -J sjat@makerfloss.eu:7576 -p 7576 sjat@10.13.0.3 # fisi → VPS → mf04
|
||||
ssh -J sjat@makerfloss.eu:7576 -p 7576 sjat@10.13.0.3 # srv07
|
||||
```
|
||||
|
||||
This path does **not** require mamba to be present.
|
||||
First use on a fresh control node: accept/pin `makerfloss.eu`'s host key.
|
||||
|
||||
### B3. ProxyJump via kuku → mamba (only when mamba is at the makerspace)
|
||||
### C2. Via mamba (when mamba is at the makerspace)
|
||||
|
||||
This is the chain the `AnsibleBaobabV4` inventory already uses for `mf04`:
|
||||
|
||||
```
|
||||
fisi → kuku (baobab WG hub) → mamba (WG peer 10.8.0.4, on the makerspace LAN) → mf04
|
||||
```
|
||||
ubongo reaches mamba anywhere over the **boma wg hub** (`ssh mamba` =
|
||||
`10.99.0.10:7576`, pinned in the boma-managed ssh config). If mamba is wired
|
||||
at the makerspace, hop onward to the rack LAN:
|
||||
|
||||
```bash
|
||||
ssh -o ProxyJump="kuku,sjat@10.8.0.4:7576" -p 7576 sjat@10.0.0.184 # mf04
|
||||
ssh -o ProxyJump=mamba -p 7576 sjat@10.0.0.183 # srv07 via mamba
|
||||
```
|
||||
|
||||
In `AnsibleBaobabV4/host_vars/mf04.yml`:
|
||||
`ansible_ssh_common_args: '-o ProxyJump="kuku,sjat@10.8.0.4:7576"'`.
|
||||
Useful when the VPS or a box's wg1 tunnel is down but the LAN is up (this is
|
||||
how srv07 was found and fixed on 2026-07-17). Breaks when mamba leaves.
|
||||
|
||||
**Constraint:** only works while mamba is physically at the makerspace, online,
|
||||
and roaming on the baobab WG plane. Breaks the moment mamba leaves.
|
||||
|
||||
### Choosing between B1/B2/B3
|
||||
### Choosing
|
||||
|
||||
| Need | Use |
|
||||
|------|-----|
|
||||
| Reach mf04/mamba over the overlay, mamba may be absent | **B1** (netbird, on-demand) |
|
||||
| Reach makerfloss1 / mf04 / VPS-side services, max isolation, no tunnel on fisi | **B2** (VPS bastion) |
|
||||
| mamba is at the makerspace and you want the existing Ansible path | **B3** (ProxyJump) |
|
||||
| The **switch** mgmt plane (`192.168.88.1`) | None of these directly — mgmt VLAN 99 is reachable only from `ether8`. Need a host *on* `ether8` (mamba, or a testmachine patched in) to forward through. See switch runbook. |
|
||||
| Reach srv07/mf01/VPS services, any time | **C1** (bastion) |
|
||||
| Box's wg1 is down / raw LAN work, mamba on-site | **C2** (via mamba) |
|
||||
| The switch mgmt plane | Neither directly — needs a host on the mgmt port; see the switch runbook and verify the post-recabling mgmt path on-site. |
|
||||
|
||||
---
|
||||
|
||||
## C. Isolation rule (important)
|
||||
## D. Netbird (legacy — kept, not used for access)
|
||||
|
||||
> **Keep the makerspace and the homelab apart.** fisi must not hold a standing
|
||||
> tunnel into the makerspace. Bring Netbird **up only for the task, down
|
||||
> immediately after**. Prefer the VPS-bastion path (B2) when it suffices, since
|
||||
> it puts no makerspace network on fisi at all. The concern is preventing any
|
||||
> compromise at the makerspace from reaching fisi/the homelab.
|
||||
The self-hosted control plane `nb.makerfloss.eu` still runs on the VPS, and
|
||||
srv07's agent is still enrolled (`100.99.133.190`). It is **no longer an
|
||||
access path**: fisi (the only Claude-side peer) is decommissioned, ubongo is
|
||||
deliberately not enrolled (isolation + boma ADR-036 doctrine: static
|
||||
WireGuard, config-as-code, no coordinator). Operator decision 2026-07-17:
|
||||
keep it running for now; do **not** enroll new peers for management access.
|
||||
Retirement is the eventual direction — that decision is logged, not taken.
|
||||
|
||||
---
|
||||
## E. Isolation rule (unchanged, important)
|
||||
|
||||
## D. Secrets & credentials (where they live — not stored here)
|
||||
> **Keep the makerspace and the homelab apart.** ubongo must not hold a
|
||||
> standing tunnel into the makerspace and must not join makerspace overlays
|
||||
> (wg1, Netbird). The bastion path (C1) puts no makerspace network on ubongo
|
||||
> at all; the mamba path (C2) rides boma's own hub and ends at mamba. The
|
||||
> concern is preventing any compromise at the makerspace from reaching the
|
||||
> homelab.
|
||||
|
||||
- **Ansible vault (baobab):** `~/.ansible/vault-keys/prod.txt`; secrets in
|
||||
`AnsibleBaobabV4/group_vars/all/90-secrets.vault.yml`
|
||||
(`ansible-vault edit ... --vault-id prod@`). WG/Netbird/service secrets are
|
||||
`vault_*` keys there.
|
||||
## F. Secrets & credentials (where they live — not stored here)
|
||||
|
||||
- **Ansible vault (baobab/prod):** `~/.ansible/vault-keys/prod.txt` (present
|
||||
on ubongo); secrets in `AnsibleBaobabV4/group_vars/all/90-secrets.vault.yml`.
|
||||
WG/Netbird/service secrets are `vault_*` keys there
|
||||
(`vault_wireguard_makerfloss_peers.*` — srv07's keypair is still under the
|
||||
historical name `mf04`).
|
||||
- **Ansible vault (switch):** identity `makerfloss`,
|
||||
`~/.ansible/vault-keys/makerfloss.txt`; switch admin password in
|
||||
`MakerFLOSS_Mikrotik/group_vars/mikrotik.vault.yml`.
|
||||
- **SSH:** operator key `~/.ssh/id_ed25519`; project-wide SSH port is **7576**
|
||||
(switch SSH is on `:22`, Forgejo git on `:7577`).
|
||||
(switch SSH `:22`, flossfw `:22`, Forgejo git `:7577`).
|
||||
|
||||
Never commit secrets to this repo.
|
||||
|
|
|
|||
70
incidents/2026-07-05-mamba-loses-netbird-on-switch-cable.md
Normal file
70
incidents/2026-07-05-mamba-loses-netbird-on-switch-cable.md
Normal file
|
|
@ -0,0 +1,70 @@
|
|||
# 2026-07-05 — mamba drops its netbird link when the switch cable is plugged in
|
||||
|
||||
- **Host(s):** `mamba` (makerspace laptop), CRS310 `crs310-maker`
|
||||
- **Reported by / observed:** sjat, on-site at the makerspace
|
||||
- **Reach path used:** Claude on `ubongo` → netbird overlay → `mamba`
|
||||
(ssh `sjat@100.99.101.182 -p 7576`). access.md §A + on-demand netbird.
|
||||
|
||||
## Symptom
|
||||
|
||||
sjat is working over a netbird overlay from `ubongo` to `mamba`. The moment the
|
||||
ethernet cable to the CRS310 is plugged into `mamba` (`enp0s31f6`), the netbird
|
||||
session drops and Claude loses `mamba`.
|
||||
|
||||
## Investigation
|
||||
|
||||
- netbird on `mamba` is **relayed** via `netbird.askari.wingu.me:443`, so all
|
||||
overlay traffic egresses `mamba`'s **default route** — normally WiFi
|
||||
`OrangeMakers` (`172.17.3.60`, `dev wlp0s20f3`, metric 600).
|
||||
- On carrier-up, NetworkManager auto-activated **`Wired connection 1`**
|
||||
(autoconnect=yes, DHCP), not the purpose-built static profile `crs310-bench`
|
||||
(autoconnect=no, so it never won the race).
|
||||
- The CRS310 mgmt-VLAN DHCP hands out a **default gateway**:
|
||||
`backups/crs310-maker/export.rsc` → `/ip dhcp-server network add
|
||||
address=192.168.88.0/24 gateway=192.168.88.1`.
|
||||
- Wired links get a lower default metric (~100) than WiFi (600), so DHCP
|
||||
installed `default via 192.168.88.1 dev enp0s31f6` and it **won**. netbird's
|
||||
underlay to askari then routed through the switch — which by design has **no
|
||||
gateway/DNS/internet** — so the tunnel died.
|
||||
|
||||
## Root cause
|
||||
|
||||
The wired DHCP profile on `mamba` accepted the switch's DHCP-provided default
|
||||
gateway, stealing the default route from WiFi and cutting the netbird underlay.
|
||||
The isolated mgmt VLAN advertising a gateway it can't route is the upstream smell.
|
||||
|
||||
## Fix
|
||||
|
||||
- **Live action taken (mamba, NetworkManager — client-side, this box only):**
|
||||
```
|
||||
sudo nmcli con mod "Wired connection 1" \
|
||||
ipv4.never-default yes ipv4.route-metric 700 ipv4.ignore-auto-dns yes
|
||||
```
|
||||
Wired link still gets a DHCP address on the mgmt subnet (reaches the switch)
|
||||
but never installs a default route or hijacks DNS; WiFi stays the default and
|
||||
the netbird lifeline holds. Not captured in any repo — lives in `mamba`'s NM
|
||||
config. `crs310-bench` (static `.2`, `never-default yes`) remains as the
|
||||
alternative sticky profile.
|
||||
- **Source repo + commit:** none required for the client-side fix. Cleaner root
|
||||
fix is switch-side (see Follow-ups) and would land in `MakerFLOSS_Mikrotik`.
|
||||
|
||||
## Verification
|
||||
|
||||
With the fix applied and the cable seated (link flapped once on a loose seat,
|
||||
then stable at 100 Mbps):
|
||||
|
||||
- `mamba` `enp0s31f6` → `192.168.88.253/24` (DHCP from mgmt pool).
|
||||
- `ip route show default` → **WiFi only** (`172.17.3.1 … metric 600`); no
|
||||
`192.168.88.1` default appeared.
|
||||
- `ip route get 100.99.146.14` (ubongo) → still via `wt0` (netbird).
|
||||
- `ping 192.168.88.1` → 0% loss, ~0.85 ms. Claude never lost `mamba`.
|
||||
|
||||
## Follow-ups
|
||||
|
||||
- **Switch-side root fix (preferred):** drop `gateway=192.168.88.1` from the
|
||||
mgmt-VLAN `dhcp-server network` in `MakerFLOSS_Mikrotik` — an isolated mgmt
|
||||
plane with "no gateway/NTP/DNS" should not advertise one. Then *no* client
|
||||
needs the NM workaround. Branch + run-twice per that repo's lockout rules.
|
||||
- Consider making `crs310-bench` the sticky wired profile on `mamba`
|
||||
(`autoconnect yes` + higher priority, `Wired connection 1` off) so the static
|
||||
`.2` is used deterministically for mgmt work.
|
||||
60
incidents/2026-07-17-srv07-access.md
Normal file
60
incidents/2026-07-17-srv07-access.md
Normal file
|
|
@ -0,0 +1,60 @@
|
|||
# 2026-07-17 — srv07 (ex-mf04) unreachable by every path; identity settled; access re-established
|
||||
|
||||
## Symptom
|
||||
|
||||
Session goal: conclude an SSH access path to the testmachine ("mf04, now
|
||||
srv07 — we think"). Every documented path was dead: old LAN IPs
|
||||
(`10.0.0.183/.184`, `10.2.30.61`) unreachable, wg1 peer `10.13.0.3` had no
|
||||
handshake and the VPS had no route to it, the fisi/Netbird path no longer
|
||||
exists (fisi decommissioned, ubongo has no Netbird agent — boma ADR-036).
|
||||
Also unclear: whether the box is srv05 or srv07 — the naming-scheme migration
|
||||
table said `mf04 → srv05`, the physical sticker said srv07.
|
||||
|
||||
## Root cause
|
||||
|
||||
Two physical-layer facts, no software fault:
|
||||
|
||||
1. **The box's network path was unplugged.** pp02:8 (srv07 eth0) had lost its
|
||||
rear patch when sw01:7 was repurposed as the sw06 uplink — already recorded
|
||||
in `MakerFLOSS/docs/hardware/pp02.md`, but not connected to "that's why
|
||||
nothing answers." The machine itself had been up 9+ days.
|
||||
2. **The naming table was wrong.** The physical box wearing the srv07 sticker
|
||||
is the Ansible/OS host mf04 (i3-4360T/16 GB/500 GB HDD — matches neither
|
||||
srv04's nor srv05's recorded i5-3570K specs). `mf04 → srv05` was a
|
||||
mis-mapping made when the placeholder pages were renamed.
|
||||
|
||||
## What was done (operator on-site with mamba; Claude from ubongo via the boma hub → mamba)
|
||||
|
||||
- Operator re-plugged the box and wired mamba into the rack switching; the box
|
||||
immediately took its old lease (`10.0.0.183`) and its wg1 tunnel re-handshook.
|
||||
- Renamed the host live to match the sticker: hostname `mf04 → srv07`,
|
||||
`/etc/hosts`, wg unit `makerfloss-mf04 → mf-srv07`. Gotcha: wg-quick
|
||||
interface names are capped at **15 chars (IFNAMSIZ)** — `makerfloss-srv07`
|
||||
(16) fails with a misleading `does not exist`; hence `mf-srv07`.
|
||||
- AnsibleBaobabV4 `0bce3a1`: `host_vars/mf04.yml → srv07.yml`; `ansible_host`
|
||||
re-pinned from the Netbird IP to wg1 `10.13.0.3` + ProxyJump via the VPS
|
||||
(mirrors mf01); inventory + hub peer entry renamed. Vault keys keep the
|
||||
historical `mf04` name (same keypair).
|
||||
- MakerFLOSS docs `f047332`: srv07 real specs, dated correction of the
|
||||
naming-table row, pp02 note (path restored, new patching unrecorded).
|
||||
- This repo: access.md rewritten for the ubongo era, network-map.md refreshed
|
||||
from live probing.
|
||||
- Netbird: **kept** (control plane + srv07 agent), operator decision — no
|
||||
longer an access path, not removed yet.
|
||||
|
||||
## Verification
|
||||
|
||||
- `ssh -J sjat@makerfloss.eu:7576 -p 7576 sjat@10.13.0.3` → hostname `srv07`
|
||||
(roaming path, end-to-end).
|
||||
- `ssh -p 7576 sjat@10.0.0.183` from wired mamba → works (on-site path).
|
||||
- `wg-quick@mf-srv07` active + enabled; hub handshake fresh after rename.
|
||||
|
||||
## Follow-ups
|
||||
|
||||
- Record srv07's actual rear patch / switch port (pp02.md + sw01.md).
|
||||
- Identify the rack-LAN Debian trio `.10/.11/.12` and `.158/.165/.209`.
|
||||
- Find/verify makerfloss1 (LAN and wg1 both dark).
|
||||
- Ansible play against srv07 not yet re-run — first run after the rename
|
||||
should be watched (wg config name change: the play must not resurrect a
|
||||
`makerfloss-srv07`/`makerfloss-mf04` config; host_vars now says `mf-srv07`).
|
||||
- Eventual Netbird retirement decision (logged in access.md §D).
|
||||
|
|
@ -30,4 +30,6 @@ what we found, what we changed (and in which source repo), outcome.
|
|||
|
||||
## Log
|
||||
|
||||
_(none yet)_
|
||||
- [2026-07-05 — mamba drops netbird when the switch cable is plugged in](2026-07-05-mamba-loses-netbird-on-switch-cable.md)
|
||||
— switch mgmt-DHCP default gateway stole mamba's default route; fixed with
|
||||
`never-default` on the wired NM profile.
|
||||
|
|
|
|||
|
|
@ -1,47 +1,37 @@
|
|||
# Network map (thin)
|
||||
|
||||
Pointers, not the source of truth. Authoritative data is in the source repos —
|
||||
links below. Confirm live values before acting.
|
||||
links below. Confirm live values before acting. Refreshed 2026-07-17 from
|
||||
live probing (mamba wired on the rack LAN + the wg1 hub + mf01).
|
||||
|
||||
## Subnets seen across the repos
|
||||
## Subnets seen live on 2026-07-17
|
||||
|
||||
| Subnet | Role | Source of truth |
|
||||
|--------|------|-----------------|
|
||||
| `10.2.30.0/24` | **CRS310 data VLAN 30** (the new switch). Uplink `ether1` → gateway `10.2.30.1`; access ports `ether2–7`. | `MakerFLOSS_Mikrotik/host_vars/crs310-maker.yml`, `docs/superpowers/specs/2026-06-09-crs310-flat-mgmtvlan-design.md` |
|
||||
| `192.168.88.0/24` | **CRS310 mgmt VLAN 99** — isolated, switch at `192.168.88.1`, reachable only from `ether8`. DHCP `.10–.254`. | same |
|
||||
| `172.17.3.0/24` | OrangeMakers LAN — `makerfloss1` at `.51`. | `AnsibleBaobabV4/host_vars/makerfloss1.yml` |
|
||||
| `10.0.0.0/24` | Makerspace LAN — `mf04` at `.184`. | `AnsibleBaobabV4/host_vars/mf04.yml` |
|
||||
| `10.13.0.0/24` | **makerfloss WireGuard plane (`wg1`)**. Hub `10.13.0.1` (VPS), `makerfloss1` `.2`, `mf04` `.3`, `sjat-roaming` `.5`. UDP `:51820`. | `AnsibleBaobabV4/host_vars/makerfloss.yml`, `specs/2026-05-12-makerfloss-wireguard-design.md` |
|
||||
| `100.99.0.0/16` | **Netbird overlay** (`wt0`), control plane `nb.makerfloss.eu`. Peers: mf04 `100.99.133.190`, fisi `100.99.61.26` (on-demand, normally down). | `specs/2026-05-27-makerspace-vpn-design.md` |
|
||||
| `10.8.0.0/24` | baobab (home) WireGuard plane. Hub **kuku** `10.8.0.1` (UDP `:51194`); mamba `10.8.0.4`. | `AnsibleBaobabV4` |
|
||||
| `10.20.10.0/24` | homelab LAN — **fisi** `.17`, kuku `.118`, papa `.11`. | `AnsibleBaobabV4` |
|
||||
| Subnet | Role | Notes / source of truth |
|
||||
|--------|------|-------------------------|
|
||||
| `10.0.0.0/24` | **Makerspace rack LAN** (the subnet you get wired into the rack switching). Gw `.1` (FreeBSD — likely flossfw). srv07 `.183`, mf01 `.223` (both DHCP, ssh `:7576`). Unidentified: Debian 13 trio `.10/.11/.12` (ssh `:22`, likely TaPPaaS nodes), ssh hosts `.158/.165/.209`. | live scan 2026-07-17; `AnsibleBaobabV4/host_vars/srv07.yml`, `mf01.yml` |
|
||||
| `172.17.3.0/24` | **OrangeMakers house LAN / wifi** (mamba's wifi lands here). Does **not** route to `10.0.0.0/24`. `makerfloss1` recorded at `.51` — did not answer. Seen: a Raspberry Pi `.66`, a Debian 12 box `.215` (neither is ours). | live scan 2026-07-17 |
|
||||
| `10.13.0.0/24` | **makerfloss WireGuard plane (`wg1`)**, hub `10.13.0.1` on the VPS, UDP `:51820`. Peers: `.2` makerfloss1 (down), **`.3` srv07 (up)**, `.5` sjat-roaming/mamba (dormant), `.6` sjat-work (down), **`.8` mf01 (up)**, `.9` flossfw (down). | `AnsibleBaobabV4/host_vars/makerfloss.yml` |
|
||||
| `100.99.0.0/16` | **Netbird overlay** (legacy, kept for now — see `access.md` §D). Control plane `nb.makerfloss.eu` up; srv07 agent still enrolled (`100.99.133.190`); fisi peer gone (host decommissioned); ubongo deliberately not enrolled. | `specs/2026-05-27-makerspace-vpn-design.md` |
|
||||
| `10.99.0.0/24` | **boma WireGuard hub** (homelab plane — hub on askari `.1`; ubongo `.2`, mamba `.10`). Only relevance here: it's how Claude reaches mamba from ubongo. Never bridged to makerspace nets. | `boma` repo, ADR-036 |
|
||||
| `192.168.88.0/24` | CRS310 (sw01) mgmt VLAN — switch at `.1`, reachable only from the mgmt port. Post-recabling state **unverified**; sw01 p8 (mgmt) is patched out via pp01:3 and currently non-active. | `MakerFLOSS_Mikrotik`, switch runbook |
|
||||
| `10.2.30.0/24` | Former CRS310 data VLAN 30. **Probably gone** — mf01 moved off it onto `10.0.0.x`, and nothing answered on it 2026-07-17. Treat references to it (older specs, host comments) as historical. | historical: `specs/2026-06-09-crs310-flat-mgmtvlan-design.md` |
|
||||
|
||||
## Makerspace addressing — mostly resolved (2026-06-09)
|
||||
## Open questions (check when on-site)
|
||||
|
||||
Confirmed on-site:
|
||||
|
||||
- A client on the new switch's **data ports** (`ether2–7`) gets a
|
||||
`10.2.30.0/24` lease (sjat's laptop got `10.2.30.227`); gateway `10.2.30.1`.
|
||||
- The data VLAN `10.2.30.0/24` and the existing makerspace `10.0.0.0/24`
|
||||
**inter-route**: from `mf04` (`10.0.0.183`, gw `10.0.0.1`), both
|
||||
`10.2.30.1` and `10.2.30.227` ping at <1ms. So the two subnets are different
|
||||
segments joined by the makerspace router (`10.0.0.1` ↔ `10.2.30.1`), not
|
||||
isolated from each other.
|
||||
|
||||
Still loose:
|
||||
- `makerfloss1` is recorded as `172.17.3.51` — a *third* subnet. Not yet
|
||||
confirmed whether it's still on `172.17.3.x` or has moved onto `10.0.0.x` /
|
||||
`10.2.30.x`. Confirm when next on-site.
|
||||
- **IP drift:** `mf04` is actually `10.0.0.183` (DHCP), but
|
||||
`AnsibleBaobabV4/host_vars/mf04.yml` says `ansible_host: 10.0.0.184`. The
|
||||
ProxyJump-via-mamba path there targets the stale `.184`. Either pin a DHCP
|
||||
reservation or update host_vars. (Reaching mf04 over `wg1` `10.13.0.3` is
|
||||
unaffected.)
|
||||
- **srv07's physical uplink:** connectivity restored 2026-07-17, but the new
|
||||
rear patch of pp02:8 (or a direct cable?) is unrecorded — see the note in
|
||||
`MakerFLOSS/docs/hardware/pp02.md`. Record switch + port.
|
||||
- **makerfloss1:** where is it, and is it still alive? Neither its LAN IP nor
|
||||
its wg1 peer answered.
|
||||
- **The `.10/.11/.12` trio and `.158/.165/.209`:** identify and label
|
||||
(TAPPaaS nodes? APs?). Add to the hardware docs once known.
|
||||
- **Switch mgmt path** after the rack re-cabling (sw01/sw06/sw07): confirm
|
||||
how to reach `192.168.88.1` and update the switch runbook.
|
||||
|
||||
## Public services (makerfloss VPS, `88.99.32.236`)
|
||||
|
||||
All TLS-terminated at the VPS via Traefik, certs via Gandi DNS-01:
|
||||
`docs.makerfloss.eu`, `slides.makerfloss.eu`, `forgejo.makerfloss.eu` (git SSH
|
||||
`:7577`), `mail.makerfloss.eu` (Poste.io), `discourse.makerfloss.eu`,
|
||||
`snipeit.makerfloss.eu`, `nb.makerfloss.eu` (Netbird).
|
||||
`snipeit.makerfloss.eu`, `nb.makerfloss.eu` (Netbird, legacy).
|
||||
Source: `AnsibleBaobabV4/host_vars/makerfloss.yml`.
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue