personal data lake
Infovore
Self-hosted infrastructure for digital sovereignty: personal data archive (Infovore), backup, web presence, chat, and RSS/worklog serving. The goal is to disentangle from corporate cloud and run as much as possible from the MacMini via TailScale.
Sub-projects
-
HeresiarchHalogen active
Possibly the old pre-Claude attempt at andrewcondon.com/heresiarch.com, superseded by the Claude-authored version on Cloudflare. Needs investigation to confirm — if it is the old version, should be marked defunct.
-
Infovore Data Sources 2 active
Personal data archive consolidating 58 years of digital material — music (149K tracks), photos (~145K), books, podcasts, apps, manuals, recipes, quotes, garden — and, since June 2026, the **system of record for life-realm projects**: `life-projects/{woodworking,house,garden}/*.md`, one frontmatter-markdown file per project. That folder is the bare reference ProjectSource in the press-vs-title model — Marginalia's server reads it straight off disk at request time and federates it into the newspaper Register alongside the engineering DuckDB, so life projects live here but still appear in the paper. Frontmatter carries title/domain/status/created/updated plus tracker_id (the federated identity) and optional parent_tracker_id (nesting). **MacMini is the only running instance.** It hosts the Crucial4TB SSD with the actual media files, runs `tools/webapp/server.py` on `*:8090` as the `net.hylograph.infovore` LaunchAgent, exposes the dashboard via Tailscale at http://andrews-mac-mini:8090, and is the only machine BackBlaze covers. Its checkout pulls over SSH (deploy key added 2026-06-10) and is what the marginalia press reads life-projects from. **MBP is the code editor — nothing runs there.** Repo at /Users/afc/work/afc-work/infovore-larder-db. Workflow: edit → `scripts/deploy-to-macmini.sh` (rsync, excludes `data/`) → kickstart the LaunchAgent → test via Tailscale URL. The MBP carries no `data/` by design. Per-category sqlite DBs in `data/<category>/*.db` summarised by `/api/stats`; each viewer is a self-contained static dir under `tools/<category>/<tool>/` mounted at `/t/<slug>/`. Adding a category: ingester populates the DB, register a tool tile, dashboard auto-renders. Near-term: Music Finder viewer (FTS5 search across 149K tracks, dual-mode playback). Eventually: real personal-tracking feeds (finance from bank export, health) land here in the per-domain SQLite idiom — the tracker's finance/sports rows are demo fixtures, not data. --- Personal data archive consolidating 58 years of digital material — music (149K tracks), photos (~145K), books, podcasts, apps, manuals, recipes, quotes, garden, projects. **MacMini is the only running instance.** It hosts the Crucial4TB SSD with the actual media files, runs `tools/webapp/server.py` on `*:8090` as the `net.hylograph.infovore` LaunchAgent (since 2026-04-26), exposes the dashboard via Tailscale at `http://andrews-mac-mini:8090` for laptop and phone access, and is the only machine BackBlaze covers (data + code). **MBP is the code editor — nothing runs there.** Repo lives at `/Users/afc/work/afc-work/infovore-larder-db`. Workflow: edit → `scripts/deploy-to-macmini.sh` (rsync, excludes `data/`) → `launchctl kickstart -k gui/501/net.hylograph.infovore` on the MacMini → test via the Tailscale URL. The MBP carries no `data/` directory by design — keeping it would mean drifting unbacked-up state. Per-category sqlite DBs in `data/<category>/*.db` summarised by `/api/stats`; each viewer (Walk Map, Photo Finder, Apps Triage, Books, Manuals) is a self-contained static dir under `tools/<category>/<tool>/` mounted at `/t/<slug>/`. Adding a category means: ingester populates the DB, register a tool tile, the dashboard auto-renders. Near-term: Music Finder viewer — fast FTS5-backed search across the 149K tracks plus dual-mode playback (HTML5 audio in browser, server-side `afplay` on the MacMini for stereo).
-
nextcloud-mutual-backup active
Mutual backup plan with Paul Harrington: symmetric discs (4TB Crucial SSD each), send him a disc for his setup in Boston. Also aspiration to run Nextcloud on the MacMini. Need to investigate current status and whether Nextcloud and the mutual backup strategy are compatible or should be separate projects.
-
worklog-server active
Python server that serves worklogs and knowledge base docs from various projects as RSS/Atom feeds, validated with NetNewsWire. Dovetails with the Project Tracker — should surface KB content into project cards. The RSS approach (drip-drip-drip of Claude session output as feed items) is appealing. Needs brainstorming session on how to integrate with the project tracker.
-
MacMini service supervision someday
A DeepStar-like supervisor for the MacMini's always-on services — motivated by the whisper sidecar silently crash-looping ~294,000 times over a month (May–June 2026) with nothing noticing until dictation was wanted. The mini runs six-plus LaunchAgents with no health surface: marginalia api/frontend/whisper, infovore, infovore-files, plus Tailscale itself (which sometimes just stops, severing the tailnet while LAN stays up). Shape: along the lines of DeepStar (#191, the live-coding rig's tier-1 supervisor — up/down/status/restart/logs over a service roster), running on the mini and watching its roster: is each agent up, is its port answering, is Tailscale connected; surface a status page (candidate: a Deployments/health row in the newspaper) and possibly auto-remediate the known failure modes (Tailscale up when stopped). Related: the launchd wrapper backoff scripts already record failure counts that nothing reads — a supervisor could alert on them. Cheap detection over silent rot, same instinct as the Brunel hygiene axis.
-
Shared human/Claude KB conventions someday
Plan a small KB design pass: where Claude deposits notes, frontmatter discipline, how the index regenerates, how stale entries get demoted. Triggered by a 2026-04-26 review of LightRAG (https://lightrag.github.io) — the retrieval substrate is the easy part; the hard part is the human-and-agent write-loop, conventions, and avoiding entropy. Andrew flagged disk-space concern: without a way to compact learnings + conversations, the KB plus Claude's auto-memory could grow unbounded. Pickup target: next week (2026-05-04 or later). See KB note: purescript-polyglot/docs/kb/research/lightrag-and-shared-kb-2026-04-26.md.
-
Connector inventory idea
Make a complete inventory of all connectors and adapters: - PSUs for computers and phones, USB converters etc, Ethernet USBC - stereo to mono, headphone splitters - audio cables - plug adapters - IEC cables and the like Need to print and attach labels to all this stuff, create a map of the connections covered and understand what is and isn't present.
-
de-Google, de-Apple, Deee-lite idea
A project to track the various ways in which i'm tied in to the American tech giants and/or just tied into services where i don't control my data * Proton mail replacement for Google gmail (in progress) * Proton or other replacement for 1Password * Instapaper (done) * Quotebook (done) * Apple Notes (mostly done) *
-
Multi-host topology — schema + tooling idea
Make Marginalia, SDI, and the `/marginalia` skill aware that services live on different hosts (MBP, MacMini, Cloudflare, …) instead of assuming local single-machine deployment. Existing schema has a vestigial `environment` text field on `project_servers` that's defaulted to NULL = 'mbp-native' and not consumed anywhere; promote to first-class with structured values. Scope: add `host` (tagged enum: `mbp`/`macmini`/`cloudflare`/…) and `tailscale_name` (routable address) to `project_servers`. Backfill known services. Update SDI to spawn only ports whose `host` matches its own machine and respond with a redirect for others. Update the marginalia slash command so its default API URL comes from config rather than hardcoded `localhost:3100`. Add a JSON cache layer in SDI and the marginalia skill so transient inability to reach the canonical Marginalia API (degraded Tailscale, machine asleep) doesn't break local tooling — last-known registry is good enough for spawn decisions. Unblocks the in-place migration of Marginalia itself to MacMini (sibling project). Order: schema first → tooling consumers updated → migration → cache layer hardened.
-
Narrow-grant file daemon for Infovore-served files idea
Generalised approach to giving Infovore (and similar Python services on the MacMini) access to TCC-protected paths without granting Full Disk Access to `python3` itself. Instead of wrapping the FastAPI server as a code-signed `.app` bundle (the original framing — see below), build a small dedicated Go daemon (`infovore-files`) whose sole job is HTTP file serving from configured roots like `/Volumes/Crucial4TB/Music`. Infovore proxies or 302-redirects file requests to the daemon. The daemon is the only thing that needs FDA — auditable in ~100 lines, single-purpose, narrow stable signing identity. Generality: the daemon's config is a simple `prefix → absolute-path` map. Photos, videos, GPX, anything else that lives under `/Volumes/Crucial4TB/...` and needs HTTP serving plugs in by adding a root. One TCC grant covers all current and future viewers. First use: unblocks Music Finder's browser playback (paused in `project_music_finder_paused` memory). After that, can fold in any future viewer that needs protected-path bytes.
-
Marginalia migration to MacMini (in-place replace demo) done
Move Marginalia from MBP-canonical to MacMini-canonical. The MacMini already runs the three Marginalia LaunchAgents (api/frontend/whisper) but pointed at a demo DuckDB created when showing it to someone else. In-place replacement: stop the demo, swap in the real `tracker.duckdb` from the MBP, restart, confirm Tailscale accessibility from MBP and phone. Why: same logic as Infovore — MacMini is the always-on machine, gets BackBlaze coverage, accessible via Tailscale from any device, and Marginalia is critical infrastructure being relied on daily. Trade-off: MBP-side tooling pays a Tailscale hop (sub-5ms) on every Marginalia call. Dependency: should land after the multi-host schema + tooling work (sibling project) so MBP-side consumers (`/marginalia` skill, SDI) already understand a non-local API URL by the time the move happens. Hard cutover; keep the MBP Marginalia DuckDB as a backup until validated.
-
mattermost-tailscale dormant
MatterMost running in Docker on the MacMini, served via TailScale. Migrated from private Slack with Paul Harrington. Working and in use. Long-term, would prefer something more tightly integrated with AI — possibly converging with the Project Tracker.