home/cloud-search
Watch
1
0
Fork
You've already forked cloud-search
 0
Code Issues Pull requests Projects Releases Packages 2 Wiki Activity Actions
No description
11 commits 1 branch 0 tags 396 KiB
  • TypeScript 99.6%
  • Dockerfile 0.4%
Find a file
Exact
Repository files (latest commit first)
Filename Latest commit message Latest commit date
Nick Kratzke ce211cdc6e
All checks were successful
CI/CD / Typecheck & test (push) Successful in 23s
Details
CI/CD / Deploy to Dockhand (push) Successful in 23s
Details
CI/CD / Re-host meilisearch image (push) Successful in 6s
Details
CI/CD / Build & push app image (push) Successful in 6s
Details
feat: pause/resume the indexing pipeline from the web UI 
- store/pause.ts: shared pause switch backed by the kv table (persists
  across restarts); default running, pause()/resume()/toggle()
- drain-queue & poll-delta skip their ticks while paused (checked at tick
  start: in-flight work finishes, then polling + draining stop)
- POST /admin/pause + /admin/resume (session-gated, logged)
- /status now exposes indexing: paused|running
- search UI header: Pause indexing / Resume indexing button (state
  refreshed from /status on load)
- test/pause.test.ts: round-trip, idempotence, KV-level flag

109 tests pass, tsc clean.
2026-08-26 08:38:21 +02:00
.forgejo/workflows fix: Meilisearch v1.53 API (POST /indexes, PATCH settings), real healthcheck, deploy diagnostics 2026-08-22 22:55:48 +02:00
docker fix: prod baseline blockers (meili id, db lock, fd leak) + leveled logging 2026-08-23 23:32:25 +02:00
specs feat(vision): downscale images to VISION_MAX_EDGE before the VLM call 2026-08-24 00:46:55 +02:00
src/mastra feat: pause/resume the indexing pipeline from the web UI 2026-08-26 08:38:21 +02:00
test feat: pause/resume the indexing pipeline from the web UI 2026-08-26 08:38:21 +02:00
.dockerignore v0.1: deployable skeleton — search UI, auth, state, deploy pipeline 2026-08-22 22:47:40 +02:00
.env.example feat(vision): downscale images to VISION_MAX_EDGE before the VLM call 2026-08-24 00:46:55 +02:00
.gitignore v0.5: full indexing pipeline (delta poll → extract → LLM → Meilisearch) 2026-08-23 00:31:15 +02:00
bun.lock feat(vision): downscale images to VISION_MAX_EDGE before the VLM call 2026-08-24 00:46:55 +02:00
package.json feat(vision): downscale images to VISION_MAX_EDGE before the VLM call 2026-08-24 00:46:55 +02:00
README.md feat: pause/resume the indexing pipeline from the web UI 2026-08-26 08:38:21 +02:00
tsconfig.json v0.1: deployable skeleton — search UI, auth, state, deploy pipeline 2026-08-22 22:47:40 +02:00

README.md

cloud-search

Indexes a personal OneDrive into a locally hosted Meilisearch index. A continuously running agent (Bun + Mastra) polls the Graph delta API and indexes every file:

  • all files — metadata: filepath, type, dates, size, location (EXIF GPS)
  • images — LLM summary + OCR text from the photo
  • documents (pdf, docx, …) — extracted text content + LLM summary
  • text files (csv, txt, log, md, …) — content (when small) + LLM summary

Query surface

  • https://cloudsearch.barbara8.de — public, session-gated search UI (operator password login; sign in at /login, connect OneDrive at /auth).
  • Tailscale-only raw API: Meilisearch at http://100.96.0.81:17700 (master-key protected), app ad-hoc at http://100.96.0.81:18124 (/healthz, /status, /search, /auth).
  • Session-gated admin: POST /admin/pause + POST /admin/resume (toggle the indexing pipeline from the UI header — poll and drain ticks skip while paused; state persists across restarts), POST /admin/reindex (rebuild Meilisearch from the extraction cache — no downloads/LLM) and POST /admin/requeue-failed (reset terminally failed queue rows).

Status: v0.5 — indexing pipeline live (in code). The stack (app + Meilisearch) deploys and serves the UI, and the full pipeline runs: poll-delta (every 5 min, resumable Graph delta walk behind a single-flight lock) → drain-queue (every minute, bounded-concurrency worker: etag-dedup → download → extract → one LLM call → Meilisearch). The index fills once OneDrive is connected (/auth) and the baseline walk completes — see the spec's rollout plan (baseline ≈ 35k photos is v1).

One-time setup (spec §"First-time bootstrap"): register the Azure redirect URIs https://cloudsearch.barbara8.de/auth/callback and http://100.96.0.81:18124/auth/callback, then connect OneDrive at /auth once.

  • Spec: specs/cloud-search.md — read this first
  • Stack: Meilisearch v1.53.1 (re-hosted on the Forgejo registry) + Bun/Mastra indexer + libSQL state; LLM via the local vLLM endpoint (mac/qwen3.6-35b-a3b on llm.schwanke.it)
  • Deploy: Dockhand stack cloud-search on UM890 (env 3); Pattern D CI (.forgejo/workflows/ci.yml) builds both images and deploys on push to main
  • Related: home/agent — appointment-detection agent; reuses its OneDrive auth, libSQL, and LLM/session patterns