Architecture Overview

RingFacts tracks a watchlist of subjects — currently three MMA fighters, called Subject A (Daniel Donchenko), Subject B (Yaroslav Amosov) and Subject C (Ilia Topuria) throughout this page — for a fan group on Telegram. There is deliberately no UI: the Telegram group is the entire product surface. One repo, one container image, two entry points: a webhook service (the future conversational rail) and the hourly hunter job — same image, different --command/--args.

About 2,000 lines of production JavaScript, and roughly the same again in tests. There is no framework and no build step: plain ES modules, node hunter.js. The shape is one orchestrator and a set of single-purpose modules it calls — every decision the pipeline makes lives in hunter.js; every module under lib/ answers one question and holds no opinion about the run. That is the rule to keep in your head when reading any of this.

The core idea of the claims layer, live since 2026-08-08: the thing the group cares about is not an article — it's the underlying fact. The red corner is the evidence world: raw articles, immutable, exactly what the hunter collects. The blue corner is the fact world: canonical claims distilled from evidence, living records whose status changes as evidence accumulates.

The problem this solved in production: three-then-seven sightings of one Masvidal quote about Subject B (MMA Junkie original → XSPORT translation → heavy.com retelling → …) — retellings kept slipping the article-level gates, and corroboration (the most important signal in news!) was being silently discarded as "duplicate."

Three tables, all live in Neon. items carries the evidence plus a full dedup audit trail (why was this held, against what, at what similarity); claims and claim_sources carry the fact world.

The columns that make the system honest:

• nearest_similarity / nearest_item / held_reason — the dedup audit trail. Every held item records what held it and how close it was; threshold tuning (0.85 → 0.80) came from reading these columns, not from guessing.
• tg_message_id — the Telegram message the bot posted for a claim. This is what lets the bot later reply to its own rumor post with "✅ Confirmed" — threading the story.
• supersedes — claims are never edited when reality changes; a new claim supersedes the old ("moved to June 6" supersedes "on May 9"). History stays intact. (Column exists; the flow that writes it is phase 2.)
• rss_description — Google News's own related-coverage links per item. Captured "for later mining" on 2026-08-06; the mining arrived with 2e — it now feeds the matcher as secondary evidence for archive items with no body.
• resolved_url / body / body_fetched_at (2e) — the real article address once Google's wrapper is decoded (same as url for direct-feed items), and the extracted article text (capped ~10k chars). Null body = headline-only; every consumer tolerates it. url stays the identity; both columns feed the URL gate so the same story arriving from two rails is one item.
• body_via (2e) — which rung of the extraction ladder produced body, or which failure stopped it (feed-content is the richest rung; og-description the thinnest). Quality telemetry, not decoration: it is what let the 2026-08-10 dedup fix (§5) trace a wrong post back to the specific rung that produced the doubled body.
• digest_tier (2026-08-09, lib/tier.js) — main or tangential, set only on items that actually posted. Decides whether an article earns its own headline or folds into a shared "↘ Also mentioning" line (§5).
• subject_role (2026-08-10) — the matcher's own judgment of how prominent the subject is in the article: central, supporting, or passing, reported in the same tool call as the verdict. Unlike digest_tier it is set on every matcher-seen item (wrong-subject and match rows included) — deliberately, because it doubles as the audit dataset for measuring the LLM signal against the mention-count rule (§5). Null = matcher off or failed.

Every hour at :17, the hunter sweeps a 24-hour window of two discovery rails: Google News RSS per fighter per language edition, and — since 2e — six direct publisher feeds (fetched once, filtered per fighter by name; real URLs, and an independent rail if Google sheds our IP). The window overlapping the hourly cadence ~24× is the reliability trick: any single failed run loses nothing, because the next run re-covers it. Each unseen item then descends the ladder — cheap gates first, and network only for items that earned it:

Details that matter in production:

• The body step is downstream of the free gates on purpose (2e, spec §11 shallow-before-deep): held dups and known URLs never cost network. For each matcher-bound item — decode Google's wrapper (zero-network base64 for current tokens; the batchexecute fallback has a per-run circuit breaker), catch the cross-source duplicate a decode can reveal (same story already stored from a direct feed → held as echo through the drift guard), then fetch and extract the article (zero-dependency ladder: feed content → JSON-LD articleBody → <article> → paragraphs → og:description, the winning rung logged as quality telemetry). Direct-feed items usually skip the fetch entirely — WordPress feeds ship whole articles in content:encoded.
• The flood cap applies to unseen items only (max 5 per fighter per run, newest-first, truncation logged). An earlier version capped before the URL gate — which let already-known items permanently shadow older unseen ones. Found by questioning, fixed.
• Held ≥ 0.80 inherits the claim of its nearest neighbour as an echo — corroboration recorded for free, no LLM spent. The 0.80 threshold is measured, not guessed: a real translated pair scored 0.841, unrelated pairs ≤ 0.702.
• …but inheritance is checked against the claim itself (added 2026-08-08). Inheritance is transitive — B is held against A and takes A's claim, then C is held against B and takes it too — and every hop only has to clear 0.80 against the previous headline, so a chain can walk somewhere its starting claim never was. It did: a story about Subject C's manager blasting Ali Abdelaziz rode a 0.802 → 0.869 → 0.974 chain onto an unrelated matchmaking claim. Before linking, the item is now compared to the claim's own canonical text; if some other claim fits at least 0.10 better, the item stays held but unlinked (a wrong evidence row is worse than none — phase 2 counts these rows for corroboration, and a confirmation post lists them to the group). The 0.10 came off the live data: measured across all 28 links, drifted ones sat 0.107–0.214 below the item's best-fitting claim, while correct-but-awkward ones sat 0.076–0.082 below. Matcher-made links are never second-guessed — only the cheap gate is.
• Everything is idempotent. Re-running any part (migrations, backfills, the hunt itself) is always safe — the URL gate and IF NOT EXISTS guards make reruns no-ops.

The matcher is the intelligence concentrated at the top of the ladder. For each item that survives the free gates, one Haiku call receives the headline, source, timing, a body excerpt when one exists (first 1200 chars — 2e; archive items get Google's related-coverage cluster from rss_description instead) — and all active claims for that fighter, similarity-ordered ("order, don't cut": a top-3 cutoff was measured risky because metaphor headlines bury the right candidate). Forced tool use means the answer is always a parseable verdict — no JSON scraping. Five verdicts:

Prompt-contract rules learned from live data (each one bought by a real failure in bootstrap testing):

• Reactions are NEW, not MATCH — "X reacts to Y's prediction" is different news from the prediction.
• Peripheral mentions are NO_CLAIM, not WRONG_SUBJECT — an early prompt dropped division news that involved the fighter as context; readers wanted it.
• Canonical text must be strictly evidence-supported (headline + body excerpt) — the model once invented "UFC defeat at the White House." Now: never invent; too ambiguous → UNSURE.
• Headline-blindness is fixed for new items (2e): a body that clearly concerns the fighter overrides a headline that never names them ("Usman walks through the division"). The pre-2e archive has no bodies, so old items of that shape stay dropped — accepted.
• Furniture mentions fixed (2026-08-09): the 2e watch item fired — three digest posts whose only "mention" of the fighter was invisible markup (an image alt, a URL slug) or a "LATEST NEWS" promo block. Two-layer fix: matchesSubject now searches reader-visible text only (htmlToText before matching, kills the markup vectors upstream of Gate 1), and the matcher rules gained an explicit line: a fighter appearing nowhere in the shown text, or only in site furniture, is WRONG_SUBJECT. Prose mentions ("Usman's manager targets Topuria") still post.
• Digest tier shipped (2026-08-09): not everything the matcher lets through earns a headline. WRONG_SUBJECT drops an article outright; tangential (new, downstream of the matcher) folds it into a shared "↘ Also mentioning: Source1 · Source2" line instead — still posted=true, still linked, just not a bullet of its own. Rule (lib/tier.js): headline never names the fighter, body ≥300 characters (a floor — a 141-char blurb can't support a mention count), and the body names them ≤1×. Threshold measured over the archive: claim-bearing articles name the fighter 2–12× in their body, the junk cluster 0–1×, a clean gap. Claim-producing items are exempt by construction.
• Duplicated paragraphs were feeding the tier rule a false mention count (2026-08-10): Sports Illustrated renders its own lede paragraphs twice inside <article> (measured live: 24 paragraphs, 17 unique). A story about an untracked fighter that named Subject C in exactly one passing sentence therefore counted 2 mentions — one over the demote threshold — and posted a full headline in the Subject C digest it should not have earned. The tier rule itself was never wrong; the body handed to it was. Fixed in lib/extract.js: a paragraph identical to an earlier one is dropped before the mention count runs — both where the extractor reads <article>/<p> tags directly, and at rung 0, where a publisher's own feed ships the whole article (content:encoded) and would otherwise flatten to one line before any duplicate could be spotted. lib/extract.test.js ("duplicated paragraphs (SI pathology, item #72)") guards both paths; a walkthrough of this exact item, gate by gate, is linked at the foot of this page.
• The matcher now judges prominence, and the tier listens (2026-08-10): the mention count has a blind spot the SI fix couldn't close — item #73, an article about an untracked fighter, named Subject C twice as genuine background color ("teammate of…", "in his corner") and beat the ≤1 threshold honestly. The count can't tell a bystander mention from one that matters, but the Haiku matcher, which already reads every one of these articles, can. It now reports subject_role — central | supporting | passing — in the same forced tool-use call (zero extra LLM spend), and the tier checks that first: passing folds the article, anything else falls through to the measured count rule unchanged. The LLM signal is deliberately the weaker partner: it can only add demotions, never rescue an article the count rule folds, and a headline naming the fighter still always keeps its bullet. Every verdict's role is stored on the item so audit-digest-tier.js can cross-tab LLM-vs-count agreement over real data — that dataset, not intuition, decides whether the LLM ever earns more authority.
• The "also mentioning" line stopped hiding an outlet's second story (2026-08-10): the line grouped folded articles by outlet and rendered one link each — the newest — so an outlet with two demoted stories in one run had its older one appear nowhere. That line is the only place a folded article is ever offered, and Gate 1's URL check guarantees no later run offers it again, so the story was reachable from the archive and from nothing else. It now renders both, numbered newest-first (Bloody Elbow (1) · Bloody Elbow (2)); a bare repeated name reads as a bug, and the index is what earns the second link its place. Numbering appears only where an outlet has more than one, and identical URLs still collapse — the same article reached twice is one story. The case has never occurred in the live archive, which is why it shipped with a forced integration check (verify-digest-tier.js) rather than waiting for news to produce one.

Statuses live on the claim, never on articles, and transitions are driven by rules, not an LLM — the LLM's job ends at "what does this article assert, and how is it sourced?" Phase 1 deliberately ships the conservative core: claims are born rumor (or born confirmed via an official source), and the only path to confirmed is ufc.com asserting it. Counting "two independent sources" waits for phase 2, when article bodies make independence detectable (§7).

Designed, not built — but 2e delivered its prerequisite (bodies + real URLs), so this is now buildable. "Two sources" is worthless if the second copied the first: XSPORT translating MMA Junkie is one reporting chain, not two. Corroboration hinges on telling echoes from independent reports. Signals, cheapest first:

• Attribution text (strongest). Echoes usually confess in their own body: "as reported by MMA Junkie". The matcher reads bodies in 2e anyway — an attributes_to field comes free.
• Original reporting markers. "told us", "in an interview with this publication", exclusive quotes → origin candidate.
• Timing. Earliest sighting in a cluster is the presumed origin (verified on live data: MMA Junkie preceded both echoes).
• Same-org detection. A curated outlet-family map, grown lazily as false corroborations appear.

Default when unsure: echo. The asymmetric cost rules this: a false "confirmed" tells the group fiction is fact; a delayed confirmation just waits for the promotion's own announcement, which always comes. Bias toward skepticism — the same reasoning that made phase 1 ufc.com-only.

Posting is "one message per claim event", not per article. The claim type picks the ceremony; the claim event picks the timing. Headlines post as-is when they're in the group's languages (uk/en); anything else is translated to English by Gemini and labeled "(translated from es)" — one language, never both. The headline is plain text; the source name is the clickable link.

The rumor→confirmed thread, concretely (this is what the armed Subject A trap will produce):

The bot runs unattended; every failure mode found in production has a specific answer:

Total: still under two dollars a month. 2e added a discovery rail (6 outlet feeds, fetched free) and a body-fetch step (plain HTTP GETs to publisher sites, also free) — neither touches a billed API, so the wider funnel shows up only as more Haiku matcher calls, and even doubling that volume barely moves the total. The escalation-ladder economics hold: the archive can grow 100× and only the (free) pgvector search notices. Fits the planned $10 Anthropic cap with a wide margin. Exact vector scans are fine at this scale; an HNSW index becomes worth it around ~100K+ rows — years away.

The system maintains itself. A scheduled self-improvement session runs every ~6 hours (~18:47 / 00:47 / 06:47 / 12:47 PT — between hunter runs) and:

• Reads its continuity first: docs/checkin-log.md (the append-only run log — each session's trace, newest on top), TODO.md, and recent git history.
• Audits the last window: run results, matcher verdicts, held items, claim births, feed health — data first, opinions second.
• Acts under the decide-act-report policy: when in doubt, take the best decision for the project and report it post-operationally for correction — don't stall. Ask-first is reserved for: destroying schema/data, adding paid services, contradicting a documented decision, or changing what the group experiences in an undesigned way.
• Commits every session — even a no-change run appends its log entry. Then posts a concise chat report: 📊 data findings · 🔧 changes · 💡 proposals · 👁 watch items.

Hard rules baked into every run: never post to the group or any DM; never print secret values (command substitution only); never delete data. The first autonomous run already shipped real code — the normalizeVerdict gate (§5), including catching the pg-bigint-vs-JSON-number id comparison bug live.

Parked evolutions (specs written, deliberately not built): docs/sandboxed-autonomy.md — moving this loop into an ephemeral sandbox (GitHub Actions cron) with scoped credentials so even a fully poisoned run is harmless; and hourly database snapshots to an always-free GCS bucket (TODO), since Neon's free tier keeps only a 6-hour restore window.

Build sequence ahead:

• 2e — SHIPPED 2026-08-08 (see §4): six direct outlet feeds, wrapped-URL decode, zero-dep body extraction, matcher reads excerpts, full claims re-bootstrap (claims #9–16, drifted links cleaned, argmax mismatches 7→2). Phase 2's prerequisite is delivered.
• 5 phase 2 — the full lifecycle, NOW UNBLOCKED: independence-counted corroboration (attribution text is readable in bodies), denials, supersede flow, corroboration display (edit vs. reply — taste decision pending).
• Step 6 — conversational responder with memory and web search ("where to watch?"), through the dormant webhook service. This is where Mastra + TypeScript enter — single structured calls (the matcher) never needed an agent framework; a tool-using conversational agent does.
• Deploy automation: GitHub remote + Actions (also the prerequisite for the sandboxed autonomy spec and phone-started cloud sessions).

Standing watch items (tracked by the autonomous loop):

• 2e telemetry: decode success rate (fast path 3/3 at launch), per-outlet fetch 403s, dup-hold volume with two rails on. Which extraction rung wins is now QUERYABLE (body_via, shipped 2026-08-09) instead of console-only — the gap that hid Sherdog's 4/4 ~75-char og-description blurbs from every prior audit.
• Peripheral body-mention items from outlet feeds: the furniture-mention leak was fixed 2026-08-09 (visible-text gate + WRONG_SUBJECT rule for site furniture); still watch that genuine prose-mention stories keep posting and real news isn't over-dropped.
• Digest tier (2026-08-09): tangential articles fold into an "also mentioning" line rather than a bullet (§5). Watch the demotion rate against the ~17% measured on the archive — it was measured on a backfilled, not organically-fetched, sample, and since 2026-08-10 the subject_role check adds demotions of its own, so expect the rate to move; audit-digest-tier.js cross-tabs the two signals against each other. Cross-rail feed reuse (read a Bloody Elbow article's own feed text instead of a 403ing direct fetch, when Google discovers the same story first) is designed but backlogged — deliberately, until it can be done without hardcoding per-outlet fetch behaviour, since adding athletes may change which outlets/feeds/ extraction strategies matter at all.
• 503s under the 75s retry; direct feeds now cover a Google blackout, so escalation is only about lost Google-exclusive discoveries.
• The 0.80 embedding threshold — revisit if false holds appear.
• Canonical-name flubs (an Islam/Khabib mix-up observed once) — Sonnet escalation if it becomes a pattern.
• Multi-claim articles (v1 extracts one primary claim), cross-fighter claims ("Amosov vs Donchenko" would touch two watchlists), storyline threading above claims, a lazy source blocklist for keyword-stuffing aggregators, the 14-vs-30-day staleness window — all observed, parked until the data insists.