AlphaForge — System Handbook

The complete architecture & decision guide: what trades, why, which layers decide, what the system learns on its own, and what it is machine-forbidden from doing. Written for both a financial reader and an engineer.

Educational research — not financial advice Paper trading only (code-locked) Living document — see changelog Last full revision: 2026-11-05
Reading order. Part I is the investor/desk view: strategy, evidence, risk, and the bar for real capital. Part II is the machine: every layer, every decision gate, every self-improvement loop. When this document and config/base.yaml disagree, the config is the truth — this is the map, not the territory. Companions: docs/GUIDE.md, docs/SPEC.md, BEAST.md, README.md.

1What this system is

AlphaForge is a self-improving systematic trading research platform that runs unattended on a small VPS. It operates two markets:

US equities — fully automated (paper)
One decision per trading day at 14:55 ET on a dedicated Alpaca paper account. The accepted live strategy is a regime-sized S&P 500 core (SPY) with conditional 1.25× leverage behind a crash-entry veto — not a stock-picking book (see §2 for why).
Tunis Stock Exchange (BVMT) — advisory
Daily transparent factor ranking of all ~310 listings with per-stock verdicts. Deliberately not ML and deliberately not auto-traded: its own 14-year backtest says concentrated picking did not beat owning the liquid market (§16).

The defining design choice: improving the system is itself a subsystem. Signal weights adapt daily from measured accuracy; strategy changes must survive a purged 2015–2024 walk-forward and ten statistical gates; a nightly OODA loop reviews each day's real decisions and proposes one experiment per night; every lesson is written down permanently and machine-enforced where possible. Nothing changes on opinion.

2Where the edge is — a measured answer, not a belief

Months of gated experiments converge on one conclusion: allocation is the edge; stock selection has not been.

The decisive measurement (2026-07-01 ablation, 2015–2024, identical frictions for every arm):

ArmCAGRMax drawdown
SPY buy & hold+13.0%−33.7%
SPY × regime sizing (GMM), out-of-sample refit+16.1%−28.9%
Random 6-stock books × same sizing (30 seeds, mean)+9.4%−44.5%
Full ML stock-picking system (then-baseline)+6.6%−26.4%

The trained picker earned less than random selection under the same sizing. Every accepted experiment in the ledger is allocation-side (bear gate, continuous GMM sizing, concentration caps, pure benchmark core, reversal sizing, conditional leverage). Consequently the live book holds the market itself, sized by regime — and the individual-name machinery runs in advisory shadow mode, graded daily (§15), until a names-strategy beats the core through the full gate battery. Nothing is discarded; it is held to evidence.

The live baseline (exp0057, adopted 2026-07-10; de-levered 2026-08-05). 100% of sized gross in SPY. Conditional leverage (target 1.25× when drawdown-guard tier 0, GMM bull-probability ≥ 0.75, and 20-day realized vol in the calm half of its trailing 3-year distribution all cleared) was disabled 2026-08-05 after the DSR gate sweep (reports/dsr_sizing_sweep_2026-07-31.md) found no leverage candidate cleared the 0.95 deflated-Sharpe bar — 1.25× and 1.0× were statistically indistinguishable (DSR 0.637 vs 0.640) while 1.0× cut max drawdown by 1.4pp for 0.6pp less CAGR. Human sizing decision, not an evolve acceptance: gross is now capped at 1.0× unconditionally (config/baseline_overlay.json: risk.drawdown_guard.max_gross = 1.0, risk.conditional_leverage.enabled = false). A broker-side 15% disaster stop still sits under the core at all times.
Trend-following satellite activated live, 5% of capital (2026-08-08) — a deliberate override, not an evidence-backed acceptance. benchmark_core.weight 1.0 → 0.95, satellite_selection.mode: trend_following. This exact config's own 2015-2024 walk-forward (reports/ momentum_sleeve_sweep_2026-08-07.md) shows it underperforming the pure-core baseline by ~0.80pp/yr, broadly (not one bad year). Activated anyway on an explicit, informed user decision in service of a competitive goal, overriding config/LIVE_BASELINE_FREEZE.md (whose clock reset to 2026-11-08 as a result — see memory/HISTORY.md's 2026-08-08 governance entry for the full reasoning, never to be "rediscovered" as an oversight). §9 and §15 below describe the mechanism this activates for real; the shadow tracker (scripts/trend_sleeve_shadow.py) keeps running in parallel at zero capital as an independent forward comparison. benchmark_core.rebalance_band tightened 5%→2% in the same change — found via dry-run testing that a 5% satellite weight sits almost exactly on the default 5% band, silently stalling the initial core sell that funds the satellite's first buy. Rollout is two ticks, not one: the core sells down to 95% on the first tick after this deploys; the satellite deploys into that freed cash on the next tick once the account's cash balance reflects the sale.
Satellite split 50/50 trend / ML screen (2026-10-26) — exp0108, an evolve-ACCEPTED change adopted live. satellite_selection.mode trend_following → split (trend_following: 0.5, ml_score: 0.5, 5 positions, merged by rank). Still 5% of capital, gross still ≤ 1.0× (the 2026-07-28 no-leverage decision holds). Walk-forward 2015–2024: 10.95% CAGR, +0.59pp vs the then-baseline, max DD −17.4%. Purpose: a deliberate, small increase in stock-specific risk on the user's request, taken where the gates and the forward counterfactual ledger (screen top-N +$9.6k vs live −$32.4k over 2026-W29..W43) both point, instead of via leverage or a bigger trend sleeve (sat10pct backtest: −1.3pp). Resets the config/LIVE_BASELINE_FREEZE.md clock; see memory/HISTORY.md's 2026-10-26 governance entry.

3Track record & honesty rules

MeasurementResultContext
Walk-forward 2015–2024 (accepted baseline)+11.15%/yr, max DD −17.4%, Sharpe 0.91vs SPY +13.0%/yr at −33.7% DD — the strategy trails SPY on raw CAGR and wins on risk-adjusted terms (Sharpe 0.91 vs 0.78, half the drawdown)
2025 locked holdout (spent 2026-07-12)+4.4%, DD −15.9%Formally inside the promised range, but SPY did +18.2% — and the unlevered predecessor scored better (+5.4%) on the same year. The leverage knob's WF gain did not transfer; live paper is the arbiter (§15).
Live paper recordsince 2026-07-13judged in months, not days; graded daily by the counterfactual ledger
BVMT strategy backtest (14.4y)FAILtop-10 5.2%/yr at −40% DD vs equal-weight market 6.0% at −28% — published in the UI on purpose

The honesty rules that produce these numbers

  • Point-in-time everywhere: survivorship-free S&P membership (delisted names included), EDGAR fundamentals keyed by filing date, purged/embargoed training windows, frozen per-year models.
  • Live-only signals never enter backtest evidence (law L10) — no retroactive credit for data that has no honest history.
  • 2025 is a locked holdout the optimization loop cannot touch, and holdout results never feed hypothesis selection.
  • A rising evidence bar: the deflated-Sharpe threshold increases with every experiment tried — the loop gets harder to impress the longer it searches (multiple-testing correction).
  • Costs are modeled: 8bps one-way + square-root market impact + 5%/yr margin financing on any levered day.

4Risk architecture (defense in depth)

Broker reconciliation (V2 P0.2, 2026-11-05). Every tick reconciles the broker before planning and again after trading (execution/reconcile.py). It cancels any resting sell with no long position behind it, or larger than the shares held, and covers and pages on any negative position while shorting is disabled. Wrong-size stops are re-armed at the same price. Every order carries a source tag af-<date>-<leg>-<symbol>-<id>. Positions missing from the broker are journaled as vanished and paged, and broker rejects are paged. Stop-cascade circuit breaker (P0.3): now live, seeded from persisted stop_dates (engine parity). Five hard stops within the window pause new entries and rotation.

Position level — stop ladder Catastrophic −10% • regime hard stop (6% bull → 4% bear) • 15% trailing stop from the since-entry high-water mark • 2×ATR(14) volatility stop (advisory) • time exit. Broker-side OCO/bracket stops mean protection survives even if every process dies. The HWM is ratcheted on each bar's high by both the backtest engine and the live runner — live once ratcheted on its 14:55 snapshot instead, silently running a looser trailing stop than walk-forward validated (fixed 2026-08-20, pinned by tests/test_live_hwm_parity.py). An optional profit ratchet (risk.profit_ratchet, default OFF) tightens the trailing width as a position's peak gain clears tiers, so a windfall is not given back at full 15% width; keyed on the HWM gain so a stop never loosens as a trade deteriorates.
Portfolio level — tiered drawdown guard −5% warn → −8% halve new sizes → −12% cap gross at 50% → −15% liquidate + 20-day cooldown. The high-water mark is cycle-based (resets after cooldown) — an all-time HWM once trapped the predecessor in cash for five years (law L6). Stop-cascade breaker: 5 stops in 10 days pauses new entries.
Sizing level — regime throttle Continuous GMM bull-probability interpolates exposure 0.5×–1.75× (clamped to 1.0× unless the leverage veto clears); bear regime caps the book at ONE position; Monte-Carlo capital allocation (10,000 paths) can only ever tighten what regime sizing asks.
Infrastructure level — dead-man switches Watchdog kills all live positions if a tier goes silent; clock-drift beyond threshold halts new entries; a degenerate screen universe (< 20 names) aborts the tick rather than trade garbage (the 06-29 incident, law-grade rule); reconciler blocks entries while broker truth and local state disagree.

5The real-money bar

Hard-coded lock. The broker adapter refuses the real-money endpoint unless ALPHAFORGE_ALLOW_REAL_MONEY=1 is deliberately set — and the written bar must be met first: max drawdown ≤ −25%, worst year ≥ −15%, and 3–6 months of live paper results. Earliest possible green light: mid-October 2026. The switch is manual by design; no autonomous loop can reach it (the execution layer is a machine-protected path, §14).

6Architecture map

WhereRuns what
Contabo VPS (Ubuntu, 6 vCPU / 12GB, Tailscale-only access) THE executor: the daily tick (host venv + systemd), the full Docker stack (QuestDB, Redis, tiers T1–T4, watchdog, web UI), ~13 systemd timers, auto-deploy (polls origin/beast every 5 min), nightly OODA, daily backups. Single committer of live state (journal, counterfactuals).
Desktop (Windows)Research only: evolve experiments, walk-forwards, threshold sweeps, BVMT deep-history backfill (the data source 403s datacenter IPs — residential IP required).
GitHub beast branchSingle source of truth. Push = deployed within ~5 minutes. The deploy gate runs the test suite before swapping containers and rolls back on failure; a sha that failed a gate is remembered (artifacts/reports/.deploy_bad_sha) and never retried — one failure page, then quiet until a new push (2026-08-31; before this, one bad commit re-failed and re-paged every 5 minutes indefinitely).
INFRA    QuestDB (bitemporal store) · Redis Streams (signals/kill/heartbeats) · Watchdog
TIER 3   Daily Switch (nightly): universe → regime → fundamentals → sentiment → capital allocation
TIER 2   AI Brain (every 10 min): freshness gate → xreg → TimesFM quantiles → decision engine
TIER 1   Live Risk (continuous): P&L ticks · noise band · risk matrices · kill switch · reconciler
TIER 4   Research (nightly): attribution → scorecard → drift → hypotheses → shadow backtest
         → champion/challenger → promotion gate (human-approved only)
─────────────────────────────────────────────────────────────────────────────
CLASSIC  The validated daily strategy (host venv): screen → verdicts → orders at 14:55 ET

The two surfaces share repo, config, and credentials but run independently: the classic daily strategy is the validated track that trades; the Beast is the always-on research/monitoring stack whose intraday engine must still earn promotion (§12, §14).

7Data layer

Partial-bar fix (V2 P0.4, 2026-11-05). Today's bar is served to the 14:55 tick but never saved before 16:15 ET, and every refresh re-fetches the last 7 days, so a bar captured mid-session is overwritten by the real close. Previously a mid-session bar was frozen as that day's close on alternating days (error up to 2.6%).

SourceWhatCadence
Yahoo Finance (yfinance)Daily OHLCV, ~700 US names + SPY + factor ETFs, dividend-adjusted (auto_adjust=True) → parquet cachedaily at tick + on demand
SEC EDGAR XBRLFundamentals with true filing dates (point-in-time; dead companies included)on refresh
AlpacaAccount/positions/orders (paper), quotes, price fallback for next-day gradingcontinuous
Alpaca news stream + sweepsOwn 24/7 headline collector → news_log.jsonl (yfinance news is IP-blocked on the VPS)24/7
Options flow, insider filings, PEAD eventsPer-signal live-only fetchersat tick
sp500_pit.csvSurvivorship-free S&P membership history; Monday PIT refresh adds new index membersstatic + weekly
tunis-stockexchange.com / ilboursa.comBVMT reference + ~16y deep history per real equity (desktop-IP only, shipped by scp)daily / monthly
QuestDBThe Beast's bitemporal store: every row carries event_time + knowledge_time, so "what did we know when" is a query, not a guesssynced Mon–Fri 16:30 ET
Data hygiene is law-grade here. ~10% of cached names once carried >40% fake single-day spikes (L19) — every return analysis masks spikes first. A single NaN latest bar once silently collapsed the tradable universe to 2 names (L20) — hence quorum-based as-of dates, no forward-filled closes, a self-healing cache, and the <20-name circuit breaker. Rule: confident garbage is worse than a crash.

8The five thinking parts

#ModelTypeRoleRetrained?
1US scoring stack (live_current.pkl) LightGBM + XGBoost + CatBoost behind an out-of-fold meta-learner, isotonic-calibrated, blended over 5/10/20-day horizons (50/30/20) Scores every US stock 0→1 daily; drives ranking and verdicts Weekly (Sun 10:00 ET) on all history through the latest close
2TimesFM 2.5 (200M params)Google time-series foundation modelThe Beast's intraday engine: quantile price-path forecasts with uncertainty deciles (§12)No — but its decision thresholds recalibrate monthly from graded evidence
3FinBERTFinance-language model Headlines → per-stock sentiment from the system's own news logNo
4Signal overlay layer~9 hand-built signals Each nudges the final score with a self-learning weight (§13) Daily IC adaptation
5BVMT factor screenTransparent factor math Ranks ~310 Tunisian listings; momentum 40 / value 35 / technical 25 Factor mix re-validated monthly vs 14y history

Plus ~10 frozen wf_*.pkl walk-forward snapshots used only to measure historical performance honestly. They never trade.

9US daily decision pipeline — layer by layer

1 · Universe~47-name liquid US set (static list, min $10 price, min $50M ADV); a point-in-time top-200-by-ADV mode exists for honest backtests.
2 · FeaturesSet "v8plus": 26 features (20 base + 6 cross-sectional), one vectorized implementation for training AND live — train/serve skew is impossible by construction. Every feature carries a registry status (proven/candidate/banned) and an orthogonality category.
3 · LabelsCross-sectional top-30% forward-return quantile at a 5-day horizon (floor law L4: never below 0.30 — thinner labels starved early training years).
4 · ML stackPer horizon (5/10/20d): Optuna-tuned LGBM + XGB + CatBoost on purged time folds → out-of-fold logistic meta-learner → isotonic calibration. Horizon blend 50/30/20.
5 · Overlay signals final = 0.5 + 0.60·(ml−0.5) + Σ wᵢ·ICᵢ·(sᵢ−0.5) — sector RS 0.10, PEAD 0.06, rel-vol accumulation 0.05, news sentiment 0.05, insider 0.04, fundamentals 0.04, options flow 0.03. TimesFM is disabled in the daily blend (measured −9.87pp, exp0022). Each weight is multiplied by a rolling 60-day accuracy factor (§13).
6 · Regime sizing (never features — law L0) GMM bull-probability on SPY → continuous exposure 0.5×–1.75× with asymmetric smoothing (slow to upgrade, fast to downgrade). Discrete caps: bull 6 positions / sideways 4 / high-vol 3 / bear 1.
7 · Portfolio construction Entry floor score 0.55 · rotation edge 0.05 · 3–30 day holds · inverse-vol sizing · the non-ML 40% of the blend is deliberately mean-reversion (anti-correlated with the momentum-leaning ML — law L1).
8 · Risk stackStop ladder + drawdown guard + cost model (§4) applied identically in backtest and live — the engine is shared code.
9 · What actually trades (exp0057 overlay, de-levered 2026-08-05, satellite activated 2026-08-08, split 2026-10-26) benchmark_core.weight = 0.95 → 95% of sized gross goes to SPY (2% rebalance band, 15% backstop stop, gross capped 1.0× — conditional leverage disabled per the DSR gate sweep). The remaining 5% is a split satellite since 2026-10-26 (exp0108, portfolio/split_selection.py): half ranked by the ML/overlay blend, half by the trend_following selector (portfolio/trend_following.py): intermediate 12-1 momentum gated by an SMA50/SMA200 golden-cross, up to 5 positions, 60-day min hold — activated live on a deliberate override, not because it cleared the gates (it didn't; see the note above §2). Trim-to-fund since 2026-11-12 (portfolio.sat_trim_to_fund, portfolio/funding.py plan_trims, engine + live): when a new target can't be bought for lack of cash, held names above their slot (sleeve ÷ max positions) are sold back toward it, pro rata and only as far as the new names need; the kept shares stay stop-protected at the old stop price. Steps 1–7 still feed the advisory ML screen, Ticker Inspector, and the counterfactual ledger exactly as before; the satellite's actual live selection comes from the trend_following module instead, not that screen. Prior to 2026-08-08, "0 stock buys" was the accepted design; it no longer describes current behavior.

10How a daily verdict is made (Ticker Inspector)

For any ticker, the UI reconstructs the exact live-engine reasoning:

  1. Score vs floor — final blended score against the 0.55 entry floor, with the full layer attribution table (exactly which signal added or subtracted how much).
  2. Rank vs regime budget — is it inside today's top-N for the current regime tier?
  3. Rotation edge — an incumbent is only displaced if the challenger beats it by ≥ 0.05 (churn costs money).
  4. Stop ladder — the regime hard stop, catastrophic stop, trailing stop and time exit drawn on the chart.
  5. AI-brain second opinion — T2's independent TimesFM-based verdict (§12) plus a CONFIRMED/CONFLICT consensus banner between the two engines.

Verdict vocabulary: BUY / WATCH / HOLD / SELL / AVOID (US, advisory while the core trades) and STRONG BUY → SELL (BVMT, always advisory, always carrying the backtest caveat).

The same tab carries two further modes. Smart search: a Compass-style filter-query bar across the US screen and BVMT factor screen (webui/search.py). Monte Carlo (webui/montecarlo.py, /api/montecarlo/{ticker}): simulate up to 10,000 possible price futures for any US or BVMT name, resampled from that ticker's own cached daily history — models are i.i.d. bootstrap, block bootstrap (preserves volatility clustering), or GBM; controls cover horizon, path count, lookback, annualized drift/vol overrides, invested amount, and seed. Output is percentile fan bands, a terminal-return distribution, and tail stats (P(loss), VaR/CVaR 95, max drawdown). Data-error days (|move| > ~50%, the known spike/zero-row cache artifacts) are dropped and counted, and a sub-one-year sample triggers an explicit "drift estimate is noisy" warning. It is a visualization of the historical return distribution only — deliberately advisory, feeding no score, no verdict, and no live state.

11The Beast — four tiers on four clocks

PARKED 2026-11-04. All tiers (infra_wd, T1–T4) sit behind the compose profile beast; the deploy no longer starts them, and they were stopped by hand on the VPS. Reason: T1 shares the live Alpaca account, and its kill switch protects only SPY. Broker history shows 73 of 91 satellite exits between 08-24 and 10-15 were market sells sent outside the daily runner's window. Redis and QuestDB stay up, bound to 127.0.0.1. Do not re-enable without a separate broker account. The description below is kept for reference.

TierClockJobKey mechanics
T3 · Daily SwitchnightlySet tomorrow's contextTop-40 universe → GMM regime + confidence → PIT fundamental node → FinBERT overnight sentiment → Monte-Carlo capital allocation (10,000 paths, ruin tolerance 5% at the guard's halt threshold; can only tighten sizing) → writes daily_context.
T2 · AI Brainevery 10 minRe-decide every nameGate chain in §12. Output: one decision row per symbol per cycle — BUY/SELL published to T1; every HOLD recorded with its reason. Runs thread-capped (2 cores) so inference can never starve the box.
T1 · Live Risk~5s loopGuard live positionsConsumes T2 signals + kill messages. 10,000-path Monte-Carlo noise band (95%) separates noise from signal moves; fixed- precedence risk matrices (KILL > shock-absorber 15min/−3% > stagnation 30min/−2% > preservation 60min/−5% → KILL; short squeeze-shield 10min/+2%); order lifecycle with broker-side bracket stops; batch reconciler blocks new entries on any broker/local mismatch.
T4 · ResearchnightlyGrade & propose Attribution grades every matured decision (including HOLDs — "what would that decision have earned?"); benchmark scorecard (capture ratio, FN/FP miss labels — plumbing-failure HOLDs excluded so outages are never graded as strategy); drift monitor (PSI 0.20 alert, decision-mix SPC, calibration, alpha decay); miss clustering → hypotheses; PIT shadow backtest with leakage assertions; champion/challenger runs shadow-live and never executes; promotion needs a human.
Watchdog (the dead-man switch). Per-tier silence thresholds (T1 30s×3 · T2 600s×3 · T3/T4 1d×2). On silence: publish kill ALL (T1 flattens), Telegram alert, then restart the silent container (one restart per tier per 15 min; detection-only without the Docker socket). Weekend nuance: a kill while the market is closed queues market orders for the next open — by design "flatten ASAP", but it means a false weekend kill leaves a pending liquidation to catch (see §17, 2026-07-25 incident).

12T2 decision engine, gate by gate

The intraday engine is a challenger, not the validated strategy — its daily-blend cousin measured −9.87pp and was removed (exp0022). It must earn promotion through T4 with out-of-sample evidence. Its live record started 2026-07-17 after threshold recalibration.

Gate 0 · FreshnessBars staleness ≤ 120h and no gap > 4 trading days, else hold:stale/hold:incomplete — the engine refuses to infer on missing data.
Covariates (xreg)Live sentiment (own news log → FinBERT) · order-flow proxy (CVD + imbalance from daily bars — reduced fidelity until a real tick feed exists) · technical mix weighted by regime (trend weight 0.75 bull → 0.25 bear).
ForecastTimesFM 2.5 (200M), last 512 closes per name, batched, 120s timeout, model resident in memory — output is the full decile path q10…q90, not a point guess.
Gate 1 · Confidence confidence = 1 − (q10–q90 width / price) / 0.25, floor 0.45 — a forecast whose own uncertainty band is too wide for its price is not acted on.
Gate 2 · Cost|expected move| must exceed estimated transaction cost + 5bps margin — a signal that can't pay its own toll is a HOLD (hold:below_cost).
Gate 3 · Noise bandBUY needs q40 > 0; SELL needs q60 < 0 (band [0.4, 0.6], horizon 10 trading days). History: the original [0.2, 0.8] band at h=20 demanded ≥80% directional certainty — real forecasts essentially never assert that, and the engine made 1 BUY per 1,267 HOLDs. The 2026-07-18 sweep (36 combos × ~125 historical dates, identical forecasts re-graded) picked the current combo: 386 trades, 57.8% hit, best per-day return of all horizons.
Sizing hintmin(10% of equity, slot × confidence), regime sizing inherited from T3's context. Decisions land in QuestDB; only live cycles publish to T1.
Self-recalibration, gated. A monthly sweep (first Saturday) may move these thresholds only if: ≥80 forecast dates, winning combo has ≥200 trades, hit rate ≥52%, positive mean, and ≥25% total edge over the current combo. Changes apply via a config overlay that walk-forwards can never see.

13Self-learning — seven loops, each with a written gate

LoopWhenWhat may changeEvidence gate before ANY change
Signal IC adaptationdailyPer-signal weight multipliers (0–1.5×)60-day rolling Spearman IC, ≥200 obs; IC < −0.01 → ×0.5 decay; IC > 0.02 → ×1.25 recovery
Counterfactual gradingdaily, post-tickNothing — builds evidenceGrades actual vs no-veto vs unlevered-core vs top-N vs flat at +1/3/5/10/20d; never grades a partial close (§15)
Nightly OODAMon–Fri 21:00 ETThe live baseline (config), only through the full gauntletNominations need ≥10 graded days + effect-size floors; one evolve iteration/night: precheck → full walk-forward → gates G1–G10 → lesson recorded either way
OODA code patchsameSignal/feature/portfolio/model/analytics code + testsMachine-protected paths (§14); full test suite + isolated-worktree walk-forward before any commit
Weekly reportSat 09:00 ETNothing — reporting—
US model retrainSun 10:00 ETlive_current.pkl weightsNone needed — weights only, never strategy
T2 recalibration1st Sat monthlyT2 decision thresholds§12 note

The evolve loop (structural learning)

baseline → propose (backlog + OODA nominations + registry candidates + optional LLM)
        ↳ the LLM proposer is the ONLY replenishing source: once the static
          backlog is spent, an LLM outage means "pool exhausted" and the loop
          exits rc=0 having learned nothing. Model id: alphaforge/llm.py
          (one definition, GROQ_MODEL overrides) — see the 2026-08-21 changelog.
        → guardrails (static veto: banned features, regime-in-model, momentum saturation)
        → precheck (1-year WF, minutes — catches crashes & catastrophes)
        → full purged walk-forward 2015–2024
        → gatekeeper G1–G10 (two acceptance routes: CAGR or risk)
        → accepted: overlay merges into baseline, git commit, LIVE runner parity required
          rejected: overlay dropped, never applied
        → lesson appended to memory/LESSONS.md either way — feeds the next iteration

The recursion is deliberately conservative: the ledger dedups every overlay ever tried (the ledger is part of the migration payload — L24), rejected categories sort to the back of the queue, and the deflated-Sharpe bar only rises. The loop cannot edit its own gates.

14Gates, guardrails & laws

Gatekeeper — ten gates, two routes

GateRequirement
Route A (CAGR)G1 ΔCAGR ≥ +0.3pp · G2 worst-year MaxDD not worse by >2pp · G3 worst year not worse by >2pp · G4 median per-year Δ ≥ 0 (broad, not one lucky year) · G5 gains not concentrated in beta-pumped 2020/21
Route B (risk)Worst year improves ≥5pp AND full-path MaxDD improves ≥3pp AND CAGR give-up ≤1.5pp
Both routesG6 deflated Sharpe above a fixed bar, currently 0.60 (lowered from 0.95 on 2026-08-08 — the bar used to rise with trial count instead, a bug fixed 2026-07-27; now the DSR value is what falls as n_trials grows, checked against the fixed bar) · G7 run completed (no early stop) · G8 no year regresses >15pp into negative territory · G9 full-path MaxDD ratchet (≤2pp worsening) · G10 investable ceiling: MaxDD ≤ −35% forbidden, Calmar ≥ 0.5 (with improves-on-baseline escape)
G6 lowered 0.95 → 0.60 (2026-08-08). Data-driven, not a guess: every honestly n_trials-corrected DSR reading in this repo's ledger (15 experiments, n_trials 24-100) is 0.306-0.640 except one early outlier (0.937 at n_trials=48) — at current n_trials (~100), 0.95 had become close to unreachable for anything, including the live baseline itself (0.637-0.640). 0.60 sits just below the live baseline's own honest reading and above the noise cluster every genuinely-rejected experiment landed in. G1/G4/G9/G10 are unchanged. Because this makes an autonomous evolve accept meaningfully more likely, the loop's commit path (evolve/loop.py) is now freeze-aware: it will no longer write baseline_overlay.json/live_baseline.json while config/LIVE_BASELINE_FREEZE.md exists, regardless of what the gates decide — an accepted-but-frozen candidate stays in the ledger only, and a human must consciously lift the freeze to activate it (evolve adopt --version <ver> at the freeze-lift review). Since 2026-08-31 a frozen accept also leaves the ledger's baseline pointer untouched: the first frozen accept (exp0108, 2026-08-24) moved the pointer while the disk overlay stayed put, so the stale-baseline guard correctly saw a divergence and killed every subsequent nightly with rc=2 — the loop stayed bricked 08-25..28 until a human rebaselined. While frozen, every candidate is graded as an independent delta against the config the book actually trades.

Laws (machine-enforced where possible; each cost real money to learn)

#LawCost of learning it
L0No VIX/SPY/market features in the signal model — regime lives in sizing onlyAUC 0.73 illusion, destroyed P&L
L1The fallback blend stays mean-reversion (anti-correlated with the momentum-leaning ML)−5.1pp
L2Momentum is saturated — no new momentum features2015 −9.4%
L4Label quantile never below 0.30catastrophic 2015/16
L5Bear regime = 1 position maxsweep-validated
L6Drawdown HWM is cycle-based, resets after cooldown5 lost years of compounding
L7Stress-test every defensive change on recovery years (2019/2023)V19 regression
L8/G5Distrust gains concentrated in 2020/2021—
L10Live-only signals never enter backtest evidencemisattributed alpha
L17/L18High raw IC ≠ walk-forward gain; single-knob optima are two-sidedmultiple burned nights
L21Allocation is the measured edge; selection must beat the core to earn capitalthe ablation, §2
L24Gate state (ledger, IC state) is part of any migration payload; "the loop ran" ≠ "the loop learned"5 silent nights, §17
Machine-protected paths. The autonomous patcher can never modify: the gatekeeper, guardrails, the backtest engine (the measuring instrument), execution, the risk guard, the watchdog, alerting, deployment scripts, or itself. Enforced in code (patcher.PROTECTED_PREFIXES), unit-tested. Also standing: never docker compose down -v (the volumes hold the bitemporal store); 2025 stays locked; VPS is the single committer of live state.

15Reports & observability

Daily recap and outcome log: the V2 learning loop (P1, 2026-11-05)

After each tick, alphaforge/recap/daily.py writes artifacts/live/recap/daily/DATE.json (schema recap.v1) with:

  • health: tick ran, reconciled, orders not from the runner, negative positions, model age;
  • the book;
  • every decision with its reason;
  • today's broker orders;
  • P&L against SPY and against exposure-matched SPY;
  • SPY 5-day volatility forecasts (EWMA, rv20, HAR, Kronos when available, and the GMM as a stress label);
  • a copy of every ranked name's signal scores (scores/DATE.csv.gz).

A human .md recap, answering the five daily questions, goes to Telegram. Each night outcomes.py appends realized values for matured forecasts and decisions (outcome.v1, append-only). Every Friday grade_week() writes recap/weekly/YYYY-Www.json: forecaster accuracy, each signal's 5-day rank IC (including weight-0 "log-only" layers) and decision excess returns, computed from these files and prices only. A tick day with no recap, or a failed back-fill, pages.

Weekly report SPY window fixed (2026-11-05): SPY is now measured over the same dates as the live equity marks. The old window gave SPY 6 sessions to live's 5: W43 showed +1.15pp, the true gap was +0.11pp.

The counterfactual ledger — the "should we ever pick names?" mechanism

Every real decision day is graded next-close against the implementable alternatives, so vetoes and capital budgets become statistics instead of anecdotes:

ArmQuestion it answers
actualWhat did the tick really earn?
no_vetoWhat if the crash-entry veto never fired? (prices the veto)
core_unleveredWhat if we never levered at all? (prices the conditional-leverage decision itself — added 2026-07-25 after the holdout suggested leverage subtracts out-of-sample)
topN_eq / topN_sizedWhat if the screen's top names got the capital? (prices stock selection daily)
flat / best_nameCash floor / non-implementable hindsight ceiling

Each day's top-N is also re-graded at +1/3/5/10/20-day horizons as closes mature — a swing that fades by day 5 should not count as regret.

Daily counterfactual gap report (added 2026-08-07)

AlphaForge's version of the predecessor project's (ai_portfolio_experiment_v2) daily "Counterfactual Gap Analysis" recap, adapted to data this project actually has — computed from real journal/ledger/OODA records, never estimated. Runs automatically every trading night inside the nightly OODA pass (right after the counterfactual ledger grades whatever became newly gradable that night — a decision day needs the NEXT trading day's close before it can be graded, so this always has fresh numbers, never runs on its own ungradable same-day). Writes reports/daily-adjustment/ {date}.md: prior/EOD equity, day P&L vs the do-nothing floor, the real close-to-close counterfactual policies from the ledger above, fired reason codes, dev==prod deployment check, cumulative track record. The satellite-capital wording in this report is derived from benchmark_core.weight at render time, not hardcoded — it spent 2026-08-08..09-06 asserting the satellite was "0% live capital" after the sleeve had already gone live at 5%, which under-rated how much of the topN_eq ceiling was already fundable inside an approved budget. Self-healing — any date missed on a prior night gets its report the first night it matures. Unlike the general reports/ tree (mostly one-off analysis, gitignored), this specific subdirectory is git-tracked and synced nightly so it's readable on the desktop, same tier as journal/state/counterfactuals.

Trend-following satellite — forward shadow tracker (added 2026-08-07)

Diagnosis: the live book never holds a multi-quarter secular winner (e.g. PLTR) because (a) the blend's non-ML 40% is deliberately mean-reversion (§2's L1, a tested two-sided optimum, not reblendable) and (b) the two existing satellites (§9 below) are both short-horizon (3-30 day) setups, too short to ever hold a trend that plays out over months. alphaforge/portfolio/trend_following.py is a THIRD, orthogonal satellite mode: intermediate (12-1 month) momentum gated by a long-horizon SMA50/SMA200 golden-cross structure, meant to run at a much longer min_hold_days (~60) than the other two when active.

Superseded 2026-08-08 — this sleeve is now LIVE at 5% of NAV (benchmark_core.weight: 0.95, satellite_selection.mode: trend_following; see §2). The paragraph below describes the shadow-only period 2026-08-07..08 and is kept for the reasoning, not as a description of today's book. The trend_sleeve_shadow.py tracker still runs nightly as an independent check on the live sleeve.

Shadow-only at the time of writing, and not through the evolve loop. config/LIVE_BASELINE_FREEZE.md forbade editing baseline_overlay.json/live_baseline.json before 2026-10-13 (a 3-month clean-paper-record policy; the clock was consciously reset to 2026-11-08 by the 2026-08-08 activation itself), and the ledger's own strongest lesson (exp0081) says five prior satellite-mechanism attempts all failed the same gates regardless of selection philosophy, concluding the gates may reject the mechanism itself, not any one selector — so this was deliberately tested and tracked, not silently activated. scripts/momentum_sleeve_sweep.py answers the historical (2015-2024 walk-forward) question directly: does ANY nonzero satellite weight (swept 5/10/15/20%) clear the gates with this genuinely orthogonal signal — see reports/momentum_sleeve_sweep_*.md for the honest result. Separately, scripts/trend_sleeve_shadow.py tracks what the sleeve would do FORWARD, starting 2026-08-07 — real (not backtested) evidence, originally at zero live capital and now running alongside the live 5% sleeve as an independent shadow. Runs nightly via the OODA pass, writes artifacts/live/trend_sleeve_shadow.json, never touches the frozen config files, never submits an order.

Learning Lab: shadow stock-picking books (SHADOW, added 2026-11-05)

Owner request: keep learning to pick stocks "in the dark" with every layer, without touching the live 5% satellite. alphaforge/lab/shadow_books.py runs one paper book per layer and grades them all the same way. Nothing in it places orders, and nothing the live tick reads for sizing or selection depends on it.

  • Books: every logged layer (final_score, ml_score, each sig_*); two price ranks computed from closes up to the score day (reversal_5d, sector_rev_5d against the sector median of config/sector_map.csv); the Claude agents (llm_insider, claude_analyst); and three blends pre-registered before the first back-fill (blend_quant_rev, blend_ml_rev, blend_layers — mean percentile rank; a change is a new blend, never an edit).
  • Rules: daily top-5, equal weight, entry at the close of the first session after the score day, 21-session cohorts, 10 bps per side charged to the book; excess against the equal-weight universe of that day's ranked names (gross) and SPY.
  • Inputs: recap score files, falling back to the tick's screen JSON (last per day, ≥100 names); llm_signal.v1 files; cached closes with unadjusted splits back-adjusted and other >75% one-day drops (spin-offs) dropping the name from that cohort.
  • Statistics: Newey-West t (20 lags) of the mean cohort excess, hit rate, Spearman rank IC at 21 and 5 sessions with rolling timelines, sessions-to-significance (t grows with √n), and a G11-style paired test: daily (book − EW) returns on the same cohorts, 21-session block bootstrap, MDE at 80% power, Bonferroni over eligible books, from 126 paired sessions.
  • Status ladder: learning (<60 matured cohorts, or inconclusive) → promising (t ≥ 2, excess > 0) → candidate (promising and the G11-style test passes); dead at t ≤ −2. A candidate is a research lead for the evolve loop, never a live change.

Rebuilt from scratch every night in the OODA recap step (a failure sends a WARN). Output: artifacts/live/lab/shadow_books.json and shadow_books_timeline.json, host-local and gitignored.

Claude analyst (alphaforge/agents/analyst.py, budget purpose analyst, 1 call a night, config llm.agents.analyst): from the 20:20 ET agents timer, Claude works through the equity-research skill steps (earnings quality and surprise, catalysts, valuation against peers, thesis and risks, insider/ownership) on the top 15 names by final score. It returns 5 picks and up to 5 vetoes as strict JSON, validated against the candidate list, and logs them to recap/llm/analyst_DATE.json (llm_signal.v1, claude_analyst). A refused or failed call is logged as unavailable and sends a WARN.

Back-fill (85 full score days, 2026-06-12..11-04, closes to 11-04): sig_sector_rs +6.46% per 21 days vs EW, t 2.98, 57 of 60 cohorts; reversal_5d +3.17% (t 2.64) and sector_rev_5d +2.95% (t 2.56) are promising; final_score (the live blend) +0.82%, t 0.21; sig_relvol_accum −3.44%, t −2.81. None is eligible for the G11-style test yet (98 of 126 paired sessions). The Learning UI page (/api/lab/progress, RBAC page id still lab) opens with whether the loop is producing verdicts, then shows the leaderboard with long and short results and concentration per book, IC movers, the analyst's latest picks, forecaster calibration from the weekly scoreboard and the promotion pipeline, all labelled SHADOW.

Today: the landing page (added 2026-11-06)

GET /api/today (webui/today.py, page id today) is the default page and the first tab. It answers the owner's three morning questions in plain words, then shows the evidence: the US book with each stop and its distance (LIVE), yesterday's top-10 ranking with a warning when the scores are nearly tied, the Claude analyst's picks and vetoes and the insider agent's findings (SHADOW, newest successful run only), news on those names (36 h), index, sector and VIX moves with the coming economic events, and the BVMT top-10 next to the published list's forward record. Prices come from Alpaca daily bars (IEX feed) and VIX from Yahoo, fetched on demand, held 15 minutes and refreshed in the background; nothing is written to the data cache. Read-only.

The cockpit (added 2026-11-05)

Primary nav is Today, Cockpit, Book, Performance, Learning, Inbox and BVMT. Every other page sits under More; page ids, deep links such as #inspector and RBAC grants did not change. Four new page ids exist (cockpit, book, performance, inbox); admins get them automatically, other users need them granted on the Access page and otherwise land on their first allowed page.

GET /api/cockpit (webui/cockpit.py) answers the five daily questions from primary sources and writes nothing:

  • Alive? The latest recap's health, its age, the guard and the learning check. No recap means unknown, never a tick.
  • What did it do, and why? Recap decisions with reasons, anomalies with severity.
  • Beating SPY? From the journal: book against SPY times the share of equity invested at the previous tick, (equity - cash) / equity, so the satellite's market exposure is not credited as skill. Shows the Newey-West t-stat (5 lags), a 95% interval, sessions to t = +2 at today's effect, and the recap's backtest-fidelity percentile. The recap's own bench.exposure_matched and the fidelity band use core exposure only, which runs a few points lower, so the numbers differ slightly. Equity is marked at the tick and SPY at the close, which adds noise, not bias.
  • What did it learn? Ledger verdicts, top shadow books, calibration.
  • What needs my decision? Failed checks, recap anomalies, book vs broker differences, promotion candidates, accepted experiments not adopted, active overrides. It is derived on each load and has no acknowledge action, because the page changes nothing.

Learning status comes from outcomes. data.learning_liveness() reads the ledger: three business days without a valid verdict (accepted, rejected or no_effect) is "not learning". A row in nightly_runs.jsonl proves only that the timer fired. On Self-Learning, green now means a verified outcome; an artifact that merely exists shows a neutral "ran".

Book compares what the runner believes it holds (state.json, core share count, each satellite's last journalled buy) with the recap's broker snapshot, flags resting sells with no long behind them using the rule in execution/reconcile.py, and shows each new buy's stop-stress odds and suggested stop.

Every number carries its source: a solid LIVE tab, an outlined BACKTEST exp tab, or a dashed SHADOW tab. Backtest figures come from data.running_backtest(), which follows recap.fidelity_reference (now exp0108_20260824), not live_baseline.json, whose version names the model artifacts (exp0057_20260710). If that run's results are missing on a host, the UI says so and shows no other experiment's numbers.

Visual system: webui/static/styles.css holds light and dark tokens, following the OS setting unless the viewer picks one. State is encoded in form as well as color: a muted check for verified, an amber triangle for a warning, a red octagon for urgent, a dashed ring with hatching for unknown. Below 760px the rail becomes a bottom tab bar plus a More sheet that holds sign-out and theme.

Everything else

  • Weekly report card (Sat 09:00 ET, committed to git, rendered in the UI's Reports tab): live vs SPY, experiments run, commits, reason codes, out-of-universe movers. Both sides are measured Friday→Friday: the live equity series is seeded with the last journal mark strictly BEFORE the week, the same window _spy_ret() uses, so the two are comparable (fixed 2026-09-06 — see the changelog).
  • Telegram: tick done/skipped/crashed, watchdog kills and remediations, clock drift, boot notifications, self-learning outcomes.
  • Web UI (Tailscale-private + public HTTPS front door with RBAC): the cockpit above, with every older page under More.
  • Daily report (scripts/daily_report.py): the row once labelled "Cumulative alpha" is now "Return difference vs SPY (price, not exposure-adjusted)", because that is what it computes.
  • The one log that matters daily: artifacts/reports/live_run.log.

16BVMT (Tunis) pipeline

A transparent factor screen — deliberately not ML (too little honest point-in-time data to validate one):

  • Factors: momentum 0.40 (tanh-blended multi-window) · value 0.35 (1/PE, 1/PB, dividend yield percentiles) · technical 0.25 (price vs SMA20/60 + up/down-volume balance). Liquidity floor 2,000 TND 20-session dinar volume plus a trading-frequency filter (a print on ≥ half the trailing 60 sessions) that halved backtest drawdown by excluding rarely-traded names that cleared the dinar floor on one chunky print.
  • Verdicts: top-5 liquid with all factors ≥ 0.6 → STRONG BUY · top-10 liquid → BUY · illiquid top-decile → WATCH · bottom quartile → SELL. Every BUY carries the backtest caveat.
  • The honest card, shown in the UI: 14.4 years, monthly top-N vs equal-weight market, 60bps/side — still FAIL (5.2%/yr at −40% DD vs market 6.0% at −28%). Same lesson as the US: ownership beat picking. The ranking is research, not a money machine.
  • Real benchmark: the actual TUNINDEX (PX1, 2010→present) is cached for honest regime work; headline-tone sentiment was tested and found actively misleading (PR-framed "resilience" headlines scored positive while earnings fell 43%) — it is deliberately NOT wired in.

BVMT Lab: shadow stock picking (added 2026-11-05)

A page (bvmtlab, its own permission) that puts every BVMT layer side by side and grades each one forward. It is shadow and advisory only: there is no Tunisian broker, and its order ticket is text for the owner to place by hand.

  • Shadow books (alphaforge/lab/bvmt_lab.py, written to artifacts/live/lab/bvmt_books.json after each daily screen): momentum, value, technical, reversal, composite, liquid composite (identical to the published top-N on all 47 back-filled decisions) and claude_bvmt_analyst, top-5 and top-10, graded at 5/10/20 sessions. Rebuilt from the archived screens, so it is deterministic.
  • Rules: one decision per distinct market date, because frozen-feed days re-ranked the same data and would otherwise count as new evidence; partial feeds skipped; fill at the close of the first session after the screen; ±6.09% limit (limit-up entries unfilled, limit-down exits slip); price return only, dividends excluded on both sides; moves no daily limit allows are masked as price breaks, not graded.
  • Costs (an assumption, not a measurement): 0.60%/side at ≥ 50k TND/day, +0.25% at 10k–50k, +0.75% below 10k.
  • Status: learning / promising / candidate / dead at 10 sessions, on the less extreme of the Newey-West and independent-window t, because overlapping windows are not independent evidence. A book cannot reach promising or candidate with more than 50% of its slots in one sector, or if its excess disappears without its largest contributor or best winner (Concentration column).
  • Claude agents (nightly, inside the D8 budget; log-only, and a refusal is recorded and alerted): bvmt_docs extracts dated facts from official French disclosures, up to 3 calls; bvmt_analyst writes an equity-research review with 5 picks and vetoes, 1 call. Output goes to artifacts/live/recap/llm/bvmt_*_DATE.json. Neither agent can change a ranking.
  • First back-fill (47 decisions, 2026-07-24 to 11-03, only 4 independent 10-session windows, TUNINDEX not cached for this period): no book beats the equal-weight universe after costs. Best is technical top-10 at −1.08pp net per 10 sessions (+0.45pp gross); the published top-5 is worst at −3.61pp net (−1.98pp gross) with 59% of its slots in banks. Composite rank IC is positive (+0.06), so the ordering carries some information that the top of the list does not capture. This is too short to call a verdict.

17Operations & failure modes (learned the hard way)

Gated tick and release freeze (V2 P0.5, 2026-11-05). live_daily.sh no longer runs its own git pull, which used to bypass the deploy poller's pytest, smoke and rollback gates. The tick runs whatever the gated poller deployed. The poller skips deploys Mon–Fri 14:00–16:30 ET (ALPHAFORGE_DEPLOY_FORCE=1 overrides for emergencies).

Timer table (systemd, self-installing from git)

TimerWhenDoes
tickMon–Fri 14:55 ETTHE trading tick (Persistent — a missed fire catches up at next boot; the Alpaca-clock gate makes a late fire a safe no-op after close)
news24/7 + 08:00 ETheadline collection
bvmt-screenMon–Fri 15:00 TunisBVMT ranking
infradbMon–Fri 16:30 ETQuestDB bars/fundamentals sync
oodaMon–Fri 21:00 ETnightly self-review + one evolve iteration
weeklySat 09:00 ETreport card
t2-recal / bvmt-tune1st Sat / 1st Sun monthlythreshold / factor-mix recalibration
trainliveSun 10:00 ETUS model retrain
backupdaily 04:30 ETtimestamped tarball incl. hot QuestDB copy
deploy / unitsync / artifact-perms / diskguard5 min / 10 min / hourlyauto-deploy, unit self-update, root-chown guard, disk guard

Incident learnings now baked into the system

DateWhat happenedWhat now prevents it
2026-06-29One corrupt NaN bar collapsed the screen to 2 names — would have liquidated the book into themQuorum as-of dates, no forward-fill, self-healing cache, <20-name circuit breaker
2026-07-10Watchdog false-kill storm flattened live T1 every ~10 min on a T2 staleness misreadPer-tier silence thresholds
2026-07-14 / 07-23Missed ticks (trigger lapse; kernel-update reboot mid-window)Boot-notify Telegram alert; tick timer is now Persistent (catch-up fire, clock-gated safe)
2026-07-17T2 had never made a real decision on the VPS — four stacked faults each masked by the last (empty store → CPU timeout → staleness window → slow sentiment)Daily QuestDB sync, 120s forecast budget, 120h staleness window, own-news-first sentiment; the audit recipe is in reports/audit_decision_layers_2026-07-17.md
2026-07-25Unbounded PyTorch threads let T2 + QuestDB saturate all 6 cores; cycles slowed until the watchdog fired two false kills — each queuing a stray SPY liquidation for Monday's open (caught and cancelled; market was closed)T2 hard-capped (2 CPUs + thread env); scorecard excludes plumbing-failure HOLDs so outages are never graded as strategy decisions
2026-07-26The evolve ledger was never migrated to the VPS: dedup lost 73 experiments, the G6 trial-count reset, and a retired hypothesis ran 5 nights straight into timeouts — the self-improvement loop silently learned nothing for 5 daysLedger merged (additive); retired ideas removed from the code backlog; ledger.db added to the provisioning payload (L24); LLM SDKs restored ([llm] extra now in the install steps)
2026-08-21
(found 09-06)
Analysis-only symbols rot silently. QQQ.parquet sat 10 sessions stale, and its history only reached back to 2026-01-02 (170 rows) vs SPY's 4,193 — nothing on the live path ever refreshes it, because only cfg["data"]["benchmark"] (SPY) is fetched each tick. No trade was affected, but weekly_diagnosis.py and the web UI read QQQ and would have compared against a frozen series. Same bug class as 4a3a5c4 (BVMT cache never rebuilt)weekly_diagnosis.py now refreshes both benchmarks itself before reading them, so a gap here is a slow report rather than silent drift; QQQ re-backfilled to full 2010+ history
2026-08-31
(guard added 09-06)
The evolve loop refused to run for a full day (STALE BASELINE) — root-caused in 8e8f8cd to a frozen accept moving the ledger pointer. The drift was only discovered by the next night's refusing loop, costing ~11h of walk-forward recompute and one research nightSame check moved to commit time: scripts/githooks/pre-commit (installed by scripts/install_hooks.sh) blocks staging baseline_overlay.json unless it matches the ledger baseline, pointing at evolve run --rebaseline
Known open design tension. A watchdog kill while the market is closed queues market orders that execute at the next open. For a true weekend tier-death that is arguably correct ("flatten ASAP"); for a false one it is a pending unwanted liquidation. The 07-25 root cause is fixed, but the semantic question — should T1 refuse to place flatten orders while the market is closed and rely on broker-side stops + re-detection at the open? — is deliberately left as a human decision, because it changes dead-man-switch behavior.

Changelog — this document

This is a living document. Whenever functionality it covers is added or modified, the relevant section AND this changelog must be updated in the same change.

DateChange
2026-11-12§9: satellite trim-to-fund (portfolio.sat_trim_to_fund: true in baseline_overlay.json, user decision). 11-06..11-11 the satellite targeted SNDK/WDC daily but bought nothing: DD/MRNA/SMCI held the whole 5% sleeve and free cash was ~$1.2k against a ~$10.3k minimum order. Held names are now trimmed back toward their slot to fund new targets; same function in the engine and live; trim-funded buys may be 1% under the minimum order (5% sleeve ÷ 5 slots equals it exactly). Freeze clock reset. §13/§17: OODA nightly moved 17:30 → 21:00 ET, inside the Claude window; at 17:30 every evolve hypothesis call was refused and the fallback keys were dead, so nothing was tested 11-10..11-12.
2026-11-06Web UI: the app shell and /static/ files are served with Cache-Control: no-cache, so a browser revalidates them on every load and shows a new deploy at once. The fixed ?v=2 asset URLs had let browsers keep the old app.js after deploys.
2026-11-06§15: Today page (/api/today), now the landing page and first tab: the three morning answers (US book and stops, BVMT, market context) with their evidence. Read-only.
2026-11-06Live screen: the sector ETFs behind sector_rs (10% of the live blend) are now fetched fresh on every screen (sector_rs.refresh_etfs); an ETF still stale after the fetch is dropped, so its stocks get no sector score instead of a wrong one. They had been frozen at 2026-07-30, because the signal reads the cache without refreshing and the ETFs are not in the universe. Backtests are unchanged.
2026-11-06Claude agents: a refused run (cap reached, outage) no longer overwrites a successful run's results for the same day (agents.save_record). On 11-05 a second evening run replaced the 17:13 results with "unavailable"; they were restored from the nightly state commit.
2026-11-05Deleted the v2 demo app (alphaforge/backend, alphaforge/frontend, its containers); /v2 now redirects to the app. Web image build order fixed so a code change no longer reinstalls every dependency.
2026-11-05Routing: / now redirects to the app at /login/; this handbook moved to /handbook/. §13: the evolve loop gets five pre-registered V2 tests (guard tiers exempt, guard off for the core, 13% vol target, core only, re-entry cooldown), graded by G11 on the V2 engine after a rebaseline. Old-engine verdicts no longer block re-tests.
2026-11-05§16: the BVMT daily screen moves from 15:00 to 18:30 Tunis. At 15:00 the feed often still had the previous session's prices (5 of 20 recent screens, so the BVMT screen and ledger looked frozen).
2026-11-05§15: Web UI redesign, the cockpit. New default page answering the five daily questions (/api/cockpit, webui/cockpit.py), plus Book, Performance and Inbox; 19 flat tabs become six primary pages and a More group, with old page ids and deep links intact. Learning Lab is now Learning. New light and dark identity, phone layout with a bottom tab bar. Fixes from the audit: learning status now comes from ledger outcomes (the page showed green while the loop was dead); the backtest cards read the running config exp0108, not exp0057; the live benchmark on Self-Learning is since-inception and exposure-matched, not this week's raw return; "Cumulative alpha" in the daily report is renamed to what it is. Read only: no new mutating endpoint, no change to trading or sizing code.
2026-11-05§15: Learning Lab, SHADOW only: shadow stock-picking books for every layer, two price reversal ranks, three pre-registered blends and the Claude agents (alphaforge/lab/shadow_books.py, rebuilt nightly in the OODA recap step), plus the nightly US Claude analyst (agents/analyst.py, claude_analyst, budget purpose analyst). New UI page Learning Lab (lab). No order, sizing or live-config path reads any of it; the live 5% satellite is unchanged.
2026-11-05§16: BVMT Lab page, forward-graded BVMT shadow books with a concentration gate, and the Claude bvmt_docs / bvmt_analyst agents (log-only). alphaforge-llm-agents timeout raised to 100 min for the worst case of all agents in one unit. Phone-width shell layout added for all pages.
2026-11-05V2 D8 (investment committee, owner-delegated). Claude subscription budget (alphaforge/llm_budget.py, llm.budget in base.yaml):
  • at most 10 calls a night and 45 a week, only 20:00–23:30 ET (02:00–05:30 Tunis);
  • caps and models per job (sonnet by default, haiku for the recap, opus only for the weekly patcher);
  • a 5-hour cooldown after any usage-limit hit;
  • a refused call falls back to the non-AI path.
Trading, risk, money figures, gates and live config never depend on an LLM. New Claude research agents (alphaforge/agents/, hedge-project pattern: headless Claude Code with WebFetch/WebSearch on the subscription token). First agent: insider Form 4 purchases for the top-20 ranked names (owner's choice over a SEC User-Agent header), logged at weight 0 as llm_insider and graded forward. Runs from the new alphaforge-llm-agents timer, 20:20 ET Mon–Fri.
2026-11-05V2 P3, §14: new universal gate G11 (evolve/protocol.py), a paired significance test on the candidate and baseline equity curves:
  • monthly paired returns;
  • 3-month block-bootstrap p-value;
  • minimum detectable effect at 80% power;
  • Bonferroni over graded ideas in the same family.
A gain that doesn't clear the MDE and the corrected alpha is INCONCLUSIVE. G11 fails closed when a curve is missing. Replay: exp0108 (accepted, now live) would be inconclusive (+0.54pp vs MDE 1.61pp, p 0.20 vs 0.0025). Config: evolve.gates.protocol.
2026-11-05V2 P3, §15: Monte Carlo layers in the daily recap (recap/montecarlo.py, all log-only and graded):
  1. SPY 5-session VaR95/CVaR95 by block bootstrap. This reuses the method of the rejected exp0090 tail gate as a forecaster, graded on how often realized returns fall below VaR95 (target 5%).
  2. Backtest-fidelity band: live's realized active return (book minus exposure-matched SPY) as a percentile of the block-bootstrapped active returns of the reference walk-forward (recap.fidelity_reference, default exp0108). An alert fires below the 5th percentile. Real data 2026-11-04: 53rd percentile since 10-26, 26th since 07-13.
  3. Suggested stop: the tightest hard stop with 10-session hit odds of 20% or less (SMCI 19% vs the flat 6% used; KO 4%).
The Ticker Inspector Monte Carlo mode is unchanged.
2026-11-05V2 P3, §15. Stop stress test (recap/stop_stress.py): each new satellite buy in the daily recap logs its odds of touching the hard stop within 5/10 sessions, from a moving-block bootstrap of its own daily bars checked against the lows. A daily-horizon adaptation of the MIT "Time Machine" idea from the day-trader-skill pack. The outcome back-fill records the realized hits, and the Friday scoreboard reports calibration (Brier score). Diagnostic only; nothing trades on it. The Kronos SPY volatility forecast is logged in the recap from the existing shadow files.
2026-11-05V2 P2 parity decisions (user-approved):
  1. The backtest funds the satellite from cash with live's rank-order fill (portfolio.satellite_funding: cash; margin = legacy).
  2. The walk-forward carries the drawdown guard across years (risk.drawdown_guard.carry_across_years).
  3. The live regime uses the engine's causal 21-day refit.
Walk-forward results change, so the evolve baseline must be re-run before new verdicts count (memory/HISTORY.md 2026-11-05).
2026-11-05V2 P2, first slice. Benchmark-core sizing now has ONE definition (portfolio/core_sizing.py) shared by backtest/engine.py and execution/live_runner.py. Live preflight now refuses configs that enable backtest-only features (vol_target and the timesfm, kronos, vol_structure and mc_tail gates). tests/test_run_live_e2e.py drives the real run_live offline with a golden plan. Verified: 2018+2020 walk-forward byte-identical before and after.
2026-11-05V2 P0.2–P0.5 + P1 (blueprint artifact EWmhbVt3mX5psBHkjQYByh). §4: broker reconciler, order source tags, vanished-position and broker-reject pages, live stop-cascade circuit breaker. §7: partial-bar fix. §15: daily recap, outcome log and weekly grader; weekly SPY window fix. §17: tick no longer pulls, 14:00–16:30 ET release freeze. Live behaviour changes, so the freeze clock resets (memory/HISTORY.md 2026-11-05).
2026-11-04§11: the Beast is parked (compose profile beast, removed from vps_deploy.sh; containers stopped on the VPS, watchdog first). Redis and QuestDB ports now bind to 127.0.0.1. This applies the compose and deploy parts of the user-approved fd97a07 (09-30), which had never been merged into beast. V2 blueprint P0.1; freeze rationale in memory/HISTORY.md 2026-11-04.
2026-10-26§2/§9: live satellite switched trend_following → split 50/50 trend / ML screen (exp0108, evolve-accepted 2026-08-24). Still 5% of capital, gross ≤ 1.0×. User-requested small risk increase; freeze clock reset (memory/HISTORY.md 2026-10-26). config/baseline_overlay.json changed, models unchanged (exp0057_20260710). New read-only scripts/risk_ladder.py: forward return-vs-volatility slope across the counterfactual books from the weekly reports. Live buy sizing: new satellite names funded best-score-first at full size (_fill_in_rank_order) instead of proportionally scaled to free cash, which pushed every order under min_order_pct and left 3 of 5 slots and ~$21k idle (the engine scales by gross headroom, so backtest orders were ~full size — live had diverged). Split guard test narrowed to exp0108's exact accepted settings.
2026-09-06§15: the weekly report card graded itself on a different window than its own benchmark. _spy_ret() reached back to the prior Friday's close (correct), but the journal filter was d0 <= date <= d1, so the live series started at the week's own Monday tick — live_ret was Mon→Fri while spy_ret was Fri→Fri, silently discarding Monday's P&L every week. Three consecutive Mondays were losses, so the reported number flattered itself: W36 read +0.594% when the true Fri→Fri return was +0.232%, a 2.6× overstatement. The equity series is now seeded with the last non-dry-run journal mark strictly before d0, mirroring _spy_ret. W32–W36 were regenerated so the history is one methodology, not two; daily_report.py's "Prior close equity" already used this convention, so the two reports now agree.
2026-09-06§15: every daily report claimed the satellite was not live for a month after it went live. daily_report.py hardcoded "satellite is not live" / "0% live capital" while benchmark_core.weight had been 0.95 since 2026-08-08 — the sleeve demonstrably traded (DELL, MRNA, HPE, SNDK at ~$13k a name). It made the topN_eq ceiling read as unfundable when ~5% of NAV was already approved for it, and misled a weekly analysis on first read. The wording is now derived from the overlay at render time so it cannot go stale again; the matching stale comment in live_runner._baseline_version was corrected in the same pass. The §15 trend-following section, which still described the sleeve as "shadow-only, not live" and cited the pre-reset 2026-10-13 freeze date, was corrected too.
2026-09-06§17: stale-baseline drift is now caught at edit time, not by a refusing loop the next night. A tracked pre-commit hook (scripts/githooks/pre-commit → scripts/check_baseline_overlay_rebaseline.py, installed per-host by scripts/install_hooks.sh since .git/hooks/ is never version-controlled) refuses to stage config/baseline_overlay.json unless it matches this host's ledger baseline. Verified against the actual 2026-08-31 drift shape; the evolve accept path is unaffected because it updates the ledger before writing the overlay.
2026-09-06§17: analysis-only price symbols were rotting silently. QQQ.parquet was 10 sessions stale and missing 15 years of history (170 rows from 2026-01-02, vs SPY's 4,193 from 2010) — nothing on the live path refreshes anything but cfg["data"]["benchmark"]. No trade was affected; weekly_diagnosis.py and the UI would have silently compared against a frozen series. QQQ was re-backfilled and weekly_diagnosis.py now refreshes both benchmarks before reading them.
2026-08-31Infra: deploy gate failures no longer retry-loop. A gate failure's git reset --hard recreated the local≠origin diff, so the 5-min poller re-attempted the same bad commit forever — 2026-08-30/31 one commit with date-rotted tests (frozen today = 2026-08-21 fixtures in test_universe_coverage.py, fixed same day) produced ~70 identical failed deploys and Telegram pages overnight. vps_deploy.sh now records the failed sha in artifacts/reports/.deploy_bad_sha and skips it until origin/beast moves; the marker clears when both gates pass.
2026-08-31§9: frozen accepts no longer brick the loop. exp0108's gated accept (2026-08-24) moved the ledger baseline pointer while the freeze correctly kept baseline_overlay.json untouched; the 2026-08-22 stale-baseline guard then saw stored ≠ disk and refused every nightly run 08-25..28 (rc=2) — two safety features fighting. _commit() now records a frozen accept in the experiment row only: no live files, no pointer move; the yardstick stays the traded config and accepted-but-frozen candidates are independent deltas for the freeze-lift review. Regression tests in tests/test_evolve_baseline_sync.py. The stuck ledger was rebaselined against the live overlay the same day.
2026-08-21§9: the evolve loop was dead for three nights and every health signal read green — L24's "the loop ran ≠ the loop learned", new root cause. Groq retired llama-3.3-70b-versatile; the id was hardcoded in four independent call sites (evolve/hypotheses.py, ooda/patcher.py, ooda/decide.py, live/news_watch.py) so all four broke at once, and each swallows LLM errors by design (an outage must never block the nightly). Result: 2026-08-18/19/20 the OODA service finished in 41s/69s/55s instead of ~18min, logged hypothesis pool exhausted — nothing to test, exited rc=0, and wrote zero ledger rows — while systemd, the tick, the deploy and evolve_rc all still looked healthy. The loop's own staleness warning ("40.9 days since the last accepted experiment") fired nightly into a log nobody read. Fixed by centralising the id in alphaforge/llm.py (env GROQ_MODEL overrides, so the next retirement is a .env edit, not a redeploy) and moving to openai/gpt-oss-120b, verified reachable and JSON-clean on the project key (qwen/qwen3.6-27b rejected — emits <think> blocks that break the downstream JSON parsers). Pinned by tests/test_llm_model_config.py.
2026-08-21§9: liveness alarm for the evolve loop, closing the gap the entry below left open. The existing heartbeat measures days_since_last_accepted — but acceptance is rare by design (most candidates are correctly rejected), so that bar reads as normal noise: it sat above 40 days right through the outage and was tuned out. Added Ledger.business_days_since_last_experiment() — did the loop record an experiment at all, regardless of verdict — which should be true every weeknight, making its absence unambiguous. Fires at error level (vs the acceptance bar's warning) past ooda.evolve_dead_business_days (default 2). Business days because the timer is Mon–Fri: a calendar-day bar of 2 would page every Monday on a healthy loop. Deliberately outcome-based, not cause-based — the loop has now died twice from unrelated causes (an orphaned hypothesis re-queued into timeouts, then a retired proposer model); both violated this one invariant, and so will the third cause nobody has imagined yet. A best-effort _diagnose_dead_evolve() names the cause in the alert when the signature is recognisable, but never gates the alert on recognising it. Pinned by tests/test_evolve_liveness_alert.py, including the Monday false-positive case.
2026-08-20§4: live/backtest parity fix on the trailing stop's high-water mark, plus a default-OFF profit ratchet. backtest/engine.py ratchets Position.hwm on every bar's HIGH before check_exit; execution/live_runner.py ratcheted only on the once-daily 14:55 snapshot, so live never saw a session's range and ran a systematically looser trailing stop than the one walk-forward validated. Live now rebuilds the HWM from the cached daily highs since entry, which also self-heals a dropped tick (the old incremental max lost that high permanently). Surfaced by MRNA: bought 2026-08-12 by the mean-reversion blend at $64.06, +177% on 08-19 Phase 3 melanoma data, recorded hwm $163.17 against a $174.38 close, exited 08-20 by trailing stop at $134.78 (+110.4% realised, ~$7.5k below the peak). The parity bug did not change that exit — both HWMs trigger at $134.78 — but at $145 the stale HWM would have held and the correct one exits. Regression-pinned by tests/test_live_hwm_parity.py. Separately, risk.profit_ratchet (default OFF → bit-identical engine) tightens the trailing width by peak-gain tier (12%/10%/7% above +25%/+50%/+100%) and is registered as a priority-0 evolve hypothesis — not adopted live; the gatekeeper decides, and tiers are the swept knob (G6). Live baseline unchanged and still frozen to 2026-11-08.
2026-08-08§2/§9: trend-following satellite activated live, 5% of capital (benchmark_core.weight 1.0 → 0.95, satellite_selection.mode: trend_following, min_hold_days 3 → 60 while active). Deliberate override of the live-baseline freeze, not an evolve acceptance — this exact config's own 2015-2024 walk-forward showed it underperforming the pure-core baseline by ~0.80pp/yr (reports/momentum_sleeve_sweep_2026-08-07.md). Activated anyway per an explicit user decision; full reasoning in memory/HISTORY.md's 2026-08-08 governance entry, freeze clock reset to 2026-11-08. Wired execution/live_runner.py for the new mode (previously shadow-only). Found + fixed a real bug via dry-run testing before this went live: a 5% satellite weight sits almost exactly on the default 5% core rebalance band, silently preventing the core's initial sell from ever firing and starving the satellite of funding cash — tightened benchmark_core.rebalance_band 5% → 2%. Rollout is two ticks (core sells down first tick, satellite buys with the freed cash the next), not simultaneous.
2026-08-08§Gatekeeper: G6 deflated-Sharpe bar lowered 0.95 → 0.60 (config/base.yaml evolve.gates.min_deflated_sharpe), backed by the real post-correction DSR history (see the note in that section) — at current n_trials, 0.95 had become unreachable for anything including the live baseline itself. G1/G4/G9/G10 unchanged. Paired safety fix: evolve/loop.py's auto-commit-to-live-baseline path is now freeze-aware (checks for config/LIVE_BASELINE_FREEZE.md before writing baseline_overlay.json/live_baseline.json, same file the freeze protects) — added specifically because this relaxation makes an autonomous accept more likely while the freeze (through 2026-10-13) is in effect; verified with a direct unit test that the write is actually blocked.
2026-08-07§15: new trend-following satellite (alphaforge/portfolio/trend_following.py, wired into backtest/engine.py) — a third, orthogonal satellite selection philosophy (12-1 momentum gated by SMA50/SMA200 trend structure, long min_hold_days) built to test whether ANY satellite mechanism can clear the gates with a genuinely different signal type, per the ledger's own recommendation after 5 prior rejections. Shadow-only: respects config/LIVE_BASELINE_FREEZE.md (no live-config edits before 2026-10-13), never runs through the evolve loop. scripts/momentum_sleeve_sweep.py for the historical walk-forward answer, scripts/trend_sleeve_shadow.py (wired into nightly OODA) for zero-capital forward tracking starting today.
2026-08-07§15: new daily counterfactual gap report, automated — reports/daily-adjustment/{date}.md now generates every trading night inside the nightly OODA pass (previously existed as a script but was only ever run manually, 3 sparse reports). Self-healing; git-tracked exception carved out of the otherwise-ignored reports/ tree.
2026-08-05Nightly OODA/evolve timeout raised 9h → 14h (config/base.yaml: ooda.evolve_timeout_s, scripts/systemd/alphaforge-ooda.service: TimeoutStartSec 10h → 15h). Root cause: 3 of 5 recent nightly runs (07-30/07-31/08-03) were feature:* hypotheses that can't reuse pretrained models (model-affecting overlay), forcing a full 10-year WF retrain that ran to the old 9h wall every time and got killed with zero diagnostic output. Also fixed the diagnostic gap itself: evolve's subprocess output now streams to artifacts/ooda/evolve_logs/{date}.log on disk instead of an in-memory pipe, so a future timeout leaves a readable tail instead of nothing (alphaforge/ooda/nightly.py). Timer still fires 17:30 ET; >6h margin remains before the 14:55 ET tick.
2026-08-05§2/§9: live baseline exp0057 de-levered — conditional leverage (1.25× behind the crash-entry veto) disabled, gross capped at 1.0× unconditionally. Human sizing decision per the DSR gate sweep (plan item 4, reports/dsr_sizing_sweep_2026-07-31.md), which found no leverage config cleared the 0.95 DSR bar and 1.0×/1.25× were statistically indistinguishable on DSR while 1.0× cut max drawdown 1.4pp for 0.6pp less CAGR. config/baseline_overlay.json changed, model artifacts unchanged (still exp0057_20260710).
2026-07-26§10: Ticker Inspector gained a Monte Carlo mode — per-ticker forward-path simulator (bootstrap / block / GBM) for both markets with user-controllable horizon, paths, lookback, drift/vol overrides and seed; advisory-only, feeds nothing downstream.
2026-07-26Initial full revision. Incorporates the 07-25/26 incident fixes: T2 CPU cap, persistent tick timer + unitsync restart fix, scorecard plumbing-failure exclusion, core_unlevered counterfactual arm, feature-category sub-bucketing, evolve-ledger migration (L24), LLM SDK restoration.