Conversational quant backtesting agent system

ORACLE LOOP

A living Playground workbench where David can describe trading ideas in chat, then Hermes updates deterministic strategy DSL, cached-data manifests, simulation outputs, critiques, and UI visualization artifacts.

The LLM is allowed to be wrong. The execution system is not.

Current runAlgo + dataStrategy DSLCompiler guardrails

Architecture

Messy language stops at a strict compiler boundary.

01

Natural language

LLM surface

Ambiguity is allowed but must be surfaced before execution.Hallucinated indicators, missing exit, contradictory sizing.
02

Strategy DSL

Compiler boundary

Strict JSON, normalized defaults, schema-complete fields.Invalid JSON is repaired or rejected before data touches the kernel.
03

Cached data layer

Rate-limit governor

Never fetch the same symbol/timeframe twice; prefer disk cache over APIs.Cached-only mode if free providers throttle or return partial data.
04

Deterministic kernel

Execution core

No LLM, no network, no randomness unless explicitly seeded.Reject lookahead, centered windows, undefined indicators, impossible fills.
05

Report + critique

Human loop

Metrics and curves are computed; interpretation is isolated commentary.Overfit warnings, regime sensitivity, and parameter fragility are shown.

Algorithm

How a chat idea becomes a reproducible backtest.

01 · Compile

Turn chat into a falsifiable contract

The agent converts the idea into a strict Strategy DSL: assets, timeframe, indicator windows, entry rules, exit rules, sizing, and fees. Missing or ambiguous fields are completed deterministically or rejected before execution.

02 · Validate

Reject impossible or leaky rules

The compiler checks indicator vocabulary, rolling-window validity, lookahead risk, fill timing, exit coverage, and contradictory logic. This is where hallucinated indicators die.

03 · Fetch once

Resolve data through cache manifests

Every symbol/timeframe request checks memory, disk cache, and immutable dataset chunks before any external fetch. Overlapping requests coalesce into one provider call.

04 · Simulate

Run a deterministic vector engine

Signals are shifted to avoid future leakage, fills apply fees/slippage assumptions, position sizing follows the DSL, and the kernel computes trades, returns, exposure, turnover, drawdown, and equity.

05 · Critique

Attack the belief, not the spreadsheet

The report tests whether the result survives regime splits, parameter sweeps, missing-data flags, and overfit warnings. LLM commentary can interpret results but never computes them.

Seeded demo artifact

Buy SPY when RSI < 30, hold 5 trading days.

Data
SPY · 1d
Cache
cached_only_demo
Hash
spy-1d-demo-v0-e8b7c2

Overlay equity curve

7 strategies
2023-012025-07

Trade distribution

<-5%
2
-5–0%
11
0–2%
8
2–5%
7
>5%
3

Robustness critique

Regime sensitivity

Mean reversion works best in grinding bull tape; stress-test 2008, 2020, and inflation-shock windows before trusting it.

Parameter fragility

RSI 25/30/35 and hold 3/5/8 days should be swept; a single threshold is a belief, not evidence.

Data caution

Demo run uses a seeded cached artifact. Production runs should pin provider, hash, adjustment policy, and missing-bar count.

Strict output contract

Strategy DSL is the handshake between chat and execution.

Future chat updates should edit the data artifact first, then let this route render the compiled hypothesis, data manifest, deterministic metrics, and critique.

{
  "strategy_name": "rsi_mean_reversion",
  "assets": [
    "SPY"
  ],
  "timeframe": "1d",
  "entry_rules": [
    {
      "indicator": "rsi",
      "window": 14,
      "operator": "<",
      "value": 30
    }
  ],
  "exit_rules": [
    {
      "type": "time_hold",
      "days": 5
    }
  ],
  "position_sizing": "all_in",
  "fees": 0.001
}

Compiler guardrails

The execution system is allowed to reject the LLM.

  • No lookahead: signals can only use bars available before the simulated fill.
  • No undefined indicators: RSI, SMA, EMA, breakout, volatility, and time-hold are the initial allowed vocabulary.
  • No impossible exits: every strategy must include a time, signal, stop, or target exit rule.
  • No silent data holes: partial histories mark the run as biased instead of backfilling fantasy prices.
  • No repeated fetches: data requests resolve through symbol/timeframe/source/hash manifests.

Cache hierarchy

L0Session memory

Coalesce repeated in-chat requests.

L1Local parquet cache

Reusable symbol/timeframe history.

L2Compressed dataset store

Immutable chunk hashes for reproducibility.

L3External API

Yahoo/Stooq/FRED fallback fetch only for missing segments.