Methodology
Decision Lifecycle
A structured methodology for moving a decision candidate from initial generation through evidence accumulation, qualification assessment, and deployment — or retirement.
Objective
Guide a decision candidate through the full lifecycle — from initial generation to deployment or retirement — ensuring that each stage transition is gated by explicit evidence criteria. The methodology produces a deployed decision with a complete provenance trail, or a retirement record with documented reasons.
Scope
Covers
- Candidate generation and structural pre-validation
- Historical validation via backtesting (in-sample and out-of-sample)
- Forward testing via paper trading simulation
- Qualification assessment across multiple evidence dimensions
- Promotion decision and handoff to live execution
- Post-deployment monitoring and lifecycle terminal states
- Retirement and library standby transitions
Does Not Cover
- Internal mechanics of any single stage (e.g., how backtests run, how ticks evaluate — those are separate methodologies)
- Capital sizing or portfolio construction decisions
- Market regime classification methodology
- Exchange connectivity, order routing, or settlement
- Parameter optimization (covered by Continuous Optimization methodology)
Preconditions
These inputs and assumptions must be satisfied before the methodology can produce valid results.
| Input | Requirement |
|---|---|
| Generation infrastructure | Template engine and/or AI generation pipeline capable of producing valid StrategySpec candidates |
| Validation engine | Backtesting engine with IS/OOS split, composite scoring, and fee-efficiency adjustment |
| Paper trading infrastructure | Tick-based simulation with isolated capital, position tracking, and exit attribution |
| Qualification criteria | Defined thresholds for each qualification dimension (duration, trade count, divergence, regime proof) |
| Market data | Historical OHLCV data for backtesting and live market data feed for paper trading |
| Regime catalog | Classified regime periods covering historical and forward observation windows |
Procedure
Candidate Generation
Generate strategy candidates using template engine (deterministic) and/or AI generation (LLM-based). Apply diversity controls: family quotas (no single family > 60% of batch), timeframe rotation, exit profile rotation, fuzzy fingerprint dedup, and regime-aware family filtering. Tag each candidate with origin (agent/ai) and generation context.
Decision Criteria
If candidate fails structural pre-validation (unsupported indicators, invalid parameter ranges, impossible entry conditions) → reject before consuming compute. Level+crossover inversion guard catches contradictory conditions.
Structural Pre-Validation
Validate each candidate against the canonical StrategySpec contract. Check indicator support (24 supported indicators), parameter ranges, risk limits, and structural validity. Apply learning-informed filters: fee-bleeder family/timeframe combos are hard-gated, M4 indicator guidance adjusts parameters, regime-aware direction resolution enforces long/short consistency.
Decision Criteria
If spec validation fails → terminate with 'invalid_spec'. If fee-bleeder family/timeframe combo → skip with audit log. No partial credit — validation is binary.
Historical Validation (Backtest)
Execute each validated candidate against historical data. In-sample (70%) measures strategy viability; out-of-sample (30%) tests generalization. Composite scoring: Sharpe ratio (40%), return (30%), max drawdown (30%), adjusted by a trade-count confidence multiplier (0.10-1.00). Fee-efficiency multiplier (0.70-1.00) adjusts for round-trip costs.
Decision Criteria
Zero trades → terminate with 'zero_trades'. Composite score below threshold → terminate with 'failed_backtest'. OOS degradation beyond coherence threshold → flag as 'failed_oos' but may still proceed to paper trading at reduced confidence.
Paper Trading Admission
Admit validated candidates to paper trading with isolated capital. Each bot runs independently — no shared pool. Enforce diversity constraints: 15 per asset, 20 per family, 25 per timeframe, 150 total bot cap. Configure tick cadence (1-minute) and data source (Hyperliquid candle data for venue consistency).
Decision Criteria
If diversity constraints violated → queue for admission when slot opens. If system bot capacity exhausted → reject with 'capacity_full'. Foundry candidates auto-admitted (top 3 current-regime, requires 10%+ backtest return, 8+ trades, composite >= 50, not failed_oos).
Forward Performance Observation
Monitor paper trading over minimum 7-day observation period. The 6-hourly evaluation batch computes divergence metrics, behavioral signatures, regime-conditional performance, fee-bleeder detection, and M6 anomaly scoring. Do not intervene — observe only. Exit attribution captures MAE/MFE, drift, and mechanism for every closed position.
Decision Criteria
If paper ROI diverges from backtest beyond threshold (cohort p < 0.05) → flag for divergence review. If fee-bleeder pattern detected (rolling 20-trade avg < 0.3%) → advisory alert. If no trades after 14 days → not a failure, may be regime-appropriate.
Qualification Assessment
Apply multi-dimensional qualification criteria. All dimensions must pass (conjunctive — no weighted average). Dimensions: (a) minimum paper duration, (b) minimum trade count, (c) backtest-paper divergence within tolerance, (d) regime proof (positive performance in at least one classified regime), (e) no active disqualifying conditions. Re-assessed every 6 hours.
Decision Criteria
All dimensions pass → 'qualified' or 'conditionally_qualified'. Any dimension fails → remains 'unqualified' with specific failure reason. Evidence may accumulate over subsequent cycles.
Promotion Decision
User initiates promotion to live trading. System performs pre-deployment checks: exchange connection verified, fee status not suspended, capital allocation within limits, region compatibility (product_mode vs instrument_type). Qualified Scout system creates a system-owned paper reference account for multi-user parity. Live account created with deposited capital.
Decision Criteria
If exchange connection missing → block promotion. If fee status suspended → block promotion. If region incompatible (e.g., US user + leveraged perp) → block promotion. If qualified scout already exists → link to existing scout.
Post-Deployment Monitoring and Terminal States
Live strategy runs against real market data with real capital. Regime transition response adjusts trailing stops on toxic transitions. User may pause, resume, adjust capital (top-up/reduce), or kill. Kill switch closes exchange positions and revokes wallet. Outcomes feed back into learning layer via exit attribution and outcome aggregation.
Decision Criteria
If user kills → terminal state. If regime transition detected and strategy is vulnerable → tighten trailing stop (survivability-based multiplier). If drawdown exceeds circuit breaker → alert. Lifecycle never returns to paper once promoted.
Expected Outputs
Deployed Strategy
A live-trading strategy with complete provenance: generation context, backtest results, paper performance, qualification evidence, and deployment parameters
Provenance Trail
End-to-end audit trail from candidate generation through each lifecycle stage, including rejection reasons for candidates that did not advance
Retirement Record
For candidates that did not qualify: specific failure reasons, stage at which lifecycle terminated, and whether the strategy was retired or shelved to library standby
Learning Feedback
Outcome data that feeds back into the generation layer: which families, timeframes, parameters, and regimes produced qualified strategies
Interpretation Guide
How to read the outputs this methodology produces.
draft
Candidate generated, not yet validated
Action: Proceed to backtest validation
validated
Backtest completed with passing composite score
Action: Approve for paper trading
approved
Admitted to paper trading, forward observation in progress
Action: Wait for minimum observation period
active
Paper trading with live market data
Action: Monitor via 6h evaluation batch
qualified
All qualification dimensions pass
Action: Eligible for user-initiated promotion to live
promoted
Live trading with real capital
Action: Post-deployment monitoring, regime transition response
paused
Temporarily halted (user action or regime gate)
Action: Resume when conditions change or user re-enables
killed
Terminal state — positions closed, wallet revoked
Action: No further action. Outcomes feed learning layer.
library_standby
Shelved for non-current regime. Has demonstrated edge in at least one regime.
Action: Auto-activated when favorable regime arrives
retired
Terminal state — insufficient evidence of edge after adequate observation
Action: No further action. Retirement reason feeds generation filters.
Worked Example
Scenario
A momentum strategy on ETH/USDT with 4h timeframe enters the lifecycle during a trending_up regime.
Template engine generates candidate with RSI + MACD entry, ATR trailing exit profile (v2), tagged origin='agent'
→ Valid StrategySpec produced, family=momentum, direction=long
Pre-validation passes: supported indicators, valid parameters, not a fee-bleeder combo (momentum x 4h)
→ Spec validated, admitted to backtest queue
IS backtest: 12 trades, composite score 67.3. OOS: 4 trades, composite 58.1 (within 15% of IS)
→ Validation passes. Robustness label: 'moderate'
Admitted to paper trading with $1,000 isolated capital. Foundry diversity check: 8/15 ETH slots used
→ Paper account created, first tick evaluates at next 1m candle
After 18 days: 6 paper trades, 3.2% ROI. Exit attribution: 2 trailing stops, 2 take profits, 1 stop loss, 1 time decay
→ Divergence within tolerance (p=0.34). No fee-bleeder flag.
Qualification assessment: duration 18d > 14d min, trades 6 > 3 min, divergence p=0.34 > 0.05, regime proof: trending_up 4 trades +2.8% avg
→ Status: qualified
User promotes with $500 capital. Exchange connection verified, fee status active, US spot mode with 1x leverage
→ Live account created. Scout paper account linked. Capital flow event recorded.
Live trading begins. After 30 days: 4 trades, +$18.50 realized. Regime transitions to ranging — trailing tightened via survivability data.
→ Strategy active with full provenance trail. Outcomes feed weekly learning summary.
Outcome
Strategy completes full lifecycle from generation to live deployment in 49 days. Provenance trail links every decision to specific evidence: 12 IS trades, 4 OOS trades, 6 paper trades, 4 live trades. Learning feedback: momentum x 4h x ETH qualifies in trending_up.
Failure Conditions
How to recognize when the methodology is not producing valid results.
Zero trades in backtest
Entry conditions too restrictive for the historical data window
Reject at step 3. Do not proceed to paper trading — will not generate trades.
IS/OOS divergence exceeds 40%
Strategy may be overfit to in-sample period
Flag as failed_oos. May still paper trade at reduced confidence, but unlikely to qualify.
Fee-bleeder family/timeframe combo
Historical data shows >40% of strategies in this combo are fee bleeders
Hard-gate at step 2. Candidate skipped with audit log.
Regime mismatch at promotion
Strategy qualified in a regime that is no longer current
Strategy shelved to library_standby with favorable_regimes. Activated when regime returns.
Excessive paper-backtest divergence
Forward conditions materially differ from historical assumptions
Remains unqualified. May resolve with more data or may indicate structural issue.
No trades after 14+ days paper
Strategy inactive in current market conditions
Not necessarily a failure — may be regime-appropriate. Continue observation. May qualify via future regime.
ROI < -5% after 14+ days
Strategy demonstrates consistent negative performance
Eligible for retirement. Foundry sweep retires if no regime edge exists.
Quality Checks
| Check | Passes When | Fails When |
|---|---|---|
| Each stage transition requires explicit evidence gate | No candidate advances without meeting the stage's decision criteria | Candidates skip stages or advance on insufficient evidence |
| Lifecycle status accurately reflects current stage | Status transitions are atomic and auditable via strategy_events | Status is stale, backdated, or inconsistent with actual stage |
| Retirement preserves learning value | Retired candidates have documented failure reasons that feed generation filters | Candidates disappear without contributing to institutional knowledge |
| Library standby activates on correct regime | Shelved strategies resume only when their favorable_regimes match current regime | Strategies resume in regimes where they have no evidence of edge |
| Provenance trail is complete | Every deployed strategy can be traced back to generation context, backtest results, paper performance, and qualification evidence | Gaps in the audit trail — missing backtest results, unlinked paper accounts, or absent regime proof |
Handoff
Where the outputs of this methodology go next in the protocol chain.
Validation Engine
Paper Trading Infrastructure
Qualification Pipeline
Live Execution Engine
Learning Layer
Termination State
Produces
Either a live-deployed strategy with complete provenance trail, or a retired/shelved strategy with documented reasons. Both outcomes contribute to institutional learning.
Confidence
Confidence increases with lifecycle depth. A strategy that reaches qualification has survived 6 evidence gates. A strategy that reaches live deployment has survived 7. Minimum viable lifecycle duration is ~11 days (7 days paper minimum + validation time).
Next Step
Deployed strategies enter post-deployment monitoring. Retired strategies contribute to generation filters. Shelved strategies await regime activation. All outcomes feed the Evidence Loops methodology.
Replication Notes
What another team would need to reproduce this methodology.
- Generation infrastructure must support both deterministic (template) and stochastic (AI) candidate production with diversity controls
- Validation engine must support IS/OOS split with configurable ratio and composite scoring
- Paper trading requires tick-based simulation at 1-minute cadence with venue-consistent data
- Qualification assessment must be conjunctive (all dimensions pass) and re-evaluated periodically
- Library standby requires a regime catalog and automated activation logic
- The lifecycle is strictly forward-progressing — a promoted strategy never returns to paper trading
- Each stage writes immutable events (strategy_events) for full auditability
Knowledge Lineage
Introduced In
Expanded In
Modeled By
Investigated In
Related Research
Frameworks
See this methodology implemented in Orqis