Publication
Qualified Execution
The principle that execution should be earned through evidence rather than assumed through generation.
Paper
WL-QE-001
Version
0.1
Status
Living Document
Last Updated
July 2026
Discipline
Systems Engineering
Authors
Warren Labs Research
Implementation
Orqis
Citation
WL-2026-QE-01
Research Status
Key Contributions
This paper introduces:
- Qualification as a distinct architectural layer between validation and execution
- 14-criteria gate with fail-closed promotion
- Staged qualification with regime-conditional eligibility
- Trade-level attribution proving qualification separates outcomes (p < 0.0001)
- False-negative analysis quantifying the cost of conservative gates
- Evidence that qualification effectiveness holds across all 6 market regimes
Abstract
Most decision systems treat execution as the default consequence of generation. When a strategy is produced and passes basic validation, the implicit assumption is that it is ready for deployment. This paper argues that this assumption is architecturally dangerous — particularly in domains where generation is abundant and the cost of unqualified execution is high. We introduce Qualified Execution as an engineering principle: the requirement that every candidate for deployment must earn the right to execute through structured evidence accumulation, multi-criteria qualification, and regime-conditional proof. We describe a production implementation with 14 concrete qualification criteria, a fail-closed promotion gate, and staged qualification states. Trade-level attribution from 2,824 production trades demonstrates that qualification separates outcomes with statistical significance (p < 0.0001), with effectiveness holding across all six market regimes.
01
The Default Assumption
In most organizations that rely on strategic decision-making, execution follows generation with minimal intermediary infrastructure. A strategy is created, perhaps validated against historical data, and then deployed. The gap between “this idea passed a test” and “this idea deserves real resources” is either nonexistent or managed through informal human judgment.
This default assumption — that generation implies readiness for execution — is inherited from an era when generation was expensive. If producing a credible candidate required months of analysis and years of domain expertise, the generation process itself served as an implicit quality filter. The difficulty of producing an idea was a reasonable proxy for the idea’s readiness.
That proxy has collapsed. AI-assisted generation can now produce hundreds of plausible candidates in minutes. The cost of generation no longer correlates with the quality or deployment-readiness of the output. Yet most systems continue to treat generation as implicit permission to execute — a structural vulnerability that becomes more dangerous as generation scales.
Observation
The cost of generating a strategy candidate in Orqis is approximately $0.05. The minimum cost of qualifying it — seven days of paper trading with at least five closed trades — is orders of magnitude higher. When generation costs $0.05 and qualification costs seven days, the system must be designed around the assumption that most generated candidates do not deserve execution.
Observation
Most systems treat execution as the default consequence of generation.
When a strategy is generated and passes basic validation, the implicit assumption in most systems is that it is ready for deployment. Qualification — structured evidence gathering between validation and execution — is either absent, informal, or conflated with the validation step itself.
Observed internally
Counterargument
Doesn't qualification slow down deployment?
Yes — intentionally. The 7-day minimum paper trading requirement (G1) creates a deliberate information acquisition cost. But the cost of unqualified deployment is higher: never-qualified strategies average −8.7% ROI. The qualification delay is not friction — it is the minimum observation period required to distinguish robust strategies from lucky ones.
“Generation creates possibilities. Qualification creates permission.”
Core Thesis
Execution must be earned.
No candidate should move from exploration to real-world action without first passing through structured evidence gathering, multi-criteria qualification, and regime-conditional proof. The infrastructure that enforces this principle is what separates Decision Infrastructure from conventional decision systems.
02
The 14 Criteria
Qualification is implemented as 14 concrete, measurable criteria — 10 global and 4 regime-specific. Each criterion produces a machine-readable result with required and actual values. The evaluator is a pure function: it takes assembled data and returns a verdict. No database reads, no side effects, fully deterministic.
The criteria are deliberately heterogeneous. Some measure outcomes (win rate, ROI), but as binary thresholds rather than continuous scores — the system asks “did this strategy demonstrate positive returns?” not “how positive were the returns?” Others measure process quality: observation duration, evidence sufficiency, behavioral stability, fee viability. This heterogeneity prevents any single dimension from dominating qualification.
Verified Evidence — Production
14-criteria qualification gate
Every strategy must pass 10 global criteria and 4 regime-specific criteria before reaching live deployment. The gate is fail-closed: missing, stale, or negative qualification blocks promotion regardless of other signals.
- G1: Paper trading ≥ 7 days — minimum observation period
- G2: Closed trades ≥ 5 — statistical minimum
- G3: Win rate ≥ 45% — profitability threshold
- G4: ROI > 0% — must be net positive
- G5: Max drawdown ≤ 30% — risk ceiling
- G6: No HIGH severity annotations — no active degradation
- G7: Intelligence score ≥ 40 — multi-dimensional quality floor
- G8: Avg trade return ≥ 0.3% — fee viability
- G9-G10: EAS score minimums — economic viability for paper and live
- R1-R4: Per-regime proof — ≥ 3 trades, positive PnL, ≥ 40% win rate in current regime
✓ Production observed
Engineering Note
10 of 14 criteria are process-based (observation time, trade count, intelligence score, annotation status, fee viability, EAS minimums). Only 3 are outcome-based (win rate, ROI, drawdown) — and these are binary thresholds, not continuous penalties. The system evaluates readiness, not performance.
03
Fail-Closed Architecture
The most important architectural decision in qualified execution is the default behavior when evidence is uncertain. In most systems, uncertain evidence is treated as permission — if the system cannot determine that something should be blocked, it allows it. Qualified Execution inverts this: if the system cannot confirm that a candidate has earned execution, it blocks it.
This fail-closed principle runs through every layer of the architecture. The promotion gate blocks when qualification data is missing. The execution engine pauses when regime proof is absent. The lifecycle manager auto-pauses when degradation signals accumulate. Each layer independently enforces the same principle: absence of evidence is not permission to proceed.
Verified Evidence — Production
Fail-closed promotion gate
The promote-to-live route enforces a 4-step sequential gate. If any step fails, promotion is blocked — real money deployment never proceeds on uncertain evidence.
- Step 1: Qualification row must exist — block if no qualification data
- Step 2: Staleness check — block if qualification computed > 12 hours ago (sync recompute attempted)
- Step 3: Live eligibility boolean — block with unmet criteria list if not eligible
- Step 4: Pass → proceed to compatibility, fee, and balance gates
✓ Production observed
Verified Evidence — Production
21 gates across 5 layers
Qualification is not a single checkpoint — it is a defense-in-depth architecture with 21 distinct gates organized across 5 system layers.
- Layer 1 — Qualification (G1-R4): MCP server, 6h batch computation
- Layer 2 — Promote-to-Live (P1-P12): Web API, checked at promotion time
- Layer 3 — Execution Engine (E1-E5): Per-tick decision gates including regime gate
- Layer 4 — Lifecycle Management (L1-L3): Auto-pause/resume based on annotations
- Layer 5 — Allocation Intelligence: Advisory ranking (never blocks execution)
✓ Production observed
Engineering Implication
Execution engine should never read allocation intelligence. It should only enforce hard safety gates. Advisory systems inform priority. Safety systems enforce permission.
“Absence of evidence is not permission to proceed.”
04
Staged Qualification
Qualification is not a single gate that a strategy either passes or fails. It is a staged progression through seven distinct states, reflecting the continuous accumulation and potential degradation of evidence over time.
A strategy begins as unproven. As evidence accumulates through paper trading, it may achieve global qualification but remain unproven in the current market regime. When regime-specific evidence is sufficient, it reaches conditional qualification — the state that enables live deployment. If performance degrades, qualification is revoked and the strategy enters a degraded state, where it continues paper trading to potentially recover.
This staging reflects a fundamental insight: qualification is not permanent. The same strategy may be qualified in ranging markets but unproven in trending markets. It may be qualified today and degraded next week. The system continuously re-evaluates — qualification is earned and maintained, not achieved and assumed.
Research Note
Qualification is not binary. Seven distinct states form a staged progression: unproven → globally_qualified_regime_unproven → conditionally_qualified → degraded. A strategy can be qualified in ranging but not in trending, or vice versa. Qualification is regime-conditional, not absolute.
Design Principle
Qualification is not permanent.
59 strategies currently hold qualification. 135 have ever achieved it. The gap reflects continuous re-evaluation — when evidence degrades, qualification is revoked. This is the architectural equivalent of continuous integration: the system continuously validates that deployed decisions still deserve deployment.
Production observed
Research Note
A narrow regime override exists: when global criteria (win rate or ROI) fail but current-regime evidence is strong (≥5 trades, ≥50% win rate, positive PnL, proven), qualification is granted with a regime_override_applied warning. This prioritizes contextual decision quality over global statistics — a strategy with 80% win rate in ranging but 30% in trending should still qualify during ranging.
05
Evidence That It Works
The strongest test of any qualification system is whether it separates outcomes. If qualified strategies perform similarly to never-qualified strategies, the gate is adding cost without value. If they diverge, the gate is making substantively different decisions.
The Orqis qualification system has been tested through trade-level attribution on production paper trading data. The results show statistically significant separation across multiple dimensions — win rate, account-level returns, regime-adjusted performance, and false-negative analysis.
Verified Evidence — Production
Qualification separates outcomes
Trade-attributed cohort comparison using production paper trading data. Strategies assigned to cohorts based on whether they ever passed the 14-criteria gate. Trades attributed to qualification windows based on open timestamp.
- Win rate spread: +21.9pp (60.5% qualified vs 38.6% never-qualified, p < 0.0001, z = 8.94)
- Account-level spread: +14.9% (59 currently qualified at +6.2% vs 201 never-qualified at −8.6%)
- Regime universality: spread positive in all 6 regimes when controlling for intended_regime
- Evidence stability: no qualification decay within 60 days
- Capital protection: $57.3K protected, 68.8% rejection accuracy
✓ Production observed
Verified Evidence — Production
False-negative analysis
Systematic analysis of 113 rejected strategies with sufficient trade history. The system sacrifices ~8% false negatives to protect against 58% correctly identified losers.
- 66 strategies (58%) correctly rejected — average ROI −10.3%
- 38 strategies (34%) marginally profitable — average ROI +3.4%, failed robustness criteria
- 9 strategies (8%) strong false negatives — average ROI +13.8%, rejected despite profitability
- All 9 false negatives share a pattern: high return, low win rate (31–44%), profit through infrequent large wins
✓ Production observed
Verified Evidence — Production
End-to-end conversion funnel
Less than 1% of generated strategies reach live deployment. The funnel confirms the core thesis: execution is genuinely earned, not assumed.
- Generation → Validation pass: ~31% (1,158 of 3,743)
- Foundry sweep: ~80 candidates → ~6 qualify per sweep (7.5%)
- Qualification → Live eligible: 59 of 266 evaluated (22.2%)
- Live eligible → Deployed: 3 live strategies from 59 qualified (5.1%)
✓ Production observed
“The system rejects 77.8% of candidates. Rejected strategies average −8.7% ROI. The gate makes different decisions, not just slower ones.”
06
What Qualification Is Not
Defining a principle requires defining its boundaries. Qualification is frequently confused with prediction, optimization, and guaranteed outcomes. These confusions are understandable because qualification interacts with all of them. But it is architecturally distinct from each.
Distinction
Qualification is not prediction.
Prediction estimates what may happen. Qualification determines whether the evidence gathered so far warrants committing resources. A strategy can have a strong prediction (high backtest composite score) and still fail qualification (insufficient paper evidence, regime mismatch, behavioral instability).
Distinction
Qualification is not optimization.
Optimization improves parameters. Qualification determines whether the optimized result is robust enough to trust. Orqis separates these architecturally: the quant engine optimizes, the qualification evaluator qualifies. They never share a code path.
Important Distinction
Qualification is not a guarantee.
A qualified strategy can still lose money. Qualification improves the odds — the win rate spread is +21.9pp — but it cannot eliminate uncertainty. The goal is better processes, not perfect outcomes.
Limitations
Limitations & Open Questions
- Qualification selects for robustness (win rate), not profitability. Qualified strategies have worse mean return (−0.78%) than never-qualified (−0.41%) despite much higher win rate.
- Paper-to-live divergence is unmeasured. 119 live trades exist — insufficient for statistical comparison. All qualification evidence rests on paper trading outcomes.
- The false-negative rate of 8% means the system systematically excludes high-return/low-win-rate profiles. Whether this is the right tradeoff depends on the deployer's risk tolerance.
- Qualification criteria are currently domain-specific to capital allocation. Whether the same 14 criteria generalize to other domains is unproven.
- The system has been operational for approximately 3 months. Long-term stability of the qualification signal is directionally supported but not proven beyond 60 days.
References
Foundational Concepts
Related Research
Continue Reading
See these ideas implemented in Orqis