Methodology
Continuous Optimization
A methodology for improving qualified strategies through systematic parameter exploration — preserving qualification integrity while searching for better configurations.
Objective
Improve the performance of qualified strategies through structured parameter optimization while preserving the evidence trail that justified qualification. The methodology produces an optimized strategy version that is strictly better than the baseline — or confirms that the baseline is already near-optimal. Optimization never bypasses qualification; optimized versions re-enter the standard validation pipeline.
Scope
Covers
- Selection of optimization candidates from qualified strategies with paper history
- Divergence-aware candidate prioritization
- Optuna-based parameter search (TPE sampler, composite objective)
- Validation of optimized versions through the standard pipeline
- Baseline comparison and rejection of worse-than-baseline results
- Auto-optimization scheduling and capacity management
Does Not Cover
- Initial strategy generation (covered by Decision Lifecycle methodology)
- Qualification assessment (covered by Validation-First Qualification methodology)
- Paper trading simulation mechanics (covered by Simulation Before Deployment methodology)
- Exit profile design or structural changes to strategy logic
- Portfolio-level optimization or capital allocation across strategies
Preconditions
These inputs and assumptions must be satisfied before the methodology can produce valid results.
| Input | Requirement |
|---|---|
| Qualified strategy | Strategy with qualification status (conditionally_qualified or globally_qualified_regime_unproven) and paper trading history demonstrating viability |
| Optimization infrastructure | Optuna framework with TPE sampler, configurable trial count, and composite objective function |
| Validation pipeline | Standard backtest validation (IS/OOS) available for optimized version assessment |
| Paper trading data | Sufficient paper trading history to compute divergence-aware prioritization scores |
| Market data | Historical OHLCV data for Optuna trials (pre-fetched once, injected into all trials to avoid per-trial API timeouts) |
Procedure
Select Optimization Candidates
Identify strategies eligible for optimization. Candidates must be qualified (proven via paper trading) with sufficient paper history. Prioritize using divergence-aware scoring: composite backtest score, paper ROI, per-trade reliability. User-requested optimizations are processed first, then system-selected candidates (best paper performers). Strategies losing in paper trading are deprioritized. Grid trading and DCA families are excluded (consistent timeouts due to parameter space complexity).
Decision Criteria
If strategy is not qualified → skip. If strategy family is grid_trading or dca → skip (known timeout risk). If daily processing quota reached (3 per day) → queue for next cycle. User requests always take priority over system-selected candidates.
Define Parameter Search Space
For each candidate, define the Optuna search space based on the strategy's family and indicator configuration. Parameters include: indicator periods (e.g., RSI period 10-25), threshold values (e.g., RSI overbought 65-80), risk parameters (stop loss, take profit, trailing configuration), and exit profile multipliers. The search space is bounded by StrategySpec validation rules — no trial can produce an invalid spec.
Decision Criteria
Search space must be constrained enough to converge in 25 trials but broad enough to find meaningful improvements. Parameters that define the strategy's core identity (family, direction, primary indicator) are fixed — only tuning parameters are varied.
Run Optimization Trials
Execute 25 Optuna trials using the TPE (Tree-structured Parzen Estimator) sampler. Each trial generates a parameter configuration, builds a StrategySpec, and runs a backtest against pre-fetched market data. Composite objective: return (30%), win rate (20%), Sharpe ratio (20%), max drawdown (15%), trade count (15%) — same weighting as standard validation. Timeout: 480 seconds per optimization run.
Decision Criteria
If all 25 trials complete → select best trial by composite score. If timeout occurs → use best trial found before timeout. If zero trials produce valid results → optimization fails (report to caller). TPE sampler focuses on promising regions after initial random exploration (first 10 trials).
Score and Compare to Baseline
Compare the best trial's composite score to the baseline strategy's original backtest score. The optimized version must be strictly better — equal is not sufficient (avoids unnecessary churn). Compute improvement magnitude across each scoring dimension. Check that the optimized version doesn't sacrifice one dimension excessively to improve another (e.g., doubling return but halving win rate).
Decision Criteria
If optimized composite score > baseline composite score → proceed to validation. If optimized composite score <= baseline → reject optimization. Report 'baseline is near-optimal'. If any individual dimension degrades by more than 30% → flag for review even if composite improves.
Validate Optimized Version
Submit the optimized strategy version to the standard validation pipeline (trigger_validation_run). The optimized version undergoes the same IS/OOS backtest, composite scoring, and fee-efficiency assessment as any new candidate. This is critical — optimization does not grant a validation bypass. The optimized version is persisted as a new strategy_version linked to the same strategy_template.
Decision Criteria
If validation passes (composite score above threshold, non-zero trades, OOS coherent) → optimized version enters paper trading. If validation fails → reject the optimization. The baseline version remains active and unaffected.
Qualification Pipeline Entry
The validated optimized version enters the standard qualification pipeline: paper trading with isolated capital, 6h evaluation batch, qualification assessment. It must independently earn qualification through forward testing — the baseline's qualification does not transfer. If the optimized version qualifies and the user approves, it can replace or run alongside the baseline.
Decision Criteria
Optimized version must meet all qualification dimensions independently: paper duration, trade count, divergence tolerance, regime proof. Variant inheritance does not apply — optimized versions are full strategy versions, not risk-only variants.
Expected Outputs
Optimized Strategy Version
A new strategy_version with tuned parameters that passed validation and entered the qualification pipeline
Optimization Report
Trial-by-trial results, best parameters found, baseline comparison, convergence analysis
Baseline Confirmation
When optimization fails to improve on baseline: confirmation that current parameters are near-optimal for the given data
Parameter Importance Analysis
Which parameters had the most impact on composite score (from Optuna's parameter importance)
Interpretation Guide
How to read the outputs this methodology produces.
Optimized > Baseline by 10%+
Significant improvement found. Baseline parameters had meaningful room for improvement.
Action: High confidence in optimized version. Proceed through validation and qualification.
Optimized > Baseline by 2-10%
Marginal improvement. Baseline was already reasonably configured.
Action: Proceed but with lower expectations. Paper trading will reveal if the improvement is real or noise.
Optimized <= Baseline
Baseline is near-optimal for this data window. No parameter change improves the composite score.
Action: Reject optimization. The strategy's performance ceiling is structural, not parametric.
Optimization timeout (480s)
Parameter space was too large or backtest execution too slow to complete 25 trials
Action: Use best trial found before timeout. Consider reducing search space or excluding slow-to-evaluate parameters.
Individual dimension regression > 30%
Optimization improved composite but at the expense of one dimension (e.g., win rate collapsed to boost return)
Action: Review trade-off. May indicate overfitting to one metric. Consider constraining the degraded dimension.
Grid trading / DCA excluded
These families consistently timeout during optimization due to parameter space complexity
Action: Not an error. These families are excluded by design. Consider manual parameter tuning instead.
Worked Example
Scenario
A qualified mean_reversion strategy on BTC/USDT with 4h timeframe is selected for auto-optimization. Baseline composite score: 58.2. Paper ROI after 30 days: +4.1%.
Candidate selection: strategy is qualified (conditionally_qualified), family=mean_reversion (not excluded), paper ROI positive, no user-requested optimizations pending.
→ Selected as top system candidate. Divergence-aware score: 72.3 (composite 58.2 + paper ROI 4.1% + per-trade reliability 0.84).
Search space defined: RSI period [10-25], RSI overbought [65-80], RSI oversold [20-35], stop_loss_pct [1.5-5.0], take_profit_pct [2.0-8.0], trailing_width_atr_mult [1.0-2.5]. Fixed: family, direction, primary indicator (RSI).
→ 6 variable parameters, all bounded within StrategySpec validation rules.
25 Optuna trials completed in 340 seconds. TPE sampler converged by trial 18. Best trial: RSI period=16, overbought=73, oversold=28, SL=2.8%, TP=5.2%, trailing_width=1.8. Composite: 64.7.
→ Best trial composite 64.7 vs baseline 58.2 — improvement of +11.2%.
Baseline comparison: return +18% (baseline +12%), win rate 58% (baseline 52%), Sharpe 1.42 (baseline 1.21), drawdown -8.3% (baseline -9.1%). No dimension regressed > 30%.
→ Optimized version is strictly better across all dimensions. Proceed to validation.
Validation run submitted. IS: 15 trades, composite 64.7. OOS: 5 trades, composite 59.8. Coherence: 7.6% degradation (within 15% threshold).
→ Validation passes. New strategy_version created, linked to original template.
Optimized version admitted to paper trading with $1,000 capital. After 21 days: 7 trades, +5.8% ROI (vs baseline's 4.1% over comparable period).
→ Qualification assessment pending. Forward performance consistent with optimization improvement.
Outcome
Optimization improved the baseline composite score by 11.2% (58.2 → 64.7). The optimized version passed validation and entered paper trading, showing early signs of forward improvement (+5.8% vs baseline +4.1%). Total optimization cycle: 340 seconds compute + standard qualification timeline. Parameter importance: RSI period and trailing width had the largest impact.
Failure Conditions
How to recognize when the methodology is not producing valid results.
All 25 trials produce worse-than-baseline results
Baseline parameters are near-optimal for this data window and strategy structure
Not a failure of the methodology — it means optimization has confirmed the baseline. Report and move on.
Optimization timeout (480s) with zero valid trials
Strategy's backtest is too slow for the parameter space
Exclude this strategy from auto-optimization. Consider reducing the search space or pre-filtering slow indicators.
Optimized version fails validation
Optimization found better parameters for IS but they don't generalize to OOS
Reject. This is overfitting to the in-sample period. Baseline remains active.
Optimized version diverges in paper trading
Optimized parameters work historically but not in forward conditions
Normal risk. The standard qualification pipeline will catch this — optimized version remains unqualified.
Same strategy optimized repeatedly with diminishing returns
Strategy has been fully explored — no further parametric improvement available
Remove from auto-optimization candidate pool. Performance ceiling is structural, not parametric.
Quality Checks
| Check | Passes When | Fails When |
|---|---|---|
| Optimized version uses the same validation pipeline as new candidates | IS/OOS split, composite scoring, fee-efficiency multiplier — identical to standard validation | Optimized version bypasses any validation step or uses different scoring |
| Baseline is unaffected by optimization | Original strategy version and its paper/live accounts continue running unchanged | Optimization modifies the baseline version or interferes with its execution |
| Market data is pre-fetched once per optimization run | All 25 trials use the same kline data (fetched before trial loop) | Each trial fetches data independently (causes API timeouts and inconsistency) |
| Parameter search space respects StrategySpec validation | No trial can produce an invalid spec — all parameter ranges are within validation bounds | Trials produce specs that fail validation, wasting compute |
| Composite objective matches standard validation weighting | Return 30%, win rate 20%, Sharpe 20%, drawdown 15%, trade count 15% | Optimization uses different weights — optimized versions look good by one metric but fail standard validation |
Handoff
Where the outputs of this methodology go next in the protocol chain.
Validation Pipeline
Paper Trading
Learning Layer
Auto-Optimization Queue
Termination State
Produces
Either an optimized strategy version that enters the qualification pipeline (improvement found) or a baseline confirmation report (no improvement found). Both outcomes are valuable — knowing a strategy is near-optimal prevents wasted optimization cycles.
Confidence
Confidence in optimization results depends on trial convergence. If TPE sampler converges by trial 15-20 with stable best score, confidence is high. If best score is still improving at trial 25, the search space may be under-explored — consider additional trials.
Next Step
Improved versions flow into the standard qualification pipeline (validation → paper trading → qualification). Rejected optimizations are logged and the strategy is removed from the auto-optimization candidate pool for a cooldown period.
Replication Notes
What another team would need to reproduce this methodology.
- Optuna TPE sampler requires at least 10 random trials before Bayesian optimization kicks in — 25 total trials is the minimum for meaningful exploration
- Market data must be pre-fetched once and injected into all trials — per-trial API calls cause timeouts at scale
- The composite objective must match the standard validation scoring weights exactly — mismatches cause optimized versions to fail validation
- Grid trading and DCA families should be excluded from auto-optimization (consistent timeouts due to parameter space complexity)
- Divergence-aware prioritization requires paper trading data — strategies without paper history cannot be meaningfully ranked
- The 480-second timeout is calibrated for Cloud Run execution — adjust for different compute environments
- Optimized versions must enter the full qualification pipeline — optimization does not grant a validation bypass under any circumstances
Knowledge Lineage
Introduced In
Modeled By
Investigated In
Depends On
Influences
Related Research
Frameworks
See this methodology implemented in Orqis