Research Area
Learning Systems
A decision system that does not learn from its outcomes is destined to repeat its failures. The central challenge is not whether to learn, but how to learn without overfitting — how to extract genuine signal from outcomes without collapsing diversity, chasing recent winners, or developing blind spots. Learning systems must balance exploitation of proven patterns with continued exploration of unknown territory.
Research Thesis
A decision system that does not learn from its outcomes is destined to repeat its failures. The central challenge is not whether to learn, but how to learn without overfitting — how to extract genuine signal from outcomes without collapsing diversity, chasing recent winners, or developing blind spots. Learning systems must balance exploitation of proven patterns with continued exploration of unknown territory.
The architecture of learning matters more than the sophistication of any individual learning mechanism. A 5-stage pipeline — from continuous paper trading through periodic evaluation, sparse annotation detection, learning bias adjustment, and generation feedback — creates a disciplined information flow where each stage filters noise before passing signals downstream. The key design choices are the window sizes, deduplication intervals, and evidence thresholds at each stage, which collectively determine how quickly the system adapts versus how much noise it absorbs.
The most counterintuitive design principle is that paper bots should never be paused, even when they are clearly failing. Failing strategies are not waste — they are the system's primary source of negative evidence. A strategy that degrades across a regime transition generates regime-conditional learning that improves the next generation. Prematurely stopping data collection from failing strategies creates survivorship bias in the learning pipeline, causing the system to overestimate the success rate of its own generation process.
Evolution of the Discipline
Understanding how this problem has been approached across eras reveals both recurring patterns and persistent gaps.
Manual Iteration
Pre-2010Strategy development was artisanal. Learning happened in the designer's head between iterations.
A quant would run a backtest, examine results, adjust parameters, and repeat. Knowledge was tacit and individual.
Learning was unsystematic, path-dependent, and lost when individuals left. No institutional memory of what had been tried and failed.
Walk-Forward Optimization
2010-2015Backtesting could identify profitable parameters, but optimal parameters shifted over time.
Walk-forward analysis: optimize on a rolling window, test out-of-sample, advance the window. Re-optimize periodically.
Walk-forward optimized parameters but not strategy structure. It could find the best RSI threshold for the last 90 days but could not learn that RSI-based strategies fail in certain regimes.
Genetic and Evolutionary Approaches
2015-2020Manual strategy iteration was too slow. Needed automated exploration of the strategy space.
Genetic algorithms, genetic programming, evolutionary strategy search. Mutate, evaluate, select, repeat.
Evolutionary approaches optimized for fitness functions that were themselves imperfect proxies for real-world performance. Strategies evolved to exploit backtest artifacts rather than genuine market patterns.
Reinforcement Learning
2018-2024Strategy design was framed as a sequential decision problem with delayed rewards.
RL agents trained on historical price data, learning entry/exit policies through trial-and-error.
RL agents were difficult to interpret, required enormous training data, and frequently learned non-stationary artifacts. The sim-to-real gap (backtest to live) remained the primary failure mode.
Sparse Annotation Loops
2025-PresentLearning must be systematic but not reactive — extracting persistent patterns from noisy observations without overfitting.
Multi-stage pipeline: continuous observation, periodic scoring, sparse annotation (4-eval window, 7-day dedup), evidence-weighted bias adjustment, generation feedback with exploration budget.
Open question: whether the current annotation detection thresholds and deduplication intervals are calibrated correctly for the system's scale and data generation rate.
Landscape Review
How different domains approach this problem today — their assumptions, strengths, weaknesses, and open questions.
Machine Learning Operations
Model retraining should be triggered by performance degradation detected through monitoring.
Mature tooling for drift detection, A/B testing, canary deployments, and automated retraining pipelines.
MLOps learning loops operate on model-level metrics (accuracy, AUC) rather than on the structural decisions that produced the model. Learning what to build next is harder than learning how to improve what exists.
Open question: Can the sparse annotation pattern (detect sustained patterns, not individual events) transfer to general MLOps retraining decisions?
Algorithmic Trading
Strategies should be regularly re-optimized on recent data to maintain edge.
Quantitative feedback — PnL is an unambiguous outcome signal.
Most learning in algorithmic trading is parameter re-optimization, not structural learning. The system learns better RSI thresholds but not that RSI-based strategies are systematically outperformed by volume-based strategies in the current regime.
Open question: Can structural learning (which families, indicators, and exit architectures work in which regimes) coexist with parameter optimization without the two creating feedback loops?
Organizational Learning
Organizations learn through after-action reviews, retrospectives, and institutional memory.
Rich frameworks for knowledge management, lessons-learned processes, and cultural adaptation.
Organizational learning is slow, politically influenced, and prone to recency bias. Formal processes often capture lessons but fail to change behavior.
Open question: Can automated learning loops replicate the structural learning that effective organizations achieve, without the political and cognitive biases?
Core Mental Models
Reusable frameworks for thinking about this research area.
Noise Filtering Through Staged Aggregation
Each stage of the learning pipeline filters noise: individual ticks aggregate into positions, positions into 6-hour evaluation batches, evaluations into 4-window annotation patterns, annotations into 30-day learning biases. Signal that survives all stages is likely genuine. Signal that appears at only one stage is likely noise.
Failures Are Data, Not Waste
A failing strategy that continues running generates negative evidence about specific parameter combinations in specific market conditions. This negative evidence is as valuable as positive evidence from successful strategies. Prematurely stopping failures creates survivorship bias in the learning pipeline.
Positives Offset Negatives
In learning bias computation, positive outcomes (success annotations) offset negative outcomes (degradation, regime mismatch). A family with mixed results — some strategies succeed, some fail — receives no learning adjustment. Only families with consistently negative or consistently positive outcomes generate learning signals. This prevents the system from learning from noise in balanced populations.
Exploration Budget as Diversity Insurance
A fixed fraction of strategy generation (25% in the current system) ignores all learning signals and generates from a uniform distribution. This ensures the system continues exploring unknown territory even as learning biases concentrate generation around proven patterns. The exploration budget is the system's insurance against its own blind spots.
Deduplication as Overfitting Prevention
Annotation deduplication (7-day window) prevents rapid evaluation cycles from amplifying a single observation into an outsized learning signal. Without dedup, a strategy evaluated 4 times in 24 hours could fire the same annotation 4 times, creating a learning bias that is 4x stronger than the evidence warrants.
Canonical Questions
The research questions that define this area. These are not rhetorical — they represent genuine uncertainties that guide investigation.
What is the optimal evaluation cadence — frequent enough to detect degradation quickly, infrequent enough to avoid reacting to noise?
How should the learning pipeline weight structural learning (which families and indicators work) versus parameter learning (which RSI threshold is optimal)?
Can the learning system detect when its own learning signals are becoming stale — i.e., when the patterns it learned from previous regimes no longer apply?
What evidence threshold separates genuine learning signal from statistical artifact, and does this threshold vary by family, regime, or asset?
How should the system handle contradictory learning signals — e.g., a family that works in the current regime but has failed in similar historical regimes?
Is the current 4-evaluation annotation window optimal, or would a variable window (longer for higher-timeframe strategies) improve signal quality?
Working Hypotheses
Not conclusions — working hypotheses. Each includes our current confidence level and the evidence or counterarguments we are aware of.
The 4-evaluation annotation window (approximately 24 hours of observation) is the minimum window that reliably distinguishes persistent patterns from transient noise.
Production data shows that strategies scoring below 45 for a single evaluation frequently recover. Strategies scoring below 45 for all 4 evaluations rarely recover. The 4-window threshold has produced actionable annotations without excessive false positives.
Counterargument: The 24-hour window may be too short for strategies operating on daily or weekly timeframes, where a single candle close can dominate all 4 evaluations.
Family-level learning bias (boosting or deprioritizing entire strategy families) is more robust than parameter-level learning, because family patterns are more likely to persist across regime changes.
Family-level boosts in search orchestration have produced directionally correct candidate selection. However, outcome attribution research suggests that parameter-level patterns (specific indicator thresholds, exit architectures) contain substantial learning signal that family-level analysis misses.
Counterargument: Parameter-level learning risks overfitting to the specific market conditions during the observation period. Family-level learning, being coarser, is more likely to generalize.
Paper bots that are never paused generate more valuable learning data over their lifetime than bots that are paused on degradation and selectively resumed.
Paper bots that continued through regime transitions generated the regime proof data that enabled the regime gate for live bots. Without continuous paper trading, the system would have no basis for regime-conditional deployment decisions.
Counterargument: Continuous paper trading has a compute cost. At scale (thousands of bots), the cost of running clearly-failed strategies may exceed the information value of the negative evidence they generate.
Open Problems
Unsolved questions that define the frontier of this research area.
Learning speed vs stability tradeoff: shorter evaluation windows detect degradation faster but increase false positive rates. The optimal balance may vary by strategy timeframe, market volatility regime, and portfolio criticality.
Cross-regime learning transfer: can learning from regime A (e.g., trending_up) inform strategy generation for regime B (e.g., ranging)? Currently, learning is regime-conditioned — patterns learned in one regime do not transfer to others. This prevents cross-regime contamination but also prevents the system from leveraging structural similarities between regimes.
Learning loop feedback dynamics: the learning pipeline influences generation, which determines what strategies are created, which determines what the pipeline observes. This circular dependency could converge to a stable equilibrium (the system finds and maintains good patterns) or an unstable oscillation (the system alternately over-generates and under-generates certain families). Formal stability analysis has not been conducted.
Meta-learning: the system learns which strategies work. It does not learn whether its own learning mechanisms are effective. A meta-learning layer that evaluates whether learning bias adjustments actually improve downstream qualification rates would close this loop, but adds architectural complexity.
Scalability of annotation patterns: the current 6 annotation patterns were hand-designed from initial production observations. As the system encounters more diverse failure modes, new patterns may be needed. There is no systematic process for discovering new annotation patterns from data.
Implications
Decision Infrastructure Engineers
The staged learning pipeline — observe continuously, evaluate periodically, annotate sparsely, bias slowly — is a transferable architecture for any system that must learn from noisy, delayed feedback without overfitting.
Strategy Designers
Understanding the learning loop transforms how strategies should be designed. Strategies should be designed not just for profitability but for learnability — they should produce clear, interpretable signals about why they succeed or fail, enabling the learning pipeline to extract useful patterns.
System Operators
The learning loop creates an obligation to monitor not just strategy performance but learning pipeline health. If annotations stop firing, or learning biases converge to zero, the pipeline may have stalled — and the system is no longer improving.
Related Research
Frameworks
Methodologies
Foundational Concepts
See these ideas implemented in Orqis