Binary Classifier

The Binary Classifier module brings machine learning to your trading systems through an ensemble method called AdaBoost (Adaptive Boosting). It learns patterns from historical data to predict binary outcomes—such as "price goes up" or "price goes down"—and provides both a prediction and a confidence score.

---

How AdaBoost Works

AdaBoost is an ensemble learning algorithm that combines multiple "weak learners" (simple models that are only slightly better than random guessing) into a powerful "strong learner."

flowchart TD
    subgraph Training["Training Phase"]
        D1["Weak Learner 1<br/>(Decision Stump)"]
        D2["Weak Learner 2<br/>(Corrects #1's errors)"]
        D3["Weak Learner 3<br/>(Corrects #2's errors)"]
        DN["... N Learners"]
    end
    
    subgraph Ensemble["Final Ensemble"]
        V["Weighted Vote<br/>of All Learners"]
    end
    
    D1 --> D2 --> D3 --> DN --> V
    
    V --> P["Prediction: 0 or 1"]
    V --> C["Confidence: 0.0 to 1.0"]

The AdaBoost Process

1. Start with equal weights for all training samples
2. Train a weak learner (decision stump—a tree with just one split)
3. Identify misclassified samples and increase their weights
4. Train the next weak learner focusing on the harder samples
5. Repeat for N estimators
6. Combine all learners with weighted voting based on their accuracy

This iterative focus on difficult cases is what gives AdaBoost its name—it adapts to the hardest examples.

---

Mathematical Foundation

Decision Stump

A decision stump is a one-level decision tree that makes a single binary split:

$$h(x) = \begin{cases} 1, & \text{if } x_j \geq \theta \\ 0, & \text{otherwise} \end{cases}$$

Where $x_j$ is a single feature and $\theta$ is the optimal threshold.

AdaBoost Weighting

After each round $t$, sample weights are updated:

$$w_i^{(t+1)} = w_i^{(t)} \cdot e^{-\alpha_t \cdot y_i \cdot h_t(x_i)}$$

Where:

• $\alpha_t$ is the learner's weight (based on its accuracy)
• $y_i$ is the true label
• $h_t(x_i)$ is the learner's prediction

Misclassified samples get higher weights, focusing subsequent learners on the hard cases.

Final Prediction

The ensemble combines all learners:

$$H(x) = \text{sign}\left(\sum_{t=1}^{T} \alpha_t \cdot h_t(x)\right)$$

---

Training and Prediction Flow

flowchart LR
    subgraph Inputs["Feature Matrix"]
        F1["Feature 1<br/>(e.g., RSI)"]
        F2["Feature 2<br/>(e.g., MACD)"]
        F3["Feature 3<br/>(e.g., Volume)"]
    end
    
    subgraph Labels["Target Labels"]
        T["Historical Outcomes<br/>(0 or 1)"]
    end
    
    subgraph Model["Walk-Forward AdaBoost"]
        TR["Retrain every 50th<br/>observed sample"]
    end
    
    subgraph Outputs["Predictions"]
        P["Per-Sample<br/>Prediction Series"]
        C["Per-Sample<br/>Confidence Series"]
        FI["Per-Sample<br/>Importance Matrix"]
    end
    
    F1 & F2 & F3 --> TR
    T --> TR
    TR --> P & C & FI

Key Behavior: BCLS walks forward one sample at a time. prediction[t]/confidence[t]/ feature_importance[t] are always produced by a model trained ONLY on samples strictly before t - this prevents look-ahead bias at every slot, not just the last one. Retraining on every single sample would be expensive at scale, so the model refits every 50 newly observed, labeled samples and predicts every sample in between with the latest such model - a standard periodic-refit walk-forward that never uses future data.

---

Missing Data

• A sample with any missing feature is undeterminable: prediction/confidence/ feature_importance are all None at that sample's slot, and the sample is excluded from every later fit.
• A missing target label only affects training - the sample it belongs to can still be predicted once it has complete features.
• Mixed-calendar inputs (e.g. a 24/7 crypto feature alongside a business-day feature): weekend/holiday rows are excluded from training entirely, which means a 24/7 feature's weekend dynamics are not learned from when a business-day feature is also wired in. There is no fabrication-free alternative - this is the correct least-harm default. A matrix built entirely from business-day series loses nothing.
• Backfill emits the full-length walk-forward series (leading Nones while too few training samples exist yet); live emits exactly one value per output, gated only on whether the CURRENT sample is complete - a historical gap elsewhere never blocks live.
• feature_importance is CHART-ONLY: it re-types to a matrix (rows = timestamps) and cannot be wired into a downstream module as a scalar series.

---

Outputs Explained

Output	Description	Range
Prediction	The predicted class (0 or 1) for the latest sample	{0, 1}
Confidence	Probability assigned to the predicted class	[0.0, 1.0]
Feature Importance	How much each feature contributed to decisions	[0.0, 1.0] per feature

Interpreting Confidence

• Confidence = 0.95: The model is 95% certain of its prediction—high conviction
• Confidence = 0.52: Nearly 50/50—the model is uncertain, treat with caution
• Confidence thresholds: Many traders only act when confidence exceeds 0.6 or 0.7

Understanding Feature Importance

Feature importance reveals which inputs drive the model's decisions:

pie title Feature Importance Example
    "RSI" : 45
    "MACD Histogram" : 30
    "Volume Delta" : 15
    "Price ROC" : 10

Use this to:

• Validate your intuition — Are the "right" features being weighted?
• Simplify your system — Remove low-importance features to reduce noise
• Discover patterns — Sometimes unexpected features dominate

---

Practical Applications in Trading

1. Trend Direction Prediction

Train the classifier to predict whether price will be higher or lower N periods ahead:

Feature Inputs	Target Label
RSI, MACD, OBV, ATR	1 if Close[t+5] > Close[t], else 0

2. Regime Classification

Classify market conditions to select appropriate strategies:

Feature Inputs	Target Label
ADX, Volatility, Correlation	1 = Trending, 0 = Range-bound

3. Signal Filtering

Use the classifier to filter signals from other indicators:

flowchart TD
    Signal["Raw Buy Signal<br/>(e.g., Golden Cross)"]
    BCLS["Binary Classifier"]
    Confidence{"Confidence > 0.65?"}
    Execute["Execute Trade"]
    Skip["Skip Trade"]
    
    Signal --> BCLS
    BCLS --> Confidence
    Confidence -->|Yes| Execute
    Confidence -->|No| Skip

---

Configuration Parameters

Parameter	Description	Typical Value
N Estimators	Number of weak learners to combine	50–200
Decision Tree Max Depth	Depth of each decision tree (1 = stump)	1–3
Random State	Seed for reproducibility	Any integer

Parameter Guidance

• More estimators generally improve accuracy but increase computation time
• Deeper trees can capture more complex patterns but risk overfitting
• Decision stumps (depth=1) are the classic AdaBoost configuration and work well for noisy financial data

---

Advantages

• Resistant to overfitting — The weak learner approach prevents memorizing noise
• Handles complex patterns — Ensemble captures non-linear relationships
• Interpretable — Feature importance reveals what drives predictions
• Adaptable — Works with any numerical features you can compute

Limitations

• Requires labeled data — You must define and provide historical target labels
• Sensitive to outliers — Extreme samples can dominate training
• No guarantee of future performance — Past patterns may not repeat
• Black box to some degree — While feature importance helps, exact decision logic is complex

---

Further Reading

• AdaBoost - Wikipedia
• Ensemble Methods in Machine Learning
• Scikit-learn AdaBoostClassifier Documentation
• Machine Learning for Trading - Georgia Tech

---

Indicator Assessment

---
config:
  xyChart:
    width: 600
    height: 400
  themeVariables:
    xyChart:
      plotColorPalette: "#1abc9c, #e74c3c"
---
xychart-beta horizontal
    title "Binary Classifier Assessment (Scale: 1-10)"
    x-axis ["Popularity", "Reliability", "Complexity", "Versatility", "Recent Interest"]
    y-axis "Score" 0 --> 10
    bar [5, 5, 7, 8, 8]
    bar [4, 4, 7, 8, 9]

Legend: First bar (teal) = Equities/Forex | Second bar (red) = Cryptocurrency

Dimension Explanations

Dimension	Equities/Forex	Crypto	Rationale
Popularity	5	4	Machine learning classifiers are less common than traditional indicators. Growing adoption among quant traders. Less established in crypto retail trading.
Reliability	5	4	Performance depends heavily on feature selection and market regime. Risk of overfitting. Requires careful validation. Markets may shift, invalidating learned patterns.
Complexity	7	7	Requires understanding of machine learning concepts, feature engineering, and proper train/test splits. Not beginner-friendly.
Versatility	8	8	Can be applied to any binary classification problem: trend direction, regime detection, signal filtering. Highly adaptable with different feature sets.
Recent Interest	8	9	Strong growth in ML-based trading research. Increasing integration in trading platforms. Very high interest in crypto algorithmic trading communities.