Workflow

1. Initialize Sample Weights

   • All data points start with equal weights (e.g., 1/N for N points).

2. Train Weak Learner (Decision Tree Stump)

   • Select the best stump based on metrics like Entropy or Gini Impurity.

3. Calculate Total Error (TE)

   • TE = fraction of misclassified points by the stump.

4. Compute Stump Weight (α)

   • \(\alpha = \frac{1}{2} \ln\frac{1 - TE}{TE}\)

   • Higher α → better performing weak learner; α is the weight in the final prediction.

5. Update Data Point Weights

   • Correctly classified points → decrease weight: multiply by \(e^{-α}\)

   • Misclassified points → increase weight: multiply by \(e^{α}\)

6. Normalize Weights & Assign Bins

   • Weights normalized to sum ≈ 1; assigned proportional bins (misclassified points get larger bins).

7. Select Data Points for Next Learner

   • Random values between 0-1 determine selection; misclassified points are more likely to be chosen.

8. Iterate

   • Repeat training, weight updates, and selection for a predefined number of iterations, generating multiple weak learners.

9. Final Prediction

   • Weighted sum of predictions: \(f = \alpha_1 M_1 + \alpha_2 M_2 + ... + \alpha_n M_n\)

   • Class with highest total weighted score is chosen as the final output.