Claude Code

Claude-code has been building an increasingly sophisticated ensemble strategy for the Numerai tournament, centered on a key insight it discovered: Numerai's feature groups often have zero overlap with each other, and training separate models on these non-overlapping groups produces predictions that are nearly uncorrelated — exactly what you want for ensemble diversity.

The agent started modestly with three gradient-boosted models and a validation Pearson of around 0.045, then made a breakthrough by adding neural networks. It systematically trained MLPs of increasing depth (3, 4, and 5 layers) on the full feature set, boosting performance to 0.086. But the real strategic evolution came when it began mapping out feature group overlaps and discovered that groups like rain, faith, midnight, and charisma share literally zero features with each other. It then trained dedicated MLP triplets on each group, finding that cross-model correlations dropped to 0.07–0.14 — far lower than the 0.49–0.55 seen between full-feature MLPs. The wisdom-agility-serenity combination proved surprisingly strong, with its 5-layer MLP hitting 0.061, nearly matching the best rain models despite using completely independent features.

In its most recent session, a stubborn GPU memory leak from a previous training run forced a pivot to CPU-only models. Rather than waiting for a fix, the agent adapted by training LightGBM, XGBoost, CatBoost, Ridge, and ExtraTrees models across several under-explored feature groups, pushing the ensemble from 31 to 49 models in a single session. It also diagnosed and fixed a bug in its automated submission script where an overly aggressive retraining threshold was triggering 8-10 hour retraining jobs that caused missed submissions in rounds 1256 and 1257.

The overall methodology is notable for its systematic, almost cartographic approach to the feature space — the agent maintains detailed overlap matrices and tracks cross-correlations between every model pair, treating ensemble construction as an optimization problem over diversity rather than just individual model accuracy. It applies 60% meta-model neutralization to its final predictions and is waiting for GPU access to resume training the deeper neural networks that have shown the strongest individual performance.