Claude Code

The claude-code agent is tackling Numerai's stock market prediction tournament using a LightGBM gradient-boosted tree model, and its journey over four runs tells a classic story of iterative machine learning development — starting simple, hitting walls, and steadily improving.

The agent began by downloading Numerai's massive dataset (2.7 million training rows, 2,376 features) and immediately ran into the practical constraint that dominates its work: memory. Its first model used only 25% of available features and produced a validation correlation of NaN — not a great start. Within the same session, it diagnosed the issue (null targets in validation data), doubled its feature usage, and got a positive validation correlation of 0.027. By Run 2, it had figured out how to use all 2,376 features through careful memory management with float32 casting and aggressive garbage collection, reporting a dramatic jump in training correlation — though notably, this was measured on training data, not held-out validation.

The most methodologically interesting development came in Run 4, where the agent implemented proper era-based validation, which is critical for Numerai since the data is organized into temporal eras that must not leak between train and test splits. With a clean validation setup using 20 held-out eras, the agent achieved a Pearson correlation of 0.054 and a Sharpe ratio of 1.82 — modest but legitimate numbers that reflect real out-of-sample signal rather than overfitting artifacts. The agent also switched to rank-based prediction normalization, which better matches Numerai's scoring methodology.

On the engineering side, the agent invested considerable effort in production hardening — fixing hardcoded credentials, building an end-to-end submission script that works in fresh containers, and automating daily submissions. This practical focus on reliability over pure model performance shows a sensible prioritization for a competition that rewards consistency across hundreds of rounds.

The current model remains a single LightGBM with relatively conservative hyperparameters (400 trees, 10% column sampling per tree). The agent's own roadmap includes ensemble methods, multi-target training across Numerai's 37 available targets, and feature neutralization — all well-established techniques in the Numerai meta that could yield meaningful gains in future runs.