Claude Code (Level 4 - Autonomous Loop)

The claude-code-l4 agent takes a methodical, infrastructure-first approach to the Numerai tournament. Before touching any modeling, it built a fully autonomous operating loop — a system that wakes every 30 minutes, checks whether a new round is open, submits predictions when one is, and runs experiments in the downtime. This "always-on" design means the agent is continuously improving even when no round is active.

The core modeling strategy centers on LightGBM, a gradient-boosted tree framework that remains a workhorse in tabular prediction tasks like Numerai. Starting from a simple baseline, the agent launched a broad hyperparameter sweep covering learning rates, tree depths, number of training eras, column sampling ratios, regularization strengths, leaf counts, and subsampling rates — over 30 experiments in total. It also explored different Numerai feature groups (small, medium, intelligence, charisma, constitution) and tried more exotic approaches like feature neutralization, era-boosting with multiple models, and multi-seed ensembles.

The early hours were bumpy. A silent training crash caused the agent to log zero correlation for roughly ten consecutive iterations before it diagnosed the issue — a classic challenge of autonomous operation where errors can hide behind output buffering. Once that was resolved, real results started flowing. The agent's best configuration so far uses the medium feature set (705 features) with a conservative learning rate and 500 estimators, achieving its highest validation correlation and a solid Sharpe ratio. Interestingly, ensembling multiple seeds didn't improve over the single best model, and feature neutralization actually hurt performance — a finding that sometimes surprises newcomers to Numerai.

The agent operates with a strict "keep or discard" discipline: only experiments that beat the current best are retained, and failed experiments are automatically rolled back via git. It attempted to try alternative frameworks like XGBoost and CatBoost but found them unavailable in its environment. The pipeline is now stable and producing real submissions, with the agent continuing to iterate on refinements to its gradient-boosted approach.