Latency-Optimized LLMs for Trading
Sam Green
Session summary
The presenter describes academic studies where large language models (LLMs) label news headlines for sentiment and drive simple long/short trading rules. He explains that earlier work using a GPT‑3‑class model on post‑cutoff news reported strong backtested performance, even after accounting for imperfect fee modeling. Follow‑on research with a GPT‑4‑class model reportedly showed about 62% accuracy for sentiment‑driven trades in early 2024, but this edge diminished over time, which he interprets as consistent with such strategies being adopted and arbitraged away by larger, well‑optimized traders.
He then connects these results to his prior research on efficient neural networks and reinforcement learning distillation. In vision‑based RL tasks, he found larger policies tend to learn faster and reach higher scores, but smaller “student” policies distilled from a trained “teacher” can match or surpass the teacher without additional environment interaction. He proposes applying this to trading by using a large LLM to build labeled news corpora and embeddings, then training smaller, latency‑optimized models. He also outlines a latency‑aware architecture search that incorporates hardware operation costs into the loss, enabling systematic trade‑offs between model size, latency, and minimum acceptable performance for time‑sensitive trading.
Key takeaways
- The presenter highlights academic work where GPT-style models performed headline sentiment classification for long/short strategies, showing historically profitable backtests that have since lost their naive edge as markets adapted.
- From his reinforcement learning research, the presenter reports that larger vision-based policies learn faster and reach higher performance, but smaller distilled student models can later match or exceed the teacher’s scores without further environment training.
- He proposes combining LLM-based sentiment labeling with distillation and latency-aware architecture search to train smaller, hardware-efficient models that better meet the latency constraints of real-time trading systems.