# NeurIPS 2023: Our Favorite Papers on LLMs, Statistical Learning, and More

[Two Sigma](https://yomu.fyi/company/two-sigma) · Emily Majewski · Mar 21, 2024

**Type:** Explainer

## Summary

Researchers reviewed prominent machine learning papers presented at NeurIPS 2023 covering large language models and statistical learning theory. One investigation showed that claimed emergent abilities in models such as GPT-3 often result from nonlinear evaluation metrics rather than fundamental shifts in model capability. To reduce the computational burden of model adaptation, QLoRA enables 65-billion-parameter model fine-tuning on a single 48-gigabyte GPU via 4-bit NormalFloat quantization and paged optimization. Direct Preference Optimization eliminates complex reward modeling by casting reinforcement learning from human feedback into a preference classification task. Additional work resolved statistical anomalies like double descent using effective parameter counts and introduced stochastic gradient approximations for Gaussian processes.

## Context

Researchers reviewed findings from NeurIPS 2023 addressing scalability, generalization, and computational constraints across large language models and statistical learning. Key challenges examined include high hardware demands for LLM customization, complex multi-stage alignment workflows in RLHF, the mathematical validity of emergent model capabilities, and computational bottlenecks in Gaussian process sampling.

## Approach / What changed

The reviewed papers present algorithmic and theoretical frameworks: QLoRA combines 4-bit NormalFloat quantization with paged optimization for efficient fine-tuning, Direct Preference Optimization condenses RLHF into a direct classification objective, and pathwise conditioning with stochastic gradient descent approximates Gaussian process posteriors. Other studies assess metric sensitivity in LLM evaluation and introduce the DISCS benchmark for discrete sampling.

## Takeaways

- Apparent emergent abilities in LLMs can disappear when switching from nonlinear evaluation metrics like accuracy to linear metrics like token edit distance or by increasing test sample resolution.
- QLoRA enables fine-tuning a 65B parameter model on a single 48GB GPU by combining a 4-bit NormalFloat data type, double quantization, and paged optimization to mitigate memory spikes.
- Direct Preference Optimization simplifies LLM alignment by reformulating the constrained reward maximization problem of RLHF into an equivalent human preference classification task.

**Tags:** [LLMs](https://yomu.fyi/topic/llm), [Machine Learning](https://yomu.fyi/topic/machine-learning), [Performance](https://yomu.fyi/topic/performance)

[Read original post](https://www.twosigma.com/articles/neurips-2023-our-favorite-papers-on-llms-statistical-learning-and-more)