Frühere Suchanfragen

Keine früheren Suchanfragen

Suchoptionen

Nur verfügbar, wenn angemeldet.
mastodontech.de ist einer von vielen unabhängigen Mastodon-Servern, mit dem du dich im Fediverse beteiligen kannst.
Offen für alle (über 16) und bereitgestellt von Markus'Blog

Verwaltet von:

Serverstatistik:

1,5 Tsd.
aktive Profile

mastodontech.de: ÜberStatusProfilverzeichnisDatenschutzerklärung

Mastodon: ÜberApp herunterladenTastenkombinationenQuellcode anzeigenv4.4.3

#sampling

0 Beiträge0 Beteiligte0 Beiträge heute
Hacker News

Inference-Aware Fine-Tuning for Best-of-N Sampling in Large Language Models

arxiv.org/abs/2412.15287

arXiv.orgInference-Aware Fine-Tuning for Best-of-N Sampling in Large Language ModelsRecent studies have indicated that effectively utilizing inference-time compute is crucial for attaining better performance from large language models (LLMs). In this work, we propose a novel inference-aware fine-tuning paradigm, in which the model is fine-tuned in a manner that directly optimizes the performance of the inference-time strategy. We study this paradigm using the simple yet effective Best-of-N (BoN) inference strategy, in which a verifier selects the best out of a set of LLM-generated responses. We devise the first imitation learning and reinforcement learning~(RL) methods for BoN-aware fine-tuning, overcoming the challenging, non-differentiable argmax operator within BoN. We empirically demonstrate that our BoN-aware models implicitly learn a meta-strategy that interleaves best responses with more diverse responses that might be better suited to a test-time input -- a process reminiscent of the exploration-exploitation trade-off in RL. Our experiments demonstrate the effectiveness of BoN-aware fine-tuning in terms of improved performance and inference-time compute. In particular, we show that our methods improve the Bo32 performance of Gemma 2B on Hendrycks MATH from 26.8% to 30.8%, and pass@32 from 60.0% to 67.0%, as well as the pass@16 on HumanEval from 61.6% to 67.1%.
#HackerNews#InferenceAwareFineTuning#BestOfN
Whitney Loblaw

Favourite unusual real-life sounds used as a rhythmic element in music?

- Cough (6klop): 6klop-electronix.bandcamp.com/
- Running in gravel (Ez3kiel): ez3kiel.bandcamp.com/track/aki
- Table tennis (Flying Lotus and Laura Darlington): flyinglotus.bandcamp.com/track
- Basketball and shoe squeaks (Memory Tapes): youtube.com/watch?v=WacIBze4YT

What's yours? I adore those and want to find more. I also love randomly hearing music in the world when there isn't any.

6klopRendre l'esprit, by 6klop from the album Yersinia
#sampling#music
AngesagtLive-Feeds
Über