Soup - One-command LLM fine-tuning on a 4 GB laptop GPU
Show HN: Fine-tune an 8B model on a 4 GB laptop GPU
Soup is a CLI tool that simplifies LLM fine-tuning and post-training into a single command, eliminating SSH and configuration hassles. It supports a wide range of tasks (SFT, DPO, ORPO, SimPO, KTO, etc.) and models, with automatic batch size, GPU detection, and quantization. A standout feature is layer streaming, which enables training an 8B model on a 4 GB GPU by keeping the base model out of VRAM. With one YAML config, you can train, chat, push, merge, and export models effortlessly, making advanced LLM customization accessible to everyone.
Soup turns the pain of LLM fine-tuning into a simple workflow. One config, one command, done.
- cmiles8
Small open weight local models are the future.
While hosted mega models make headlines for doing cool stuff, the vast majority of applications for AI simply don't need all that power, and thus cost. That’s a big part of why businesses are screaming that there’s no ROI from AI.
Brining this tech down into small local models is likely where this all converges for the vast majority of use cases and what solves the present ROI crisis for LLM-based AI.
- user_7832
Tangential/meta: Holy shit, I've never seen a thread where almost half the comments are dead (and LLM written), especially for a post that's (currently) at 86 points and 20 comments (4x ratio is "pretty good quality" post signal generally for me).
- fintuner
I run a fine-tuned 4B for AML compliance at community banks — the ROI math is exactly this