llmspeed - Test local LLM speed before buying hardware
Show HN: Llmspeed – feel how fast a local LLM runs before you buy the hardware
llmspeed lets you feel how fast a local LLM will actually run on your chosen hardware before you spend a dime. Pick a GPU or Mac from a list spanning NVIDIA RTX 4090 and 5090, AMD Strix Halo, Mac Studio M5 Ultra, Intel Arc, and even cloud H100s, then pair it with a model like Llama 3.1 8B, Qwen, Gemma, or GPT-OSS. You instantly see tokens per second and time-to-first-token, with numbers drawn from community benchmarks on r/LocalLLaMA and localscore.ai or projected from specs. It's open source on GitHub, so you can race configs and avoid costly hardware mistakes.
Pick hardware and a model. Feel the real speed before you buy.
- lopotubez
I feel like these types of benchmarks mostly test what the mainstream tells you is possible and or works. You rarely get to hear the different implementations of inference engines, different types of setups (e.g. hybrid ones or older hardware) and their pros and cons. Also it seems these were tested with very minimal token usage.
Currently on my 9 year old laptop with a GTX 1070 Max-Q 8GB vram, 16 gb ddr4 ram and i7 8750h. I am running a finetuned Qwen 3.5-35B-A3B (Tiel-Coder-35B-A3B-GGUF-MTP) with 4-bit quant on a llamacpp fork (Turboquant) with 131k context at around 16-19 tk/s and 350-500 tk/s prefil using Pi harness. It is amazing for writing scripts and doing medium coding tasks and summarizing text for me. After around 100k context it goes down to 9 tk/s but that is to be expected.
If you ever ask any frontier LLM or these types of benchmarks, they would probably say that the best model I can run on my system is a crappy 9B model or something, which is just completely false. So I really wonder how much more you can pull out from more powerful systems and much better open weight models.
- hizlikovboy27
around this time, i was thinking about running a local LLM for the company i work for. i'm trying to figure out what kind of hardware i'd need and what level of performance i could expect. i tried a few things on your website but how much do the results reflect real world usage?
in my use case, the model might need to analyze a given task first then generate and execute a database query and finally send the results back to the client. to what extent could your setup handle this kind of workflow and how accurate are the results?
good job.
- mirekrusin
It would be great if numbers weren't off, would include optimised versions (ie. speculative decoding), context size, include allowed parallel inference (single agents use parallel inference a lot) etc.
I don't think in general people care about long list of models, they just care about the best one.
Also dual gpu is common setup, there should be ie. 2x 4090.
- gitowiec
I would like to see in such simulators:
- multi GPU configuration (hetero and homo)
- PCIe gen chooser (3.0/4.0/5.0)
- is nvlink enabled (2 cards, 2 x 2 cards, 3 x 2 cards)
- is PCIe bifurcated or CPU native (threadripper pro for example)
- kristopolous
just go rent one for a few minutes and try it out ... it'll cost you well under $0.50 if you're just trying to get a feel for if it's right for you. There's a lot of things like context window that this doesn't capture.
Depending on who you sign up for, that's within the introductory credits (usually like 5-10$) so it's actually $0.