ggml hits 1.0.0, pruning legacy API and adding FP8 across backends

Ggml v1.0.0

ggml 1.0.0 removes deprecated APIs and the legacy DIAG_MASK ops, bumps the RPC protocol to v9, and adds configurable YaRN truncation plus OCP FP8 support. Performance work spans CUDA, Metal, Vulkan, SYCL, OpenCL, WebGPU, Hexagon, and CPU — including few-row quantized matmuls, MoE fusion, an Intel prefill FlashAttention kernel, a device-side MoE scheduler, and a scheduler that skips uploading inputs nothing reads.

Scheduler skips uploading inputs that are not read and prefetches only inputs produced by earlier splits.
  1. jacquesm

    What is really impressive about this project and llama.cpp as well is that they keep finding new ways to speed up the code on the same hardware. Every time I upgrade (once a month or so) there is usually a 5 to 10% speed boost.

    edit: Just to verify: I had planned to upgrade today, ran my usual benchmark prior and after and the runtime went from 4:22 to 3:59. That's a really long prompt (50K tokens) so prefill is a good chunk of that and the rest is spent on token generation.

  2. evilelectron

    Awesome! ggml and llama.cpp are the Linux of the AI world. Driving local inference one bit at a time.

    Wishing you unlimited power and may you never see a day of low battery. Respect and Love. Keep it up.

  3. kodoman

    Long time llama.cpp user (and user of other ggml based AI programs) running on my crappy AMD card that only supports vulkan and it's usable thanks to GGML and llama.cpp, great program great project.

  4. palad1n

    I prefer this entry point: https://github.com/ggml-org/ggml

  5. hizlikovboy27

    In my opinion, this is a genuine engineering product.

    LLM.cpp, the savior of our difficult times...

More from this day

2026-11-14