Flide - Fast inference engine for Jev-style decision models
Show HN: Flide: Make Jev-like decision models up to 9.5x faster
Flide is a high-performance inference engine built for Jev-style decision models, delivering up to 9.4x faster inference than vLLM and SGLang on a single H200. Using exact state forking, it runs one sequence instead of many, computes nothing twice, and adds no extra tokens—so every answer matches what the model would give the question alone. It serves the Jev /v1/systemone API, supports dozens of model families including OneJev, Qwen3.5, Gemma 4, and Cloudflare's Clef, and handles text, images, and video. Install via PyPI, spin up a server with one command, and start sending requests in minutes.
With exact state forking, Flide runs one sequence, computes nothing twice, adds no tokens, and gives each question the answer it gets when asked alone.
- yossuf2000
do i need to host the model to do this ?
if yes what model can i host in 3050 card
- jarym
I wanted to try this - essentially for my dataset I have a number of decision questions for the same input. I haven't figured out yet which local model to try first with it!
- unikcc
Hi HN! I'm one of the Flide team.
We built Flide from scratch to make Jev-like decision model inference up to 9.5x faster than vLLM and SGLang, while keeping exactness.
Here is the source code: https://github.com/OmniJev/Flide
Look forward to hearing your feedback and ideas!