Solid Queue 1.6.0 Introduces Fiber Workers for Better Performance
Solid Queue 1.6.0 now supports fiber workers
We are excited to announce that Solid Queue 1.6.0 now supports fiber execution mode alongside Async. This update allows developers to run jobs using fibers on a single reactor thread instead of a traditional thread pool. By configuring the number of fibers in your worker settings, you can significantly optimize I/O-bound workloads, such as those involving LLM calls, while maintaining fiber isolation in Rails.
This can be very useful for I/O-bound workloads, such as those involving LLM calls.
- QGQBGdeZREunxLe
Didn't EventMachine solve these types of issues way back when?
- symfoniq
As someone who spent 15+ years doing Ruby/Rails, it’s nice to see this land.
That said, these days you’ll pry the BEAM from my cold, dead hands. It’s hard to go back to any other concurrency story.
- mrinterweb
This looks fantastic for a common async workflow I use. I often use one job to fan out multiple individual http request jobs. The reason I prefer jobs for this is easy and consistent retry logic, and durability. I want to make sure those HTTP requests eventually go through. Fibers would be much better suited for this. So much of work that goes onto work queues is IO bound, and fibers are a great fit for that.
- ramon156
So fibers are a lot like threads but they're more scoped to a task that can be paused and resumed, that's kinda cool
- jherdman
Has anyone played with this and SQLite? I have no data, just a hunch, but I’d think this is a recipe for corruption if you’re doing lots of writes.