Image: reddit.com · Video still: i directed fable5 to solve the access to fast reasoning on consumer hardware problem - we built a from-scratch CPU inference engine - it beats llama.cpp, it's o
A from-scratch CPU inference engine that outpaced llama.cpp on a laptop
A non-kernel-engineer directed Claude to write a CPU inference engine from scratch, reported at about 77 tokens a second against llama.cpp's 14 on the same laptop and model.
The full story
The poster says they are not a kernel engineer but directed Claude as a kernel researcher, set the goal and the honesty bar, and Claude wrote every line. On the same laptop, model (gpt-oss:20b) and prompt, they report llama.cpp at about 14 tokens a second and their engine at about 77, and about 92 against 82 on the Q4_K format.
They also say Claude built a Metal GPU engine from scratch after showing a hybrid idea was impossible with microbenchmarks, and admit the benchmarks were wrong four times along the way. The code is open source.
How it was done
Tools
Themes
Loading themes…
Sources
From a short post by the maker, so the details are as they tell them.
- Reddit: i directed fable5 to solve the access to fast reasoning on consumer hardware problem - we built a fr
The maker's post on r/ClaudeAI.
The subreddit's moderators removed it, but the original is still readable (it can take a few seconds to load): archived copy of Reddit: i directed fable5 to solve the access to fast reasoning on consumer hardware problem - we built a fr (opens in a new tab)