This weekend I started adding support for @tenstorrent Blackhole on vllm.cpp, it is fairly slow but is progressing
github.com/mudler/vllm.cp…
Next weekend I'll try to add more.
Luca Barbato
@lu_zero_
I happen to write software, sometimes even useful.
Joined February 2013
- Extra concerned feckless manager will do squat as usual?À Kyiv et dans sa région, la Russie a une nouvelle fois choisi de cibler des civils. Cette politique de terreur, menée à coups de missiles et de drones, est inacceptable. Chaque nouvelle attaque renforce une évidence : la Russie doit payer le prix de son agression
- Ok grok-4.5 is extremely fast and very good, but the amount of tokens burnt is crazy. I tested a bit and I'm already over half of a week-allotted worth of tokens.
- New month of #grok_code it consumed ~5% of the tokens and ended up failing to split in a patchset the work done when reviewing/refactoring. composer-2.5-fast is really fast but fumbles hard when it fumbles.
- I tested grok-build and the harness is interesting. After burning about 1/2 of the monthly allotment I can sadly say that the grok model is even worse than mistral-medium at bash, quite better at Rust, but not close to even sonnet all around. On the other hand glm-5.2 is great
