It was possible to quantize tinyLlama with Spqr.
Quantize an open-source LLM. Assignment Course AI and its Applications 2025