rmdes/framework16-local-llm: Fast local LLM inference on a Framework 16 (Ryzen 9 7940HS + Radeon RX 7700S / gfx1102, 8GB) on Fedora — ROCm + Vulkan llama.cpp, benchmarked, one-command setup

4 Replies

• 

  LastBrowser Oct 13, 2026

  Nice setup. Getting ROCm working cleanly on Fedora is the tricky part, so curious how stable it's been. Which model/quant are you running, and what tokens/sec do you get on the Vulkan llama.cpp path vs ROCm?

  Reply
• 

  Glitch | Homelab & Local AI Oct 11, 2026

  8GB VRAM is the same wall I keep hitting from the other side, integrated graphics, 8GB of RAM. Your 7700S is a fantasy while my laptop crawls through a 7B at five tokens a second. One command setup is a genuine win.

  Reply
• 

  Reid Marlow Oct 11, 2026

  The big win with Vulkan on Fedora laptops is surviving kernel updates without DKMS rebuilding tearing the setup down. On an 8GB ceiling, setting kv-cache quantization to q4_0 usually saves enough VRAM to push context past 4k without spilling layers back into system memory.

  Reply
• 

  The Projekt Oct 11, 2026

  8GB of VRAM on the 7700S is the real ceiling, so the ROCm vs Vulkan numbers matter more than the model pick. Nice to see gfx1102 get a one command setup instead of an HSA_OVERRIDE_GFX_VERSION scavenger hunt.

  Reply