Gguf on smcleod.net https://smcleod.net/tags/gguf/ Recent content in Gguf on smcleod.net smcleod.net https://smcleod.net/og-card.jpg https://smcleod.net/og-card.jpg Hugo -- 0.166.0 en-GB Fri, 09 Oct 2026 08:57:07 +1100 Patching NVIDIA's driver and vLLM to enable P2P on consumer GPUs https://smcleod.net/2026/02/patching-nvidias-driver-and-vllm-to-enable-p2p-on-consumer-gpus/ Wed, 25 Feb 2026 23:59:00 +1000 https://smcleod.net/2026/02/patching-nvidias-driver-and-vllm-to-enable-p2p-on-consumer-gpus/ NVIDIA artificially restricts peer-to-peer (P2P) GPU communication to their enterprise cards. Turns out this is a software limitation, not a hardware one. I patched my drivers to remove it, hacked vLLM to take advantage of it, and got a 15-50% throughput improvement running Qwen 3.5 35b on dual RTX 3090s. Code, Chaos, and Copilots (AI/LLM Talk July 2024) https://smcleod.net/2024/07/code-chaos-and-copilots-ai/llm-talk-july-2024/ Thu, 18 Jul 2024 01:00:10 +0000 https://smcleod.net/2024/07/code-chaos-and-copilots-ai/llm-talk-july-2024/ Integrating AI into Daily Development Understanding AI/LLM Quantisation Through Interactive Visualisations https://smcleod.net/2024/07/understanding-ai/llm-quantisation-through-interactive-visualisations/ Wed, 17 Jul 2024 01:00:10 +0000 https://smcleod.net/2024/07/understanding-ai/llm-quantisation-through-interactive-visualisations/ AI/LLM Quantisation Visualised LLM vRAM Estimator https://smcleod.net/vram-estimator/ Mon, 01 Jan 0001 00:00:00 +0000 https://smcleod.net/vram-estimator/ LLM vRAM Estimator