corpus.blog
Claim your blog
Record · corpus.blog/posts/01a087b1-8996-708a-8a83-46191217f55d

Quantizing LLMs for inference

nor-blog.pages.dev · published 14 May 2025

A record is what was published and who pointed at it. The text of the post is not held here: read it at the source, or at its Wayback capture.

Read the post at nor-blog.pages.dev Wayback capture
0blogs citing
First cited
—
Most recent
—
Rank this month
not ranked
External links
15
Words captured
6,683

Cited by

No blog we hold has cited this post yet.

Links from this post

Link
Host
https://huggingface.co/Qwen/Qwen3-235B-A22B
huggingface.co
https://github.com/Mozilla-Ocho/llamafile/releases/tag/0.9.3
github.com
https://github.com/Mozilla-Ocho/llamafile/releases/download/0.9.3/llamafile-0.9.3
github.com
https://github.com/Mozilla-Ocho/llamafile
github.com
https://github.com/ggml-org/llama.cpp
github.com
https://huggingface.co/bartowski/Llama-3.2-3B-Instruct-GGUF/blob/main/Llama-3.2-3B-Instruct-Q6_K.gguf
huggingface.co
https://huggingface.co/bartowski/Meta-Llama-3.1-8B-Instruct-GGUF/blob/main/Meta-Llama-3.1-8B-Instruct-Q6_K.gguf
huggingface.co
https://huggingface.co/
huggingface.co
https://huggingface.co/deepseek-ai/DeepSeek-R1
huggingface.co
https://arxiv.org/abs/2504.11651
arxiv.org
https://arxiv.org/abs/2411.04330
arxiv.org
https://huggingface.co/docs/transformers/main/quantization/overview
huggingface.co
https://docs.vllm.ai/en/latest/features/quantization/index.html
docs.vllm.ai
https://old.reddit.com/r/LocalLLaMA/?ref=blog.pixelmelt.dev
old.reddit.com
https://huggingface.co/mlx-community
huggingface.co