corpus.blog
41,715 blogs · 5,239,565 posts Claim your blog
Record · corpus.blog/posts/01a08c61-5d65-7392-9e8e-40fabde09913

Running private LARGE AI Models on Your Hardware: A Guide to Scaling Local LLMs

Weisser Zwerg · published 21 February 2025

A record is what was published and who pointed at it. The text of the post is not held here: read it at the source, or at its Wayback capture.

Read the post at weisser-zwerg.dev Wayback capture
0blogs citing
First cited
—
Most recent
—
Rank this month
not ranked
External links
105
Words captured
7,738

Cited by

No blog we hold has cited this post yet.

Links from this post

Link
Host
https://groq.com
groq.com
https://www.together.ai
together.ai
https://docs.cloud.google.com/vertex-ai/generative-ai/docs/model-garden/explore-models
docs.cloud.google.com
https://aws.amazon.com/de/bedrock/
aws.amazon.com
https://ai.azure.com/catalog/models/
ai.azure.com
https://docs.databricks.com/aws/en/machine-learning/foundation-model-apis/supported-models
docs.databricks.com
https://openrouter.ai/
openrouter.ai
https://fireworks.ai/
fireworks.ai
https://replicate.com/
replicate.com
https://huggingface.co/docs/inference-providers/en/index
huggingface.co
https://huggingface.co/docs/huggingface_hub/en/guides/inference_endpoints
huggingface.co
http://together.ai
together.ai
https://artificialanalysis.ai
artificialanalysis.ai
https://arena.ai/
arena.ai
https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard
huggingface.co
https://benchlm.ai/benchmarks/frontierCode
benchlm.ai
https://cognition.ai/blog/frontier-code
cognition.ai
https://massedcompute.com/
massedcompute.com
https://www.crunchbase.com/organization/massed-compute
crunchbase.com
https://www.runpod.io/
runpod.io
https://www.crunchbase.com/organization/runpod
crunchbase.com
https://docs.runpod.io/pods/storage/create-network-volumes
docs.runpod.io
https://www.macobserver.com/columns-opinions/understanding-apples-unified-memory-architecture/
macobserver.com
https://en.wikipedia.org/wiki/Asahi_Linux
en.wikipedia.org
https://www.macstadium.com/bare-metal-mac
macstadium.com
https://frame.work/de/de/blog/introducing-the-framework-desktop
frame.work
https://www.nvidia.com/en-us/products/workstations/dgx-spark
nvidia.com
https://opencode.ai/docs/models/
opencode.ai
https://docs.vllm.ai/projects/recipes/en/latest/moonshotai/Kimi-K2.5.html
docs.vllm.ai
https://www.jeffgeerling.com/blog/2025/15-tb-vram-on-mac-studio-rdma-over-thunderbolt-5/
jeffgeerling.com
https://appleinsider.com/articles/25/12/20/ai-calculations-on-mac-cluster-gets-a-big-boost-from-new-rdma-support-on-thunderbolt-5
appleinsider.com
https://www.dell.com/en-us/blog/breaking-the-multiphysics-simulation-bottleneck
dell.com
https://lenovopress.lenovo.com/lp1943-thinksystem-amd-mi300x-192gb-750w-8-gpu-board
lenovopress.lenovo.com
https://www.newegg.com/p/N82E16859155927
newegg.com
https://www.servethehome.com
servethehome.com
https://www.servethehome.com/big-cluster-little-power-the-8x-nvidia-gb10-cluster-marvell-cisco-ubiquiti-qnap-arm/
servethehome.com
https://www.nvidia.com/en-us/products/workstations/dgx-spark/
nvidia.com
https://www.nvidia.com/en-us/networking/ethernet-adapters
nvidia.com
https://mikrotik.com/product/crs804_ddq
mikrotik.com
https://github.com/NVIDIA/nccl
github.com
https://github.com/vllm-project/vllm
github.com
https://huggingface.co/moonshotai/Kimi-K2.5
huggingface.co
https://huggingface.co/moonshotai/Kimi-K2.6
huggingface.co
https://openzfs.github.io/openzfs-docs/man/master/8/zfs-snapshot.8.html
openzfs.github.io
https://opencode.ai/
opencode.ai
https://github.com/openclaw/openclaw
github.com
https://github.com/NousResearch/hermes-agent
github.com
https://github.com/turnstonelabs/turnstone
github.com
https://docs.nvidia.com/sync/latest/cluster-assistant.html
docs.nvidia.com
https://forums.developer.nvidia.com/t/fitting-a-high-quality-reap-less-glm-5-2-onto-4x-dgx-spark/374832
forums.developer.nvidia.com
https://github.com/tonyd2wild/GLM-5.2-QuantTrio-200K-4x-DGX-Spark
github.com
https://spark-arena.com/
spark-arena.com
https://github.com/theshiphq/claw-spark
github.com
https://developer.nvidia.com/cuda-gpus
developer.nvidia.com
https://en.wikipedia.org/wiki/CUDA
en.wikipedia.org
https://en.wikipedia.org/wiki/Ampere_(microarchitecture
en.wikipedia.org
https://en.wikipedia.org/wiki/Ada_Lovelace_(microarchitecture
en.wikipedia.org
https://en.wikipedia.org/wiki/Hopper_(microarchitecture
en.wikipedia.org
https://github.com/Dao-AILab/flash-attention
github.com
https://massedcompute.com/home-old/pricing/
massedcompute.com
https://www.nomic.ai/gpt4all
nomic.ai
https://ollama.com
ollama.com
https://huggingface.co/docs/text-generation-inference/index
huggingface.co
https://huggingface.co/
huggingface.co
https://github.com/
github.com
https://docs.vllm.ai/en/latest/features/quantization/index.html
docs.vllm.ai
Which Quantization Method is Right for You? (GPTQ vs. GGUF vs. AWQ)
newsletter.maartengrootendorst.com
https://docs.astral.sh/uv
docs.astral.sh
https://everything.curl.dev/usingcurl/netrc.html
everything.curl.dev
https://gist.github.com/technoweenie/1072829
gist.github.com
https://huggingface.co/docs/huggingface_hub/en/guides/cli
huggingface.co
http://huggingface.co/settings/tokens
huggingface.co
https://en.wikipedia.org/wiki/Git
en.wikipedia.org
https://git-lfs.com
git-lfs.com
https://graphite.dev/guides/how-to-use-git-lfs-pull
graphite.dev
https://github.com/open-webui/open-webui
github.com
https://docs.openwebui.com/tutorials/tips/reduce-ram-usage
docs.openwebui.com
https://massedcompute.com/best-llama-3-inference-endpoint-part-2/
massedcompute.com
https://docs.vllm.ai/en/stable/serving/distributed_serving.html
docs.vllm.ai
https://blog.runpod.io/introduction-to-vllm-and-how-to-run-vllm-on-runpod-serverless
blog.runpod.io
https://blog.runpod.io/how-to-run-vllm-with-runpod-serverless-2
blog.runpod.io
https://www.deepspeed.ai/tutorials/zero-offload/
deepspeed.ai
https://www.microsoft.com/en-us/research/blog/zero-infinity-and-deepspeed-unlocking-unprecedented-model-scale-for-deep-learning-training/
microsoft.com
https://github.com/microsoft/DeepSpeed
github.com
https://huggingface.co/docs/accelerate/index
huggingface.co
https://opencode.ai/docs/tools/
opencode.ai
Llama 3.1 - 405B, 70B & 8B with multilinguality and long context Read at the source ↗
huggingface.co
https://docs.nvidia.com/nim/large-language-models/1.7.0/function-calling.html
docs.nvidia.com
https://pi.dev/
pi.dev
https://hermes-agent.nousresearch.com/
hermes-agent.nousresearch.com
https://openclaw.ai/
openclaw.ai
https://openhands.dev/
openhands.dev
https://www.elastic.co/search-labs/blog/pre-computed-context-llm-agent-costs
elastic.co
https://github.com/headroomlabs-ai/headroom
github.com
https://github.com/Graphify-Labs/graphify
github.com
https://github.com/JuliusBrussee/caveman
github.com
https://p.migdal.pl/
p.migdal.pl
Do Qwen3.6 27B quantizations break the pelican?
quesma.com
Benchmarking Qwen3.8 27B quantizations: 4-bit holds up, 1-bit collapses Read at the source ↗
quesma.com
https://www.youtube.com/watch?v=M1j_uRqKMKI
youtube.com
https://okara.ai/
okara.ai
https://mammouth.ai/
mammouth.ai
https://blog.kubesimplify.com/docker-networking-demystified
blog.kubesimplify.com
https://docs.vllm.ai/en/stable/features/tool_calling/
docs.vllm.ai
https://github.com/anomalyco/models.dev
github.com
Elsewhere on weisser-zwerg.dev (4)