Record · corpus.blog/posts/01a07d5d-86e9-710f-b9c2-b454944dd780

Host overhead is killing your inference efficiency

Modal · published 18 November 2025

A record is what was published and who pointed at it. The text of the post is not held here: read it at the source, or at its Wayback capture.

Read the post at modal.com Wayback capture

0blogs citing

First cited: —
Most recent: —
Rank this month: not ranked
Links from this post: 19
Words captured: 1,564

Cited by

No blog we hold has cited this post yet.

Links from this post

Link

Host

https://modal.com/gpu-glossary/perf/performance-bottleneck

modal.com

https://modal.com/gpu-glossary/perf/memory-bound

modal.com

https://modal.com/gpu-glossary/perf/compute-bound

modal.com

'I paid for the whole GPU, I am going to use the whole GPU': A high-level guide to GPU utilization Read at the source ↗

modal.com

https://modal.com/gpu-glossary/device-hardware/tensor-core

modal.com

https://modal.com/gpu-glossary/perf/pipe-utilization

modal.com

https://modal.com/gpu-glossary/host-software/nvidia-smi

modal.com

https://modal.com/gpu-glossary/device-software/kernel

modal.com

https://modal.com/docs/examples/torch_profiling

modal.com

https://github.com/sgl-project/sglang/pull/9460

github.com

https://github.com/sgl-project/sglang/pull/9409

github.com

Introducing: B200s and H200s on Modal Read at the source ↗

modal.com

https://modal.com/gpu-glossary/perf/memory-bandwidth

modal.com

https://huggingface.co/Qwen/Qwen3-8B-FP8

huggingface.co

https://modal.com/blog/torch-compile-parameters

modal.com

https://github.com/sgl-project/sglang/pull/9487

github.com

https://developer.nvidia.com/blog/cuda-graphs/

developer.nvidia.com

https://modal.com/products/platform

modal.com

https://modal.com/careers

modal.com