Record · corpus.blog/posts/01a08783-2995-73ce-8954-4dc09c70b9ee

Disaggregated Prefill and Decode for LLM Serving on AWS - The KV Transfer, the Routing Threshold, and What Disaggregation Does Not Fix | hidekazu-konishi.com

hidekazu-konishi.com · published 20 August 2026

A record is what was published and who pointed at it. The text of the post is not held here: read it at the source, or at its Wayback capture.

Read the post at hidekazu-konishi.com Wayback capture

0blogs citing

First cited: —
Most recent: —
Rank this month: not ranked
Links from this post: 28
Words captured: 15,103

Cited by

No blog we hold has cited this post yet.

Links from this post

Link

Host

Self-Managed LLM Inference on Amazon EKS - Serving Open-Weight Models with vLLM, Neuron/GPU, and Karpenter | hidekazu-konishi.com Read at the source ↗

hidekazu-konishi.com

Elastic Fabric Adapter and the AWS Network Fabric - SRD, ENA Express, and Placement Group Design | hidekazu-konishi.com Read at the source ↗

hidekazu-konishi.com

Amazon Bedrock Inference Throughput and Latency Optimization - Quotas, Provisioned Throughput, Latency-Optimized Inference, Prompt Caching, and Intelligent Prompt Routing | hidekazu-konishi.com Read at the source ↗

hidekazu-konishi.com

LLM Inference Resilience Patterns on AWS - Retries with Backoff and Jitter, Circuit Breakers, and Stream Recovery | hidekazu-konishi.com Read at the source ↗

hidekazu-konishi.com

Amazon EKS Control Plane Configuration - What You Can Set, What Blocks the Way Back, and How Far Each Setting Reaches | hidekazu-konishi.com Read at the source ↗

hidekazu-konishi.com

Distributed Training Resilience on AWS - Where the Checkpoint Lives, How Far a Failure Sends You Back, and What Each Mechanism Requires | hidekazu-konishi.com Read at the source ↗

hidekazu-konishi.com

Speculative Decoding and Draft Models on AWS - Acceptance Rates, Which Methods Preserve the Output Distribution, and What the Managed Draft Model Takes Away | hidekazu-konishi.com Read at the source ↗

hidekazu-konishi.com

LLM Weight Quantization on AWS - Weight-Only and Activation Formats, What Each Costs in Accuracy, and When the Platform Quantizes for You | hidekazu-konishi.com Read at the source ↗

hidekazu-konishi.com

Batch and HPC Job Scheduling on AWS - Job Queues, Fair-Share Policies, Multi-Node Parallel Constraints, and the Slurm Route | hidekazu-konishi.com Read at the source ↗

hidekazu-konishi.com

https://docs.aws.amazon.com/sagemaker/latest/dg/sagemaker-hyperpod-model-deployment-dpd.html

docs.aws.amazon.com

https://docs.aws.amazon.com/sagemaker/latest/dg/sagemaker-hyperpod-model-deployment-ts-dpd.html

docs.aws.amazon.com

https://docs.aws.amazon.com/sagemaker/latest/dg/sagemaker-hyperpod-model-deployment-caching-routing.html

docs.aws.amazon.com

https://docs.aws.amazon.com/sagemaker/latest/dg/sagemaker-hyperpod-model-deployment.html

docs.aws.amazon.com

https://docs.aws.amazon.com/sagemaker/latest/dg/sagemaker-hyperpod-inference-release-notes.html

docs.aws.amazon.com

https://aws.amazon.com/about-aws/whats-new/2026/7/amazon-sagemaker-hyperpod-dpd/

aws.amazon.com

https://aws.amazon.com/blogs/machine-learning/disaggregated-prefill-and-decode-for-llm-inference-on-sagemaker-hyperpod/

aws.amazon.com

https://aws.amazon.com/blogs/machine-learning/introducing-disaggregated-inference-on-aws-powered-by-llm-d/

aws.amazon.com

https://aws.amazon.com/ec2/instance-types/p5/

aws.amazon.com

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/efa-acc-inst-types.html

docs.aws.amazon.com

https://aws.amazon.com/ai/machine-learning/neuron/

aws.amazon.com

https://docs.vllm.ai/en/latest/features/disagg_prefill.html

docs.vllm.ai

https://docs.vllm.ai/en/latest/features/nixl_connector_usage.html

docs.vllm.ai

https://docs.vllm.ai/en/latest/features/nixl_connector_compatibility.html

docs.vllm.ai

https://docs.lmcache.ai/api_reference/configurations.html

docs.lmcache.ai

https://docs.lmcache.ai/disaggregated_prefill/nixl/xpyd.html

docs.lmcache.ai

https://github.com/ai-dynamo/nixl/blob/main/src/plugins/libfabric/README.md

github.com

https://github.com/llm-d/llm-d/blob/main/guides/pd-disaggregation/README.md

github.com

https://github.com/llm-d/llm-d/blob/main/docs/operations/disaggregation/vllm.md

github.com