Record · corpus.blog/posts/01a08783-2996-72ba-a876-ac540e57977a

Self-Managed LLM Inference on Amazon EKS - Serving Open-Weight Models with vLLM, Neuron/GPU, and Karpenter | hidekazu-konishi.com

hidekazu-konishi.com · published 29 June 2026

A record is what was published and who pointed at it. The text of the post is not held here: read it at the source, or at its Wayback capture.

Read the post at hidekazu-konishi.com Wayback capture

0blogs citing

First cited: —
Most recent: —
Rank this month: not ranked
External links: 17
Words captured: 7,310

Cited by

No blog we hold has cited this post yet.

Links from this post

Link

Host

https://docs.aws.amazon.com/eks/latest/userguide/ml-inference-load-serve-model.html

docs.aws.amazon.com

https://docs.aws.amazon.com/eks/latest/userguide/ml-cluster-setup-cli.html

docs.aws.amazon.com

https://docs.aws.amazon.com/eks/latest/userguide/device-management-neuron.html

docs.aws.amazon.com

https://docs.aws.amazon.com/eks/latest/userguide/ml-eks-optimized-ami.html

docs.aws.amazon.com

https://docs.aws.amazon.com/eks/latest/userguide/auto-accelerated.html

docs.aws.amazon.com

https://docs.aws.amazon.com/eks/latest/best-practices/karpenter.html

docs.aws.amazon.com

https://docs.aws.amazon.com/eks/latest/userguide/ml-node-pools.html

docs.aws.amazon.com

https://docs.aws.amazon.com/eks/latest/userguide/device-management.html

docs.aws.amazon.com

https://docs.aws.amazon.com/eks/latest/userguide/aws-load-balancer-controller.html

docs.aws.amazon.com

https://docs.aws.amazon.com/eks/latest/userguide/service-accounts.html

docs.aws.amazon.com

https://aws.amazon.com/ai/machine-learning/neuron/

aws.amazon.com

https://aws.amazon.com/ec2/instance-types/inf2/

aws.amazon.com

https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-openai.html

docs.aws.amazon.com

https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_Converse.html

docs.aws.amazon.com

https://aws.amazon.com/blogs/aws/openai-open-weight-models-now-available-on-aws/

aws.amazon.com

https://docs.vllm.ai/en/latest/

docs.vllm.ai

https://aws.amazon.com/bedrock/pricing/

aws.amazon.com

Elsewhere on hidekazu-konishi.com (14)

Amazon Bedrock Model Catalog 2026 | hidekazu-konishi.com Read at the source ↗

hidekazu-konishi.com

Amazon Bedrock Glossary - A Reference for AI Engineers and Architects | hidekazu-konishi.com Read at the source ↗

hidekazu-konishi.com

Reproducible LLM Inference - Batch Invariance, the Cost of Determinism, and Where Bitwise Identity Is Actually Required | hidekazu-konishi.com Read at the source ↗

hidekazu-konishi.com

Speculative Decoding and Draft Models on AWS - Acceptance Rates, Which Methods Preserve the Output Distribution, and What the Managed Draft Model Takes Away | hidekazu-konishi.com Read at the source ↗

hidekazu-konishi.com

LLM Weight Quantization on AWS - Weight-Only and Activation Formats, What Each Costs in Accuracy, and When the Platform Quantizes for You | hidekazu-konishi.com Read at the source ↗

hidekazu-konishi.com

Distributed Training Resilience on AWS - Where the Checkpoint Lives, How Far a Failure Sends You Back, and What Each Mechanism Requires | hidekazu-konishi.com Read at the source ↗

hidekazu-konishi.com

Disaggregated Prefill and Decode for LLM Serving on AWS - The KV Transfer, the Routing Threshold, and What Disaggregation Does Not Fix | hidekazu-konishi.com Read at the source ↗

hidekazu-konishi.com

AWS Generative AI History and Timeline - From SageMaker JumpStart to Bedrock AgentCore | hidekazu-konishi.com Read at the source ↗

hidekazu-konishi.com

Programming AWS Trainium with the Neuron Kernel Interface - What the Compiler Stops Deciding for You, Which Names Do Which Jobs, and When Not to Descend | hidekazu-konishi.com Read at the source ↗

hidekazu-konishi.com

Sharing One GPU on Amazon EKS - Where the Split Is Declared, Where It Is Enforced, and What Each Method Leaves Shared | hidekazu-konishi.com Read at the source ↗

hidekazu-konishi.com

Amazon EKS Control Plane Configuration - What You Can Set, What Blocks the Way Back, and How Far Each Setting Reaches | hidekazu-konishi.com Read at the source ↗

hidekazu-konishi.com

Gang Scheduling and Dynamic Resource Allocation on Amazon EKS - Why One Half of the Vocabulary Is Still Alpha, and What Decides Which Device Allocation Path You Get | hidekazu-konishi.com Read at the source ↗

hidekazu-konishi.com

Life After Ingress NGINX on Amazon EKS - Where the Upstream and AWS Disagree About the Order of Migration, and What GatewayClass Actually Decides | hidekazu-konishi.com Read at the source ↗

hidekazu-konishi.com

LLMOps Observability and Evaluation Architecture on AWS - Tracing, Metrics, and Automated Evaluation Gates with CloudWatch and OpenTelemetry | hidekazu-konishi.com Read at the source ↗

hidekazu-konishi.com