corpus.blog
56,966 blogs · 4,417,066 posts Claim your blog
Record · corpus.blog/posts/01a0b878-238c-71f2-b154-7cf6ef6ff9f5

Defeating Nondeterminism in LLM Inference

thinkingmachines.ai · published 10 September 2025

A record is what was published and who pointed at it. The text of the post is not held here: read it at the source, or at its Wayback capture.

Read the post at thinkingmachines.ai Wayback capture
37blogs citing
First cited
30 Jan 2025
Most recent
17 Sept 2026
Rank this month
not ranked
Links from this post
20
Words captured
7,998

Cited by

Blog
In the post
Date
cio.com
AI agents should retrieve facts, not define them Read at the source ↗
17 Sept 2026
LambdaTest Blog
AI Reliability Engineering: SLOs for Model-Backed Features Read at the source ↗
13 Sept 2026
fazamhd.com
Large Language Models Read at the source ↗
10 Sept 2026
atp.fm
706: No One Is Getting a Good Deal Tonight Read at the source ↗
27 Aug 2026
hidekazu-konishi.com
Reproducible LLM Inference - Batch Invariance, the Cost of Determinism, and Where Bitwise Identity Is Actually Required | hidekazu-konishi.com Read at the source ↗
24 Aug 2026
vLLM Blog
IsoExec: Unified Execution to Eliminate Trainer-Inference Mismatch in SkyRL Read at the source ↗
21 Aug 2026
tim-schipper.nl
Did the model get worse? Comparing coding models without a benchmark Read at the source ↗
18 Aug 2026
vincentschmalbach.com
How Floating-Point Determinism Affects LLM Reproducibility Read at the source ↗
11 Aug 2026
vincentschmalbach.com
How Does Mixture-of-Experts Routing Affect LLM Repeatability? Read at the source ↗
11 Aug 2026
tensor.khalilli.ai
Floating Point Read at the source ↗
10 Aug 2026
henrypan.com
Harness Training Read at the source ↗
19 Jul 2026
coles.codes
Pydantic AI structured outputs and evals on Bedrock Read at the source ↗
13 Jul 2026
mikelev.in
Meeting in the Middle: Engineering Deterministic AI Workflows Read at the source ↗
1 Jul 2026
Quarkslab Blog
From prompt to pwned: chaining LLM and web bugs to Admin Read at the source ↗
4 Jun 2026
abacusnoir.com
Chasing Bit-Equality Through a Wasm Actor Runtime Read at the source ↗
17 May 2026
blog.doubleword.ai
Speculative KV coding: losslessly compressing KV cache by up to ~4× using a predictor model Read at the source ↗
8 May 2026
alexcbecker.net
The Myth of Deterministic Software Read at the source ↗
3 May 2026
Home · Sara Zan
Setting the temperature to zero will make an LLM deterministic? Read at the source ↗
24 Mar 2026
storagereview.com
NVIDIA Groq 3 LPX: Everything we know Read at the source ↗
19 Mar 2026
bhekani.com
What does it actually mean to run AI in production? Read at the source ↗
10 Mar 2026
elliotmassen.com
Agent UX mirrors model limitations Read at the source ↗
22 Feb 2026
conseval.com
How an Agent Works: The Infer Function Read at the source ↗
21 Feb 2026
Kevin Lynagh
Easy VM sandboxes for LLM agents on MacOS, Miami & Paris travel Read at the source ↗
1 Feb 2026
quesma.com
Vibe coding needs git blame Read at the source ↗
9 Jan 2026
cameronrwolfe.substack.com
GRPO++: Tricks for Making RL Actually Work Read at the source ↗
5 Jan 2026
blog.sshh.io
Understanding AI Benchmarks Read at the source ↗
21 Dec 2025
adamkarvonen.github.io
LLM inference is nearly deterministic. We use this to audit providers Read at the source ↗
28 Nov 2025
kipply
Digest | Jul/Aug/Sep/Oct 2025 Read at the source ↗
1 Nov 2025
atp.fm
662: Just Break the Law Read at the source ↗
23 Oct 2025
boydkane.com
Why your boss isn't worried about AI Read at the source ↗
14 Oct 2025
bionicteaching.com
Weekly Web Harvest for 2025-10-05 Read at the source ↗
12 Oct 2025
data-monger.com
Larry Ellison Knows Read at the source ↗
8 Oct 2025
mikulskibartosz.name
The Temperature=0 Myth: Why Your LLM Still Isn't Deterministic (And How to Fix It) Read at the source ↗
24 Sept 2025
vishalbakshi.github.io
Re-evaluating colbert-ai Index Artifacts Between PyTorch Versions with Precision-Based torch.allclose Tolerances Read at the source ↗
14 Sept 2025
vishalbakshi.github.io
Debugging ColBERT Index Differences Between PyTorch 2.7.1 and 2.8.0 Read at the source ↗
14 Sept 2025
timkellogg.me
Link Graveyard: A snapshot of my abandoned browser tabs Read at the source ↗
13 Sept 2025
vishalbakshi.github.io
Batch Size Causes BertModel Forward Pass Divergence Between torch==2.4.1 and torch==2.5.0 for colbert-ai. Read at the source ↗
11 Sept 2025
qouteall.fun
Traps to Developers Read at the source ↗
3 Aug 2025
vrutkovs.eu
21-31 December 2025 Read at the source ↗
30 Jan 2025
babbo.dev
Intelligence (Artificial) in a Fallen World Read at the source ↗
—
Suriya's site - Technical Writing & Life
to learn Read at the source ↗
—

Links from this post

Link
Host
https://152334h.github.io/blog/non-determinism-in-gpt-4/
152334h.github.io
https://community.openai.com/t/a-question-on-determinism/8185/2
community.openai.com
https://cookbook.openai.com/examples/reproducible_outputs_with_the_seed_parameter
cookbook.openai.com
https://docs.vllm.ai/en/v0.7.0/getting_started/faq.html
docs.vllm.ai
https://docs.sglang.ai/references/faq.html
docs.sglang.ai
https://arxiv.org/abs/2506.09501
arxiv.org
https://community.openai.com/t/a-question-on-determinism/8185
community.openai.com
https://x.com/hosseeb/status/1773146428594090473
x.com
https://en.wikipedia.org/wiki/Fetch-and-add
en.wikipedia.org
https://github.com/NVIDIA/cutlass/issues/1421
github.com
https://arxiv.org/abs/2307.08691
arxiv.org
https://github.com/NVIDIA/nccl/issues/1497
github.com
https://github.com/Dao-AILab/quack/blob/main/media/2025-07-10-membound-sol.md
github.com
https://github.com/NVIDIA/cutlass/blob/main/media/docs/cpp/efficient_gemm.md
github.com
https://docs.nvidia.com/cuda/parallel-thread-execution/
docs.nvidia.com
What Shapes Do Matrix Multiplications Like? [medium] Read at the source ↗
thonking.ai
https://github.com/vllm-project/vllm/blob/0ae43dbf8cb28a299ae724fc742b0c5bcddea868/vllm/attention/ops/prefix_prefill.py
github.com
https://github.com/thinking-machines-lab/batch_invariant_ops
github.com
https://fengyao.notion.site/off-policy-rl
fengyao.notion.site
https://arxiv.org/abs/2502.17387
arxiv.org