corpus.blog
41554 blogs · 4,450,554 posts Claim your blog
Record · corpus.blog/posts/01a0cac2-0806-72ba-85b8-8447a1083190

How Nvidia’s CUDA Monopoly In Machine Learning Is Breaking – OpenAI Triton And PyTorch 2.0

SemiAnalysis · published 16 January 2023

A record is what was published and who pointed at it. The text of the post is not held here: read it at the source, or at its Wayback capture.

Read the post at semianalysis.com Wayback capture
1blogs citing
First cited
12 Feb 2025
Most recent
26 Mar 2025
Rank this month
not ranked
Links from this post
33
Words captured
3,936

Cited by

Blog
In the post
Date
Modular
Modular: What about Triton and Python eDSLs? (Democratizing AI Compute, Part 7) Read at the source ↗
26 Mar 2025
Modular
Modular: How did CUDA succeed? (Democratizing AI Compute, Part 3) Read at the source ↗
12 Feb 2025

Links from this post

Link
Host
https://arxiv.org/abs/2212.13138
arxiv.org
https://thegradient.pub/state-of-ml-frameworks-2019-pytorch-dominates-research-tensorflow-dominates-industry/
thegradient.pub
https://pytorch.org/docs/stable/generated/torch.nn.LayerNorm.html
pytorch.org
https://pytorch.org/docs/stable/torch.html
pytorch.org
https://pytorch.org/docs/stable/generated/torch.nn.Softmax.html
pytorch.org
https://pytorch.org/docs/stable/generated/torch.nn.ReLU.html
pytorch.org
https://semianalysis.com/cxl-enables-microsoft-azure-to-cut
semianalysis.com
https://semianalysis.com/gpt-model-training-competition-heats
semianalysis.com
https://semianalysis.com/advanced-packaging-part-1-pad-limited
semianalysis.com
https://www.mosaicml.com/composer
mosaicml.com
https://developer.nvidia.com/blog/nvidia-hopper-architecture-in-depth/
developer.nvidia.com
https://www.nvidia.com/en-us/on-demand/session/gtcfall22-a41095/
nvidia.com
https://horace.io/brrr_intro.html
horace.io
https://ai.facebook.com/blog/pytorch-foundation/
ai.facebook.com
https://www.youtube.com/watch?v=ppWKVg-VxmQ
youtube.com
https://semianalysis.com/amd-to-infinity-and-beyond
semianalysis.com
https://semianalysis.com/intel-is-throwing-the-kitchen-sink
semianalysis.com
https://semianalysis.com/tenstorrent-blackhole-grendel-and
semianalysis.com
https://semianalysis.com/tesla-dojo-unique-packaging-and-chip
semianalysis.com
https://semianalysis.com/amazon-graviton-3-uses-chiplets-and
semianalysis.com
https://semianalysis.com/marvelldeepdive2022
semianalysis.com
https://semianalysis.com/meta-discusses-ai-hardware-and-co
semianalysis.com
https://semianalysis.com/graphcore-announces-worlds-first?s=w
semianalysis.com
https://www.youtube.com/watch?v=bGo-2xNvNAc
youtube.com
https://pytorch.org/blog/introducing-pytorch-fully-sharded-data-parallel-api/
pytorch.org
https://github.com/pytorch/tau
github.com
https://www.youtube.com/watch?v=rn-kJQ-7JmQ
youtube.com
https://arxiv.org/pdf/2112.08429.pdf
arxiv.org
https://dev-discuss.pytorch.org/t/torchdynamo-update-8-torchdynamo-passed-correctness-check-on-7k-github-models/663
dev-discuss.pytorch.org
https://pytorch.org/functorch/stable/notebooks/aot_autograd_optimizations.html
pytorch.org
https://github.com/pytorch/pytorch/blob/master/torch/_inductor/codegen/triton.py
github.com
https://github.com/HazyResearch/flash-attention/blob/main/flash_attn/flash_attn_triton.py
github.com
https://github.com/HazyResearch/flash-attention
github.com