corpus.blog
56,966 blogs · 4,417,066 posts Claim your blog
Record · corpus.blog/posts/01a08c55-41df-705c-913a-1abbca8b8f8d

Exploring AI CPU-Inferencing with Azure Cobalt 100

thomasvanlaere.com · published 17 October 2025

A record is what was published and who pointed at it. The text of the post is not held here: read it at the source, or at its Wayback capture.

Read the post at thomasvanlaere.com Wayback capture
0blogs citing
First cited
—
Most recent
—
Rank this month
not ranked
Links from this post
135
Words captured
15,156

Cited by

No blog we hold has cited this post yet.

Links from this post

Link
Host
https://techcommunity.microsoft.com/blog/azureinfrastructureblog/announcing-cobalt-200-azure%E2%80%99s-next-cloud-native-cpu/4469807
techcommunity.microsoft.com
https://aka.ms/Cobalt200-VM-Pr
aka.ms
https://github.com/microsoft/Build26-LAB512-hands-on-with-the-latest-cobalt-vms-from-code-to-cloud-deployment
github.com
https://newsroom.arm.com/blog/arm-official-history
newsroom.arm.com
https://en.wikipedia.org/wiki/Golden_Sun
en.wikipedia.org
https://en.wikipedia.org/wiki/HTC_Hero
en.wikipedia.org
https://en.wikipedia.org/wiki/Raspberry_Pi
en.wikipedia.org
https://www.adafruit.com/product/2733
adafruit.com
https://en.wikipedia.org/wiki/Mac_transition_to_Apple_silicon
en.wikipedia.org
https://en.wikipedia.org/wiki/Mac_transition_to_PowerPC_processors
en.wikipedia.org
https://en.wikipedia.org/wiki/Mac_transition_to_Intel_processors
en.wikipedia.org
https://learn.microsoft.com/en-us/windows/arm/overview
learn.microsoft.com
https://en.wikipedia.org/wiki/Hackintosh
en.wikipedia.org
https://www.youtube.com/watch?v=WyX8TO3awfw
youtube.com
https://en.wikipedia.org/wiki/Surface_%282012_tablet%29
en.wikipedia.org
https://en.wikipedia.org/wiki/Universal_Windows_Platform
en.wikipedia.org
https://learn.microsoft.com/en-us/windows-hardware/design/device-experiences/continuum-phone
learn.microsoft.com
https://en.wikipedia.org/wiki/Surface_Pro_X
en.wikipedia.org
https://blogs.windows.com/windowsdeveloper/2022/10/24/available-today-windows-dev-kit-2023-aka-project-volterra/
blogs.windows.com
https://www.qualcomm.com/developer/blog/2024/07/microsoft-build-2024-unleashing-the-potential-ai
qualcomm.com
https://www.windowscentral.com/hardware/qualcomm-cancels-snapdragon-x-elite-devkit-just-days-after-first-orders-arrive
windowscentral.com
https://www.qualcomm.com/products/mobile/snapdragon/laptops-and-tablets/snapdragon-x2-elite
qualcomm.com
https://en.wikipedia.org/wiki/Surface_Pro_%2811th_generation%29
en.wikipedia.org
https://learn.microsoft.com/en-us/windows/arm/apps-on-arm-x86-emulation
learn.microsoft.com
https://en.wikipedia.org/wiki/Arm_Holdings
en.wikipedia.org
https://newsroom.arm.com/news/new-arm-product-naming-architecture
newsroom.arm.com
https://newsroom.arm.com/news/announcing-lumex-css-platform-ai-era
newsroom.arm.com
https://www.arm.com/products/silicon-ip-cpu/c1-ultra
arm.com
https://www.arm.com/products/silicon-ip-cpu/c1-premium
arm.com
https://www.arm.com/products/silicon-ip-cpu/c1-pro
arm.com
https://www.arm.com/products/silicon-ip-cpu/c1-nano
arm.com
https://newsroom.arm.com/news/accelerating-the-next-generation-cloud-to-edge-infrastructure
newsroom.arm.com
https://aws.amazon.com/blogs/aws/now-available-graviton4-powered-memory-optimized-amazon-ec2-x8g-instances/
aws.amazon.com
https://cloud.google.com/products/axion
cloud.google.com
https://community.arm.com/arm-community-blogs/b/servers-and-cloud-computing-blog/posts/neoverse-compute-subsystems-css-the-fastest-path-to-production-silicon
community.arm.com
https://newsroom.arm.com/news/enabling-ai-infrastructure-on-arm
newsroom.arm.com
https://azure.microsoft.com/en-us/blog/azure-virtual-machines-with-ampere-altra-arm-based-processors-generally-available/
azure.microsoft.com
https://learn.microsoft.com/en-us/azure/virtual-machines/vm-naming-conventions
learn.microsoft.com
https://news.microsoft.com/source/features/ai/in-house-chips-silicon-to-service-to-meet-ai-demand/
news.microsoft.com
https://techcommunity.microsoft.com/blog/azureinfrastructureblog/inside-maia-100-revolutionizing-ai-workloads-with-microsofts-custom-ai-accelerat/4229118
techcommunity.microsoft.com
https://learn.microsoft.com/en-us/azure/virtual-machines/sizes/cobalt-overview
learn.microsoft.com
https://azure.microsoft.com/en-us/blog/azure-cobalt-100-based-virtual-machines-are-now-generally-available/
azure.microsoft.com
https://youtu.be/NGuVGM6amsw?feature=shared&t=292
youtu.be
https://youtu.be/7T58uxQQ1sc?si=5md_bXm8d69yAn6f&t=297
youtu.be
https://youtu.be/g6vFyuCZrVs?si=MaV8bDetWTn-gkuS&t=410
youtu.be
https://www.youtube.com/playlist?list=PLLasX02E8BPCVzPdk8e25F9XhCUmN1mbV
youtube.com
Maia 200: The AI accelerator built for inference Read at the source ↗
blogs.microsoft.com
https://learn.microsoft.com/en-us/azure/virtual-machines/sizes/general-purpose/bpsv2-series?
learn.microsoft.com
https://amperecomputing.com/blogs/Azure-Ampere-Bpsv2-Burstable-Virtual-Machines
amperecomputing.com
https://learn.microsoft.com/en-us/azure/virtual-machines/sizes/general-purpose/dpsv5-series
learn.microsoft.com
https://learn.microsoft.com/en-us/azure/virtual-machines/sizes/general-purpose/dpdsv6-series
learn.microsoft.com
https://learn.microsoft.com/en-us/azure/virtual-machines/sizes/general-purpose/dpldsv5-series
learn.microsoft.com
https://learn.microsoft.com/en-us/azure/virtual-machines/sizes/general-purpose/dpldsv6-series
learn.microsoft.com
https://learn.microsoft.com/en-us/azure/virtual-machines/sizes/general-purpose/dplsv5-series
learn.microsoft.com
https://learn.microsoft.com/en-us/azure/virtual-machines/sizes/general-purpose/dplsv6-series
learn.microsoft.com
https://learn.microsoft.com/en-us/azure/virtual-machines/sizes/general-purpose/dpsv6-series
learn.microsoft.com
https://learn.microsoft.com/en-us/azure/virtual-machines/sizes/memory-optimized/epdsv5-series
learn.microsoft.com
https://learn.microsoft.com/en-us/azure/virtual-machines/sizes/memory-optimized/epdsv6-series
learn.microsoft.com
https://learn.microsoft.com/en-us/azure/virtual-machines/sizes/memory-optimized/epsv5-series
learn.microsoft.com
https://learn.microsoft.com/en-us/azure/virtual-machines/sizes/memory-optimized/epsv6-series
learn.microsoft.com
https://learn.microsoft.com/en-us/azure/virtual-machines/trusted-launch
learn.microsoft.com
https://newsroom.arm.com/blog/arm-at-computex-2024
newsroom.arm.com
https://www.arm.com/markets/artificial-intelligence/software/kleidi
arm.com
https://onnxruntime.ai/blogs/arm-microsoft-kleidiai
onnxruntime.ai
https://github.com/ggml-org/llama.cpp/blob/40be51152d4dc2d47444a4ed378285139859895b/docs/build.md
github.com
https://github.com/vllm-project/vllm/blob/017354c0ef06c0d3bdec88ebf94e52f1c79edf97/vllm/model_executor/layers/quantization/kernels/mixed_precision/dynamic_4bit.py
github.com
https://newsroom.arm.com/news/pytorch-kleidi-integrations-cloud-to-edge
newsroom.arm.com
https://www.arm.com/products/compute-subsystems-for-client
arm.com
https://developer.arm.com/documentation/ddi0602/2025-06/SIMD-FP-Instructions/SMMLA--vector---Signed-8-bit-integer-matrix-multiply-accumulate-to-32-bit-integer--vector--?lang=en
developer.arm.com
https://community.arm.com/arm-community-blogs/b/ai-blog/posts/optimize-llama-cpp-with-arm-i8mm-instruction
community.arm.com
https://www.nvidia.com/en-in/data-center/tensor-cores/
nvidia.com
https://kib.kiev.ua/x86docs/ARM/SVE/DDI0584A_g_SVE_supp_armv8A.pdf
kib.kiev.ua
https://community.arm.com/arm-community-blogs/b/architectures-and-processors-blog/posts/neoverse-v1-platform-a-new-performance-tier-for-arm
community.arm.com
https://www.ibm.com/think/topics/quantization
ibm.com
https://www.deeplearning.ai/the-batch/microsoft-researchers-show-that-heavily-quantized-versions-of-llama-can-perform-as-well-as-near-full-precision/
deeplearning.ai
https://github.com/ggml-org/llama.cpp/releases
github.com
https://huggingface.co/microsoft/phi-4
huggingface.co
https://huggingface.co/collections/microsoft/phi-4-677e9380e514feb5577a40e4
huggingface.co
https://pypi.org/project/huggingface-hub/
pypi.org
https://azure.microsoft.com/en-us/blog/empowering-innovation-the-next-generation-of-the-phi-family/
azure.microsoft.com
https://huggingface.co/microsoft/Phi-4-multimodal-instruct/blob/main/configuration_phi4mm.py
huggingface.co
https://huggingface.co/microsoft/phi-4-gguf
huggingface.co
https://huggingface.co/microsoft/phi-4-gguf?show_file_info=phi-4-Q4_K_S.gguf
huggingface.co
https://github.com/ggml-org/llama.cpp/pull/1684
github.com
https://app.semanticdiff.com/gh/ggerganov/llama.cpp/pull/5590/overview
app.semanticdiff.com
https://huggingface.co/docs/hub/en/gguf
huggingface.co
https://community.arm.com/arm-community-blogs/b/servers-and-cloud-computing-blog/posts/neoverse-n2-delivers-leading-price-performance-on-asr
community.arm.com
https://manpages.debian.org/testing/llama.cpp-tools/llama-cli.1.en.html
manpages.debian.org
https://huggingface.co/docs/text-generation-inference/en/conceptual/flash_attention
huggingface.co
https://man7.org/linux/man-pages/man2/mlock.2.html
man7.org
https://www.databricks.com/blog/llm-inference-performance-engineering-best-practices
databricks.com
A Gentle Introduction to 8-bit Matrix Multiplication for transformers at scale using transformers, accelerate and bitsandbytes Read at the source ↗
huggingface.co
https://manpages.debian.org/testing/llama.cpp-tools/llama-bench.1.en.html
manpages.debian.org
https://github.com/ggml-org/llama.cpp/tree/master/tools/batched-bench
github.com
https://community.arm.com/arm-community-blogs/b/servers-and-cloud-computing-blog/posts/running-llama-3-70b-on-aws-graviton4
community.arm.com
https://huggingface.co/QuantFactory/Meta-Llama-3-8B-Instruct-GGUF?show_file_info=Meta-Llama-3-8B-Instruct.Q4_0.gguf
huggingface.co
https://huggingface.co/SanctumAI/Llama-3.2-3B-Instruct-GGUF
huggingface.co
https://github.com/ggerganov/llama.cpp
github.com
https://builder.aws.com/content/2pViH36qEfqDYwv0kEysJ0kqDp4/small-language-models-slms-inference-with-llamacpp-on-graviton4
builder.aws.com
https://learn.microsoft.com/en-us/azure/virtual-machines/sizes/gpu-accelerated/ncast4v3-series
learn.microsoft.com
https://learn.microsoft.com/en-us/azure/virtual-machines/sizes/gpu-accelerated/nca100v4-series
learn.microsoft.com
https://learn.microsoft.com/en-us/azure/virtual-machines/sizes/gpu-accelerated/ncadsh100v5-series
learn.microsoft.com
https://ai.azure.com/catalog/models/Phi-4
ai.azure.com
https://github.com/ggml-org/llama.cpp/discussions/15013
github.com
https://github.com/vllm-project/vllm/issues/9990
github.com
https://www.intel.com/content/www/us/en/foundry/process/18a.html
intel.com
https://www.theregister.com/2023/06/21/amd_ampere_cloud_core_competition/
theregister.com
https://research.cs.wisc.edu/vertical/papers/2013/hpca13-isa-power-struggles.pdf
research.cs.wisc.edu
https://chipsandcheese.com/p/arm-or-x86-isa-doesnt-matter
chipsandcheese.com
https://en.wikipedia.org/wiki/AMD_APU
en.wikipedia.org
https://www.intel.com/content/www/us/en/foundry/overview.html
intel.com
https://newsroom.intel.com/intel-foundry/foundry-news-roadmaps-updates
newsroom.intel.com
https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32022L2464
eur-lex.europa.eu
https://energy.ec.europa.eu/topics/energy-efficiency/energy-efficiency-targets-directive-and-rules/energy-efficiency-directive_en
energy.ec.europa.eu
https://energy-efficient-products.ec.europa.eu/product-list/servers-and-data-storage-products_en
energy-efficient-products.ec.europa.eu
https://www.twobirds.com/en/insights/2023/global/sustainability-and-the-eu-taxonomy
twobirds.com
https://publications.jrc.ec.europa.eu/repository/handle/JRC141521
publications.jrc.ec.europa.eu
https://eur-lex.europa.eu/eli/dir/2024/1760/oj
eur-lex.europa.eu
https://learn.microsoft.com/en-us/azure/carbon-optimization/overview
learn.microsoft.com
https://riscv.org
riscv.org
https://riscv.org/about/faq/
riscv.org
https://github.com/ThomVanL/blog-2025-10-exploring-ai-cpu-inferencing-with-azure-cobalt-100/
github.com
https://learn.microsoft.com/en-us/azure/aks/ai-toolchain-operator
learn.microsoft.com
https://arxiv.org/abs/2504.18415
arxiv.org
https://www.maartengrootendorst.com/blog/quantization
maartengrootendorst.com
https://www.maartengrootendorst.com/book/
maartengrootendorst.com
https://jalammar.github.io/illustrated-transformer/
jalammar.github.io
https://lilianweng.github.io
lilianweng.github.io
https://developer.nvidia.com/blog/mastering-llm-techniques-inference-optimization/
developer.nvidia.com
https://arxiv.org/pdf/2501.00032
arxiv.org
https://github.com/aws/aws-graviton-getting-started/blob/d505a4161316a386c82a8a21d548917b48a6b0f6/machinelearning/llama.cpp.md?plain=1
github.com
https://community.arm.com/arm-community-blogs/b/servers-and-cloud-computing-blog/posts/ai-inference-on-google-axion-cpu
community.arm.com
https://community.arm.com/arm-community-blogs/b/servers-and-cloud-computing-blog/posts/accelerated-llm-inference-on-arm-neoverse-n2
community.arm.com
https://newsroom.arm.com/news/ai-inference-everywhere-with-new-llama-llms-on-arm
newsroom.arm.com
https://www.arcee.ai/blog/the-case-for-small-language-model-inference-on-arm-cpus
arcee.ai