corpus.blog
41,554 blogs · 5,038,968 posts Claim your blog
Record · corpus.blog/posts/01a087de-9145-73ad-b82f-d63f436028a9

Continual training of Llama-3.1-8B for 809B tokens

severelytheoretical.wordpress.com · published 21 April 2025

A record is what was published and who pointed at it. The text of the post is not held here: read it at the source, or at its Wayback capture.

Read the post at severelytheoretical.wordpress.com Wayback capture
0blogs citing
First cited
—
Most recent
—
Rank this month
not ranked
External links
14
Words captured
1,946

Cited by

No blog we hold has cited this post yet.

Links from this post

Link
Host
https://huggingface.co/meta-llama/Llama-3.1-8B
huggingface.co
https://twitter.com/ml_perception/status/1781021776597942587?ref_src=twsrc%5Etfw
twitter.com
https://en.wikipedia.org/wiki/Frontier_(supercomputer
en.wikipedia.org
https://docs.olcf.ornl.gov/systems/frontier_user_guide.html
docs.olcf.ornl.gov
https://pytorch.org/tutorials/recipes/distributed_device_mesh.html
pytorch.org
https://github.com/pytorch/torchtitan
github.com
https://github.com/eminorhan/frontier-torchtitan
github.com
https://huggingface.co/datasets/Zyphra/Zyda-2
huggingface.co
https://huggingface.co/datasets/bigcode/the-stack-v2-train-smol-ids
huggingface.co
https://huggingface.co/datasets/HuggingFaceTB/finemath
huggingface.co
https://huggingface.co/eminorhan/smoky-llama/tree/main
huggingface.co
https://github.com/eminorhan/frontier-torchtitan?tab=readme-ov-file
github.com
MI300X vs H100 vs H200 Benchmark Part 1: Training – CUDA Moat Still Alive Read at the source ↗
semianalysis.com
https://t.co/r0uBJwQEbC
t.co
Elsewhere on severelytheoretical.wordpress.com (1)