41554 blogs · 4,417,066 posts Claim your blog

Record · corpus.blog/posts/01a087e0-819f-708f-bdbd-abf07f7c5283

Notes on training BERT from scratch on an 8GB consumer GPU

montebicyclelo · published 29 May 2023

A record is what was published and who pointed at it. The text of the post is not held here: read it at the source, or at its Wayback capture.

Read the post at sidsite.com Wayback capture

0blogs citing

First cited: —
Most recent: —
Rank this month: not ranked
Links from this post: 11
Words captured: 583

Cited by

No blog we hold has cited this post yet.

Links from this post

Link

Host

https://arxiv.org/abs/1810.04805

arxiv.org

https://huggingface.co/

huggingface.co

https://github.com/sradc/pretraining-BERT/tree/main

github.com

https://huggingface.co/datasets/sradc/chunked-shuffled-wikipedia20220301en-bookcorpusopen

huggingface.co

https://github.com/sradc/pretraining-BERT/blob/main/pretraining_BERT.ipynb

github.com

https://gluebenchmark.com/

gluebenchmark.com

https://arxiv.org/abs/2212.14034

arxiv.org

https://wandb.ai/site

wandb.ai

https://s3-us-west-2.amazonaws.com/openai-assets/research-covers/language-unsupervised/language_understanding_paper.pdf

s3-us-west-2.amazonaws.com

http://arxiv.org/abs/1810.04805

arxiv.org

http://arxiv.org/abs/1706.03762

arxiv.org