corpus.blog
41,554 blogs · 4,830,141 posts Claim your blog
Record · corpus.blog/posts/01a0879b-306a-728d-a8ef-90e54301c64f

Generative transformer from first principles in Julia

the_origami_fox · published 23 March 2024

A record is what was published and who pointed at it. The text of the post is not held here: read it at the source, or at its Wayback capture.

Read the post at liorsinai.github.io Wayback capture
0blogs citing
First cited
—
Most recent
—
Rank this month
not ranked
External links
36
Words captured
17,006

Cited by

No blog we hold has cited this post yet.

Links from this post

Link
Host
https://github.com/LiorSinai/TransformersLite.jl
github.com
https://arxiv.org/abs/1706.03762
arxiv.org
https://web.archive.org/web/20210126024542/https://cdn.openai.com/research-covers/language-unsupervised/language_understanding_paper.pdf
web.archive.org
https://www.youtube.com/watch?v=Y2F8yisiS6E&t=1202s
youtube.com
https://medium.com/@joaolages/kv-caching-explained-276520203249
medium.com
https://en.wikipedia.org/wiki/Perplexity
en.wikipedia.org
https://karpathy.ai/zero-to-hero.html
karpathy.ai
https://www.youtube.com/watch?v=kCc8FmEb1nY
youtube.com
https://github.com/karpathy/nanoGPT
github.com
https://github.com/LiorSinai/TransformersLite-examples
github.com
https://github.com/chengchingwen/Transformers.jl
github.com
https://www.gutenberg.org/ebooks/100
gutenberg.org
https://github.com/LiorSinai/TransformersLite-Examples/blob/72d0d76256fc5b8447a84855f2eb065ef05f8b27/data/Shakespeare/prepare_shakespeare.jl
github.com
https://github.com/chengchingwen/BytePairEncoding.jl
github.com
https://www.youtube.com/watch?v=zduSFxRajkE&feature=youtu.be
youtube.com
https://github.com/FluxML/Flux.jl/blob/009d9841960ac15d9a02499ac6e341e777dedf34/src/layers/basic.jl
github.com
https://github.com/FluxML/NNlib.jl/blob/1af2535d12cfdcabc6ccd2f259968c16e84c7b81/src/gather.jl
github.com
https://arxiv.org/abs/1803.02155
arxiv.org
https://arxiv.org/pdf/1809.04281
arxiv.org
https://www.youtube.com/watch?v=XdlmDfa2hew
youtube.com
https://arxiv.org/abs/2104.09864
arxiv.org
https://arxiv.org/abs/2306.15595
arxiv.org
https://arxiv.org/abs/2009.06732
arxiv.org
https://arxiv.org/abs/2205.14135
arxiv.org
https://arxiv.org/abs/2310.01889
arxiv.org
https://en.wikipedia.org/wiki/Dot_product
en.wikipedia.org
https://github.com/FluxML/NNlib.jl/blob/07833637dec96d12d0614308d3145b432fdb320a/src/attention.jl
github.com
https://github.com/chengchingwen/NeuralAttentionlib.jl/blob/master/src/mask/dataless.jl
github.com
https://github.com/FluxML/NNlib.jl/blob/07833637dec96d12d0614308d3145b432fdb320a/src/batched/batchedmul.jl
github.com
https://github.com/FluxML/Flux.jl/blob/348c56f6172c6ce838790b0ba23c5f4c58d93b83/src/layers/basic.jl
github.com
https://github.com/FluxML/Flux.jl/blob/master/src/layers/attention.jl
github.com
https://github.com/FluxML/NNlib.jl/blob/master/src/attention.jl
github.com
https://github.com/LiorSinai/TransformersLite-Examples/blob/main/examples/GPT/generate_batches.jl
github.com
https://arxiv.org/abs/1412.6980
arxiv.org
https://github.com/LiorSinai/TransformersLite-Examples/blob/main/common/training.jl
github.com
https://en.wikipedia.org/wiki/Tensor
en.wikipedia.org