Record · corpus.blog/posts/01a087dd-c181-702f-96ef-cec88f5be523

Welcome to the world of AI - Learning about the Decoder-Only Transformer - From scratch with NumPy

securitynik.com · published 7 March 2026

A record is what was published and who pointed at it. The text of the post is not held here: read it at the source, or at its Wayback capture.

Read the post at securitynik.com Wayback capture

0blogs citing

First cited: —
Most recent: —
Rank this month: not ranked
Links from this post: 30
Words captured: 9,641

Cited by

No blog we hold has cited this post yet.

Links from this post

Link

Host

https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjPUhCfWoCmnw-7EQpyUkJ3r_GgYHp0HZfqZqmCgr1b9ECKBcX10NuJRxzHHimIRHCU9iWu4PbwMATGnrlbD4MoqDh6E505-tDq3WyJwQfzXsUY0KXae8BGDWRCmkdEUzXckn51GRhJHJdFOxsf82HfIK48BR0KeXxXve_v-Q15yz_O4cbunjSkH-lr0OI/s1032/pandas-stoi.jpg

blogger.googleusercontent.com

https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhnEIurh1WpOQnQ77uWimbwgsdfv2WNtQ2LbAhY-JxelbQZn2EwwNE29Epks-s70-Zapm4W1r7DZ8CNmjCTAOrduejl1_h_wwhIfFxQxg4J4tTUzIH1L_KPfsDOT6eA0W99Q2U79oswKSxk96ehrNsvYo6GAuFs7zuj8QgS-NjE8vzcQz0u7lUzHwsiNrc/s1050/pandas%20-%20itos.jpg

blogger.googleusercontent.com

https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh-gP7QHXSqE1gjadpE2rpQJEPQI6AEHu2j7wsTZu5GsxZ9VU4C81yfHetqibJ3hIhSPBA7nql0b4Hzsu3nKrKDJN2URtIebJr5O5GxHA5GeYgzZy8dJKHJ97a3e5dfQRS6vF9ZGGMc2lC5VRuogLIDFGK5cIyRECQBPbIJ9OshAi8l0eAXoJ82Yn3g9TA/s2582/token%20plot%20indiies.jpg

blogger.googleusercontent.com

https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhyWwAIGZcC15urH4LwBV-jmSzd_fXfB-2TwZrrNnQ794lZmComxsJWF42iMGWvN-4aXLQR01CKeUe8tyEsoH6cf9J4VnPNGWCskpTbZK1PY8_hIpuEubUG15M6LRV2KhFhqDpjWPBd4PfT5VwiERhzC0u1MMWdW8sb8fNYNx-gOz-ZsrgqR9fqEHv_rA0/s1358/softmax-sensitivity-to-temperature.jpg

blogger.googleusercontent.com

https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgC3NckURpex1K9llHDTqgtjoyeMef9sSsVLVqbPMi8kqZXgSR0qU1c6keB_vziFloE9A22ldQeD6z3SOms3l_H2zmNgkSDVIsbgq6kvTpjNlAaSQaGEBzYUlqXNpwdpDVVVzypg8RFP5L8L_FHvshfuA1Bo55Yn_NafumVcjQGiE80LVvuJgpV69alt9s/s2382/layernorm.jpg

blogger.googleusercontent.com

https://www.cs.toronto.edu/~rsalakhu/papers/srivastava14a.pdf

cs.toronto.edu

https://arxiv.org/pdf/1606.08415

arxiv.org

https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj8Y3nAfVc9XkPQFpcSKsxzmg9sET5dy2U6h9djXUDNvv_H61c0ufPxmUENg7UG_CoMRwqQSzxdkVvMdBo0Ns2DgmXxmbY08GcTRSV6fTVcAGb6qNayOmUVnB7Aecl8wTnia_hQiDhMpKYLM2e758uu-I0q6mqF0AvHALmgAikVGibkzUut8dlLWIWsOSs/s1634/GELU.jpg

blogger.googleusercontent.com

https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiVpS70MWdzkQsxn_VpUxamyhswanm9I-YCrTVJqR8VP0ay-PrvuoEWu9yQHbYT0W3phpRypdW0usGqwUt_p9o21KhnPhQ-lHWprUs9gBJdPHbHaHtxDd1hhuST2EUsC8QQyNhMQnFVn6NMJtz-YNkP7qVAhxanMgbRhRvirOODy35A09L4_MsQv7xCiUQ/s1622/learned%20positional%20embeddings.jpg

blogger.googleusercontent.com

https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjuRR2QBRRvdPR0_5GAGWoJyrE8tMU8Nr13ilJF9WSvmm7KGbiyNyVJF5Q-_gMkCN42ofA_G3JBC8-82gQlH0BLT70WNYxTJVTzeJlKLtvubgZCzhtQT8PHIu5toJI7mCfl1xagbKd3ALarm4iumGWl_52b44cD1nPjYV_iOst2Ndsl-6MF-uwpseDaSXk/s1718/tokens%20vs%20position%20norms.jpg

blogger.googleusercontent.com

https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhg9-2C4H6QbwWOlme4pS3Bkkcj59foDiV3lMd_2pAf12b6eEx4oLqqgCgtw_vkKWnBdD5MSmE8yFPfG49BCJivsujxxLjVjxX9oif9C4ryyg4mNa1LVbrvEjoQDcXZIYB_16Bb_c4S67ODIRev3VH-urNj48AdKsLcJi0tfwOK5xt2sMAe6gPYzi5vgRg/s2520/pca.jpg

blogger.googleusercontent.com

https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjeSToyauTbKXktwLa13lhQb8lncExyLM8rY1YLWhpfM0PPRmDORt9ZexjYX0z1v1zNqAN290fhVlhRgBb1Q3Y9wsq5SOOjbNeMt4jxshzjo5RMiF3gDo2JHMKibQttWNd5EBmlyUTZ60J4CsYDGSdI0XLqDYTxetnE9JX48jPTQf5XkCG4EMj5NbKpdlE/s2548/query%20-%20key%20plots.jpg

blogger.googleusercontent.com

https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEifuJn-ARcuV_Zsy6IMoKeHT39-mIK350LrlLWz-PMECjyT7RFS1RkKpwR3Q7E9-fWQ70dppJQ0MKVZS8u5VM40TvK5hGQOyajeGuUGIE6SuZEvCXchZCMlbL-0Op87tATkRz3-Fgxi57b3skOS0iL7Qg9lzj72-N6yrnPaH1E9gO4moWNc9E1_1hZzut8/s2066/attn%20distribution%20for%20positions.jpg

blogger.googleusercontent.com

https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj3gd9Kv7AxfVMK1J6pXj6lcv971ObU7LEXBp56om85OflWciEdbZNA_ngd0zFwKEJOZuqrX2HZPnfbP0eYsZgF-s5u-vKqiba17_S-kdLVQSEgeNyqH3GQmb4mamnnAa_ZFh-5RZ_si2uLlCh1KNEw2NzNoGlUVIcaCTcDcna5X18L8iWaIjlU74TGllQ/s1414/norms.jpg

blogger.googleusercontent.com

https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhyf424_cKupVI6VAxMY3oLpxNS4Au41whMxwXzGjkK6LXPRhvKRyGxhj5Y2MKE4f1IBhOkAVJMlRzmPt9k-hWf4TTly8UEEwmVomYShDo4ZHif2bAft_FulYajsCzy67qzC6UdNVDS4w0ag8fFBIL6YgPGDKPpheO5K1HaYknMxI_wdp-4qkcdQAF5edU/s2542/heads%20visualized.jpg

blogger.googleusercontent.com

https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgeDns1emeuHPvBeMhvyE1WE1k2VShGaumx2mKqpsu40jpUM3_birti53FLs9A8qe2krQbZkUyLE3mJKMcVpLWDImZ_asMK4t1ZgWUBNgHeQwLfLz_YaDumaFmZS8oNC4g-oJ8lDlcNykcxO7e2IVtNQ8BzNwzfwHBXjVWYFIIcQ7GyZMkG4rvfiqpZSt4/s1440/variance%20mha.jpg

blogger.googleusercontent.com

https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjYLQ9Vuc88zNGxOcqT5ozO7ikXqr0TzTGetSeIFqy1BMoEwV5y3tjuaqOcxmWH_U3oCL9XTJG6wVLmB3j1weH1ntOSCh5bWBYbRtrlmXt5P_ZuoYJK5SK4dyqwyMdTwHn0ri2SiZjZoCUMO2_M6HxGS6XShA9DsbhzxTYebr9Sp1ecP2p5J-wxtp9H-_s/s2402/gelu%20-%20pre%20and%20post.jpg

blogger.googleusercontent.com

https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg99ivf6mfuhLKbw9iue9uOUbsXzxkKpp4Ti4SDLzt_h8sGYt4T1348xU2m3JPRjp568n5ig1I4-crfqF4ajq5iB6UohDdOsXhngxJV5rRL6oiNZl5cZgHSxe_NAlCU7HDTD7BRKY67vsyZJgrh6T1BC2DBrUoTZmiUOt_LFfQFoKeTzvQlku8pVZZtYzw/s1472/ffn%20plot.jpg

blogger.googleusercontent.com

https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEigth_so9dlAw8fiTH8X1H90NsrlU0CziX-38DngO2v-d-02nrcSRrJOYqhlnjKNbO4t4C0glmVfCy14yYIwckrDCzehfBznCFiBSZ7ImjXlRaZQjdstJLD3b0bZiVJp6JbRcksw0wazUMvCDEuEx1DI_s2-8WSzkeIJCqd6aSqFYThMMU2KrYbffmkik0/s1424/norms%20vs%20mha%20out.jpg

blogger.googleusercontent.com

https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiEM0tHQ9GHFAjSWwiVioj17Pqr_wUyHoTRRpwb1OInkrhtd69empfIEbQK0uCVIt_z2agqnEAiyo2N5paOK8lf8HykTgQ68Qw96-cJzsng8piTxJrRaJAAy_BQ8f_9eAwexf_sJhix5fKT5nzRnZkLrcGJiQUyUbKok-08qA7dkDiPolm7MEh1g_YG80A/s1450/decoder%20stack%20-%20norms.jpg

blogger.googleusercontent.com

https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgZSn_NWn6TkpG_RgeqLOKA2IFhQ21xWJjJfkzX9Qyhr99z6kNtXqPzZKqSktdqnKrr9WPDSH8rShM_IYpi2osf-QmGYQestm_8HcSVZYGY3kAwRXLJcpFOqZ2addp8t1TYSGwijFcDSnML4kL1NKB5PTZI2f4kz-SoqEixNLI2tb3z8uoWivSFAPXD9hg/s1412/mha%20-%20ffn%20norm.jpg

blogger.googleusercontent.com

https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiABym4u57WyBCii8Z5UceUTbTm9OC2Oi-0skYcUckLa0Gg6U4ljXZahlhqwkxKFNeNauY3dItcMIjWC3dpBPyH1VrNaTxw7qN6w5eB3iFcZOjPhyphenhyphenaP6x7CzHKg4zmJ2zJ4kXmond_mU1_trCLjmejVWtx2qkkSEmnp-MXvxSRq4tUYUEjEB8FTb2uAL_Y/s1456/residual%20stream.jpg

blogger.googleusercontent.com

https://www.youtube.com/playlist?list=PLAqhIrjkxbuWI23v9cThsA9GvCAUhRvKZ

youtube.com

Welcome to the world of AI - Understanding temperature, top_p and top_k Read at the source ↗

securitynik.com

https://github.com/SecurityNik/Data-Science-and-ML/blob/main/llm/temperature_top_p_and_top_k.ipynb

github.com

https://github.com/SecurityNik/Data-Science-and-ML/blob/main/llm/continuing-transformers.ipynb

github.com

Welcome to the world of AI - Learning about the Decoder-Only transformer - From scratch with PyTorch Read at the source ↗

securitynik.com

https://github.com/SecurityNik/Data-Science-and-ML/blob/main/llm/continuing-transformers-baby-names.ipynb

github.com

Welcome to the world of AI - Putting it all together. Building and training fully functional Decoder-Only transformer Read at the source ↗

securitynik.com

https://github.com/SecurityNik/Data-Science-and-ML/blob/main/llm/baby_name_gpt.py

github.com