Skip to content
corpus.blog
Sign in Claim your blog
Record · research.google

Constructing Transformers For Longer Sequences with Sparse Attention Methods

Google Research · 14 July 2026 · 7 min read

A post on research.google, published 14 July 2026, has not been cited by any blog yet (corpus.blog, measured 13 October 2026).

A record is what was published and who pointed at it. The text of the post is not shown here: read it at the source, or at its Wayback capture.

Read at research.google (opens in a new tab) Wayback capture (opens in a new tab)

0blogs citing

Citing blogs and links by period
Period Blogs Links
All time00
Last 90 days00
Last 30 days00
Last 7 days00
Figures for this page
Words blogs link it with0
External links50
Length1,599

Cited by

No blog we hold has cited this post yet.

Links from this post

Link
Host
https://en.wikipedia.org/wiki/Natural_language_processing Who cites this page
en.wikipedia.org
Transformer: A Novel Neural Network Architecture for Language Understanding Read at ai.googleblog.com (opens in a new tab)
ai.googleblog.com
Open Sourcing BERT: State-of-the-Art Pre-training for Natural Language Processin Read at ai.googleblog.com (opens in a new tab)
ai.googleblog.com
https://arxiv.org/pdf/1907.11692 Who cites this page
arxiv.org
Exploring Transfer Learning with T5: the Text-To-Text Transfer Transformer Read at ai.googleblog.com (opens in a new tab)
ai.googleblog.com
https://arxiv.org/abs/2005.14165 Who cites this page
arxiv.org
https://en.wikipedia.org/wiki/Seq2seq Who cites this page
en.wikipedia.org
https://ai.googleblog.com/2019/10/exploring-massively-multilingual.html Who cites this page
ai.googleblog.com
https://arxiv.org/pdf/1906.04165 Who cites this page
arxiv.org
https://arxiv.org/pdf/1911.03829 Who cites this page
arxiv.org
https://arxiv.org/pdf/1903.09588 Who cites this page
arxiv.org
https://arxiv.org/pdf/1911.03894 Who cites this page
arxiv.org
https://arxiv.org/pdf/1908.08167 Who cites this page
arxiv.org
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.676.4320&rep=rep1&type=pdf Who cites this page
citeseerx.ist.psu.edu
https://lilianweng.github.io/lil-log/2018/06/24/attention-attention.html Who cites this page
lilianweng.github.io
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.90.7842&rep=rep1&type=pdf Who cites this page
citeseerx.ist.psu.edu
https://arxiv.org/pdf/1804.05685 Who cites this page
arxiv.org
https://www.biorxiv.org/content/10.1101/353474v3.full Who cites this page
biorxiv.org
https://arxiv.org/abs/2004.08483 Who cites this page
arxiv.org
https://2020.emnlp.org/
2020.emnlp.org
https://arxiv.org/abs/2007.14062 Who cites this page
arxiv.org
https://nips.cc/Conferences/2020/ Who cites this page
nips.cc
https://en.wikipedia.org/wiki/Question_answering Who cites this page
en.wikipedia.org
https://en.wikipedia.org/wiki/Automatic_summarization Who cites this page
en.wikipedia.org
https://en.wikipedia.org/wiki/Graph_(discrete_mathematics Who cites this page
en.wikipedia.org
https://en.wikipedia.org/wiki/Complete_graph Who cites this page
en.wikipedia.org
https://arxiv.org/abs/1803.02155 Who cites this page
arxiv.org
https://nlp.cs.washington.edu/triviaqa/ Who cites this page
nlp.cs.washington.edu
https://www.mitpressjournals.org/doi/full/10.1162/tacl_a_00276
mitpressjournals.org
https://arxiv.org/pdf/1809.09600 Who cites this page
arxiv.org
https://www.mitpressjournals.org/doi/pdf/10.1162/tacl_a_00021
mitpressjournals.org
https://arxiv.org/pdf/1911.02671 Who cites this page
arxiv.org
https://en.wikipedia.org/wiki/F-score Who cites this page
en.wikipedia.org
https://en.wikipedia.org/wiki/Turing_completeness Who cites this page
en.wikipedia.org
https://arxiv.org/pdf/1912.10077 Who cites this page
arxiv.org
https://arxiv.org/pdf/1901.03429 Who cites this page
arxiv.org
https://en.wikipedia.org/wiki/Universal_approximation_theorem Who cites this page
en.wikipedia.org
https://arxiv.org/abs/1604.06174 Who cites this page
arxiv.org
https://en.wikipedia.org/wiki/ROUGE_(metric Who cites this page
en.wikipedia.org
https://arxiv.org/abs/1906.03741 Who cites this page
arxiv.org
https://arxiv.org/abs/1804.05685 Who cites this page
arxiv.org
https://www.frontiersin.org/articles/10.3389/fgene.2019.00286/full Who cites this page
frontiersin.org
https://www.nature.com/articles/nmeth.3547 Who cites this page
nature.com
http://en.wikipedia.org/wiki/SIMD Who cites this page
en.wikipedia.org
https://arxiv.org/abs/2011.04006 Who cites this page
arxiv.org
https://github.com/google-research/bigbird/blob/master/README.md Who cites this page
github.com
https://blog.tensorflow.org/2020/08/introducing-tensorflow-coder-tool.html Read at blog.tensorflow.org (opens in a new tab)
blog.tensorflow.org
https://ai.googleblog.com/2021/03/progress-and-challenges-in-long-form.html Who cites this page
ai.googleblog.com
http://goo.gle/research-etc-model
goo.gle
https://github.com/google-research/bigbird Who cites this page
github.com

Similar posts

Attention Is All You Need
jameshoward.us
1 Jul 2026
What Is Sub-Quadratic Sparse Attention? How SubQ's 12M Token Context Works
mindstudio.ai
20 Jun 2026
Nous Research Proposes Lighthouse Attention: A Training-Only Selection-Based Hierarchical Attention That Delivers 1.4–1.7× Pretraining Speedup at Long Context
marktechpost.com
16 May 2026
What Is Sub-Quadratic Sparse Attention? How SubQ's SSA Architecture Changes Long-Context AI
mindstudio.ai
19 Jun 2026
GLM 5.2 Architecture Deep Dive: Index Share, Sparse Attention, and Multi-Token Prediction
mindstudio.ai
26 Jun 2026
The Transformer
zackproser
18 Jul 2026
The Attention Mechanism: Why Your LLM Forgets the Middle
duncanleung.com
10 Aug 2026
Chapter 9: Ultra-Long Context Mastery: Dual-Chunk Attention, YaRN & Sparse Routing
kenhuangus.substack.com
20 Sept 2026
Inventing Transformers
alexcbecker.net
30 May 2026
Linear Attention, Visualized
snowchord.com
25 Jul 2026

Something wrong here? Report a problem