corpus.blog
41554 blogs · 4,417,066 posts Claim your blog
Record · corpus.blog/posts/01a08756-1e96-73f9-9eb0-bb1c2a5d1006

Building a web search engine from scratch in two months with 3 billion neural embeddings

blog.wilsonl.in · published 10 August 2025

A record is what was published and who pointed at it. The text of the post is not held here: read it at the source, or at its Wayback capture.

Read the post at blog.wilsonl.in Wayback capture
8blogs citing
First cited
16 Aug 2025
Most recent
20 May 2026
Rank this month
not ranked
Links from this post
127
Words captured
9,758

Cited by

Blog
In the post
Date
turbopuffer
Training SID-1 to beat GPT-5 at search with 1k+ QPS RL Read at the source ↗
20 May 2026
Jonathon Ready
I Built a Geocoder for AI Agents Because I Couldn't Afford Google Maps Read at the source ↗
14 Jan 2026
i5heu
Building a web search engine from scratch in two months with 3 billion neural embeddings Read at the source ↗
17 Nov 2025
kipply
Digest | Jul/Aug/Sep/Oct 2025 Read at the source ↗
1 Nov 2025
flyingcroissant.ca
Sketch: Using RSS, LLMs, and Semantic Search to Follow News Topics Read at the source ↗
1 Oct 2025
Architecture Notes
Arc Notes Weekly #100: Hundo Read at the source ↗
24 Aug 2025
bneo.xyz
new chair Read at the source ↗
16 Aug 2025
Sean's Blog
Search is hard Read at the source ↗
16 Aug 2025

Links from this post

Link
Host
https://blog.wilsonl.in//serp-rocksdb.png
blog.wilsonl.in
https://en.wikipedia.org/wiki/Long_tail
en.wikipedia.org
https://blog.wilsonl.in//serp-paragraph-cropped.png
blog.wilsonl.in
https://huggingface.co/sentence-transformers/multi-qa-mpnet-base-dot-v1
huggingface.co
https://en.wiktionary.org/wiki/needle_in_a_haystack
en.wiktionary.org
https://html.spec.whatwg.org/multipage/
html.spec.whatwg.org
https://www.nngroup.com/articles/browser-and-gui-chrome/
nngroup.com
https://developer.mozilla.org/en-US/curriculum/core/semantic-html/
developer.mozilla.org
https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA/Reference/Roles
developer.mozilla.org
https://developer.chrome.com/docs/chromium/headless
developer.chrome.com
https://fasttext.cc/
fasttext.cc
https://en.wikipedia.org
en.wikipedia.org
https://github.com/WebKit/WebKit/blob/main/Source/WebCore/page/Quirks.cpp
github.com
https://ogp.me/
ogp.me
https://schema.org/docs/gs.html
schema.org
https://schema.org/docs/full.html
schema.org
https://developers.google.com/search/docs/appearance/structured-data/search-gallery
developers.google.com
https://en.wikipedia.org/wiki/Knowledge_Graph_(Google
en.wikipedia.org
https://en.wikipedia.org/wiki/Steve_Jobs_(film
en.wikipedia.org
https://en.wikipedia.org/wiki/Steve_Jobs_(book
en.wikipedia.org
https://en.wikipedia.org/wiki/Steve_Jobs
en.wikipedia.org
https://en.wikipedia.org/wiki/Ontology
en.wikipedia.org
Chunking Strategies for LLM Applications Read at the source ↗
pinecone.io
https://jina.ai/news/long-context-embedding-models-are-blind-beyond-4k-tokens/
jina.ai
https://spacy.io/api/sentencizer
spacy.io
https://huggingface.co/distilbert/distilbert-base-uncased
huggingface.co
https://blog.wilsonl.in//statement-labeller.png
blog.wilsonl.in
https://blog.wilsonl.in//admin-statement-chain-debug.png
blog.wilsonl.in
https://www.anthropic.com/news/contextual-retrieval
anthropic.com
https://jina.ai/news/late-chunking-in-long-context-embedding-models/
jina.ai
https://docs.aws.amazon.com/AmazonS3/latest/userguide/mpuoverview.html
docs.aws.amazon.com
https://blog.wilsonl.in//poc-when-should-i-use-multipart-uploads.png
blog.wilsonl.in
https://www.psychologytoday.com/us/blog/understanding-the-anxious-mind/202303/are-you-a-life-optimizer-what-to-do-about-perfectionism
psychologytoday.com
https://blog.wilsonl.in//poc-is-perfectionism-worth-it.png
blog.wilsonl.in
https://blog.wilsonl.in//poc-im-charged-for-invisible-space.png
blog.wilsonl.in
https://blog.wilsonl.in//poc-race-conditions.png
blog.wilsonl.in
https://blog.wilsonl.in//poc-can-i-use-lua.png
blog.wilsonl.in
https://blog.wilsonl.in//poc-what-do-i-pay-for.png
blog.wilsonl.in
https://blog.wilsonl.in//poc-how-can-i-attach-some-human-english-comment-to-a-file.png
blog.wilsonl.in
https://blog.wilsonl.in//poc-can-i-upload-without-knowing-size-ahead-of-time.png
blog.wilsonl.in
https://blog.wilsonl.in//poc-can-uploads-be-interrupted.png
blog.wilsonl.in
https://blog.wilsonl.in//poc-what-permissions-do-i-need-to-upload.png
blog.wilsonl.in
https://en.wikipedia.org/wiki/Slowloris_(cyber_attack
en.wikipedia.org
https://blog.wilsonl.in//multi-level-crawl-queues.png
blog.wilsonl.in
https://publicsuffix.org/list/
publicsuffix.org
https://en.wikipedia.org/wiki/Hostname
en.wikipedia.org
https://en.wikipedia.org/wiki/C0_and_C1_control_codes
en.wikipedia.org
https://www.postgresql.org/docs/current/datatype-character.html
postgresql.org
https://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/API_SendMessage.html
docs.aws.amazon.com
https://blog.wilsonl.in//pipeline.png
blog.wilsonl.in
https://doc.rust-lang.org/std/vec/struct.Vec.html
doc.rust-lang.org
https://en.wikipedia.org/wiki/String_interning
en.wikipedia.org
https://en.wikipedia.org/wiki/Zstd
en.wikipedia.org
https://en.wikipedia.org/wiki/Trie
en.wikipedia.org
https://aws.amazon.com/sqs/pricing/
aws.amazon.com
https://github.com/wilsonzlin/queued
github.com
https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-visibility-timeout.html
docs.aws.amazon.com
https://www.oracle.com/cloud/networking/virtual-cloud-network/pricing/
oracle.com
https://www.oracle.com/cloud/pricing/
oracle.com
https://docs.aws.amazon.com/AmazonS3/latest/userguide/optimizing-performance.html
docs.aws.amazon.com
https://www.postgresql.org/docs/current/storage-toast.html
postgresql.org
https://www.tigerdata.com/blog/boosting-postgres-insert-performance
tigerdata.com
https://www.sqlite.org/atomiccommit.html
sqlite.org
https://www.citusdata.com/
citusdata.com
https://www.postgresql.org/docs/current/mvcc.html
postgresql.org
https://wiki.postgresql.org/wiki/Full_page_writes
wiki.postgresql.org
https://rocksdb.org/
rocksdb.org
https://github.com/facebook/rocksdb/wiki/Memtable
github.com
https://docs.rs/rocksdb/latest/rocksdb/
docs.rs
https://github.com/facebook/rocksdb/wiki/Iterator
github.com
https://github.com/facebook/rocksdb/wiki/BlobDB
github.com
https://github.com/facebook/rocksdb/wiki/Compaction
github.com
https://en.wikipedia.org/wiki/Consistent_hashing
en.wikipedia.org
https://xxhash.com/
xxhash.com
https://msgpack.org/index.html
msgpack.org
https://github.com/djkoloski/rust_serialization_benchmark
github.com
https://coredns.io/
coredns.io
https://www.beyondcorp.com/
beyondcorp.com
https://www.runpod.io/
runpod.io
https://github.com/PyO3/pyo3
github.com
https://github.com/huggingface/candle
github.com
https://github.com/nmslib/hnswlib
github.com
https://www.cloudflare.com/en-au/application-services/products/argo-smart-routing/
cloudflare.com
https://en.wikipedia.org/wiki/Wide_area_network
en.wikipedia.org
https://en.wikipedia.org/wiki/Packet_loss
en.wikipedia.org
https://developer.mozilla.org/en-US/docs/Glossary/SSR
developer.mozilla.org
https://en.wikipedia.org/wiki/Time_to_first_byte
en.wikipedia.org
https://support.google.com/knowledgepanel/answer/9163198?hl=en
support.google.com
https://dumps.wikimedia.org/
dumps.wikimedia.org
https://en.wikipedia.org/api/rest_v1/page/summary/Australia
en.wikipedia.org
https://www.wikidata.org/wiki/Help:Items
wikidata.org
https://en.wikipedia.org/wiki/Search_engine_results_page
en.wikipedia.org
https://blog.wilsonl.in//serp-protein-in-chicken.png
blog.wilsonl.in
https://blog.wilsonl.in//serp-blogs-theirs.png
blog.wilsonl.in
https://blog.wilsonl.in//serp-blogs-mine.png
blog.wilsonl.in
https://blog.wilsonl.in//serp-crypto.png
blog.wilsonl.in
https://blog.wilsonl.in//serp-paragraph.png
blog.wilsonl.in
https://blog.wilsonl.in//serp-ai-s3.png
blog.wilsonl.in
https://blog.wilsonl.in//serp-guarantee-clause.png
blog.wilsonl.in
https://blog.wilsonl.in//serp-ai-go-init-map.png
blog.wilsonl.in
https://blog.wilsonl.in//serp-parted.png
blog.wilsonl.in
https://en.wikipedia.org/wiki/Post/Redirect/Get
en.wikipedia.org
https://en.wikipedia.org/wiki/Natural_language_processing
en.wikipedia.org
https://en.wikipedia.org/wiki/Network_theory
en.wikipedia.org
https://www.google.com/intl/en_us/search/howsearchworks/how-search-works/ranking-results/
google.com
https://github.com/hagezi/dns-blocklists
github.com
https://en.wikipedia.org/wiki/Jaccard_index
en.wikipedia.org
https://en.wikipedia.org/wiki/Trigram
en.wikipedia.org
https://blog.wilsonl.in//serp-demo.png
blog.wilsonl.in
https://groq.com
groq.com
https://search.wilsonl.in
search.wilsonl.in
https://arxiv.org/pdf/2407.21783
arxiv.org
https://platform.openai.com/docs/pricing
platform.openai.com
https://cohere.com/pricing
cohere.com
https://jina.ai/embeddings/
jina.ai
https://community.openai.com/t/pricing-discrepancy-for-embedding-models-between-pricing-page-and-model-docs/1346972
community.openai.com
https://commoncrawl.org/
commoncrawl.org
Train 400x faster Static Embedding Models with Sentence Transformers Read at the source ↗
huggingface.co
https://onnx.ai/
onnx.ai
https://umap-learn.readthedocs.io/en/latest/
umap-learn.readthedocs.io
https://man7.org/linux/man-pages/man7/io_uring.7.html
man7.org
https://groq.com/
groq.com
https://www.cerebras.ai/
cerebras.ai
https://wilsonzlin.kit.com
wilsonzlin.kit.com
https://blog.wilsonl.in/rss.xml
blog.wilsonl.in
https://x.com/wilsonzlin
x.com
Exploring Hacker News by mapping and analyzing 40 million posts and comments Read at the source ↗
blog.wilsonl.in