corpus.blog
41,554 blogs · 5,038,968 posts Claim your blog
Record · corpus.blog/posts/01a087a3-16f8-70e6-ac6a-b42de8f25a3d

Deduplication: Exact, Near-Duplicate, and Substring Methods

mbrenndoerfer.com · published 22 March 2026

A record is what was published and who pointed at it. The text of the post is not held here: read it at the source, or at its Wayback capture.

Read the post at mbrenndoerfer.com Wayback capture
0blogs citing
First cited
—
Most recent
—
Rank this month
not ranked
External links
0
Words captured
8,052

Cited by

No blog we hold has cited this post yet.

Links from this post

This post only links within mbrenndoerfer.com.

Elsewhere on mbrenndoerfer.com (24)
https://mbrenndoerfer.com/writing/perplexity-language-model-evaluation-metric
mbrenndoerfer.com
https://mbrenndoerfer.com/writing/transfer-learning-nlp-pre-training-fine-tuning
mbrenndoerfer.com
https://mbrenndoerfer.com/writing/llama-architecture-design-training-efficiency
mbrenndoerfer.com
Data Curation and Synthetic Data Read at the source ↗
mbrenndoerfer.com
https://mbrenndoerfer.com/writing/minhash-algorithm-jaccard-similarity-lsh-deduplication
mbrenndoerfer.com
https://mbrenndoerfer.com/writing/web-crawling-common-crawl-strategies-robots-txt
mbrenndoerfer.com
https://mbrenndoerfer.com/writing/the-pile-open-source-training-dataset-large-language-models
mbrenndoerfer.com
https://mbrenndoerfer.com/writing/gpt-2-scaling-language-models-zero-shot-learning
mbrenndoerfer.com
https://mbrenndoerfer.com/writing/stochastic-gradient-descent-neural-network-optimization
mbrenndoerfer.com
https://mbrenndoerfer.com/writing/normalization-feature-scaling-min-max-machine-learning-guide
mbrenndoerfer.com
https://mbrenndoerfer.com/writing/text-normalization-unicode-nlp
mbrenndoerfer.com
https://mbrenndoerfer.com/writing/fasttext-subword-embeddings-character-ngrams
mbrenndoerfer.com
https://mbrenndoerfer.com/writing/text-preprocessing-nlp-tokenization-normalization
mbrenndoerfer.com
https://mbrenndoerfer.com/writing/human-preference-data-collection-rlhf-alignment
mbrenndoerfer.com
https://mbrenndoerfer.com/writing/t5-architecture-text-to-text-transformer
mbrenndoerfer.com
https://mbrenndoerfer.com/writing/character-encoding-ascii-unicode-utf8-nlp
mbrenndoerfer.com
https://mbrenndoerfer.com/writing/bloom-open-access-multilingual-language-model-democratization-ai-research
mbrenndoerfer.com
https://mbrenndoerfer.com/writing/benchmark-contamination-llm-detection-mitigation
mbrenndoerfer.com
https://mbrenndoerfer.com/writing/dense-passage-retrieval-retrieval-augmented-generation-rag
mbrenndoerfer.com
https://mbrenndoerfer.com/writing/long-context-models-processing-million-token-sequences-language-ai
mbrenndoerfer.com
https://mbrenndoerfer.com/writing/document-chunking-rag-strategies-retrieval
mbrenndoerfer.com
https://mbrenndoerfer.com/writing/inference-scaling-llm-deployment-optimization
mbrenndoerfer.com
https://mbrenndoerfer.com/writing/data-quality-outliers-measurement-error-missing-data
mbrenndoerfer.com
https://mbrenndoerfer.com/writing/history-shannon-ngram-language-model
mbrenndoerfer.com