corpus.blog
41,554 blogs · 5,149,205 posts Claim your blog
Back to confessionsofadataguy.com
Blog · corpus.blog/blogs/confessionsofadataguy.com/posts

confessionsofadataguy.com

confessionsofadataguy.com

2026

Why AI is a Mirror of Your Engineering Processoriginal ↗
1 Oct 2026
Databricks Model Serving: Everything You Need to Knoworiginal ↗
1 Oct 2026
The Future of Data Engineering in the Age of AIoriginal ↗
1 Oct 2026
Those Son-Of-A-Guns Laid Him Off! Ex-Big-Tech Worker.original ↗
9 Sept 2026
Rebuilding Apache Spark in Rust?! Sayyyyyy Whattttt?!!! Lakesail.original ↗
9 Sept 2026
Welcome to Omnigent. The One Harness to Rule Them All.original ↗
9 Sept 2026
Rust + DuckDB – Fast meets Faster. Yowza.original ↗
9 Sept 2026
How to Orchestrate LLM Calls in Your Data Pipelinesoriginal ↗
27 Aug 2026
DuckDB Labs acquired by AWS. What does it mean?original ↗
26 Aug 2026
Writing Rust … by Hand. No LLMs.original ↗
7 Aug 2026
Solution > Problem.original ↗
6 Aug 2026
Cloudflare as a Data Platform? Yeah. No.original ↗
6 Aug 2026
Don’t say “I found a problem.” Say … “I found a problem, here’s a solution .. and proof.”original ↗
5 Aug 2026
Agentic Data Engineering Is Here — But Can It Close the Loop?original ↗
24 Jul 2026
What Happens When a Software Engineer Builds a Company Aloneoriginal ↗
24 Jul 2026
The Creator of Pandas on AI, Apache Arrow, and the Future of Software Engineering | Wes McKinneyoriginal ↗
13 Jul 2026
AWS Lambda MicroVMs: The Compute Nobody Asked For?original ↗
4 Jul 2026
From COBOL to Copilot: 30 Years of Data, BI, and AI with Dave Langeroriginal ↗
1 Jul 2026
Semantic Intelligence is About to Transform Analytics with with David Jayatillakeoriginal ↗
1 Jul 2026
I Watched Every Databricks Data + AI Summit 2026 Announcement So You Don’t Have Tooriginal ↗
1 Jul 2026
The Truth About Lakehouse Scalability and Governance | Rust and Delta Lakeoriginal ↗
1 Jul 2026
Escaping the Token Tax: How Open Models Are Rewriting the Economics of AIoriginal ↗
11 Jun 2026
Delta Lake + DuckDB. Catalog Commits with Unity Catalog. Unlocking Concurrent Ingestion.original ↗
11 Jun 2026
From Failure to AWS: What Actually Makes a Great Engineeroriginal ↗
10 Jun 2026
Delta Lake + DuckDB: Catalog Commits & Concurrent Ingestion (Does It Actually Work?)original ↗
7 Jun 2026
The Reality of Running Data at Scale | How Real Data Engineers Thinkoriginal ↗
4 Jun 2026
What is Apache Arrow Flight?original ↗
3 Jun 2026
Databricks Zerobus: Event Streams + The Lakehouseoriginal ↗
1 Jun 2026
The Truth About The Modern Data Stack | DuckDB Insideroriginal ↗
27 May 2026
Databricks Zerobus Streaming Ingestion for Delta Lake Houseoriginal ↗
23 May 2026
Academic → CTO: What Actually Matters in Data (Matthew Housley)original ↗
19 May 2026
Apache Arrow + DuckDB (the GOAT + the GOAT)original ↗
19 May 2026
Apache Arrow as Data Interchangeoriginal ↗
14 May 2026
Spark is Dead. Long Live DuckDB.original ↗
12 May 2026
AI Isn’t Replacing Curious Developers | It’s Changing Who Winsoriginal ↗
7 May 2026
AI is Changing Data Engineering Fast!original ↗
30 Apr 2026
Delta Lake and Databricks Expert – Inside Lookoriginal ↗
30 Apr 2026
Most Data Teams Are Doing It Wrong | Podcastoriginal ↗
30 Apr 2026
How to Actually Break Free From Notebook Engineeringoriginal ↗
10 Apr 2026
Why I Finally Pulled the Plug on Polars and Moved to DuckDBoriginal ↗
10 Apr 2026
He Quit Apple After 13 Years | Here’s Whyoriginal ↗
2 Apr 2026
The Data Engineering Revolution | Spark, AI, and What’s Coming Nextoriginal ↗
27 Mar 2026
5 Steps to Become an AI Engineer (Without the Hype)original ↗
27 Mar 2026
Databricks Metric Views and the Reality of the Semantic Layeroriginal ↗
24 Mar 2026
Agent Bricks and the Commoditization of AI Systemsoriginal ↗
24 Mar 2026
Polars’ Streaming Engine Is a Bigger Deal Than People Realizeoriginal ↗
24 Mar 2026
Transactions Are Coming to the Lakehouse… Whether You Like It or Notoriginal ↗
24 Mar 2026
DuckDB, AI, and the Future of Data Engineering | with Staff Engineer, Matt Martinoriginal ↗
21 Mar 2026
Decades in Software Engineering: What Actually Makes Great Engineers? | John Crickettoriginal ↗
11 Mar 2026
Metric Views in Databricks | The Semantic Layeroriginal ↗
11 Mar 2026
Data Engineering, AI, and Career Growth – Podcast Deep Dive with Yuki Kakegawaoriginal ↗
11 Mar 2026
Will AI Kill Data Engineering (or Software Engineering)?original ↗
26 Feb 2026
Spark, Lakehouse & AI: A Deep Conversation with Bart Koniecznyoriginal ↗
25 Feb 2026
Introduction to Databricks SQL Temporary Tablesoriginal ↗
23 Feb 2026
From ClickOps to DevOps | Journey into Infrastructure with Maxine Meureroriginal ↗
18 Feb 2026
Temporary Tables in Databricks SQL | Do You Actually Need Them?original ↗
17 Feb 2026
Migrating to Databricks – A Guideoriginal ↗
13 Feb 2026
Why Declarative (Lakeflow) Pipelines Are the Future of Sparkoriginal ↗
11 Feb 2026
Robin Moffatt on the Evolution of Data Engineering: From Batch Jobs to Real-Time | Podcast Intervieworiginal ↗
11 Feb 2026
The Lakehouse Architecture | Multimodal Data, Delta Lake, and Data Engineering with R. Tyler Croyoriginal ↗
3 Feb 2026
Building Credible Data Systems | Hoyt Emerson on The Full Data Stackoriginal ↗
30 Jan 2026
Data Engineering Career Path: From Circuits to Pipelinesoriginal ↗
30 Jan 2026
Apache Airflow vs Databricks Lakeflow | The Orchestration Battleoriginal ↗
30 Jan 2026
This One Polars Pattern Makes Code 10x Cleaneroriginal ↗
30 Jan 2026
Embeddings and Vector Databases – Lance + DuckDB + LangChainoriginal ↗
19 Jan 2026
Apache Arrow ADBC Database Driversoriginal ↗
16 Jan 2026
Polars Pipe Operator in Action.original ↗
7 Jan 2026
Databricks Spark adds Excel data source.original ↗
5 Jan 2026

2025

DuckDB beats Polars for 1TB of data.original ↗
28 Dec 2025
Scott Haines on the Future of Data Engineeringoriginal ↗
17 Dec 2025
The Age of Agentic AI | LangChain and LangGraph style.original ↗
16 Dec 2025
PySpark vs DuckDB vs Polars: The Resultsoriginal ↗
16 Dec 2025
The Evolution of Databricks Compute – Serverless is Winningoriginal ↗
16 Dec 2025
Parquet Killer? Introduction to the Lance File Format.original ↗
3 Dec 2025
SQL Data Modeling with One Big Table (OBT)original ↗
3 Dec 2025
Migrate (hundreds) Delta Lake Partitioned Tables to Liquid Clusteringoriginal ↗
3 Dec 2025
PyArrow for Large Dataset Processingoriginal ↗
14 Nov 2025
Lazy Execution with Polars and DuckDBoriginal ↗
13 Nov 2025
Cluster Fatigue. Polars and PyArrow to Postgres and Apache Iceberg (streaming mode)original ↗
4 Nov 2025
Introduction to Databricks Asset Bundles.original ↗
21 Oct 2025
Fivetran buys DBT. People get mad.original ↗
21 Oct 2025
Running Llama 3.1 8B Locally (LangChain and SQLite)original ↗
13 Oct 2025
DuckDB on Mother Duck. Lake House Glory?original ↗
9 Oct 2025
The Semantic Layer – Real or Not?original ↗
9 Oct 2025
The Era of the YAML Engineeroriginal ↗
3 Oct 2025
DuckDB vs Polars. Wait. DuckDB and Polars.original ↗
25 Sept 2025
Full vs Incremental Data Loads Explainedoriginal ↗
18 Sept 2025
Apache Iceberg Writes with DuckDB (or not)original ↗
18 Sept 2025
How to tune Spark Shuffle Partitions.original ↗
12 Sept 2025
Is Data Modeling Dead?original ↗
8 Sept 2025
The Fastest Way to Insert Data to Postgresoriginal ↗
30 Aug 2025
Polars on GPU: Blazing Fast DataFrames for Engineersoriginal ↗
28 Aug 2025
The Medallion Architecture Farce.original ↗
27 Aug 2025
DuckDB … Merge Mismatched CSV Schemas. (also testing Polars)original ↗
22 Aug 2025
polars.exceptions.ComputeError: schema lengths differoriginal ↗
20 Aug 2025
Becoming a Senior+ Engineer in the Age of AIoriginal ↗
15 Aug 2025
What is SQLMesh and how is it different from dbt?original ↗
14 Aug 2025
Duplicates in Data and SQLoriginal ↗
29 Jul 2025
What Are Deletion Vectors (DLV)?original ↗
27 Jul 2025
Solving a “Fill Forward” NULL problem with Polarsoriginal ↗
22 Jul 2025
Next