corpus.blog
41554 blogs · 4,417,066 posts Claim your blog
Record · corpus.blog/posts/01a08768-744a-726f-adde-01f59e070e2d

Data pipeline monitoring 101: Tracking health and performance across the data stack

Datadog Engineering · published 14 August 2026

A record is what was published and who pointed at it. The text of the post is not held here: read it at the source, or at its Wayback capture.

Read the post at datadoghq.com Wayback capture
0blogs citing
First cited
—
Most recent
—
Rank this month
not ranked
Links from this post
74
Words captured
4,134

Cited by

No blog we hold has cited this post yet.

Links from this post

Link
Host
https://www.datadoghq.com/blog/data-observability/
datadoghq.com
https://www.datadoghq.com/blog/data-streams-monitoring/
datadoghq.com
https://learn.microsoft.com/en-us/azure/architecture/databases/guide/big-data-architectures
learn.microsoft.com
https://www.getdbt.com/blog/extract-load-transform
getdbt.com
https://www.datadoghq.com/blog/understanding-dbt/
datadoghq.com
https://www.databricks.com/blog/what-is-medallion-architecture
databricks.com
https://aws.amazon.com/blogs/big-data/how-yelp-modernized-its-data-infrastructure-with-a-streaming-lakehouse-on-aws/
aws.amazon.com
https://www.datadoghq.com/blog/data-lineage/
datadoghq.com
https://www.ibm.com/think/topics/data-integrity
ibm.com
https://dagster.io/glossary/data-backpressure
dagster.io
https://opentelemetry.io/
opentelemetry.io
https://opentelemetry.io/docs/concepts/context-propagation/
opentelemetry.io
https://www.w3.org/TR/trace-context/
w3.org
https://openlineage.io/
openlineage.io
https://airflow.apache.org/docs/apache-airflow-providers-openlineage/stable/index.html
airflow.apache.org
https://docs.datadoghq.com/data_streams/
docs.datadoghq.com
https://docs.datadoghq.com/data_streams/schema_tracking
docs.datadoghq.com
https://docs.datadoghq.com/data_streams/dead_letter_queues/
docs.datadoghq.com
https://docs.datadoghq.com/data_observability/quality_monitoring/
docs.datadoghq.com
https://docs.datadoghq.com/data_observability/jobs_monitoring
docs.datadoghq.com
https://docs.datadoghq.com/data_observability/data_catalog/
docs.datadoghq.com
https://docs.datadoghq.com/tracing
docs.datadoghq.com
https://docs.datadoghq.com/serverless
docs.datadoghq.com
https://docs.datadoghq.com/synthetics
docs.datadoghq.com
https://docs.datadoghq.com/real_user_monitoring
docs.datadoghq.com
https://www.datadoghq.com/blog/monitor-otel-with-w3c-trace-context/
datadoghq.com
Building highly reliable data pipelines at Datadog Read at the source ↗
datadoghq.com
https://airflow.apache.org/docs/apache-airflow/stable/administration-and-deployment/logging-monitoring/check-health.html
airflow.apache.org
https://www.databricks.com/blog/what-is-dag
databricks.com
https://airflow.apache.org/docs/apache-airflow/stable/administration-and-deployment/logging-monitoring/metrics.html
airflow.apache.org
https://docs.datadoghq.com/data_observability/jobs_monitoring/
docs.datadoghq.com
https://docs.datadoghq.com/integrations/airflow
docs.datadoghq.com
https://docs.datadoghq.com/integrations/dagster-plus/
docs.datadoghq.com
https://docs.datadoghq.com/integrations/databricks
docs.datadoghq.com
https://www.datadoghq.com/blog/key-metrics-for-airflow-monitoring/
datadoghq.com
https://www.datadoghq.com/blog/monitoring-kubernetes-era/
datadoghq.com
https://debezium.io/documentation/reference/stable/connectors/index.html
debezium.io
https://nightlies.apache.org/flink/flink-cdc-docs-master/docs/connectors/flink-sources/overview/
nightlies.apache.org
https://docs.confluent.io/platform/current/connect/index.html
docs.confluent.io
Mastering Postgres Replication Slots: Preventing WAL Bloat and Other Production Issues Read at the source ↗
morling.dev
https://dev.mysql.com/doc/refman/8.4/en/replication-gtids-concepts.html
dev.mysql.com
https://docs.datadoghq.com/integrations/kafka/?tab=host
docs.datadoghq.com
https://docs.datadoghq.com/integrations/kafka-consumer
docs.datadoghq.com
https://docs.datadoghq.com/integrations/amazon-kafka/
docs.datadoghq.com
https://docs.datadoghq.com/integrations/confluent-cloud/
docs.datadoghq.com
https://docs.datadoghq.com/integrations/amazon-kinesis/
docs.datadoghq.com
https://docs.datadoghq.com/integrations/airbyte/
docs.datadoghq.com
https://docs.datadoghq.com/integrations/flink/
docs.datadoghq.com
https://www.datadoghq.com/blog/monitoring-kafka-performance-metrics/
datadoghq.com
https://www.datadoghq.com/blog/monitor-amazon-msk/
datadoghq.com
https://www.datadoghq.com/blog/confluent-cloud-monitoring-datadog/
datadoghq.com
https://www.oreilly.com/library/view/the-informed-company/9781119748007/c10.xhtml
oreilly.com
https://iceberg.apache.org/docs/latest/spark-procedures/
iceberg.apache.org
https://docs.datadoghq.com/data_observability/
docs.datadoghq.com
https://docs.datadoghq.com/integrations/amazon_s3/
docs.datadoghq.com
https://docs.datadoghq.com/integrations/google_cloud_storage/
docs.datadoghq.com
https://en.wikipedia.org/wiki/Query_plan
en.wikipedia.org
https://greatexpectations.io/
greatexpectations.io
https://docs.snowflake.com/en/user-guide/warehouses-overview
docs.snowflake.com
https://docs.cloud.google.com/bigquery/docs/slots
docs.cloud.google.com
https://docs.datadoghq.com/logs/
docs.datadoghq.com
https://docs.datadoghq.com/integrations/dbt-cloud/
docs.datadoghq.com
https://aws.amazon.com/what-is/materialized-view/
aws.amazon.com
https://docs.aws.amazon.com/redshift/latest/dg/c_workload_mngmt_classification.html
docs.aws.amazon.com
https://www.ibm.com/think/topics/capacity-planning
ibm.com
https://learn.microsoft.com/en-us/fabric/data-engineering/tune-file-size?tabs=sparksql
learn.microsoft.com
https://blog.fabric.microsoft.com/en-US/blog/announcing-optimized-compaction-in-fabric-spark/
blog.fabric.microsoft.com
https://docs.datadoghq.com/database_monitoring/
docs.datadoghq.com
https://docs.datadoghq.com/integrations/snowflake/
docs.datadoghq.com
https://docs.datadoghq.com/integrations/google_cloud_big_query/
docs.datadoghq.com
https://www.datadoghq.com/blog/snowflake-monitoring-datadog/
datadoghq.com
https://developers.google.com/machine-learning/guides/rules-of-ml
developers.google.com
https://docs.datadoghq.com/tracing/
docs.datadoghq.com
https://docs.datadoghq.com/security/cloud_siem/
docs.datadoghq.com