Tag:data engineering
Articles tagged "data engineering", page 3.
- 26 MIN READ•Jul 6, 2026
Deterministic Data Engineering With AI Harnesses: Using Claude Code, Codex, Antigravity, and OpenCode for Data Work You Can Actually Trust
How to use AI agent harnesses for data engineering without losing determinism, reproducibility, and trust in your data pipelines and analytics.
data engineeringAI agentsClaude Code - 14 MIN READ•Jun 8, 2026
Apache Iceberg v4 Roadmap: Adaptive Metadata Trees, Single-File Commits, and the Delta Convergence
A deep technical breakdown of Apache Iceberg v4's proposed architecture: adaptive metadata trees, one-file commits, relative paths, column families.
apache-icebergopen-table-formatsdata-engineering - 16 MIN READ•May 31, 2026
Data Platform Native AI Agent Tooling in 2026
A comprehensive comparison of AI agent tooling across Dremio, Snowflake, Databricks, Microsoft Fabric, AWS, Google Cloud, ClickHouse, VeloDB, SpiceAI.
AI AgentsData PlatformsMCP - 24 MIN READ•May 23, 2026
Single-Node Data Engineering: DuckDB, DataFusion, Polars, and LakeSail
Optimize single-node data engineering with DuckDB, DataFusion, Polars, and LakeSail. Compare architectures and learn when to transition to Dremio MPP.
DuckDBApache ArrowDataFusion - 27 MIN READ•May 22, 2026
Apache Iceberg SCD Type 2 and CDC Patterns: Building Historical Lakehouse Tables
A deep dive into implementing Slowly Changing Dimension Type 2 (SCD Type 2) patterns and Change Data Capture (CDC) pipelines on Apache Iceberg, using.
apache icebergcdcscd type 2 - 24 MIN READ•May 22, 2026
Apache Iceberg Catalogs Explained: REST, Glue, Hive Metastore, Polaris, Nessie, and Snowflake
A deep dive into Apache Iceberg catalog architecture, comparing REST catalogs, AWS Glue, Project Nessie, Polaris, and Snowflake.
apache icebergcatalogsNessie - 24 MIN READ•May 22, 2026
Maintaining Apache Iceberg Tables: Compaction, Snapshot Expiration, and Orphan File Cleanup
An in-depth guide to orchestrating maintenance operations on Apache Iceberg tables, covering bin-packing, sort-based, Z-Order compaction, snapshot.
Apache IcebergCompactionData Engineering - 24 MIN READ•May 22, 2026
Apache Iceberg with Spark: Create, MERGE, Upsert, and Evolve Tables End to End
A comprehensive developer guide to configuring Apache Spark with Apache Iceberg, executing transactional writes, and managing schema evolution.
apache sparkapache icebergdata engineering - 21 MIN READ•May 22, 2026
Common Misconceptions About Data Lakehouse and Apache Iceberg
Addressing common search queries and reader confusion about Data Lakehouse architectures, Apache Iceberg catalogs, partitions, and lock-in.
data lakehouseapache icebergdata engineering - 5 MIN READ•Feb 18, 2026
Data Engineering Best Practices: The Complete Checklist
Best practices documents are easy to write and hard to use. They list principles without context, advice without prioritization, and rules without explaining...
data engineeringbest practiceschecklist - 5 MIN READ•Feb 18, 2026
Data Modeling Best Practices: 7 Mistakes to Avoid
A bad data model doesn't announce itself. It hides behind slow dashboards, conflicting numbers, confused analysts, and AI agents that generate wrong SQL. By ...
data modelingdata engineeringbest practices - 7 MIN READ•Feb 18, 2026
Semantic Layer Best Practices: 7 Mistakes to Avoid
Semantic layers don't fail because the technology is wrong. They fail because of design decisions made in the first two weeks : choices that seem reasonable ...
semantic layerdata engineeringbest practices