Engineering

Data Engineering in the AI Era - Taming Fragmented Agent Trace Pipelines with a Unified Lakehouse

AI accelerates application development, but it often leaves data teams with longer pipelines, more components, and inconsistent definitions across clouds. A unified, object-storage-based data platform can simplify this architecture by preserving raw data, processing changes incrementally, isolating workloads, and producing governed datasets for evaluation, analytics, and training.

wubx

Aug 28, 2026

Data Engineering in the AI Era - Taming Fragmented Agent Trace Pipelines with a Unified Lakehouse

Data Engineering in the AI Era - Taming Fragmented Agent Trace Pipelines with a Unified Lakehouse

Databend Cluster Key Best Practices: Choosing Columns, Key Order, and Granularity

From Traditional Partitions to Micro-Partitions—How Snowflake and Databend Reduce Data Scanning

From Kafka to Databend Cloud: Engineering a Trillion-Scale Agent Trace Ingestion Pipeline

From Global Sorts to Sketches

Six Practical QUALIFY Patterns

Proving an AI-Written AST Visitor Migration Did Not Break SQL Semantics

Orchestrating Reliable Data Ingestion from S3 with Airflow

Small Bitmap, Big Gains: Optimizing Large-Scale Bitmap Aggregation in Databend

Building a 2KB Histogram with Sub -0.2% Percentile Error

Trace Is Evals: Data Engineering for Agent Trace Analysis and Attribution

Building an Event-Driven CDC Pipeline: MySQL to Databend via AWS DMS and S3

Snowtree: Databend's Best Practices for AI-Native Development

Is Parquet becoming the bottleneck? Why new storage formats are emerging in 2025 (Lance, Vortex, and more)

Why We Built Our Own SQL Parser From Scratch: A Rust Implementation Story

Products

Resources

Community

Company

Solutions

Partners