This Week in Databend #80

Databend is a modern cloud data warehouse, serving your massive-scale analytics needs at low cost and complexity. Open source alternative to Snowflake. Also available in the cloud: https://app.databend.com .

What's New

Check out what we've done this week to make Databend even better for you.

Features & Improvements ✨

Meta

add databend-meta config grpc_api_advertise_host (#9835)

AST

select from stage with files/pattern (#9877)
parse decimal type (#9894)

Expression

add Decimal128 and Decimal256 type (#9856)

Functions

support array_indexof (#9840)
support array function array_unique, array_distinct (#9875)
support array aggregate functions (#9903)

Query

add column id in TableSchema; use column id instead of index when read and write data (#9623)
support view in system.columns (#9853)

Storage

ParquetTable support topk optimization (#9824)

Sqllogictest

leverage sqllogictest to benchmark tpch (#9887)

Code Refactoring 🎉

Meta

remove obsolete meta service api read_msg() and write_msg() (#9891)
simplify UserAPI and RoleAPI by introducing a method update_xx_with(id, f: FnOnce) (#9921)

Cluster

split exchange source to reader and deserializer (#9805)
split and eliminate the status for exchange transform and sink (#9910)

Functions

rename some array functions add array_ prefix (#9886)

Query

TableArgs preserve info of positioned and named args (#9917)

Storage

ParquetTable list file in read_partition (#9871)

Build/Testing/CI Infra Changes 🔌

support for running benchmark on PRs (#9788)

Bug Fixes 🔧

Functions

fix nullable and or domain cal (#9928)

Planner

fix slow planner when ndv error backtrace (#9876)
fix order by contains aggregation function (#9879)
prevent panic when delete with subquery (#9902)

Query

fix insert default value datatype (#9816)

What's On In Databend

Stay connected with the latest news about Databend.

Why You Should Try Sccache

Sccache is a ccache-like project started by the Mozilla team, supporting C/CPP, Rust and other languages, and storing caches locally or in a cloud storage backend. The community first added native support for the GitHub Action Cache Service to Sccache in version 0.3.3, then improved the functionality in v0.4.0-pre.6 so that the production CI can now use it.

Now, opendal, open-sourced by Datafuse Labs, acts as a storage access layer for sccache to interface with various storage services (s3/gcs/azlob etc.).

Learn More

Blog | Why You Should Try Sccache

What's Up Next

We're always open to cutting-edge technologies and innovative ideas. You're more than welcome to join the community and bring them to Databend.

Try using `build-info`

To get information about git commits, build options and credits, we now use vergen and cargo-license.

build-info can collect build-information of your Rust crate. It might be possible to use it to refactor the relevant logic in common-building.

pub struct BuildInfo {
    pub timestamp: DateTime<Utc>,
    pub profile: String,
    pub optimization_level: OptimizationLevel,
    pub crate_info: CrateInfo,
    pub compiler: CompilerInfo,
    pub version_control: Option<VersionControl>,
}

Issue 9874: Refactor: Try using build-info

Please let us know if you're interested in contributing to this issue, or pick up a good first issue at https://link.databend.rs/i-m-feeling-lucky to get started.