This Week in Databend #116

Databend is a modern cloud data warehouse, serving your massive-scale analytics needs at low cost and complexity. Open source alternative to Snowflake. Also available in the cloud: https://app.databend.com .

What's New

Stay informed about the latest features of Databend.

Feature Preview: Managing Background Tasks with SQL

Previously, Databend introduced Background Service to allow Databend to run one-off background jobs or cron jobs in daemon mode. This simplifies the complexity of managing data maintenance tasks.

To make it easy to create, manage and maintain background tasks, Databend recently added support for a series of SQL statements including CREATE TASK, ALTER TASK, SHOW TASK, etc.

For example, the following SQL statement instructs Databend to execute a task called MyTask1 every day at 6am PST to insert (1, 2) and (3, 4) into table t:

CREATE TASK IF NOT EXISTS MyTask1 SCHEDULE = USING CRON '0 6 * * *' 'America/Los_Angeles' COMMENT = 'serverless + cron' AS insert into t (c1, c2) values (1, 2), (3, 4) 

The Background Service requires Databend Enterprise Edition. Please contact the Databend team for upgrade information.

If you are interested in learning more, please check out the resources below:

Code Corner

Discover some fascinating code snippets or projects that showcase our work or learning journey.

Understanding Configuration Mapping in Databend

For a complex database service like Databend, supporting many configurable options helps developers manage and tune the system. A recently published blog explains the mappings between command line options, environment variables, and config files by reading the Databend code.

Databend currently supports three configuration methods in decreasing order of priority:

Command line options: Override configurations set elsewhere.
Environment Variables: Provide configuration flexibility for Kubernetes clusters.
Config files: A recommended approach for recording and managing configurations.

Please note that Databend also supports common environment variables from storage services such as AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY through the rollback mechanism in opendal. You can directly utilize these variables when working with Databend.

If you are interested in learning more, please check out the resources below:

Blog | Navigating Databend's Configuration Maze: A Guide for Developers and Operators

Highlights

We have also made these improvements to Databend that we hope you will find helpful:

Added a new table function fuse_encoding .
Added new string functions split and split_part.
SQLsmith now supports for MERGE INTO.
databend-metactl supports viewing cluster status.

What's Up Next

We're always open to cutting-edge technologies and innovative ideas. You're more than welcome to join the community and bring them to Databend.

Support More Type Comparisons in z3 Solver

Previously, Databend introduced z3.rs to solve constraints by finding solutions that satisfy given conditions. Z3 from Microsoft Research is a theorem prover commonly used to solve SMT problems. However, currently Databend's z3 solving only supports integer comparisons and needs to cover more types.

For example, after string comparison is supported, t3.f >= '2000-08-23' in select * from t1 left join t3 on t3.e = t1.a where t3.f >= '2000-08-23'; can be pushed down to table t3.

Issue #13236 | Feature: z3 supports more type comparison

Please let us know if you're interested in contributing to this feature, or pick up a good first issue at https://link.databend.rs/i-m-feeling-lucky to get started.