This Week in Databend #133

Databend is a modern cloud data warehouse, serving your massive-scale analytics needs at low cost and complexity. Open source alternative to Snowflake. Also available in the cloud: https://app.databend.com .

What's New

Stay informed about the latest features of Databend.

Understanding Engine Support for Open Table Formats

Databend supports different types of open table formats through table engines to meet the advanced analytical needs of various technology stack data lake solutions.

Currently, Databend offers support for Apache Iceberg and Delta Lake, two of the most popular open table formats, through its table engines. Refer to the following for usage:

--Set up connection
CREATE CONNECTION my_s3_conn 
STORAGE_TYPE = 's3' 
ACCESS_KEY_ID ='your-ak' SECRET_ACCESS_KEY ='your-sk';

-- Create table with Open Table Format engine
CREATE TABLE test_engine 
ENGINE = [Delta | Iceberg]
LOCATION = 's3://testbucket/admin/data/' 
CONNECTION_NAME = 'my_s3_conn';

If you would like to learn more, please contact the Databend team or refer to the resources listed below:

Code Corner

Discover some fascinating code snippets or projects that showcase our work or learning journey.

TPC-H Benchmark: Databend Cloud vs. Snowflake

The TPC-H benchmark is a standard for evaluating decision support systems, focusing on complex queries and data maintenance, and includes 22 queries. In this analysis, we compare the performance and cost of Databend Cloud with Snowflake using the TPC-H SF100 (SF1 = 6 Million Rows) dataset.

Category	Data Loading	Cold Run	Hot Run
Databend Cloud	446s, $0.25	166s, $0.09	124s, $0.07
Snowflake	695s, $0.77	207s, $0.23	138s, $0.15

If you are interested in the specific details and reproduction steps of the test, you can read TPC-H Benchmark: Databend Cloud vs. Snowflake for more information.

Highlights

We have also made these improvements to Databend that we hope you will find helpful:

Added support for the functions div0 and divnull.
Added support for creating tables with the GEOMETRY type.
Added support for the statement VACUUM TEMPORARY FILES [RETAIN <number> SECONDS|DAYS] [LIMIT <number>]; to clean up temporary files.

What's Up Next

We're always open to cutting-edge technologies and innovative ideas. You're more than welcome to join the community and bring them to Databend.

Supporting `DESC[RIBE] [TABLE] <name>`

Databend now supports the DESC[RIBE] <name> statement to describe relevant information about a table.

We now aim to expand this syntax to DESC[RIBE] [TABLE] <name>, on the one hand, to maintain consistency with the familiar syntax of other database systems, and on the other hand, to further clarify the type of object being described.

After this improvement, we expect to support both of the following ways to obtain information about table t1:

desc t1;
desc table t1;

Issue #14713 | feat: DESC[RIBE] [TABLE] \<name>

Please let us know if you're interested in contributing to this feature, or pick up a good first issue at https://link.databend.rs/i-m-feeling-lucky to get started.