This Week in Databend #101

Databend is a modern cloud data warehouse, serving your massive-scale analytics needs at low cost and complexity. Open source alternative to Snowflake. Also available in the cloud: https://app.databend.com .

What's On In Databend

Stay connected with the latest news about Databend.

Tutorial | Create External Stage with AWS IAM User

AWS Identity and Access Management (IAM) allows you to control access to AWS services and resources by specifying authorized users or entities. It also enables centralized management of detailed permissions, as well as analysis of access patterns for refining permissions across the entire AWS platform.

Databend supports creating External Stage using AWS IAM User, making access to AWS S3 object storage more secure and reliable.

If you are interested in learning more, please check out the resources listed below.

Docs | Create External Stage with AWS IAM User

Tutorial | Create External Stage on Cloudflare R2

Cloudflare R2 gives you the freedom to create the multi-cloud architectures you desire with an S3-compatible object storage.

You can use Databend to create external storage on Cloudflare R2 and further efficiently mine the value of data.

If you are interested in learning more, please check out the resources listed below.

Docs | Create External Stage on Cloudflare R2

Code Corner

Discover some fascinating code snippets or projects that showcase our work or learning journey.

New System Table: `system.backtrace`

async-backtrace provides efficient and logical "stack" traces for asynchronous functions.

Databend has built the ability of async-backtrace into the system table. Developers can obtain trace information by querying the system.backtrace system table through SELECT statement, which will help further debugging and troubleshooting.

Furthermore, if you are in cluster mode, executing a query against system.backtrace on any node will return the call stack of the entire cluster.

If you are interested in learning more, please check out the resources listed below:

feat: add system.backtrace table

Highlights

We have also made these improvements to Databend that we hope you will find helpful:

Added bitmap_not_count, bitmap_union and bitmap_intersect aggregate functions.
Added cume_dist window function.
Added support for ATTACH TABLE.
Added system.metrics table and metrics for measuring spill and transformation operations.
Added support for converting stored computed column into a regular column.
Implemented task suggestion for Serverless Background Service.

What's Up Next

We're always open to cutting-edge technologies and innovative ideas. You're more than welcome to join the community and bring them to Databend.

Add Random Seed for Random Engine

Databend previously implemented a random engine that supports generating random tables using random data. In order to obtain deterministic and reproducible test results, we plan to introduce random seed.

CREATE table(t int, time timestamp) ENGINE=RANDOM(1000)

Issue #11863 | Feature: Support to add random seed on random engine

Please let us know if you're interested in contributing to this feature, or pick up a good first issue at https://link.databend.rs/i-m-feeling-lucky to get started.