This Week in Databend #94
PsiACEMay 21, 2023
Databend is a modern cloud data warehouse, serving your massive-scale analytics needs at low cost and complexity. Open source alternative to Snowflake. Also available in the cloud: https://app.databend.com .
What's On In Databend
Stay connected with the latest news about Databend.
Computed Columns
Computed columns are generated from other columns by a scalar expression. There are two types of computed columns: stored and virtual.
A stored computed column computes and stores the result value when a row is inserted. Use this SQL syntax to create one:
column_name <type> AS (<expr>) STORED
While a virtual computed column is calculated at query time and does not store the result value. To create one, use this SQL syntax:
column_name <type> AS (<expr>) VIRTUAL
VACUUM TABLE
The
VACUUM TABLE
- Snapshots associated with the table, as well as their relevant segments and blocks.
- Orphan files. Orphan files in Databend refer to snapshots, segments, and blocks that are no longer associated with the table. Orphan files might be generated from various operations and errors, such as during data backups and restores, and can take up valuable disk space and degrade the system performance over time.
VACUUM TABLE
If you are interested in learning more, please check out the resources listed below:
Code Corner
Discover some fascinating code snippets or projects that showcase our work or learning journey.
Enable Cache in Python Binding
Databend supports data caching and query result caching, which can effectively accelerate queries. The Python bindings of Databend also support these features, albeit with slight differences.
For query result caching, SQL statements can be used to set it up, which is very convenient.
>>> from databend import SessionContext
>>> ctx = SessionContext()
>>> ctx.sql("set enable_query_result_cache = 1")
For data caching, it can be enabled through environment variables.
>>> import os
>>> os.environ["CACHE_DATA_CACHE_STORAGE"] = "disk"
>>> from databend import SessionContext
>>> ctx = SessionContext()
>>> ctx.sql("select * from system.configs where name like '%data_cache%'")
┌────────────────────────────────────────────────────────────────────────────┐
│ group │ name │ value │ description │
│ String │ String │ String │ String │
├─────────┼──────────────────────────────────────────┼─────────┼─────────────┤
│ 'cache' │ 'data_cache_storage' │ 'disk' │ '' │
│ 'cache' │ 'table_data_cache_population_queue_size' │ '65536' │ '' │
└────────────────────────────────────────────────────────────────────────────┘
Feel free to use it in your data science workflow:
Highlights
Here are some noteworthy items recorded here, perhaps you can find something that interests you.
- Read Docs | Date & Time - Formatting Date and Time to learn how to precisely control the format of time and date.
- Added support for transforming data when loading it from a URI.
- Added support for replacing with stage attachment.
- Added bitmap-related functions: ,
bitmap_contains
,bitmap_has_all
,bitmap_has_any
,bitmap_or
,bitmap_and
, etc.bitmap_xor
- Supported operator
intdiv
.//
What's Up Next
We're always open to cutting-edge technologies and innovative ideas. You're more than welcome to join the community and bring them to Databend.
Remove if_not_exists
from the Meta Request
if_not_exists
In
CreateIndexReq
CreateTableReq
if_not_existed
pub struct CreateIndexReq {
pub if_not_exists: bool,
pub name_ident: IndexNameIdent,
pub meta: IndexMeta,
}
The
if_not_exists
Therefore, it will be more effective for
SchemaApi
Issue #11456 | Moving if_not_exists out of meta request body
Please let us know if you're interested in contributing to this issue, or pick up a good first issue at https://link.databend.com/i-m-feeling-lucky to get started.
New Contributors
We always open arms to everyone and can't wait to see how you'll help our community grow and thrive.
- @silver-ymz made their first contribution in #11487. Added five bitmap-related functions.
- @Jake-00 made their first contribution in #11503. Modified duplicate test case for syntax.
SOUNDS LIKE
- @gitccl made their first contribution in #11507. Added five bitmap-related functions and fixed panic when calling with empty bitmap.
Changelog
You can check the changelog of Databend Nightly for details about our latest developments.
Full Changelog: https://github.com/datafuselabs/databend/compare/v1.1.38-nightly...v1.1.43-nightly
Subscribe to our newsletter
Stay informed on feature releases, product roadmap, support, and cloud offerings!