DMALL: Querying a Billion Operational Log Records with Databend

DMALL, a retail SaaS provider, evaluated Databend for querying replenishment calculation logs. This English summary covers the customer account originally published in July 2023 by Su Lin, head of DMALL’s supply-chain replenishment team. It describes that historical test, not a new benchmark.

The workload

Merchants wanted four months of searchable logs, while the existing service supported one month. The team reported about four million log rows per day, or 120 million per month. With anticipated business growth, it planned for approximately 250 million rows per month and one billion rows across four months. Those growth figures were planning estimates, not measured ingestion throughput.

The existing resource-constrained MongoDB cluster could not support the required billion-row storage and query workload. DMALL evaluated an object-storage-based analytical layer rather than expanding the operational database for historical queries.

The data flow

Calculation tasks continued to write logs to MongoDB. Scheduled jobs synchronized those logs into Databend for queries from the application. This was scheduled synchronization, not a measurement of end-to-end streaming freshness.

The test schema included tenant and batch identifiers, day, shop and product codes, supplier information, order quantities, category fields, and semi-structured order and log details.

Reported test conditions and result

• Compute: one node with 4 CPU cores, 16 GB RAM, and a 100 GB disk, as described in the original account.
• Storage: object storage, charged separately according to usage.
• Dataset: one billion log records.
• Result: At a scale of one billion records, common queries returned responses in seconds.

The account does not establish a universal latency target or publish a concurrency level, percentile distribution, complete query suite, ingestion-rate benchmark, or a comparison under identical resources. Its query-timing figure should not be read as checkout latency or a current Databend Cloud service-level guarantee.

Why the pattern matters

Retaining operational logs on object storage gives historical analysis a separate storage and compute path. Teams can preserve their operational write system while building an analytical history for merchant-facing queries.

For a current evaluation, test your own schema, record sizes, filters, retention period, query concurrency, and synchronization interval. The 2023 customer result is evidence for this particular workload, not a performance promise for every retail application.