Faster Workbooks on Every Warehouse: Introducing Universal Result Cache
Today we're excited to announce that our result reuse caching mechanism, Universal Result Cache, is available to customers in public beta.
Universal Result Cache brings result reuse to every warehouse Sigma supports. This functionality ensures high performance on queries in Sigma, regardless of the caching approach of your warehouse.
With Universal Result Cache, you run a query once, and Sigma serves later requests for the same result from cache instead of running the query again in the warehouse.
The result: workbooks open faster and your warehouse runs fewer duplicate queries. After a one-time setup, Sigma handles caching automatically, and the only thing users notice is faster workbooks.
How does a result cache work with different warehouses?
The most popular Sigma workbooks at your business get opened over and over in the same day. Without some kind of result caching, when 40 people open the same workbook in a morning, the warehouse runs the same queries 40 times and returns the same results 40 times. Users get slower load times, and the data team gets a higher compute bill.
Result reuse needs the warehouse to store query results and hand them back on request. Some warehouses, like Snowflake and BigQuery, do this natively: they persist every query result and let Sigma retrieve it later by its query ID.
Other warehouses may approach caching differently, by prioritizing data freshness over performance. This means that the data warehouse automatically invalidates caches when underlying tables change. This approach is great for data freshness, but can result in more warehouse queries for frequently updated tables. Universal Result Cache ensures that workbooks connected to these frequently updated warehouse tables can leverage Sigma's caching mechanism to reduce costs and load speeds.
Similarly, there are also warehouses that offer no caching. For those warehouse connections, Universal Result Cache provides a caching mechanism where previously there was none.
No matter which strategy your warehouse employs, Sigma's Universal Result Cache helps you get faster load speeds and reduced costs on your most popular workbooks.
The benefits of Universal Result Cache
Universal Result Cache stores a copy of each query result in a cloud storage bucket you own, using Sigma's external storage integrations. When the same result is requested again, Sigma serves it from the bucket instead of sending the query back to the warehouse. That one idea translates into three benefits for your team:
1. Workbooks load faster
When someone requests a result that Sigma has already processed, Sigma serves it straight from the cache, with no warehouse job and no waiting. On average, cache hits in Sigma are more than 10x faster than running the same query on the warehouse, and users don't do anything differently to get that speed.
2. Your data stays in your cloud
Universal Result Cache stores results using Sigma's external storage integrations. Designate a bucket you own in Amazon S3, Google Cloud Storage, or Azure Blob Storage, and Sigma handles the rest. Sigma reads and writes to your bucket with temporary scoped credentials, and cached results never leave your cloud account.
3. You control freshness
Some warehouses cache results on their own, but they also decide when that cache expires. On Databricks, for example, any change to the underlying data invalidates the cache, no matter how small the change or how little it matters to your analysis. With Universal Result Cache, that decision is yours. Set the cache duration on a workbook: a short window for fast-moving data, a longer one for workbooks that only need to be current as of this morning, or leave it at the default 10 minutes.
How much faster is result cache versus warehouse evals?
As part of our work into Universal Result Cache, we analyzed 50 million requests to understand performance on different types of queries. These numbers come from live production data: real customer queries across the full range of result sizes, comparing cache hits (blue) against the same queries running on the warehouse (yellow).
Averaged across all queries over a two-week period, cache hits were shown to save about 1 second per request. The gain is biggest for small and medium results, which make up the majority of queries in practice. In short, the cache is fastest exactly where your team queries the most.
Get started
Universal Result Cache is available in public beta. To turn it on:
- Set up an external storage integration pointing to a bucket in your cloud account.
- Enable Universal Result Cache on a connection.
Not a Sigma customer yet? Request a demo to learn more.
FAQ
Which warehouses does Universal Result Cache support? Snowflake and BigQuery connections keep using native result persistence, and Universal Result Cache covers every warehouse that doesn't offer it natively, like Databricks, Redshift, Postgres, Trino, MySQL, Clickhouse, SQL Server 2022, and Azure SQL Managed Instance.
How is Universal Result Cache different from result reuse on Snowflake and BigQuery? For your users, it isn't. Workbooks open the same way, and Sigma manages the cache transparently. The difference is where results live: in a cloud storage bucket you own instead of the warehouse.
What do I need to enable it? To set up Universal Result Cache, you'll need a bucket in Amazon S3, Google Cloud Storage, or Azure Blob Storage, plus a storage integration pointing to it. Admins can then enable the cache on a connection.
Will my users see stale data? The data queried from the cache is only as stale as you allow. You set the cache duration on a workbook, so results refresh on the interval you choose. For example, a workbook showing active customers may only need a refresh once a day, while a workbook dedicated to active inventory might need a refresh at least once an hour. It's completely up to your team to decide the interval.
Is cached data secure? Yes. Cached results live in a bucket you own and control, inside your own cloud account. Sigma uses temporary scoped credentials to read and write, and nothing is stored outside your infrastructure.
