December 4, 2023

Thinking about Sigma on Databricks? Here Are the Top 20 Questions We Get

Sigma is the cloud data workspace for everyone. Capable of analyzing immense volumes of data while keeping compute costs low, Sigma ensures low cost per user in large scale deployments. Sigma will never hold or persist any of your data. Familiar spreadsheet-like interactions return results securely while machine-generated SQL is executed against cloud data intelligence platforms like Databricks.

We get a lot of questions about what makes Sigma unique and how it all works. Here are some of the top questions our go to market teams answer every day.

How does Sigma sit in the modern data stack with Databricks?

Sigma solves for the last mile in the modern data stack. After data has been extracted from applications, then loaded and transformed in the warehouse, Sigma makes that data accessible to business users. This can be the most important point in proving the success of a data strategy, because it is when many users resort to extracts. Keep your data secure and governed in the modern stack and allow non SQL savvy users to ask detailed, ad hoc questions of Databricks with Sigma.

How easy is it to connect to Databricks?

Sigma can connect to Databricks in seconds. Additional features of write access and AWS/Azure Private Link can be set up with the assistance of an admin. There are additional configurations possible per each Databricks connection.

How does Sigma integrate with Databricks Unity Catalog?

Sigma leverages the permissions established in Databricks' Unity Catalog by inheriting them via authentication token. Typically within Databricks, administrators will organize individual users into groups. These groups have different permissions within Databricks to read, modify, create, etc. on various data objects and compute resources. We recommend creating one service principle for each of these groups then using that access token to make a connection within Sigma. This connection will then be shared via teams in Sigma with only the members of the group established in Unity Catalog.

Can I really deploy Databricks powered Sigma workbooks to thousands of employees at a time?

Sigma is enterprise-deployment proven. Sigma has been established in some of the most data security-focused organizations in the world. Unlike tools with on-premise deployment models, Sigma is always centrally managed, controlled, and governed. When a Sigma user builds a workbook with a wide audience application, production releases to viewerships in the thousands are both simple and secure.

Can I model Databricks data in Sigma?

Sigma can connect directly to pre-modeled tables in the lakehouse without an additional layer abstraction. When users want to perform modeling within Sigma, they are covered. Sigma supports all types of joins, unions, and even spreadsheet-like lookups. Joins in Sigma Workbooks help to dramatize the results of traditional SQL code.

Ever second guessed what would result from a Left Outer Join vs. an Inner Join? Modeling results are immediately available for data checks like “Column Details” that provide row count, null value counts, distinct values, and statistics.

Does Sigma integrate with Databricks' customers using dbt?

Many modern data stack engineers work directly in their warehouse with dbt. Sigma does not require a redundant metadata layer on top of dbt, and Sigma can integrate with dbt to provide access to docs and metadata generated from dbt directly in Sigma.

Sigma users can also be reassured of the freshness of their data from the last point of dbt modification as well as if their tables contain dbt data tests.

‍

How does embedding work with Sigma and Databricks?

Workbook embedding allows you to display your workbooks and data elements in other internal or external-facing applications. Embedded content updates in real time, in response to data updates in your warehouse. Sigma embeds via iFrame where full workbook functionality such as “Drill Anywhere,” “Ad hoc Calculations,” and “Save As” can be extended or restricted on a per user basis. Even business user-friendly JSON parsing is possible within the embedded experience. You can productize your Databricks data in no time with Sigma!
‍
Embed one workbook with the full Sigma experience, and allow end users to create endless variations for their own purposes. Take a look at our Embedding Sigma quickstart for more detail.

‍

Can I make pivot tables in Sigma? How is it better than my current Databricks extract workflow?

Sigma pivot tables sort, drill, calculate, and maintain accurate percent of totals up and down hierarchies. Conditional formatting adds clear performance indicator gradients. Where other spreadsheet tools become stale, overloaded, and disconnected, Sigma pivot tables stay enterprise-governed, live, and performant. Take a look at our Sigma pivot table quickstart guide here.

What spreadsheet functions does Sigma support?

Sigma supports a library of over 200+ spreadsheet functions. Aggregate, Date, Logical, Text, and even Pass-Through functions that can call custom user defined functions from the warehouse like “XIRR” are available. Scalar transformations of aggregations like
SUM() / SUM(), or SUM() / COUNT() are available for any combination via the Sigma function bar.

Can I perform cross-level calculations in Sigma?

Sigma excels at making cross-level calculations easier than traditional BI tools. Percent of total calculations, Countifs, and Contains across multiple groupings can be learned without having to perfect cumbersome “Level of Detail” syntax or DAX statements.

‍What does “Drill Anywhere” mean?

“Drill Anywhere” describes the ability for Sigma users to navigate into deeper levels of analysis by simply right clicking a visualization and selecting a dimension—without a defined path prebuilt for them.

A net new visualization built on a higher level of aggregation—like Sales Region—can immediately be taken to a lower level of analysis available—like Product Type. All of this happens without additional lift from an Analytics team.

This is especially useful for embedded workbooks where varied end users may have unexpected follow up questions. Build one flexible workbook rather than maintain many underutilized unique dashboards.

How can I work with my team so we do not overwrite each other's changes?

Sigma workbooks are collaborative with a built-in “Edit, Draft, and Publish” component. That means multiple users can edit the same workbook live. This multi-user live edit capability allows users to work seamlessly with version control. Never have to merge BI content from multiple users again!

Plus, Version History shows all of the granular changes made to each pending draft of a workbook. See who did exactly what within a workbook before publishing for a wider audience.

I have my own data I want to bring into Databricks with Sigma. How does that work?

Sigma will never hold or persist any of your data. Sigma instead facilitates write back to Databricks that allow users to “Bring their own data.” Sigma input tables allow end users to bring last-minute data sets and input variables into defined warehouse writeback schemas so that they can get their final answer working alongside governed data.

I want to add my own data alongside existing tables. Is that possible?

In addition to input tables, Sigma features “linked input tables.” These optional writeback columns allow Sigma users to add their own cell level data alongside existing tables in Databricks. Work on top of governed tables and add data or select from dropdowns to create business user-friendly workflows, all while keeping underlying tables secure and unaltered.

How are Sigma Lookups different from lookups in spreadsheets?

Sigma Lookups work just like lookups in spreadsheets. The difference is Sigma takes the spreadsheet lookup workflow and translates it to machine-generated SQL that executes live against the warehouse. This is how Sigma can perform a lookup on a billion row table in Databricks.

Does Sigma provide any pre-built templates?

Sigma templates are prebuilt workbooks that come free with every Sigma instance. They launch in seconds on top of commonly available warehouse tables, like Salesforce Opportunities or Hubspot Email Sends.

Templates are completely customizable for additional calculations, tables, and visualizations.

Create your own templates for your organization from “Quarterly Business Review” templates to dynamic “Period Over Period” analysis.

Does Sigma support SSO and SCIM?

Absolutely. Sigma can integrate with your existing SAML 2.0 Identity Provider and provide centralized management of users and teams via SCIM. Deploying potentially for your entire organization is lightweight and secure.

What visualizations are available?

Sigma features 14+ Visualization types out of the box that are highly configurable with data labels, trend lines, tooltips, reference marks, and more.

Can I schedule reports with Sigma?

Sigma can schedule exports to Email, Slack, Google Sheets, Google Drive, Cloud Storage, and API Endpoints via Webhook. Exports can be run with the permissions and data access of the recipient. Schedules can be set according to multiple times a day or triggered off threshold conditions in the data. Sigma admins can monitor scheduled exports across all users via usage dashboards.