Why Tableau Plateaus At Cloud Analytics Scale & How Sigma Helps Move Your Business Forward
Sr. Content Marketing Manager, Sigma
Tableau is the industry leader in reporting tools because of its ability to generate beautiful data visualizations. But when it comes to meeting the demands of modern analytics at cloud scale, Tableau bites off more than it can chew: loading, joining, modeling, and analyzing today’s large data sets in Tableau can cause it to slow or crash.
The average company today manages 162.9TB of data, and billion-row data sets are the norm, not the exception. Organizations need a BI solution that can quickly process data at this scale to unlock its full value and accelerate organization-wide data-driven decision-making. But to take advantage of the power, scale, and speed of the cloud, you need a solution that was not only purpose-built for this task, but can also complement traditional analytics tools like Tableau.
Here we discuss how Sigma completes your modern analytics solution set by taking Talbeau’s functionalities the extra mile to meet the demands of cloud analytics at scale. Along the way, we’ll also share examples of organizations using Sigma and Tableau together to get ahead.
Why Is the Cloud Ideal for Analytics?
The scale at which data is produced and collected by today’s organizations is mind-boggling. Over 2MB of data is created every second by every person in the world –– and data-driven organizations are collecting it from applications, APIs, websites, smart devices, and more with the intention of using it to improve their products and services.
In the traditional analytics ecosystem, the on-premises enterprise data warehouse (EDW) was the single source of truth for reporting and analytics. In this system, as data was created, it was sent to the EDW to be stored before being cleaned and modeled by data and BI teams for consumption by business experts.
This process was repeated for every new data source, and it worked — until now. Take marketers, for example: Marketing teams today have over 8,000 tools at their fingertips, many which generate data in JSON format. This proliferation of new data sources combined with the velocity of business, the decrease in storage costs, and the rise of on-demand cloud computing processing power have changed the game.
To harness the value of this high-velocity data and turn it into actionable insights that drive business outcomes, organizations need to embrace a new, cloud-native approach to analytics. That’s why leading companies are leveraging new technologies that accelerate time to insight at cloud-scale.
Modern cloud data analytics has many benefits, like the ability to:
- Collect and store customer data from disparate locations and view it in one unified, secure, and accessible repository: the cloud data warehouse, sometimes referred to as a cloud data platform.
- Synthesize information from on-premises and cloud-based applications and streaming data from websites, social media, and the IoT.
- Analyze petabyte-scale data faster, in a fully-governed solution –– no extracts necessary!
- Explore and analyze billions of rows of live data down to granular, row-level details to identify micro patterns to power macro business decisions.
Traditional BI solutions like Tableau are retrofitted for the cloud ecosystem and have limited ability to connect directly to cloud data warehouses. In addition to requiring a separate data prep tool that creates data sets as extracts, some versions of Tableau also struggle to process large data sets. And because business users can’t independently explore and ask questions of the live data in detail, Tableau falls just short of true “self-service” analytics.
However, when combined with Sigma, Tableau can deliver truly powerful dashboards that harness the full potential of your company’s data, with the ability to drill into the details behind the visualizations.
3 Ways Sigma Helps Tableau Meet the Demands of Cloud-scale Analytics
1. Analytics > reporting: Sigma delivers ad hoc, self-service analyses
Tableau generates rich data visualizations and dashboards, but your team’s journey with the data ends there. While Tableau allows users to drill down from its dashboards with the Hierarchy Chart feature, users are restricted to pre-configured leaf levels that prevent true “What if?” follow-up analyses without the help of data and BI teams.
This process adds days, if not weeks, to arrive at a useful insight beyond what’s already in the dashboard. In turn, this leads to risky data extracts (more on that later).
Sigma takes an alternate approach. Because it connects directly to the cloud data warehouse, organizations can take full advantage of this investment by getting their teams far closer to the full data than with Tableau.
With Sigma, business teams can seamlessly transition from viewing a Tableau visualization directly to exploring the underlying data in Sigma just a few clicks –– something not possible with Tableau alone. They can then use Sigma’s familiar UI to slice, dice, filter, and calculate data using the same format, functions, and formulas as traditional spreadsheets –– giving users the ability to drill down to record-level detail without writing SQL.
Sigma’s intuitive spreadsheet-like interface delivers true self-service analytics that minimizes ad hoc reporting requests by empowering non-technical business teams to explore data and find answers to complex data questions independently. With Sigma + Tableau, domain experts can analyze and explore data to find answers to their most pressing questions without having to wait in the BI team’s queue or learn SQL.
Spotlight: Wholesale food distributor uses Sigma + Tableau to optimize logistics, operations, and demand planning
A major US food and beverage distributor used Tableau to report on various operational areas, including inventory management, warehouse capacity, and vendor performance.
But their business users would still make ad hoc requests, which was unsustainable since these reports were too large to be manually analyzed with data extracts. The company needed a solution that allowed users to drill deeper into the data than Tableau’s pre-built filters.
The data team embedded a link in the Tableau dashboard to the Sigma worksheet underpinning the analysis. This gave over 2000+ employees direct access to the live, fully governed, multi-billion row dataset via Sigma.
From there, users could dive directly from a Tableau dashboard into the full dataset in Sigma to independently conduct any follow-up analyses.
2. Performance and scalability: Sigma crunches through hundreds of billions of rows of data
Organizations invest in the cloud data warehouse to manage massive volumes of data across hundreds of sources in a centralized, secure, fully-governed repository. But since Tableau cannot directly connect to your cloud warehouse without slowing down or crashing with datasets in the millions of rows, it can’t take advantage of its scale and processing power.
Tableau separates data modeling and preparation from dashboard creation and generally requires data team expertise to get data ready for analytics at cloud scale. Tableau also requires the use of prepared aggregations or subsets of data due to architectural limitations, rather than working directly against your cloud data warehouse. This prevents users from getting a detailed, granular view of data –– and worse, the data they do have access to is out of date the second it’s extracted.
As a cloud-native analytics solution, Sigma operates on top of the cloud data warehouse, allowing anyone to directly explore and query live data at scale it in real-time –– no copies or extracts required. Sigma can even automatically parse semi-structured data (like JSON) on the fly, making it easier to join additional data from sources like SaaS applications, websites, IoT, and more. Sigma leverages the virtually unlimited storage and processing power of your cloud data warehouse to scale on demand as needed to quickly analyze billions of rows of data and serve large numbers of users.
Spotlight: E.W. Scripps uses Tableau + Sigma to monetize market data at scale
Although ratings drive their business, the E.W. Scripps Company couldn’t monetize market data at scale because Tableau was unable to handle the volume of data in the CDW.
Tableau became too rigid/inflexible with the millions or billions of records that required additional cleansing or transformation. The process to ingest from various sources was taking 40+ minutes, plus extra hours for every “filter” applied before the complete extraction.
Today, the company uses Tableau for visualizations and reporting, while Sigma is used for data modeling and exploration.
The team can do 5-10 analytics views per week, dashboards are automated, and decision-makers at every station have live, shareable dashboards.
3. Modern data governance: Sigma strikes the perfect balance between access and control
Unrestricted data extracts open businesses up to regulatory compliance, security, and governance risks, leading data and BI teams to lock data down behind SQL and complicated BI tools. But when domain experts are unable to access or explore data without the BI team’s help, they grow frustrated and turn to data extracts to get their hands on the data they need to make timely decisions. A vicious cycle ensues.
Leveraging Sigma alongside Tableau allows your organization to embrace a more modern approach to data governance –– one that strikes a balance between data access and control. Sigma can be set up to take advantage of the security, authentication, and other user permissions that have been configured in your cloud data platform. This pass-through security management allows simplified, complete, and consolidated control of who can access data and what they can do with it.
When using Sigma, data teams, IT, and legal departments don’t have to worry about managing security for the same data set in multiple places or being exposed to risks through copied desktop extracts ever again.
Spotlight: A leading real estate firm uses Tableau and Sigma for debt ratio analysis to meet compliance regulations.
The firm’s global accounting team needed simultaneous access to the company’s 500M row general ledger to conduct debt ratio analysis and ensure compliance.
It took 2+ hours to filter the report in Tableau and then extract the data to Excel for analysis, costing the company millions in wasted time and resources — not to mention, opening them up to security and compliance risks due to the potential of mismanaging customer data.
After integrating Sigma into Tableau, anyone on the team can securely access the complete general ledger in the cloud data warehouse in a governed fashion, and it only takes 2 minutes to filter.
The team’s dashboard allows them to monitor issues in real-time, review transactions, and share insights with other groups.
Sigma’s modern approach to data governance keeps our data safe and secure, but does it in a way that enables data access, visibility, and insight, rather than forcing our team to act as gatekeepers.
Infrastructure & DevOps Analyst, Payload
Bonus: Sigma enables organizations to build a collaborative data ecosystem
Organizations that take a community-driven approach to analytics make faster, more effective data-driven decisions, which help them excel in today’s highly competitive and ever-changing marketplace. But building a collaborative data ecosystem where everyone can contribute their expertise to a company’s BI starts with building, curating, and modeling datasets and analyses for shared consumption.
This process is complicated with Tableau and other traditional tools because models and visualizations are less flexible and must be created on a one-off basis — not to mention how challenging it is for less technical users to get to the actual data. Because of the ease of use of Sigma, however, data modeling and analysis is a collaborative process that can involve business leaders and non-technical team members who traditionally only see the data through the lens of pre-built dashboards.
When these users have the freedom to explore governed data and incorporate their knowledge directly into the analyses, it minimizes the back and forth it takes for data teams to give them what they need. It also offers users the ability to answer complex ad hoc “What if?” questions more quickly.
Sigma enables business users to share, comment on, and iterate on their colleagues’ analyses without worrying about overriding and ruining their work or using the wrong version of the data. This empowers teams to collaborate to create a more complete picture of what’s going on with their business in detail, instead of just a summary or snapshot.
It’s a much more collaborative process. At least half of the time now, I can just point someone to a resource in Sigma that they can leverage or modify to answer their question. We can also sit down with someone to build a data set together and after that, they’re up and running. This used to take weeks, but now it takes about 30 minutes. Some people don’t even need to reach out to me anymore for help because they have access.
Sr. Director of Data Science and Analytics, Agero
Sigma Brings the Power, Speed, and Scale of Cloud Analytics to Tableau
Many features and benefits of the cloud just can’t be utilized with traditional BI tools designed during the on-prem era. Unlike Tableau, Sigma is purpose-built to take full advantage of everything the cloud has to offer.
When combined, Sigma and Tableau can accelerate the journey to making your organization more data-driven. Sigma transforms how your teams work with your data and complements Tableau dashboard reporting –– completing the last mile of your cloud data modernization journey.