From Power (BI) User to Sigma Champion
When tasked with “making a company more data-driven,” you have to ask yourself two questions. Which users can ensure that happens, and what skills do they have to get you there?
I found myself in that position just under five years ago, when I joined a company looking to expand its BI offerings. And with that, I embarked on a journey building out a Power BI implementation across 15 disparate applications, providing a level of data accessibility that was never achieved at this company prior.
My name is Shawn Namdar, and I have five years of experience building out accessible data solutions. And this is my story of how I was converted from a Power BI evangelist to a Partner Engineer at Sigma Computing.
I enjoyed Power BI when I was using it.
It accomplished a lot of the things that I needed it to do, and for the most part, it looked good doing it, but I continually found that as reports became more and more complex, cracks began to form. They manifested in primarily two ways: maintainability and performance.
And even though the product wasn't perfect, we continued to use it. After all, Power BI is a part of Microsoft Office, and that ecosystem is hard to justify leaving.
Over a few years, we built out a lot of data content, including:
- Workspaces for different departments and teams
- Dataflows to provide curated access for all enterprise systems
- Embedded reports to promote easier front line access
The end product had a tremendous impact by embedding Power BI reports in Sharepoint and Teams; the company began to leverage data that was previously never available to the front-line workers. Data accessibility was at an all-time high, and people knew where to find their data.
However, those were curated reports and dashboards hosted on the Power BI Service, and less than 5% of the company had ever used the desktop app. So when a user had a question that the report left unanswered, their only option was to export the data and open it in what they knew – their handy dandy spreadsheet.
Despite finding and enabling champions in each department that could “own” their scope of data and support their teams with unanswered questions, most of my time was spent troubleshooting issues in existing reports. As a functioning data engineer, my time would be better served unlocking new data sources or producing new content.
These complex issues stemmed from the difficulty of M and Dax, Power BI’s modeling languages.
Much of Power BI’s selling point was its ease of use. Microsoft claims that “if you know Excel, you’ll be able to use Power BI.” Well, that's true to a point, but there should be a disclaimer; once you get past what can be done via their various UI elements and wizards, you venture into uncharted territory for most, writing code. Some business users will be able to figure out what they need as they go along, but that is a slow, inefficient, and error-prone method – and precisely why my team ended up spending a lot of time helping out.
This was arguably my biggest qualm with Power BI. How did Microsoft, the inventor of Excel, the company that helped make a billion people on this planet fluent in spreadsheets, neglect to provide a quality tabular interface into Power BI and lock the majority of its power behind languages that many in the business would never learn.
My goal was always to lower the barrier for the business users to answer their questions with the data, but I didn't feel like we were quite there. We were still missing the last mile and I wanted my next role to cross that bridge.
Sigma Computing set out to solve the problem that had eluded me for the last few years. How do you get technical and non-technical users access to all their data while providing them the true Excel-like experience that they need to answer their questions?
Our field team at Sigma Is often asked: What does Sigma do that Power BI can't?
I'll answer that one with another question. When you think about self-service data, does that stop or start at the curated report? I see self-service data not as a nicely constrained report with highly defined drill paths and filters but as a starting point to allow for ad-hoc exploration.
With Power BI, if the creator of a report has not defined a hierarchy or slicer, the consumer is left up a creek without a paddle. They will need to either reach out to the creator and request that they add those features or export the underlying data and perform their analysis in excel.
That couldn't be further from the truth with Sigma. Sigma allows the end-user to filter or drill anywhere without the need for the creator to infer the requirements. This means that the data does not have to leave the system for the end-user to perform their analysis. Removing the technical barriers to entry, Sigma promotes true self-service ad-hoc exploration.
Put your data cubes on ice
Another key differentiator is speed of iteration. What good is a BI tool to a fast-moving business if it takes hours if not days to modify or update? In legacy BI tools to build pivot tables and charts on top of billion+ row record sets the use of OLAP Cubes was a necessity. This adds a layer of complexity for administration, an additional time-consuming step, and further separates the end-user from the underlying data.
In the Microsoft ecosystem, OLAP Cubing is handled in SSAS (SQL Server Analysis Services). This tool unlocks the potential of massive datasets for Power BI however, it comes at a cost. Data engineers must build cubes in the proprietary and complex language MDX, which is both time-consuming and an additional place where future updates must be made. SSAS acts as a source for Power BI and only presents the computed model, obfuscating the underlying records. All of these determinants culminate in the lack of agility in traditional BI.
Sigma is always running off of the records in the CDW and computing the necessary models in real-time. With optimized SQL fed to it, a CDW can easily crunch the numbers and report back the values for a pivot table in real-time.
Sigma capitalizes on this capability. No longer do engineers have to manage a semantic model, nor do they have to work in a complex language. They can build whatever pivot tables they need in real-time on the underlying data. See for yourself below.
The difference between Sigma and Power BI...
Varies with some features going in completely opposite directions and others being extremely similar. Below I will go through several areas and discuss how the two products stack up.
Power BI leverages a hybrid model, requiring both on-prem and SaaS instances. Power BI provides its own compute in most implementations limiting its scalability and performance. Sigma is and will always be a SaaS application. By being cloud-first, it was architected to rely on the CDW’s compute power to run queries enabling true cloud-scale and unparalleled performance. Power BI is designed to support a non-centralized data architecture. This means that it can take unstructured data from disparate sources, prep and blend them together. Sigma believes that the CDW should be the center of the data ecosystem, and as such, Sigma only connects to CDWs. Leave ingesting and blending to ETL/ELT partners working in the CDW. Power BI has two methods of curating modeled data, Dataflows and Datasets. Dataflows allow the user to model freely off of the data whereas Datasets provide a locked down model that can be consumed in reports. In Sigma Datasets, a creator can model out data as they see fit with a variety of transformations. These datasets can also be materialized back to the CDW to reduce cost and optimize performance. Power BI’s basic modeling capabilities can be accessed through the UI or via wizards. Any complex modeling relies heavily on the use of the proprietary languages M and or Dax. In Sigma all modeling is performed via an Excel style formula bar or with simple UI interactions. At no point are you expected to learn SQL or a proprietary modeling language. Power BI is optimized for static reporting and in turn does not provide a quality ad-hoc exploratory tabular interface. Fields are locked down, hierarchies are established, and once published the end user has very little flexibility to perform their own analysis in report.All data in Sigma starts and ends in tables. A quality spreadsheet interface is a core pillar of Sigma’s offering. At any point you can add a new column, sort, filter, drill down or group by all with a right click; enabling the end user to perform any ad-hoc analysis as they see fit.
Power BI’s Matrix visualization is a strong pivot table on the front end but behind the scenes it has the same data scale limitations that plague the rest of the platform. To pivot datasets with billions of rows the use of precomputed Cubes is a necessity. For pivot tables in Sigma the power of optimized SQL running on the CDW unlocks unlimited scale in real-time. Meaning that if you need to change your pivot tables on the fly, Sigma enables you to literally pivot without skipping a beat. Power BI has a large library of visuals that will fit most needs. Not all of them will necessarily support the drill downs that are needed. None of them will grant you access to the underlying records behind the aggregate calculations. Sigma’s goal is not to go after pixel perfect reporting. Instead the goal is to provide a robust analytics platform that can enable the end user to access the underlying data of visualization. Data in Power BI is persisted, It has to pass through Power BI’s service or desktop app to be computed. This inherently adds a vulnerability to a data architecture.At no point is a Sigma persisting data. Instead by leveraging the CDW to perform the queries Sigma only passes the data along and in turn removes vulnerabilities in the system. Additionally thanks to Sigma’s ad-hoc experience, users can stay in the governed application to perform analysis.
Power to the users
From what I have experienced, living and breathing Power BI is a great tool, but it is a jack of all trades. In trying to be something for everyone, it fails to do some of the most critical tasks as well as it should.
It is an excellent fit for locked-down top-level reporting in some organizations, but those organizations tend to be massive and have a tremendous amount of data. In that scenario, Power BI and Sigma play well together. An aggregated enterprise report can provide hyperlinks to take the user to a filtered Sigma Workbook. This can provide all the ad-hoc flexibility that the end users want in a controlled and governed environment that the business needs. Other smaller organizations might not have the budget for both Sigma and Power BI, in which case Sigma and the modern data stack will be the better choice.
When you look at where Sigma shines over Power BI you can single out four distinct areas: administration, scale, governance, and a lower technical barrier to entry.
Administration:
IT orgs supporting Power BI must manage desktop installs, any legacy Power BI Report Server instances, and nasty webs of sync schedules. As a full SaaS application, Sigma does not have installed software to administrate, and since all reports directly query the CDW, there are no sync schedules that have to be managed.
Scale:
By leveraging the CDW and their per-minute billing, resources can be scaled as needed, meaning Sigma will not run into scale issues. Power BI premium SKUs force companies to either build-up to the high water mark or to accept that during peak seasons their queries won't perform well.
Governance:
By never persisting the user's data or using desktop applications, Sigma is inherently simpler to govern than Power BI.
A lower technical barrier to entry:
A billion people on this planet know how to use spreadsheets, that is not the case for SQL, or any data modeling language out there. By building off of what people already know and use in their businesses Sigma enables users to do more while having to know much less.
Let's Sigma together! Schedule a demo today.
