Your Data Analytics Workflow is Broken. Here’s How to Fix it Today and What to Strive for Tomorrow
Data Evangelist, Sigma
Demand for analytical insights is at an all time high. The volume of data companies are collecting and have access to is increasing exponentially every day. Data insights are helping organizations make better decisions, develop more innovative products, and successfully meet the needs of their customers. In fact, studies show that data-driven companies are 162% more likely to surpass their revenue targets.
But the workflows in place to surface these insights are broken. Which begs the question: how much more impactful could these insights be if they were generated faster, with greater accuracy, and at scale?
The Traditional Analytics Workflows
For most companies, the everyday analytics workflow is a complicated patchwork of emailed requests, downloaded CSVs, broken Excel files, ad-hoc reports, stagnant dashboards, and SQL databases that are effectively inaccessible to the people that need them most.
This chasm between the line of business users that need to make decisions based on data, and their ability to access and analyze that data, remains as vast as ever. It’s called the data language barrier and it’s causing frustration on both sides.
Non-technical Knowledge Workers Struggle to Find Answers
More and more, line of business users are being required to make data-driven decisions to perform their jobs. But current workflows mean that they have to collect, organize and analyze data on their own and very few know how. As a result, more and more line of business users are forced to rely on stale dashboards built by their BI team that make it nearly impossible to dig deeper or ask follow-up questions. A curious business user’s only option is to make a data request and hope they get answers back in a reasonable timeframe. But these ad-hoc requests often balloon into multi week projects where work is repeated and time is wasted trying to better answer the initial question. The pressure to deliver results can be so intense that non-technical knowledge works take matters into their own hands.
Data is downloaded onto laptops, Excel worksheets are created and spreadsheet sprawl ensues. In addition to the obvious security risks this creates, the results of these rogue analyses are often inaccurate or conflicting due to old data or incorrect calculations.
Frustrated Analytics and BI Experts Struggle to Deliver Results
The few that do understand how to collect, organize and analyze data find themselves at the mercy of the needs of the many. They’re inundated with never-ending queues of ad hoc projects that suck up all of their time, preventing them from focusing on the impactful, high value work they love. It’s called report factory hell and it’s nearly impossible to escape from.
In addition to the mountain of tedious, one-off projects and repetitive work they have to accomplish, they’re often set up for failure. Non-technical knowledge workers come to them with abstract problems, undefined guidelines, and vague goals. But the data experts aren’t the ones who deal with these business problems daily and don’t understand all of their nuances.
They’re often left trying to make sense of what their business counterparts are really looking for and trying to come up with answers that will meet their needs. Without that domain expertise, the solutions they come up with are educated guesses at best.
And it’s not just non-technical folks contributing to spreadsheet sprawl. Even BI analysts grow frustrated with their overly complex BI tools and resort to extracting the data into CSVs for less cumbersome analysis.
There Is Another Way
The old waterfall workflows of the past don’t cut it in our fast-paced, always-on world. Today’s knowledge workers need to make data-driven decisions on demand and can’t afford to waste weeks waiting for answers from their BI team or trying to get insights from stale dashboards.
And data and BI teams can’t continue languishing in report factory hell. Their time is better spent building data models, opening up new data sources, curating data, and conducting complex analyses, not cranking out one-off reports. The solution is self-service analytics for everyone at the company. By enabling business users to go beyond the dashboard and ask questions of their data on their own, data and BI teams are free to do organization-level work that truly moves the needle.
This modern data analytics workflow transforms self-service analytics from a a pipe dream into a reality.
The Modern Data Analytics Workflow
- STEP 1 A non-technical knowledge worker from marketing, finance, sales, etc. has a question, hypothesis, or wants to explore the data to discover valuable insights.
- STEP 2 They open their cloud-native analytics tool to get direct, governed access to all of the live data inside their cloud data platform.
- STEP 3 Using the powerful spreadsheet interface that they know and love, they are able to join data sources together, calculate, filter, sort, do what-if analysis, create visualizations, collaborate with teammates, and get the answers they need. Within minutes, they’ve been able to conduct a thorough and sophisticated analysis without any additional lift from their BI team.
What Tomorrow Holds
At Sigma, we’re on a mission to open up data investigation and analysis to everyone so they can re-imagine their data analytics workflows and take a data-driven approach to their work. Three major trends on the horizon are guiding our work now and our shaping our thinking for the future:
Dashboards are only the starting point
Traditional, KPI dashboards are not enough to build a data-driven company. At best, dashboards only alert us to areas that are worth digging into deeper.
To truly surface impactful insights requires the ability to access the underlying data powering those dashboards for analysis. It is at this level of detail that businesses find competitive advantage and can create personalized customer experiences today. The devil is in the details.The next step is a true self-serve analytics powered by a scalable, cloud-native solution like Sigma.
Data analysis skills will be commonplace
All new technological skills go through a specialist phase until everyone learns how to do them capably. There was once a time when typing, slide presentations, basic image editing, and spreadsheet skills were all specializations that required specific training and years of experience. Now? Those are table stakes for every knowledge worker. As data-driven decision-making becomes the de facto way to run a team and carry out most job functions, data analysis will join the ranks of those basic skills everyone is expected to know and be fluent in.
Tools will enable people to focus on the most impactful work
Mastering the wizardry of the darkroom used to be essential skills for photographers. To achieve certain looks or even just to develop a picture at all was a very time-intensive, mechanical process.
Today, photographers can snap a picture on their phone, edit and adjust the photo right on their device, and send it off for any number of professional uses. The tools have evolved and photographers are free to focus on other parts of photography like composition, elements, symmetry, and so on.
Similarly, data analytics tools will do the heavy lifting for us so we can focus on more impactful parts of the work. The platform will replace the need for best practices. For example, the ability to join two data sources together is still pretty hard right now. In SQL, there are multiple join types.
People shouldn’t have to think about those things — the tool or system should assist you in doing that. In the future, layers of abstraction will become so robust that people will focus less on data “plumbing” problems and more on analysis and application of data insights.
Data Is the Future
Data-driven decision making fuels the successful operation of nearly every department and business function. By 2025, it’s estimated that 463 exabytes of data will be created each day. The broken analytics workflows of today will not cut it in what is sure to be a data rich future.
Leading organizations across industries are throwing out their old practices and building their companies on modern analytics workflows that are enabling them to gain more market share, displace incumbents, and disrupt their industries.