How To Build a Data-Driven Company From the Ground-Up
Data Evangelist at Sigma
Uber and Lyft Are now bigger than taxis and rental cars combined. With 25% of Americans tuning in monthly, Spotify has greater listenership than any radio station. Tesla has recently become the world’s most valuable automaker. 500 million users around the globe use Duolingo, making it the most popular way to learn a language.
How did these once tiny companies build themselves, displace powerful incumbents and completely disrupt their industries? While there are several unique factors in their rise to success, the common link can be summed up in one word: data.
Data insights are the competitive advantage
Data has helped each of these brands better target their customers, develop better products and services, and deliver more seamless experiences. And data doesn’t lie: According to a recent survey, data-driven companies are 162% more likely to surpass their revenue goals versus companies that lag behind. They also see great spikes in customer trust and have an easier time complying with regulations.
It’s logical then that startups seeking to follow in the path of these disruptors and achieve their own meteoric success want to build a data-driven foundation. But that’s easier said than done.
Becoming a truly data-driven company is not just a technical hurdle, it’s also a cultural one. That’s why it’s critical to get the infrastructure and workflows right from the beginning before any limiting habits, tools, behaviors, or processes take root.
7 Common Challenges on the Path to Building a Data-driven Company
1. Using on-prem solutions to store your data
There’s no way to dance around it — the business world is moving to the cloud. The truth is that most small, and even many medium and large companies, simply don’t have the resources, staff, and expertise to manage an on-prem data solution.
For one, there are the upfront costs associated with setting up to build your own data infrastructure. Then there’s the security vulnerabilities that come with managing your data onsite. Lastly, there’s the expense of dedicated IT and data engineering resources to maintain software and scale the system up and down depending on usage and company needs.
Small and nimble teams need something better — a cost-effective solution that handles those headaches for them and scales with their growth.
2. Keeping data siloed across tools and platforms
The average 200-500 person company uses 123 SaaS applications these days, each generating valuable data every day. The problem is this data remains locked away in each of these tools and platforms. It’s estimated that 68% of this data will go completely unused.
When data remains siloed, companies don’t see the whole picture. They miss out on opportunities and can fail to see rising threats or challenges. Additionally, when teams don’t have full visibility into their data and can’t collaborate on analyses, they get bogged down with redundant work and reporting.
3. Picking traditional BI tools vs cloud-native tools
Even for a company who has invested in consolidating all of their data in a cloud data platform, traditional BI tools built for the on-prem era make it impossible to harness its full value. Most require mastery of SQL or some other proprietary coding language to use, which prevents business users from independently analyzing data and puts all the heavy lifting on the shoulders of the BI team. It’s difficult to collaborate effectively and by the time data reaches decision makers it’s grown stale and lost most of its business value.
Scale is another issue. Not only do traditional BI tools tend to choke on large datasets, but they also often require extracts to prepare data. Dashboards and reports are delivered as subsets or aggregates that make it impossible for business teams to drill into data at a more granular level without more work from the BI team.
4. Allowing your BI team to languish in report factory hell
Because only those with coding skills can effectively collect and analyze data, the needs of the many in the company fall on the few. Data and BI find themselves to be the bottleneck in the organization with an endless queue of ad-hoc requests, otherwise known as “report factory hell.”
This tedious and unfulfilling work takes away from their ability to take on more impactful and pressing data projects such as to build more sophisticated models and identify new data sources. Not to mention it’s incredibly inefficient from a business perspective.
5. Not building data literacy across the organization
The data language barrier puts a halt to promising insights and crushes an organization’s ability to maximize the ROI of their data analytics efforts. Without a common understanding, teams are paralyzed. Business users struggle to convey their questions and needs. BI teams struggle to interpret what team leads and department heads are asking them. The results are massive delays between reports and slow insights.
The solution is data literacy training. But with no one held accountable to make it happen, tools that are barriers in and of themselves, and company cultures that minimize the importance of data-driven decision making, the problem goes unresolved.
6. Relying on dashboards to influence decision-making
The solutions most companies implement are to build or use analytics dashboards. These tools visualize data in ways most people can understand and can provide high-level insights. But while they can be useful, simply having dashboards doesn’t make your company “data-driven.” Dashboards have all kinds of limitations that prevent organizations from making impactful decisions.
For one, you can’t ask further questions or explore the data unless you know code. Instead, you’re left putting in another ad-hoc request of your BI team and hoping they get back to you in a timely manner. Another problem is that dashboards rarely reveal anything new. While they can be great for analyzing past performance, it’s difficult or impossible to forecast future results or scenarios.
Lastly, they aren’t collaborative. They consist of a fixed set of metrics and data points that were chosen in advance, in many cases, by people several degrees away from the problems you are trying to solve.
7. Letting spreadsheets run rampant
Because the data is inaccessible to them and ad-hoc reports can take weeks to fulfill, many business users take matters into their own hands. Using data extracts and good ol’ Excel, they conduct their own rogue analyses. But this introduces all kinds of problems.
Data extracts downloaded across unprovisioned laptops are one of today’s top security risks. Just ask the folks at Boeing. In 2016, an improperly emailed spreadsheet ended up costing them $15 million.
Additionally, when analysis is being run on ungoverned, old, or inaccurate data, the value of any insights gleaned from it are completely moot. The impact of decisions made on that data can have far reaching reaching implications that can negatively affect your bottom line.
The Anatomy of the Data-driven Company
Building a data-driven company from the ground up is easier than it sounds. With the right focus and investments, you can build an organization that incorporates data analysis across every role to optimize results. It starts by investing in 3 key areas: infrastructure, access, and culture.
Data-driven companies are bypassing traditional approaches to BI and reporting in favor of leveraging new technologies that accelerate time to insight and do more with less. They build their business and outpace competitors by building modern cloud analytics stacks that empower everyone in their company to harness data in advanced and unprecedented ways.
Harnessing the automation and elasticity of cloud services eliminates mundane system maintenance so data teams can focus to build innovative and influential data projects. For business users hungry for on-demand answers to open-ended questions, modern cloud analytics delivers true self-service ad hoc data exploration that fuels business outcomes.
The Modern Cloud Data Analytics Stack Defined
The modern cloud data analytics stack consists of three layered technologies and cloud-based services that collect, store, and analyze data. Together, these tools allow organizations to unlock the full value of their data and fuel smarter decision making for all.
Layer 1: The data pipeline
Data must be collected and integrated across applications, databases, files, and more so it can be easily accessed, modeled, and holistically analyzed. The modern data pipeline automatically connects and normalizes data from across sources in real-time, preparing it for storage and querying using analysis-ready schemas. Plus, it’s true self-serve as with only a few clicks, anyone can start pulling data into their data platform.
Layer 2: The data platform
Most companies wrestle with disparate data: some is structured, some semi- or unstructured, and there is no single source of truth from which they can reliably consolidate data and correlate analytics. Cloud data platforms serve as a centralized repository for all of the data in an organization and provide elastic infrastructure, unlimited scale, cost-effective risk mitigation, security management, and other cloud-specific benefits traditional on-prem warehouses do not.
Layer 3: Data exploration
To maximize the value of data and enable data-driven decision-making, companies must empower employees of all technical abilities to independently interact with data at cloud speed and scale. Cloud-native BI solutions (like Sigma) give everyone the ability to directly query live data from the cloud data platform, down to row-level detail — no manual SQL or proprietary coding required — while maintaining strict data governance. Teams can build visualizations, join data sources on the fly, unravel JSON, do rapid what-if analysis, and more via user interfaces that resemble tools they already know and love, like spreadsheets. In turn, BI experts can escape report factory hell and focus on the innovative and strategic projects they love.
Interested in learning more about the modern cloud analytics data stack?
Watch our free webinar here
Data-driven companies don’t keep data siloed — they open up access to as many people in the organization as possible. Instead of a walled garden, great effort goes into making data available to explore safely and efficiently. It’s a modern approach to data governance, striking a balance between access and control, empowering employees of all technical abilities to interact with data and quickly generate insights independently.
In these environments, data and BI teams in a company are not just gatekeepers but serve as guides to promote exploration by opening up new data sources and helps to build better models. Tools like Sigma enable advanced users to explore raw tables or curate endorsed datasets for others.
At the same time, security and compliance are never compromised. Because modern cloud data infrastructure keeps the data in the cloud data platform, the data is never downloaded onto laptops, stored in cache, or in transit. Data and BI teams maintain control over access levels and permissioning.
Interested in making your data more accessible without opening up your organization to security or compliance issues?
Download our free ebook “5 Steps to Modernize Your Data Governance Strategy”
In these leading organizations, data access is achieved by not just building some basic dashboards but by breaking down the data language barrier itself. Very few business experts know SQL, but they do know spreadsheets. In fact, 85% of people use a spreadsheet in their work. And even amongst people that understand and use SQL, 88% still turn to a spreadsheet when exploring data. Modern cloud data exploration tools like Sigma, provide the ability to explore and analyze data through a powerful spreadsheet.
Infrastructure and access are not enough — people need to change their thinking and behavior. Non-technical business users often have an incomplete understanding of the way data is collected, organized, and analyzed. They may make assumptions about how much or what data the company has access to. There’s also a tendency to misunderstand or oversimplify an analysis. This lack of data literacy may prevent them from taking full advantage of data insights in their decision making.
One way organizations are overcoming this is by implementing a data literacy program. The goals of your program should be to help everyone in your organization learn what data is available, how to collect, build, analyze, and handle it, and most importantly, how it can help them improve their daily decision making.
Establishing a data literacy program is not as complicated as it sounds. By narrowing your focus on a few critical skills, you can get everyone in your company incorporating basic data analysis into their everyday workflows.
How to Build a Data-Driven Company From the Ground Up in 5 Steps
There are five basic skills necessary to make data-driven decisions. Here are some practical suggestions for building up each skill area and developing greater data literacy in your organization:
Understand the data
For people to begin using data in their work, they must understand what is available and how it can impact their roles. They must fully grasp the tangible benefits of using data into their day-to-day decision making. This requires getting a clear picture of their current level of understanding and build bespoke trainings that speak to the specific levels of need in your organization.
Find and obtain the data
With a clear understanding of what data is available and how it can be used, you can now begin to teach people where data is located and how to access it. Start by mapping out your company’s data ecosystem, centralizing it in the cloud data platform. From there, try giving non-technical users a sandbox to practice accessing, identifying, and exploring data.
Read, interpret and evaluate the data
Data must be read, interpreted, and evaluated within the context of specific business goals or opportunities for it to have value. A great place to start is putting it in a format people understand and feel comfortable with: the spreadsheet. Dashboards are another great tool. Despite their limitations, they can start people on the road to making data-driven decisions. Build eye-catching dashboards with relevant metrics and KPIs so teams can quickly begin getting value from data.
Manage the data
As their knowledge and comfort level increase, people need to learn how to responsibly manage data to maintain security standards, avoid mistakes, and protect data privacy. Build a data governance framework by developing standards, definitions, and policies for how data is to be handled in your organization, and ensure they align to current best practices and regulations.
Build and use the data
This is the culmination of the skills in the previous four steps. At this stage, users should feel equipped to start generating their own actionable insights. To help them get started, give your users access to report templates for the most common business use cases, as well as datasets with linked data sources, column descriptions and definitions, predefined calculations etc.
For a breakdown of these steps and strategies for building your own data literacy training program, download our free ebook:
Building a Data-Driven Company Starts with Sigma
Data-driven companies better target their customers, develop and build better products and services, and deliver more seamless experiences. The result is increased market share, more loyal customers, and greater revenue.
To follow in the footsteps of these disruptors requires avoiding the most common pitfalls and investing in infrastructure, access, and culture. A modern, cloud data analytics and exploration tool is a critical part of this strategy. This key piece of the cloud data analytics stack opens up access and encourages a culture of data-driven decision making.
Are you ready to build your company on a foundation of open
Join our hands-on lab and get started harnessing the full power of your data today.