January 21, 2021

7 Questions to Ask When Evaluating Business Intelligence Software

The Quest to Become Data-Driven

The stakes to transform into a data-driven business have never been higher. From the sales floor to the C-Suite, every department wants to use data to its advantage. 88% of executives say [1] they feel the urgency to invest in big data initiatives — and it makes sense. Companies that embrace analytics and business intelligence (A& BI) continually outperform those that don’t. Data-driven companies are 23 times more likely to acquire customers ^[2].

Companies that embrace analytics and business intelligence continually outperform those that don’t.

The shift to embrace data has led to record A& BI spending. 55% of companies report investing $50M or more into big data initiatives [1]. No matter how large your BI budget is, choosing the right software is crucial to your success. But let’s face it: The business intelligence software landscape is gigantic. With hundreds of vendors out there, knowing who to evaluate and what to look for in a solution can feel overwhelming.

The BI software vendor landscape is gigantic, ranging from new entrants to megavendors.

If you’re starting to evaluate software, you may be unsure where to start. What’s the difference between all these BI software vendors? Which capabilities and features are important to have? How should you approach the process? What’s changed since your last BI purchase?

In the following pages we explore the top 7 questions every company should ask during the evaluation process — and break down some of the top BI trends and features to consider along the way.

A Brief History of Analytics and Business Intelligence Software

If you’re new to the world of business intelligence, or revisiting after a hiatus, it’s helpful to have a solid background of where we’ve been — and where we’re headed.

Dashboards reigned supreme

Business intelligence has historically focused on tracking and curating data, usually in the form of dashboards or reports. Early BI tools were designed to surface high-level insights in dashboards on a scheduled basis, but lacked real-time reporting capabilities.

These dashboard-focused tools usually answered pre-determined questions executives raised in advance. If questions changed, or additional information was required to make decisions, the dashboards needed modification. Only technical people with knowledge of SQL or other coding languages could build them, which was handled by a member of the data or IT team — effectively making them the data gatekeepers.

On-prem constraints stunted insights

In the past, implementing a business intelligence solution meant building an on-premise data center and hiring an army of IT talent to manage it. Because data storage and compute was relatively expensive compared to today, data analysis was limited.

Outdated data was often deleted to save on massive storage costs, preventing long-term historical analysis. Computing and resource constraints, combined with business teams’ inability to dig into the data behind these dashboards and reports, limited ad hoc analyses to situations when it was absolutely necessary — leaving many questions unanswered. And without the ability to ask those questions on the fly, mission critical insights remained hidden.

What’s Changed?

The datasphere looks very different today. The volume, variety, and velocity of data produced is unparalleled. Thanks to the internet, mobile devices, and the Internet of Things (IoT), more data gets created and collected than ever before. In the time it takes to walk to the water cooler and back, more than 3.8 million queries are submitted to Google and close to $1M is spent online [3].

We now live in a data economy

The velocity and volume of data shows no signs of slowing down. IDC expects the global datasphere to surpass 175 zetabytes by 2025 [4]. The pace data and customer needs change in today’s business environment requires the right tools to keep up — tools that can provide all employees with real-time access to relevant data and insights.

Annual size of the global datasphere

Source: Data Age 2025, sponsored by Seagate with data from IDC Global Datasphere, November 2018

This new data economy is powered by the modern cloud data warehouse (CDW).Modern CDWs collect data from any source and scale elastically to support nearly infinite users and ad hoc analytic workloads. This includes support for unstructured and semi-structured data such as JSON. Storage and compute costs have also come way down, meaning historical data doesn’t have to be tossed, and technology can meet — and even exceed — the demand for insights.

It’s no surprise that analysts expect 83% of enterprise analytical workloads to be cloud-based this year. But despite the wealth of data available and many opportunities to harness it to drive decisions, 73% of companies fail to put it to use ^[⁵^]. Meanwhile, decision makers can’t access this data or uncover insights in a timely or efficient manner.

So where do you go from here? You need a BI tool built to thrive in the new data economy. But how do you know which one to choose?

7 Must-Ask Questions for Evaluating BI Tools

If you’re comparing BI software, we’ve put together a list of questions to help guide your evaluation criteria and ensure you choose the best solution for your business’ needs.

1. Is the software solution built for cloud
data warehouses?

Why it matters

As you evaluate analytics and BI software solutions, make sure to look at cloud-based tools that capitalize on CDW capabilities. Things change more quickly than ever, and teams need real-time data access to make sound — yet rapid — decisions.

Today’s volume and variety of data is much better managed in the cloud — not stuck in a slow on-prem database or sitting in an extract on someone’s PC. That’s why 68% of database market growth is in the cloud[6].

Unfortunately, many companies that have invested in CDWs still use BI tools meant to meet the needs of the pre-CDW era. These solutions fail to maximize the value of CDWs by requiring data extracts prior to analysis, making it difficult to analyze semi-structured JSON data, and presenting other roadblocks that slow down time to data insight.

What to look for

Most analytics tools available today have some form of cloud offering, but few were built for the cloud data warehouse. Seek a BI solution that gives teams direct access to data inside the CDW.

These modern BI solutions accelerate time to business insight by querying data live against your CDW and leveraging the compute power and speed of the cloud to quickly analyze massive datasets in real time. They also capitalize on cloud benefits such as elasticity, real-time data access, sharing, and usage-based pricing.

2. Does the tool require SQL or other proprietary coding knowledge to use?

Why it matters

The desire to arm employees with insights has 62% of companies claiming self-service business intelligence is essential in 2020^[⁷^]. Self-service BI tools are meant to give business experts (like marketing VPs, sales ops directors, and product managers) the ability to find and analyze data without having to rely on IT professionals or dedicated data analysts to create reports.

But many “self-service” BI tools fail to deliver on this promise because they require SQL or other proprietary coding knowledge, effectively restricting data access and analysis to technical analysts. This has led to BI teams getting stuck in “Report Factory Hell” turning out constant ad hoc analyses to answer pressing business questions. Not only does this leave data experts unsatisfied, but it’s also a poor use of their time — and the company’s money. As more companies make data a bigger part of their culture and aim to take a data-driven approach to decision making, new tools are emerging that make good on the promise of self-service analytics. These tools aim to put data in the hands of any employee, and tend to have shorter learning curves, don’t require manual SQL writing, and meet the needs of people without a background in data science or extensive analytics training.

What to look for

Consider how you want people to engage with data. Will they only be viewing pre-built dashboards, or do you want domain experts in marketing, sales, finance and other departments to be able to ask follow up questions and directly gain insights from your analytics solution? If so, you’re going to need a real self-service tool that allows business people without a background in SQL to visually explore and query data.

Look for tools that provide intuitive interfaces (such as a spreadsheet), visual analytics capabilities for the less technical, and SQL runners for those who prefer to code. These types of tools drive adoption and productivity because people with a range of backgrounds can engage with data and make insight-driven decisions in real-time.

Because Sigma feels like a spreadsheet, users haven’t hesitated to dive right into Snowflake data for faster insights.” – How Clover Improved Time to Data Insights by 90% with Sigma

Alex Mora

Data Engineer, Clover Networks

3. How difficult is it to analyze semi-structured data?

Why it matters

Your company is likely sitting on a ton of semi-structured data, such as JSON. JSON has become the preferred data interchange format for mobile devices, web applications, online services, and sensors. This includes some of today’s most popular websites like Facebook and Google — and the fast-growing market of wearables and IoT devices.

These services and devices produce an unprecedented amount of data in our digital economy. Unstructured and semi-structured data now make up 80% of the data collected by enterprises. And that number is only expected to grow. This data is a potential treasure trove for companies able to harness it effectively.

But combing through JSON in real time to find patterns, emerging trends, and insights has historically been challenging with BI tools. Extracting nested JSON rows and analyzing them for insights still requires a deep technical background — meaning it’s usually off-limits for those outside the BI team. Even for those versed in SQL, the process can prove to be time-consuming.

What to look for

To make the most use of JSON, you need to parse out its nested structure and analyze the relevant fields. Look for BI software that makes it possible to easily identify and parse relevant JSON, preferably without having to write SQL. This saves data experts valuable time while empowering business users to create data views that unlock the value of semi-structured data. It also helps to be able to join JSON with existing datasets for deeper analysis.

See how it’s helped companies like Clover, Blue Bottle Coffee, and Volta Charging become data-driven. See all customer stories →

4. Does it protect my data?

Why it matters

Organizations spend millions on their data warehouses, security solutions, and compliance initiatives. But all that spend can instantly be rendered useless by everyday business workflows like downloading data to a Microsoft Excel spreadsheet.

It might seem like downloading data to a personal computer is a trivial issue. But real-world events suggest otherwise. The average data breach costs companies $3.86 million.

Business experts aren’t looking to circumvent enterprise governance practices. They’re trying to get the answers they need to make better business decisions. And because they lack the coding expertise or extensive training required to work with data directly in most BI tools, they are often powerless to answer the questions raised in that last meeting or email. So they turn to what they know best: the spreadsheet

What to look for

Avoid shadow IT scenarios that lead to data breaches by investing in BI software that connects directly to your CDW and doesn’t migrate data or rely on CSV extracts. Employees always have guided access to relevant data and can generate insights without putting the company at risk.

You should also look for security and compliance features like object and row-level security, single sign-on, and user access permissions by team or role. Additionally, compliance certifications such as SOC II, GDPR, CSA, CCPA, HIPAA, and Privacy Shield help protect your data from falling into the wrong hands.

5. Does it have last-mile data prep capabilities (i.e. semantic modeling)?

Why it matters

By the time data gets to the analysis stage, it’s already been collected, transformed, and modeled at the warehouse layer. But semantic data modeling and final clean up is often required before it’s useful. This is especially important if your company uses a cloud data warehouse.

CDWs combine many aspects of traditional enterprise data warehouses and data lakes, meaning that some data will be modeled and ready to analyze — while other data will require some “last-mile” data prep. This last mile is where the focus shifts from technology to people. Here, data is inserted into people’s everyday workflows to help influence decision making, and usually includes extracting semi-structured data, filtering out values, deduplication of data, linking datasets, and more.

Unfortunately, central data models often require updates as business teams raise ad hoc questions. Despite the fact that line of business teams are often the ones closest to company processes and the data they generate, they’re typically excluded from the data modeling conversation because they lack data access and coding skills.

Because of this, initial models are often based on data experts’ guesswork around business needs. So it takes a lot of time — and back and forth between these data and business experts — to get the model to a place where it’s relevant and usable for teams.

What to look for

Look for a cloud-native tool that provides both the ability to prep and analyze data. This gives business users the freedom to explore ad hoc ideas without waiting on BI and data teams to update the central model. Ideally, domain experts can contribute to building highly contextual models by adding definitions and calculations without writing SQL. These collections of modeled data can serve as reusable bases for more detailed analyses in the future, and should be easy to update as data processes evolve.

Some tools also allow BI and data teams to pre-model joins between data sources and models in a way that gives non-technical users a guided, endorsed path for exploration downstream. This way, data experts and business teams can build centralized models that make data usable and insightful for everyone.

Learn how Sigma is is taking a new, collaborative approach to data modeling that frees data experts from lengthy modeling processes in this weekly webinar.

6. Can you go beyond dashboard visualizations?

Why it matters

Data visualization has emerged as one of the best ways to absorb large amounts of information, present data to key stakeholders, and tell a compelling story. While the human mind is capable of incredible feats, most of us cannot easily understand complex statistical models or digest large datasets.

However, we are adept at spotting patterns within visualized data. That’s why BI dashboards are so popular. When done right, data visualizations give the viewer insight into the trends, goals, and metrics contributing to the business.

While dashboards are a great way to get a high-level view of what’s happening, they aren’t going to answer every question. In fact, dashboards usually raise more questions. Why is website traffic slumping? Why was there such a significant drop in sales leads from Texas? Why did we see an increase in customer service requests in Q2? You get the idea.

These follow up questions lead to deeper insights and help business leaders steer decisions. Unfortunately, many BI tools don’t provide users with the ability to drill down into the data presented in a dashboard. To get their answers, they have to go back to the BI team. This effectively turns dashboards into stop signs, not the starting points they should be.

What to look for

First, seek out a solution that doesn’t require manually writing SQL to make interactive data visualization dashboards. This opens the door for business and non-technical users to create their own dashboards instead of relying on data and BI teams each time a change is needed. Real-time, interactive dashboards allow you to drill down and answer follow up questions raised when you see a trend in the visualizations. These dynamic insights act as a launchpad, propelling teams toward deeper questions, smarter answers, and closer collaborations.

It’s also important that dashboards are easily shareable — either through embedding or public share links. A good BI solution makes it possible to share insights not only internally across teams, but also externally with partners and customers. With easy access to dashboards, people can keep track of key metrics and performance.

7. Can teams collaborate and build on each other’s work?

Why it matters

Businesses save valuable time and make better decisions when teams share knowledge and build on each other’s work. No one knows everything, and as the old adage goes, two heads are better than one.

Unfortunately, due to technical user experiences that alienate business teams, mainstream BI tools have left business and data users with two equally bad options:

Business teams have to learn specific technical skills to be able to answer a question.
Data and BI teams have to develop a level of understanding about a business area to develop an appropriate query.

Most of the time, neither team has the correct level of technical knowledge and domain expertise to do both jobs — and they shouldn’t have to.

Data silos across business teams are also an issue. Joining multiple data sources and getting a full, up-to-the-minute picture of performance across applications isn’t easy — especially when it requires coding expertise. Many teams turn to BI tool extracts or resort to using pre-built dashboards and reports from their favorite applications, both of which fail to take the larger business context into consideration.

When teams are unable to share, reuse, and build on each other’s analyses, insights are lost, poor decisions are made, work is duplicated, and productivity suffers.

What to look for

Seek a BI tool that strives to bridge the data language gap between data and business experts by empowering both teams to put their expertise to use. The tool should be intuitive enough for business teams to answer data questions for themselves, but powerful enough for BI teams to dig in and do complex analyses. For example, newer BI tools have changed the modeling conversation, making it possible for business users to work alongside data and BI teams to visually add business context to data models without manually writing SQL.

Your BI tool should serve as a single source of truth for your entire organization. This means selecting a solution that makes it easy to join and analyze data across multiple data sources in real time. Workspaces — combined with robust, role-based data access permissions — also promote unity by organizing relevant data sets and analyses in one central location by team.

When teams have access to each other’s analyses, they’re able to build on one another’s work rather than recreating the wheel every time they have a question. Choose a tool that enhances team productivity by enabling teams to repurpose or reuse analyses, and providing templates for the most common BI use cases.

Some BI tools even have the ability to serve as a single source of truth for a company’s entire data ecosystem. These solutions write modeled data sets back to the cloud data warehouse as SQL views. These views can then be queried and reused to power analyses and visualizations across internal and external applications.

BI Features and Functionality Checklist

Cloud Functionality

Cloud-native (SaaS) BI solution

Real-time queries live against the cloud data warehouse

Scalable analysis across billions of data rows

Real-time joins across multiple sources in a few clicks

User Experience

Visually analyze data with automated SQL generation

SQL authoring capabilities for coders

Zero proprietary coding knowledge required

Familiar, intuitive user interface

Ability to upload and share.CSV files, and join to existing data

Data Governance and Security

Zero data migration, extracts, copies, storage, or cacheing required

Robust, role-based data access permissions

Single sign-on (SSO)

Object and row level security

Privacy certifications including SOC II, GDPR, CSA, CCPA, HIPAA, and Privacy Shield

Data Modeling

Visual, semantic data modeling — no SQL writing required

Ad hoc data exploration without updating central models

Quickly and easily update models over time

Write models back to the cloud data warehouse for reuse across applications

Semi-structured Data Analysis

Ability to visually identify and parse nested JSON data

Data Visualization

Shareable charts and dashboards

Ability to drill down into underlying dashboard data

Dynamic parameters and filters

Private and public dashboard embedding & signed embedding

Collaboration

Shared team workspaces

Single source of truth across teams

Easy sharing and permission settings

Reusable data sets and analyses

Pre-built templates

Next Steps

Hopefully this guide has given you a better understanding of the features and functionality to look for as you identify the right BI software solution.