Executive Summary

Analytics and Data

The processes and technology of the analytics and data software industry continue to play an instrumental role in enabling an organization’s business units and IT to optimally utilize data in both tactical and strategic ways. To accomplish this, organizations must provide technology that can access the data, generate and apply insights from analytics, communicate the results and support collaboration as needed.

Analytics utilizes mathematics to create measurements and metrics that enable data to be evaluated in whatever form, in whatever tool or application is needed to provide insights and guide decision-making. In today’s data-driven world, organizations must use analytics to understand and plan the details of their operations across every department and across the lines of business and IT. Organizations use analytics to track costs, create staffing plans, assess employee and supplier performance, identify variances and plan corrective actions. Analytics also helps inform employees and facilitates communication throughout the organization to coordinate actions toward a common mission and specific objectives. Operating without analytics would be like flying a plane without an instrument panel.

While analytics as a modern business tool dates back more than three decades, the elements of business intelligence (BI) today have expanded well beyond query, reporting, analysis and publishing. The tools now include the sourcing and integration of data and the use of analytics for planning and forecasting, as well as dashboards that present analytics in a variety of visualizations. Analytics today also enables presentations in the form of natural-language narratives with some vendors able to support multiple languages. The collaborative sharing of insights is helping to reduce the time to take action and make decisions.

Artificial intelligence and machine learning (AI/ML) extend analytics, enabling it to classify, predict and suggest behaviors that will help improve business operations. Vendors have also begun using ML to analyze product usage data to enhance and streamline interactions, anticipating the best next step in the analytical process and then performing or recommending that step. Advanced analytics, which incorporate AI and ML, has become a staple in analytic processes. Organizations that analyze their data using machine-learning technology state that they gain a competitive advantage, improve customer experiences, increase sales and respond faster to opportunities. In light of these benefits, it is no surprise that nearly two-thirds of organizations report using machine learning today, and three-quarters of organizations in our research said they plan to increase their use of machine learning.

The ultimate goal of analytics is to help organizations make and implement decisions that improve their operations and bottom line. The range of analytics capabilities has become known as decision intelligence. Decision intelligence involves analyzing historical performance, determining potential courses of action, evaluating the results of those actions and then identifying the best path forward.

Ideally, these processes can be conducted on a self-service basis. Self-service analytics continues to be a goal for most organizations, and those that can achieve it report greater satisfaction with their use of analytics. Organizations that can access analytics without the assistance of IT are more satisfied than those who require the assistance of IT.

Organizations seeking to provide self-service analytics also need to provide self-service data preparation. This is one of the areas where we have seen significant additional capabilities from multiple vendors in this evaluation. In many of our research studies, preparing data is reported as the most time-consuming part of the analytics process, and our research finds that a majority of organizations are not comfortable allowing business users to work with data that has not been integrated or prepared for them by IT. We expect self-service data preparation capabilities will reduce the often-necessary involvement of IT.

While AI and ML still require highly specialized skills, vendors have used elements of AI and ML to provide augmented intelligence capabilities such as automated insights and key driver analyses that require little or no input from the line-of-business personnel using the tools. These augmented intelligence capabilities make it easier for a larger portion of the workforce to gain insights they might not see otherwise. Augmented intelligence also ensures more consistency in an organization’s analytics discipline because many of the analyses are automated ensuring the same types of information are available to all.

For analytics to be effective, they must be accessible. Many vendors now use natural language processing to make it easier to access and find information via search and to understand information through narratives explaining the analyses. Analytics also need to be available to line-of-business personnel as needed in their normal course of conducting business, which means providing rich mobile access to analytics to support a workforce seeking to conduct business in any location at any time. Workers today expect these mobile capabilities, so organizations must make choices to provide analytics and data platforms that can deliver.

Collaboration in conjunction with analytics has finally become much more commonplace. Two-thirds of organizations report they are using or plan to use collaboration with analytics. Vendors now provide many ways to enable collaboration ranging from commenting on analyses to rating data sources. Others provide ways for organizations to assign tasks and track them to completion, helping to ensure that the value of analytics results in specific actions taken by the organization.

Analytics should also lead to action. Organizations use a variety of operational applications where the decisions resulting from analyses are implemented. Embedding analytics directly into these systems makes it easier for line-of-business workers to access the information they need without having to use a different system, reducing the need for additional training.

As vendors continue to build out rich APIs that provide access to nearly all the functionality of their products, some vendors are also providing prebuilt connections delivering analytical outputs into operational systems. Ventana Research asserts that by 2026, more than two-thirds of line-of-business personnel will have immediate access to cross-functional analytics embedded in their activities and processes, helping to make operational decision-making more efficient and effective.

Analytics must also be timely. Organizations often operate 24/7. Information streams into business operations from a rapidly growing number of devices and sources. Without the ability to analyze this information as it occurs, organizations risk missing the opportunity to respond in the moment. Our research shows that one-half of organizations consider it essential to process streaming data and event information in seconds or milliseconds.

As organizations seek expand the spectrum of their analytic requirements, a transition to enterprise-class analytics is an essential step forward. Vendors have responded to these broadening needs with additional capabilities. In some cases, those vendors have invested in developing additional capabilities themselves. In others, they have acquired software vendors that offer complementary capabilities to their existing portfolio. Not only have the capabilities expanded, but the number of vendors has proliferated as well. Despite this expansion, there are still few vendors that attempt to provide the entire spectrum of capabilities we evaluate in this assessment. You will likely need more than one vendor to meet all your analytic needs.

This research evaluates the following vendors that offer products that address key elements of analytics and data as we define it: AWS, Cloud Software Group, Domo, GoodData, Google, IBM, Idera, Incorta, Infor, insightsoftware, Microsoft, MicroStrategy, Oracle, Pyramid Analytics, Qlik, SAP, SAS, Sigma Computing, Sisense, Tableau (Salesforce), ThoughtSpot and Zoho.

Buyers Guide Overview

For over two decades, Ventana Research has conducted market research in a spectrum of areas across business applications, tools and technologies. Ventana Research has designed the Buyers Guide to provide a balanced perspective of vendors and products that is rooted in an understanding of the business requirement in any organization. Utilization of our research methodology and decades of experience enables our Buyers Guide to be an effective method to assess and select technology vendors and products. The findings of this research undertaking contribute to our comprehensive approach to ranking and rating vendors in a manner that is based on the assessments completed by an organization.

This Ventana Research Buyers Guide: Analytics and Data is the distillation of over a year of market and product research efforts. It is an assessment of how well vendors’ offerings will address organizations requirements for analytics and data software. The index is structured to support a request for information (RFI) that could be used in the RFP process by incorporating all criteria needed to evaluate, select, utilize and maintain relationships with technology vendors. An effective product and customer experience with a technology vendor can ensure the best long-term relationship and value achieved from a resource and financial investment.

In this Buyers Guide, Ventana Research evaluates the software in seven key categories that are weighted to reflect buyers’ needs based on our expertise and research. Five are product-experience related: Adaptability, Capability, Manageability, Reliability, and Usability. In addition, we consider two customer-experience categories: Validation, and Total Cost of Ownership and Return on Investment (TCO/ROI). To assess functionality, one of the components of capability, we applied the Ventana Research Value Index methodology and blueprint, which links the personas and processes for TOPIC to an organization’s requirements.

The structure of the research reflects our understanding that the effective evaluation of vendors and products involves far more than just examining product features, potential revenue or customers generated from a vendor’s marketing and sales efforts. We believe it is important to take a comprehensive research-based approach, since making the wrong choice of a TOPIC technology can raise the total cost of ownership, lower the return on investment and hamper an organization’s ability to reach its potential performance. In addition, this approach can reduce the project’s development and deployment time, and eliminate the risk of relying on a short list of vendors that does not represent a best fit for your organization.

To ensure the accuracy of the information we collected, we asked participating vendors to provide product and company information across the seven product and customer experience categories that, taken together, reflect the concerns of a well-crafted RFI. Ventana Research then validated the information, first independently through our database of product information and extensive web-based research, and then in consultation with the vendors. Most selected vendors also participated in a one-on-one session providing an overview and demonstration, after which we requested they provide additional documentation to support any new input.

Ventana Research believes that an objective review of vendors and products is a critical business strategy for the adoption and implementation of analytics and data software and applications. An organization’s review should include a thorough analysis of both what is possible and what is relevant. We urge organizations to do a thorough job of evaluating TOPIC systems and tools and offer this Buyers Guide as both the results of our in-depth analysis of these vendors and as an evaluation methodology.

How To Use This Buyers Guide

Evaluating Vendors: The Process

We recommend using the Buyers Guide to assess and evaluate new or existing technology vendors for your organization. The market research can be used as an evaluation framework to establish a formal request for information from technology vendors on their products and customer experience and will shorten the cycle time when creating a RFI. The steps listed below provide a process that can facilitate best possible outcomes.

1. Define the business case and goals.
   Define the mission and business case for investment and the expected outcomes from your organizational and technology efforts. 
2. Specify the business needs.
   Defining the business requirements helps identify what specific capabilities are required with respect to people, processes, information and technology.
3. Assess the required roles and responsibilities.
   Identify the individuals required for success at every level of the organization from executives to front line workers and determine the needs of each. 
4. Outline the project’s critical path.
   What needs to be done, in what order and who will do it? This outline should make clear the prior dependencies at each step of the project plan. 
5. Ascertain the technology approach.
   Determine the business and technology approach that most closely aligns to your organization’s requirements. 
6. Establish technology vendor evaluation criteria.
   Utilize the product experience: Adaptability, Capability, Manageability, Reliability and Usability, and the customer experience in TCO/ROI and Validation. 
7. Evaluate and select the technology properly.
   Weight the categories in the technology evaluation criteria to reflect your organization’s priorities to determine the short list of vendors and products.
8. Establish the business initiative team to start the project.
   Identify who will lead the project and the members of the team needed to plan and execute it with timelines, priorities and resources. 

The Findings

All of the products we evaluated are feature-rich, but not all the capabilities offered by a technology vendor are equally valuable to all types of workers or support everything needed to manage products on a continuous basis. Moreover, the existence of too many capabilities may be a negative factor for an organization if it introduces unnecessary complexity. Nonetheless, you may decide that a larger number of features in the product is a plus, especially if some of them match your organization’s established practices or support an initiative that is driving the purchase of new software.

Factors beyond features and functions or vendor assessments may become a deciding factor. For example, an organization may face budget constraints such that the TCO evaluation can tip the balance to one vendor or another. This is where the Value Index methodology and the appropriate category weighting can be applied to determine the best fit of vendors and products to your specific needs.

Overall Scoring of Vendors Across Categories

The research finds SAP first on the list with IBM in second place and Oracle in third. Companies that place in the top three of a category earn the designation of Leader. Oracle and SAP have done so in five of the seven categories; IBM in four; Qlik and Domo in two; and Sisense and Tableau in one category.

The overall representation of the research below places the rating of the Product Experience and Customer Experience on the x and y axes, respectively, to provide a visual representation and classification of the vendors. Those vendors whose Product Experience have a higher weighted performance to the axis in aggregate of the five product categories place farther to the right, while the performance and weighting for the two Customer Experience categories determines their placement on the vertical axis. In short, vendors that place closer to the upper-right on this chart performed better than those closer to the lower-left.

The research places vendors into one of four overall categories: Assurance, Exemplary, Merit or Innovative. This representation classifies vendors overall weighted performance.

Exemplary: The categorization and placement of vendors in Exemplary (upper right) represent those that performed the best in meeting the overall Product and Customer Experience requirements. The vendors awarded Exemplary are: Domo, IBM, Oracle, Qlik, SAP and Tableau.

Innovative: The categorization and placement of vendors in Innovative (lower right) represent those that performed the best in meeting the overall Product Experience requirements but did not achieve the highest levels of requirements in Customer Experience. The vendors awarded Innovative are: Idera, Microsoft, MicroStrategy, Pyramid Analytics and SAS.

Assurance: The categorization and placement of vendors in Assurance (upper left) represent those that achieved the highest levels in the overall Customer Experience requirements but did not achieve the highest levels of Product Experience. The vendors awarded Assurance are: GoodData, Incorta, insightsoftware, Sisense and Zoho.

Merit: The categorization for vendors in Merit (lower left) represent those that did not exceed the median of performance in Customer or Product Experience or surpass the threshold for the other three categories. The vendors awarded Merit are: AWS, Cloud Software Group, Google, Infor, Sigma Computing and ThoughtSpot.

We warn that close vendor placement proximity should not be taken to imply that the packages evaluated are functionally identical or equally well suited for use by every organization or for a specific process. Although there is a high degree of commonality in how organizations handle analytics and data, there are many idiosyncrasies and differences in how they do these functions that can make one vendor’s offering a better fit than another’s for a particular organization’s needs.

We advise organizations to assess and evaluate vendors based on their requirements and use this research as a reference to their own evaluation of a vendor and products.

Product Experience

The process of researching products to address an organization’s needs should be comprehensive. Our Value Index methodology examines Product Experience and how it aligns with an organization’s life cycle of onboarding, configuration, operations, usage and maintenance. Too often, vendors are not evaluated for the entirety of the products; instead, they are evaluated on market execution and vision of the future, which are flawed since they do not represent an organization’s requirements but how the vendor operates. As more vendors orient to a complete product experience, the more robust of an evaluation can be conducted.

The research based on the methodology of expertise identified the weighting of Product Experience to 80% or four-fifths of the total evaluation. Importance was placed on the categories as follows: Usability (20%), Capability (20%), Reliability (15%), Manageability (15%) and Adaptability (10%). This weighting impacted the resulting overall rankings in this research. IBM, SAP and Oracle were designated Product Experience Leaders as a result of their commitment to analytics and data technology. While not Leaders, vendor rankings for Microsoft, Qlik, Domo and Tableau show that they meet a broad range of enterprise analytics and data product experience requirements and placed them in the top third.

Many organizations will only evaluate capabilities for those in IT or administration, but the research identified the criticality of Usability (20% weighting) across a broader set of usage personas that should participate in analytics and data processes.

Customer Experience

The importance of a customer relationship with a vendor is essential to the actual success of the products and technology. The advancement of the Customer Experience and the entire life cycle an organization has with its vendor is critical for ensuring satisfaction in working with that vendor. Technology providers that have Chief Customer Officers area more likely to have greater investments in the customer relationship and focus more on their success. These leaders also need to take responsibility for ensuring the marketing of their commitment is made abundantly clear on website and in the buying process and customer journey.

Our Value Index methodology weights Customer Experience at 20%, or one-fifth, as it relates to the framework of commitment and value to the vendor-customer relationship. The two evaluation categories are Validation (10%) and TCO/ROI (10%), which are weighted to represent their importance to the overall research.

The vendors that rank the highest overall in the aggregated and weighted Customer Experience categories are Oracle, SAP and Qlik. These category leaders in Customer Experience best communicate their commitment . Vendors such as Tableau, Domo, IBM and Zoho were not Overall Leaders, but have a high level of commitment to customer experience.

Many vendors we evaluated did not have sufficient information available through their website and presentations. While many have customer case studies to promote their success, they lack depth on their commitment to an organizations’ journey. This makes it difficult for organizations to evaluate vendors on the merits of their commitment to customer success. As a result, half of the vendors’ performances were scored at less than 50% of the maximum in our evaluation. As the commitment to a vendor is a continuous investment, the importance of supporting customer experience in a holistic evaluation should be included and not underestimated.

Appendix: Vendor Inclusion

For inclusion in the Ventana Research Analytics and Data Buyers Guide for 2023, a vendor must be in good standing financially and ethically, and have more than 50 dedicated workers, at least $50 million in annual or projected revenue verified using independent sources, sell products and provide support on at least two continents, and have at least 100 customers. The principal source of the relevant business unit’s revenue must be software-related and there must have been at least one major software release in the last 18 months. The product must be actively marketed as an analytics product and capable of accessing data from a variety of sources, modeling the data for analysis, analyzing the data using a variety of techniques, communicating the results in a variety of ways and supporting the data and analytics processes within an organization.  

The research is designed to be independent of the specifics of vendor packaging and pricing. To represent the real-world environment in which businesses operate, we include vendors that offer suites or packages of products that may include relevant individual modules or applications. If a vendor is actively marketing, selling and developing a product for the general market and is reflected on its website that it is within the scope of the research, that vendor is automatically evaluated for inclusion. 

All vendors that offer relevant analytics and data products and meet the inclusion requirements were invited to participate in the research evaluation process at no cost to them.

Eleven of the 22 vendors responded positively to our requests for additional information and provided completed questionnaires and demonstrations to help in our evaluation of their analytics and data products. We categorize participation as follows:

Complete participation: The following vendors actively participated and provided completed questionnaires and demonstrations to help in our evaluation of their product: Domo, GoodData, IBM, insightsoftware, Oracle, Pyramid Analytics, Qlik, SAP, Sisense, Tableau and Zoho.

Partial participation: The following vendors provided limited information to help in our evaluation: Incorta, Infor and MicroStrategy.

No participation: The following vendors provided no information or did not respond to our request: AWS, Cloud Software Group, Google, Idera, Microsoft, SAS, Sigma Computing and ThoughtSpot.

Vendors that meet our inclusion criteria but did not completely participate in our Buyers Guide were assessed solely on publicly available information. As this could have significant impact on their classification and rating, we recommend additional scrutiny when evaluating those vendors.

Products Evaluated

Vendors of Note

We did not include vendors that, as a result of our research and analysis, did not satisfy the criteria for inclusion in the Buyers Guide. These are listed below as “Vendors of Note.”