Vendor Participation in the Buyers Guide - Ventana Research

About the ISG Buyers Guide

Each ISG Buyers Guide is the distillation of a year of software provider and product research efforts by ISG Software Research (previously from Ventana Research). Built on a foundation of 20 years of business and technology research, this unbiased, fact-based guide is the first such industry undertaking to assess the value of software designed specifically for the business process or function on which it focuses.

The ISG Buyers Guide is our assessment and research of how well software providers’ offerings meet buyers’ requirements for a particular category of software. The Buyers Guide is not an abstraction; we use a carefully crafted best practices-based methodology called the Value Index that reflects how organizations actually assess software providers. We have designed the Buyers Guide to ensure that it provides objective independent research and guidance to organizations looking to assess and evaluate their applications for business and IT needs. The structure of the Buyers Guide reflects our understanding that the proper evaluation of providers and products involves far more than just examining product features, potential revenue, or customers generated from marketing and sales. We believe it is a service to both organizations and software providers to take this approach since making the wrong decisions can raise the total cost of ownership for an organization, lower the return on investment and hamper its ability to reach its performance potential. On the other hand, an organization with a successful deployment is a satisfied customer.

The Buyers Guide evaluates the software in seven key categories. Five are product-related: usability, manageability, reliability, adaptability, and capability. Two are customer assurance-related: validation and total cost of ownership and return on investment (TCO/ROI). These seven categories are explained on the following pages.

After gathering the requested information from software providers, we use our research-based analytics and Value Index methodology to generate the Buyers Guide percentages. We then build them into a set of chart indicators that are the graphic representation of the Ventana Research Buyers Guide. In these indicators, the degree of fill and the color reflects our assessment of the value to the potential buyer. The Buyers Guide thus is both an analytic and a graphic representation of the value of your and competing offerings based on an evaluation of what each can deliver.

The profile of user needs for which the Buyers Guide has been developed is derived from our experience in working with organizations and from our benchmark research. We use the extensive body of our benchmark research as a framework to validate the prioritizations contained in and the focus of each Buyers Guide assessment. This approach of using benchmark research to guide the assessment of providers and their products for use in deployment decisions is unique, blending our firm’s experience, research, and comprehensive provider and product framework to yield the only reliable research-based tool for determining what technology users need and what products they should consider.

Our Buyers Guide provides an important service; if an organization’s goal is to improve its maturity across people, processes, information, and technology, it is critical that it be able to select the provider and product that will do the job best. These guides serve as valuable marketing documents as well: They are our independent evaluation of how well-suited your offerings are to an organization’s needs.

The ISG Buyers Guide is designed to provide a balanced perspective that is rooted in an understanding of the real-world business drivers and needs of employees. Thus, using the Buyers Guide will significantly improve an organization’s capability to achieve the levels of efficiency and effectiveness it needs to optimize its performance. This value is increasingly being recognized, making the Buyers Guide a key performance optimization and product selection tool and more and more a companion to the RFP.

About the Process

The ISG Buyers Guide is an assessment and research of how well providers’ offerings will address buyers’ requirements for a particular category of software. The Buyers Guide, which we have prepared for more than two decades, is structured to replicate an RFI/RFP process by incorporating all criteria needed to evaluate, select and deploy technology and maintain relationships with providers.

Ventana Research is committed to an objective review of the providers and products in each area of the enterprise on which we focus. To prepare the Buyers Guide, we have drawn on our research-related work with organizations over the last 20 years, which has included benchmarking and advising thousands of organizations. This provided the context of the needs of buyers, while our research on technology suppliers, knowledge of the market, and thought leadership on this topic, balanced the framework for the Buyers Guide.

You have been invited to participate in our Buyers Guide because you market and sell relevant products. Participation is subject to these terms and requirements:

You may submit as many products as you wish; each will be evaluated to determine individual category compliance and ranking in the functionality evaluation.
Any package of products submitted must be listed on your public website and available to prospective buyers.
You must complete the questionnaire files supplied and detail the specific functional requirements and capabilities of the products submitted.
Functionality must be verified through product documentation and/or a demonstration of the actual product.
You must respond to questions about specific criteria across the seven evaluation categories to enable us to gain free-form feedback on your ability to meet specific customer criteria.
You must participate in a one-hour phone discussion with a Ventana Research analyst to clarify any questions and explore issues the submitted questionnaire raises. The framework of the discussion will be an RFP-like discussion of your product offerings.
You must provide customer references for independent interviews on the relationship and products being assessed.

We will use our benchmark research, the questionnaire, interviews with you, and reviews of the products themselves as the baseline for our research. Across the seven categories, we will review each response to determine the score for the category. After validation, we will aggregate the scores to determine your product’s score. If you submit more than one product for evaluation, we will include the product with the best score in our product functionality evaluation.

To arrive at the overall Value Index for each provider, we will weight each category to reflect its relative contribution to the value as realized by a purchasing organization. We have determined the weighting of the evaluation categories based on our experience and prioritizations derived from our benchmark research.

We have made every effort to encompass in the questionnaire for this Buyers Guide the appropriate functional requirements and capabilities, following a blueprint that we believe reflects what a well-crafted RFP should contain. We will validate the information independently through our database of product information and extensive web-based research, and after that in consultation with you. We may also conduct customer interviews to validate our assessments.

Evaluation Categories

We have designed the Buyers Guide evaluation categories to reflect the breadth of the real-world criteria incorporated in a request for proposal to providers such as yourself. We will evaluate your submissions in seven categories, five relevant to the product experience or package being evaluated and two to the providers grouped into a focus on customer experience. These categories and criteria are as follows:

Product Experience

The process of researching products to address an enterprise’s needs should be comprehensive. Our Value Index methodology examines Product Experience and how it aligns with an enterprise’s life cycle of onboarding, configuration, operations, usage and maintenance. Too often, software providers are not evaluated for the entirety of the products; instead, they are evaluated on market execution and vision for the future, which are flawed since they do not represent an enterprise’s requirements but how the provider operates. As more software providers orient to a complete product experience, the more robust an evaluation can be conducted that considers the needs of an enterprise.

Product Experience: Adaptability of the Product

The Adaptability category applies evaluation criteria designed to ensure products can be configured and customized to meet the needs of a given enterprise. The evaluation criteria include the configurability of the product technology that enables interfacing with business and IT, to what extent it can be customized and whether it supports user interface technologies and systems used by particular lines of business. The evaluation criteria also include how well the product operates across data-related process and workflow systems, whether it can interface well with applications and databases and what investments are required to ensure adaptability.

Product Experience: Capability of the Product

The Capability category involves criteria to evaluate the fit between the capabilities of the products and the functional needs of various groups within business and IT, from executives to line managers and frontline workers. The capabilities of a software product are essential to its inclusion in the Buyers Guide. The criteria examine functionality to the specific Buyers Guide and how the products provide support across the enterprise and as required to for consumers, customers, suppliers and vendors. 

Product Experience: Manageability of the Product

The Manageability category involves evaluation criteria intended to ensure that the products meet business and IT needs for installation, deployment and administration. The evaluation criteria include the support that the product provides for IT administration and for business administration. They also include how sophisticated the security provisions built into the application are with respect to user identity and role and access, how effective the data security is that the application provides, to what extent it supports auditing and compliance, what the license options are, how use is audited and what investments are required in licensing or subscription and maintenance.  

Product Experience: Reliability of the Product

The Reliability category involves evaluation criteria intended to ensure that products can deliver the performance and scalability required. The criteria include the nature of the product’s support for an organization’s IT architecture at the levels of the enterprise, the network, the server and the data, and how sophisticated its development and customization capabilities are. They also include the extent to which it supports access by remote and mobile users, how well and quickly it performs server processing, how well it scales in terms of number of users, volume and complexity of data and server demand, and what investments are required to ensure reliability.  

Product Experience: Usability of the Product

The Usability category involves evaluation criteria intended to assess the value of an investment by looking at its utility for varied levels of business and the diverse ages and competencies found in enterprises. The evaluation criteria include the extent to which the product provides the support needed by each of the functional roles involved in enabling the particular business and IT process — executives (CxO), management, managers, directors, analysts and frontline workers. They also examine the sophistication of a product’s mobile, Web and voice technologies support, including the extent to which the product design enables its usage by workers of different generations.

Customer Experience

The importance of a customer relationship with a software provider is essential to the actual success of the products and technology. The advancement of the Customer Experience across the entire customer journey and life cycle with an enterprise has with its software provider is critical for reaching the best possible satisfaction. Software providers that have chief customer officers are more likely to have greater focus and investments in the customer relationship and successful outcomes. These leaders also need to take responsibility for ensuring the marketing of their commitment is made abundantly clear on the website and in the buying process and overall customer journey.

Customer Experience: TCO/ROI

The TCO/ROI category applies evaluation criteria designed to assess the value a provider delivers through products and support in relation to an enterprise’s ability to achieve expected TCO/ROI. The evaluation criteria include the extent to which the provider is focused on and committed to this product line and how sophisticated it is in demonstrating product value, TCO and total benefit of ownership. The criteria also include an evaluation of the tools and documentation it provides to enable customer evaluation of TCO and ROI and what the provider cites as its investment in optimizing the benefits and costs for its products. 

Customer Experience: Validation

The Validation category applies evaluation criteria designed to assess the provider’s commitment to the market segment and its products along with the breadth of its communication of relevant information. The evaluation criteria include the extent to which the provider is focused on and committed to this product line, the stability of the provider’s management and finances, and what existing customers say about the software provider and its products. The evaluation criteria also include whether the provider can provide a clear roadmap of the product line’s development and direction, what services it provides to support deployment and the quality of its product support.

Inclusion Policy

Providers that offer a relevant product and meet the inclusion requirements are invited to participate in the Buyers Guide evaluation process, at no cost to them. If a provider does not respond to or declines the invitation, a determination is made whether to include it in our analysis based on our defined set of inclusion criteria. These criteria are designed to ensure we include in our evaluation a provider's geographic operations, customer base, revenue, and all relevant aspects of the products’ fit for the category being evaluated. If a provider is actively marketing, selling, and developing a product as reflected on its website that is within the scope of the Buyers Guide, it is automatically evaluated for inclusion. We have adopted this approach because we view it as our responsibility to assess all relevant providers whether they choose to actively participate or not.

To ensure the accuracy of the information we collect and that the Buyers Guide reflects the concerns of a well-crafted RFP, we require participating providers to provide evaluation data across all seven categories. Ventana Research then validates the information, first independently through our knowledge base of product information and extensive web-based research, and then in consultation with the providers.

The Buyers Guide is designed to be independent of the specifics of providers packaging and pricing and whether the products are priced or sold as part of a suite or bundle or individually. To represent the real-world environment in which businesses operate, we include providers that offer suites or packages of products, as the relevant individual modules or applications must still be evaluated by those responsible for those business processes. We take no position on the offering approach of the products or packages being offered; where options exist, organizations using the Buyers Guide will need to decide whether they choose a suite of products or individual applications that best meet their requirements.

Completed surveys and customer references for a Buyers Guide are typically due within six weeks of receiving the submission package.

Please upload the submission package and all supporting documents in our FileCloud portal. For all questions regarding FileCloud, please contact Project Manager Pavan Siddarth at pavan.siddarth@isg-one.com

We ask that you provide five customer references along with your completed submission package. You can enter this information in the appropriate fields in the Product Data form or you can provide it directly to the research program manager. We send each reference an email message with the questions below; alternatively, we are happy to have a phone call if the reference prefers to answer the questions that way. In all the process should not require more than 10 minutes of a reference’s time. All responses are confidential; while customer feedback is a valuable part of our evaluation, we do not share the content of reference interviews with the provider or the general public.

Below is a list of questions we ask references. Analysts may follow up with an email message or phone call for clarification or further details.

How long have you been working with the vendor, and using this product in particular?
Which specific version of the product do you currently use?
How many people use the product across your organization?
What makes the product most valuable to your organization’s needs?
What makes the product most unique to the needs of the organization?
What were the main challenges during the deployment of the product?
What are the main challenges (if any) since deploying and using the product?
What areas would you like for the vendor to improve (relative to product or relationship overall)?
If you’ve shared that feedback with the vendor, were you generally satisfied with the actions taken?
What’s your level of satisfaction with the vendor’s products, support and partnering approach?

We strongly encourage full participation in the Buyers Guide, but if this is not possible, we suggest you provide whatever documentation you have on your product — release notes, any publicly available information, and anything else that will help provide an accurate picture of the product for the analyst. We also suggest you provide customer references and schedule a briefing with the analyst during the scoring period.

No, there is no cost to participate and thus we encourage all providers to participate. There are licensing opportunities for the reports that vary in cost depending on the package you license. Please reach out to us for more information on licensing.

Prior to initiating any new Buyers Guide, our leadership team develops criteria that include the nature of the product, product marketing, overall revenue, and countries of operation. All products that meet the criteria are included in the Buyers Guide. In our view, this policy is essential for ensuring our evaluation is unbiased and that it accurately represents the state of the marketplace and the needs of business executives pursuing RFPs.

Yes, briefings are required for the purpose of confirming the analysts’ assessment and clarifying any outstanding issues. The analyst or research program manager will contact you to set this up during the scoring period.

We ask that you imagine the briefing as a final pitch for a $1 million deal. Please plan for an hour and understand that schedules often prevent extending these sessions. We suggest you structure the briefing as follows, allotting about 15 minutes per segment: company overview, product overview, demo, and discussion of the product as relates to the Buyers Guide evaluation categories, and Q&A.

More information about the Ventana Research Buyers Guide is on our website here. To purchase a Buyers Guide, please contact sales@ventanaresearch.com.

We will share the courtesy copy of the Buyers Guide Report 72 hrs prior to publication.