General Criteria
for Assessing a Healthcare Analytics Vendor
Embarking on an assessment with the knowledge of key, general criteria can
help you determine whether a vendor has the philosophy, experience and
viability that can lead to a successful outcome for your organization. By evaluating
vendors according to the following, you can narrow down the list considerably.
Completeness of
Vision
What lessons does the vendor bring from the past healthcare analytics
market and how have they adjusted their current strategy and products
accordingly? Can they bring lessons from
other industries that are more advanced in their adoption of analytics? What is
the vendor’s understanding of the present market and industry requirements?
What is their vision of the future for healthcare analytics? Look for vendors
who can clearly outline how they have evolved to meet—and anticipate—industry
needs.
Culture and Values
of Senior Leadership
It’s no cliché, but rather the precise truth: the overall culture of a
company starts at the top. Get to know
the senior leadership of the vendors you are evaluating. Insist on meeting several members of their
executive team. Simply put, do the
culture and values of a vendor’s senior leadership align with yours? When
interacting with individual members of the vendor’s team, ask yourself, “Would
I be excited to hire this person into our company?” If the answer is consistently no, find
another vendor. More than technology is required to leverage analytics to drive
real, sustainable change. Cultural transformation will be required throughout
your organization. If your culture and values don’t mesh with that of your
vendor, you will encounter significant roadblocks to success.
Specific Experience & Ability to Execute
Nothing is more important than the vendor's specific, related experience to the problem you are trying to solve, and their track record. The HealthCare.gov web site fiasco is a classic example of a vendor who had general experience building web sites, but not specific experience of the type required by the project. Make sure the vendor has a track record for delivering value and satifaction
to their clients, on projects that look and feel precisely like yours. What do KLAS, Gartner, Chilmark, and the Advisory Board have
to say about this vendor? Find at least three, preferably five, referenced
accounts. Do not accept referenced
accounts that are pre-screened and selected by the vendor—every client of the
vendor should be open to serve as a reference.
Do the vendors you are considering have solid referenced accounts that
are similar in size and demographics to your company? Ask these references very
simply: How satisfied are you with the vendor’s products, services, and overall
value? Would you hire them again?
Technology
Adaptability and Supportability
The reality is, in today’s connected world, all businesses, including
healthcare, move at the speed that their sofotware can adapt-- either fast or
slow-- to new processes and business models.
Therefore, the underlying engineering and architecture of that software
is critically imporant. You must peel
back the covers of the vendor’s products and evaluate their software
engineering for modern design patterns like object oriented programming, service
oriented architectures, loose coupling, late binding, and balanced granularity
of software services. Glossing over this
assessment is akin to buying a multimillion dollar office building without
assessing the modularity of the walls and soundness of the foundation,
plumbing, and electrical systems. How fast can the system adapt to the market
and your unique needs for differentiation? Data standards, vocabularies and
analytics use cases are changing rapidly in the healthcare industry, literally
everday, with no signs of slowing down. Find a vendor whose software
engineering can keep up. Analytic
agility is critical. Executives in your
organization can’t wait weeks and months anymore for a new report to inform a
critical decision. The industry is changing too fast.
Total Cost of
Ownership
The best solution in the world is of no value if it’s not affordable. To assess affordability, you must understand
the total cost of the vendor’s solution. Measuring Total Cost of Ownership (TCO)
is easy—add up the three-year labor costs, licensing fees (including third
party), support fees, and hardware costs associated with a vendor’s solution.
Many old-school analytics vendors require a significant upfront investment with
no guarantee of value for two years or more.
Your TCO over three years should be evenly distributed, not front-end
loaded, and your contract should be structured with escape clauses if the
vendor’s solution cannot prove value in the first year. In today’s market,
clients should expect initial value from analytics vendors in less than six
months, preferably three. If a vendor
cannot or will not commit in their contract to this timeframe for delivering
value, look for another vendor.
Company Viability
The key question here is: Will the vendor be around in nine years (the
average life span of a significant IT investment)? If not, can you live without them? Take
advantage of evaluations by neutral third-party analysts like Gartner,
Chilmark, KLAS and The Advisory Board. What are these analysts saying about the
vendor’s prospects in the market? Is the
vendor in solid financial shape? What’s
their monthly burn rate vs. income? How many days cash-on-hand do they
maintain? What’s their sales pipeline
look like? Does the vendor’s executive leadership team have a track record for
jumping from one company to another or do they have a track record of longevity
and success? How much is the vendor
spending on sales staff in comparison to engineering and product development
staff? The best products are supported
by a very lean sales staff—great products sell themselves.
Technology and Cultural Change
Technology is vital to the success of an analytics initiative, but it is
only one part of the solution. The
meaningful use of analytics is one of the most difficult things for
organizations to achieve, culturally. A
successful analtyics implementation establishes the technology as well as the
sustainable cultural changes required to turn the insights from data into
improvements in patient care and reductions in cost.
Technology
When evaluating a vendor’s technology, be sure to look at the following:
Data Modeling and
Analytic Logic
Different vendors’ analytics solutions feature different data models. Which
data model they use can have a significant effect on the cost, scalability
and—especially-- the adaptability of your analytics solution to support new use
cases. Rapidly adaptable and very flexible, a bus architecture is the best
data-modeling option for healthcare. Most vendors utilize a healthcare-specific
enterprise data model at the heart of their solution, but these enteprise data
models are difficult to load and map initially, and slow to evolve
subsequently, when faced with new use cases and source system data content.
Over-modeling data is the single most significant contributor to data warehouse
and analytics failure in healthcare. My
advice is simple: Stop modeling your
data and start relating it. Relating
data is what analytics is all about.
These enterprise data models come in three basic flavors, so be aware of
them. They are: (1) Dimensional star schema; (2) Comprehensive, enterprise data model; and (3) I2B2.
In addition to the issue of data modeling, the analytic logic associated
with the content of data marts and reporting is critically important. To learn more about the role of data modeling
and “binding” data to business and clinical logic in healthcare analytics, read
this white paper: The Late Binding Data Warehouse.
Master
Reference/Master Data Management
The ability to incorporate data from new and disparate sources into your
analytics solution requires significant expertise in master data management. What
is the vendor’s strategy for managing unique patient and provider identifiers? How
does the vendor accommodate international, national, regional and local master
data types and naming conventions? Do they support mappings to RxNorm, LOINC,
SNOMED, ICD, CPT and HCPCs? How tightly
does the vendor bind your data to the vocabularies that change regularly? The tighter the binding, the less flexible
the analtyic design will be to accomodating changes in the vocabulary and
analytic use cases based on those vocabularies.
Metadata
Repository
An effective metadata repository is the single most important tool for the
widespread utilization and democratization of data in an organization. Look for a vendor that provides a tightly
integrated, affordable, simple repository with their overall analytics
solution. The most valuable content in a
metadata repository is not computable—the most valuable content is subjective
data that comes from the data stewards and analysts who have interacted most
with the data. Look for vendors that have the ability to maintain this
subjective data through a wiki-style, wisdom of crowds contribution model. A web-searchable
metadata repository should provide information such as the source of the data,
how often it is updated, examples of the data, natural language descriptions of
the physical data tables and columns, any known data quality issues, and the contact
information for the associated data steward. The ability to quickly establish the
origins and lineage of data in a data warehouse is also a critical component to
an effective repository. Analytic vendors
tend to operate in one of two extremes: (1) They either oversell very
complicated and expensive metadata repositories that require an overwhelming
level of support and maintenance in return for a declining return on
investment; or (2) They offer no solution for metadata management, which is
disasterous to a long term analytic strategy.
Find a vendor that offers a simple, low cost, pragamatic solution
between these extremes.
Managing “White
Space” Data
Does your analytics solution offer a data collection alternative to the
proliferation of desktop spreadsheets and databases that contain analytically
important data?
White space data is the data that is collected and stored in desktop
spreadsheets and databases that it is not being collected and managed in
primary source systems, especially EMRs, or it is being collected in clinical
notes and must be manually abstracted for reporting and analysis. This desktop data fills in the missing “white
space” of analytic information that is important to the organization. For example, these desktop data sets are
commonly found in support of Joint Commission reporting, internal KPIs, finance
analytics, and clinical researchers. It
is not unusual for healthcare organizations to have hundreds of these desktop
data sources that are critically important to the analytic success of the
organization. However, because the data
resides on desktop computers and shared drives, it cannot be integrated with
other mission critical analytic data that is being stored in the Enterprise
Data Warehouse from the primary source systems.
Data synergy suffers as a result.
White space data also poses information security risks. Analytics vendors must provide a tool for
attracting the management of white space data into the content of the EDW. Look for a white space data management tool
that is web based, as easy to use as a spreadsheet or desktop database for the
collection of data, and makes is easy for end users to convert and upload their
existing desktop data sets. Also, look
for a security model in the EDW that allows for the isolation and stewardship
of these white space data sets.
Visualization
Layer
The best analytics solutions include a bundled visualization tool, and this
tool should be affordable and extensible if licensed for the entire
organization. However, the analytics
visualization layer is very volatile.
The leading visualization solution today will not be the leader
tomorrow. Therefore, look for an analytics
vendor that can quickly and easily decouple the underlying data model and data
content in the data warehouse from the visualiztion layer, and swap the
visualization tool with a better alternative when necessary. Also realize that
a single visualization tool will not solve all of your organization’s
needs. Data analysts will want to use a variety
of tools to access and manipulate data in the enterprise data warehouse. The underlying data models in the data
warehouse must be capable of supporting multiple visualization tools at the
same time. Ask vendors if their data model is decoupled from the visualization
tool. Does the data model support multiple visualization tools and delivery of
data content?
Security
As always in healthcare IT, the privacy and security of patient data are
paramount. Here are some important questions to ask a potential analytics vendor
about security:
·
Are there fewer than 20 roles in the initial deployment? Contrary to popular belief, more roles can
actually lead to lower security and will definitely lead to higher overhead
adminisrative support costs.
·
Does the solution employ database-level security, visualization-layer
security or some combination of both?
The vendor’s solution should support both.
·
What is the vendor’s model for protecting patient identifiable (protected
health information (PHI)) data and the more sensitive subsets of PHI that are
typically defined at the local State-level, such as mental health data, HIV
data, and genomic/familial data?
·
What type of tools and reports are available for managing security and
auditing access to patient identifiable data?
ETL
A robust ETL process—how analytics technology extracts data from source
systems, applies the required transformations and writes data into the target
database—is fundamental to the success of your chosen solution. Ask vendors to
demonstrate how their ETL measures up in terms of reliability, supportability
and reuse. At present, Microsoft’s ETL
tool—SQL Server Integration Services (SSIS)—is by far the most cost effective
ETL tool in the market, offering the highest value per dollar.
Performance and
Utilization Metrics
As you implement and continue to use an analytics solution, you will need
to generate metrics about who is using the system, how are they using it, and
how well the system operates. Can the vendors’ solution track basic data about
the environment, such as user access patterns, query response times, data
access patterns, volumes of data and data objects? This kind of information
will be essential to you as you refine and organize the data content and analytics
services you provide from the data warehouse.
Hardware and
Software Infrastructure
Does the vendor use Oracle, Microsoft or IBM for its hardware and software
infrastructure? These three are the only viable options in today’s healthcare
market and data ecosystem. Hadoop and
its associated open source tools is not an appropriate analytic and data warehousing infrastructure
option at this time in healthcare (accept for gene sequencing). Microsoft is
the most integrated, affordable and easiest to manage of these technology
platforms and now makes up 70% of all new sales in the analytics and data
warehousing market, across all industries, far outpacing Oracle in new sales,
it’s closest competitor. Rumors about Microsoft’s ability to scale up to large,
multi-terabyte data warehouses are totally unfounded. Microsoft’s parallel data warehouse platform
can scale to the petabyte level, far beyond the largest data warehouse needs in
the healthcare provider space. Ask any
data engineer that has worked extensively on either Microsoft or Oracle, “Which
platform is the easiest to use and most efficient for delivering quick,
adaptable analytics solutions?” Their
answer will almost certainly be Microsoft.
Cultural Change
Management
As mentioned previously, technology is important—but it is only a part of
the equation in creating a successful analytics program. A vendor’s solution
must also include processes and real-world experience for helping your manage
sustainable change in your organization, driven by analytics. Nothing is more politically or culturally
disruptive than the spotlight of analytics, not even the deployment of an
EMR. You want a vendor that has been
there, in the trenches of cultural transformation driven by the enlightment of
data.
The Healthcare Analytics Adoption Model
The model outlines
eight levels of analytics adoption an organization passes through as it gains
sophistication in using its data to drive improvement. Like a course curriculum for college studies,
following this model with discipline will lead to the successful adoption
of analytics in your organization, culturally
and technically. Use this model for
evaluating vendors’ capabilities in each level—have the vendor demonstrate
their products and services for each level. The model also provides a roadmap
for your organization to measure your progress of analytics adoption. Ask yourself, “How fast do we want to achieve
the highest levels of adoption in this model?” With the right analytics vendor
as a partner, organizations can achieve Level 5 within 18 months of
implementing and following the model, and some organizations can make it in 12
months. Level 7 is easily achievable within 24-30 months.
This model provides a standard to help you move beyond a patchwork of point
solutions to a robust, data-driven health delivery system capable of tailoring
care while optimizing efficiency and cost. Does a vendor’s solution support
each level in the model? Ask them to
prove it.
The details behind this model, including a self-inspection guide, can be
found at: Healthcare Analytics Adoption Model
Conclusion
Healthcare is at the threshold of the next revolution in data
management—being able to analyze and make informed decisions based on the data that
organizations have been collecting and sharing. This is a critical time to set
your organization on a pathway for data-driven improvement. The criteria
outlined in this paper will help you choose an analytics partner with the
expertise, processes and technology to help you achieve this objective.
No comments:
Post a Comment