Time put towards planning and performing research at the beginning of the technology assessment process is time well spent. Decisions made later in the assessment process will largely be directed by the work done in these critical first steps. There are really two primary areas to focus on in these initial efforts – determining what your organization needs, and determining what the current market can offer your telehealth program.
Step 1: Establish Requirements
Organizations may find themselves starting the process of technology assessment with little more than a high-level goal or position statement. More likely, however, an organization will have a researched need, documented disparity in available services, or established telehealth program that they wish to replicate. Regardless of the level of information gathered, work will need to be done to compile this information in a manner that is useful to those who will be performing the technology assessment. It is important to pull together people from relevant areas of your organization at this early stage of the process, so that clinicians, technical experts, and organizational administrators can provide input.
Needs Assessment
The need to perform a technology assessment does not spontaneously develop; there is a need – often clinical or financial – that can only practically be met through some variety of telehealth service. While this initial need is often fairly instructive, work should be done to clarify precisely what is desired in the implementation of a new telehealth service or program. The following processes can help to clearly identify the needs your selection will have to address.
Position Statements
Position statements provide a high-level view, but help explain why your organization is starting an endeavor and what the end goal is. Position statements typically address who is being served, why they will be interested in your services, what your service will offer, why that is important, and how it is different from other organizations or other ways of providing care.
For example, if your organization plans to start a teledermatology program, it may be useful to further define what exactly is intended. Your position statement may be “we will provide teledermatology to patients in remote locations that lack access to broadband connectivity, who live in underserved locations that lack dermatology specialty services. We will provide 20 hours of specialty services a week to our partner organizations, who otherwise would not have access to dermatology services without incurring a significant cost due to travel.”
The people who started the initiative may understand who the program will serve, and why it is important. A statement like this can give people who join the project later on a clear vision for this endeavor, and helps provide some guidance to future decisions. This process, which may appear to be too high-level to be of much use, will be helpful later on. The above statement clarifies that there are connectivity problems, that participating clinics are partnered with your organization, and that you are focusing on places that do not already have access to dermatology services.
User Profiles
The creation of user profiles will help to define a set of needs that require consideration during your technology assessment activities. These profiles can be as detailed or as broad as an organization wants, and the level of detail that is most useful to an organization may vary. At the highest level of detail, a profile might simply specify the titles of intended users. More detailed descriptions may include known concerns within a user base, such as discomfort with technology, specific workflows, prejudices and biases, etc. Regardless of which level of detail that your organization chooses to provide with each user profile, it is important to spend time thinking about everyone who will be using this technology, and how they will interact with it.
Use Cases
As with user profiles, use cases can be as high-level or as detailed as necessary. While user profiles look at who will use the system, use cases look at how they will use it. Your organization may want to consider the following when creating use cases:
- Are there particular physical constraints on the technology, such as the need to use devices with one hand, the need to frequently move equipment between rooms and other physical locations?
- Does the equipment need to interoperate with existing technology or systems?
- How will the technology fit into existing infrastructure and workflows?
- How will different users (possibly specified in user profiles) interact with the system? Will people use the devices in different ways, depending on their role?
Requirements Gathering
Use cases and user profiles are helpful starting points in the creation of clear requirements for your technology. Create a list of features that the devices must have, as well as a list of features that are desired and those that are either unnecessary or detrimental to meeting the needs identified in the user profiles and use cases. This initial list of requirements that are gathered will likely lead to the development of additional requirements. Some questions to discuss with your evaluation team include:
- Why is that feature needed? Is it really needed, or is it optional?
- How will that requirement impact users?
- Have the needs of all people who will interact with the device been considered – clinicians, patients, technical staff, maintenance staff, cleaning staff?
SWOT Analysis
SWOT (Strength, Weakness, Opportunity, Threat) analysis, while not absolutely critical to a successful device evaluation, can help establish a list of resources and concerns for consideration in your assessment process. SWOT assessments will require input from many different areas of your organization. Again, the importance of an interdisciplinary team cannot be emphasized enough at this point. This assessment looks at strengths (things that your organization does well), weaknesses (things that your organization needs to improve), opportunities (external factors that your organization should act on or could benefit from), and threats (external factors that pose a risk to your organization).
There are many resources in print and online that look at how to properly perform a SWOT analysis. Additionally, skillful SWOT analysis is applicable to a variety of organizational functions, and not simply technology assessment. Performing a SWOT analysis can help further refine what you are trying to do, and why.
This process may need to be repeated after additional information is acquired. Should, for example, a network engineer join the assessment team after the initial SWOT process is performed, it can be useful to ask the new team member to think through what strengths, weaknesses, opportunities, and threats may exist in the area of their expertise.
Step 2. Market Review
Ideally, you organization will have a firm grasp of what the general requirements are at the end of the needs assessment process. The next step is to look at what is available on the market. This review should include discussions with vendors and manufacturers, review of various third-party assessments of the devices (including other telehealth programs that have implemented similar programs and technology), product specifications, and additional research into fundamental principles behind the technology.
Vendors and Manufacturers
Working with the people who build and sell any particular technology can be an interesting balance between gathering information and processing sales pitches. Many manufacturers are excited to talk about the fundamental principles being their technology. Having a person in your organization who can clearly communicate with a manufacturer’s engineering staff can be useful when gathering information about how technology works.
Discuss the options that exist within a manufacturer’s product line, and see if there are important distinctions between various offerings. Most manufacturers and vendors will be able to provide a product cut sheet that lists specifications and additional information on how the device works. Be wary of manufacturer-specific terminology; request that any unclear terms be defined and described so that objective comparisons can be made between specification sheets from different manufacturers.
These initial interactions can be very useful in the information-gathering process. Clearly explain what you are doing regarding technology assessment, and let the vendor or manufacturer know why you are considering implementing a technology. The vendor or manufacturer may be able to provide advice about how to test the device, and may point you to the appropriate product or products in their catalog that will meet your needs.
Third-Party Assessments
Depending on the particular technology being reviewed, third party organizations may be able to provide some input as to how the devices work. Consumer technologies tend to have a variety of options available, such as Consumer Reports or other product-specific review sites. Mobile devices (tablets, phones, etc.) are great examples of a technology with extensive third-party reviews. The problem with some consumer review sites is that they may not consider the clinical application of various devices; the image captured from a mobile device that can make a person look great in front of the Grand Canyon might not produce an image that is clinically accurate, or useful.
Telehealth-specific review sites are less available than those that address consumer markets. Talk to other programs that have implemented the technology in the past to see how their experience with a device has worked out. Also consider communicating with your regional telehealth resource center to see if they have information that might be useful in the market review process.
Product Specifications
The cut sheets and other documentation provided for a device can be useful. The information may help highlight trends or features that were not considered in the initial requirements gathering process. Performing side-by-side comparisons between specification sheets can show areas that may need more research.
The TTAC has found it useful to create a product specification comparison table when performing this portion of the market review, as it helps to establish feature parity between devices, or indicate where there might be differences. The initial comparison document will often change by the end of the review, as new features are discovered and an understanding of the technology is increased.
Technology Overview Documentation
A set of product specifications and list of available products on the market may be sufficient for an organization to move into test planning, but there may still be some utility in further refining this information into a format that is more easily used by those who will be involved in the testing process. A document that explains how the technology works can be useful in ensuring that testers know what to expect, and can help explore the ties that may exist between a technology, its intended use, and the current state of the market.