The digital camera holds a unique place in the telehealth technology market. On the one hand it is a common device used by many people, typically taken from the consumer market and used in a clinical setting. These devices are relatively cheap, easy to use, and familiar to most users. On the other hand, the digital camera market is extremely volatile, with annual changes in product availability, usability, and price, as manufacturers put out newer models, retire the old, and change available features and functions.
Choosing the right camera for a telehealth program can be difficult without some guidance. This whitepaper outlines which factors should be considered in the course of choosing a camera. Five primary elements will be discussed, including determining minimum requirements, performing a market review, determining “Ease of Use” and basic mechanical functionality, evaluating image quality, and performing a final selection for your unique telehealth program.
Determining Minimum Requirements
Establishing a base set of requirements is critical when starting out on any technology assessment. This is especially true in the case of digital cameras, where numerous clinical applications may be served by a single device, requiring the camera to perform different functions in a variety of settings.
Define the Users
Knowing which users will be utilizing the digital camera is an important first step in creating the minimum requirements. Questions to ask include:
- Which specialties will use the camera?
- Dermatologists may need high color accuracy and the ability to image both dry and moist tissues
- Otolaryngologists and dentists may need to image intra-oral cavities and capture mucous membranes
- Physical therapists may need to capture “wider” images and videos that demonstrate active range of motion and mobility
- Home care specialists may need to use the camera for wound management, while radiologists may image high-contrast radiographs
- What is the level of expertise of the user?
- Rare or infrequent users may not retain proficiency in manually adjusting settings and navigating menus
- Novice users may be intimidated by complex cameras and may need guidance in how to acquire quality images
- Frequent users may be able to adjust a camera to fit their imaging needs
- Experienced users may not need additional training on how to use a digital camera
- Specially trained imaging technicians may be able to properly use Digital Single Lens Reflex (DSLR) cameras
Define the Deployment
Establishing where the cameras will be deployed, as well as how they are supported, can help set limits and expectations for how to purchase, configure, and support the camera in a telehealth program.
- How many cameras are needed?
- Some departments or clinics may share a camera, which can be cheaper overall, but may cause more cameras to disappear as individuals move, use, or lose the camera
- All individuals may need a camera, which puts responsibility for maintaining and protecting the camera on the individual provider
- Some telehealth platforms may be given their own camera, which may have a variety of implications depending upon the platform used
- How will the cameras be deployed and supported?
- Individual organizations may be responsible for purchasing and supporting the cameras, resulting in a lower cost to the telehealth program or project office, but possibly resulting in non-standard devices and configurations in the field
- A telehealth project office may be responsible for managing the purchase and support of the cameras, resulting in the possible purchase of hundreds of cameras for a single deployment
Define the Required Functionality
There are dozens of features and terms used in the marketing and sale of digital cameras. Knowing which of these functions will be useful in a clinical setting is important to ensuring that the right camera is chosen for your telehealth program.
- Point-and-shoot capabilities
- Do users need to easily capture images?
- Do users need a light-weight camera?
- Do users need to be able to perform imaging with only one hand?
- Auto mode
- Can users acquire images without having to adjust settings for each environment?
- Does the camera automatically know when to flash, how to white balance, and when to shoot in macro mode?
- Does the automatic flash provide appropriate levels of illumination?
- Does the automatic white balance compensate for various light sources?
- Does the automatic macro mode provide images in sharp focus?
- Does the macro function allow images to be taken closer than 10cm to the subject?
- Image Quality
- Are the colors produced by the camera accurate?
- Is there over-saturation or color casting in different lighting conditions?
- Are images sufficiently detailed to allow zooming and magnification of the image?
- Ease of Use
- Does the camera have controls and menus that are easily accessible?
- Are the camera’s cables, ports, and docking stations easily accessible?
- Is the camera appropriately sized for users’ hands?
Hypothetical Case Study
For the sake of demonstration, we’ve included a hypothetical evaluation scenario that focuses on cameras being deployed to multiple users with different skill levels within the same large organization. Rural clinics with minimal imaging capacity and infrequent camera use are assumed, as are expert imagers who use the cameras on a near-daily basis. Clinical applications for the cameras are diverse, with a heavy emphasis on dermatological conditions and intraoral examinations, but with the expectation that the cameras will be used for other purposes in both small clinics and large hospitals. 400 cameras are needed, with a $400 limit per camera due to budget constraints. The cameras will be configured and maintained by the project office. The diverse user base requires a camera that has strong automatic functions and is easy to use. Image quality and accuracy are very important for the selection of the camera.
The Market Review
Being equipped with a set of minimum requirements will make the market review process much simpler. That being said, there is still a lot of work that must be done before cameras should be ordered for review.
The first step in a market review for digital cameras is to look at which cameras have been slated for release in the current year. Cameras have a short lifetime on the market, so choosing a camera that is a recent arrival is a good way to ensure that it will be available during the course of a deployment. It can be very frustrating to complete an evaluation only to find out that the camera selected is no longer available by the time the bulk purchase order is ready to be submitted.
Most manufacturers do not make the information about their release dates easy to find. The Telehealth Technology Assessment Center tries to maintain a list of cameras released each quarter within its Digital Camera toolkit. Digital Photography Review, an online site that hosts camera reviews, also includes a chronological list of cameras that have been released by all manufacturers.
The process of elimination can begin once a list of cameras has been compiled. Cameras that do not meet the minimum requirements should be removed from the list. It may be beneficial to choose only one or two cameras from the same manufacturer if the manufacturer has multiple cameras that meet the minimum requirements.
The list of cameras should be reduced to a manageable number by this point. If the list is greater than ten, consider reducing the number by using online resources, camera specifications, and existing sample images to further narrow down the number of cameras slated for assessment.
Hypothetical Case Study
Our hypothetical team has identified Macro-imaging capacity of 10cm or less as a key requirement, as dermatologists are a core demographic group for this telehealth program. Additionally, there is a strong need for a camera that rates highly in terms of Ease of Use, as the intended users are expected to have limited training in complex camera features.
A minimum price point is set at $200 USD, given that previous digital camera assessments have shown that the cameras beneath that threshold do not meet clinical imaging quality requirements. Other assessments have also led to a preference for optional docking stations.
In looking at the 72 digital cameras released in 2009 that met the $200-400 price point, there is a problem; only two cameras in the lineup meet the requirement of having a docking station. Docking stations, which had been a standard part of many cameras only two years earlier, are now not an option. The minimum requirements must change to reflect the reality of the current market, so our team eliminates docking stations as a requirement in the camera lineup.
As a result, nine cameras are selected as candidates to bring in for a hands-on assessment. These are the Canon PowerShot SD970 and the A1100 IS, the Casio Exilim EX-FC100, the FujiFilm FinePix FR200EXR, the Nikon Coolpix S630, the Olympus Stylue-7000, the Panasonic Lumix DMC-ZS3, the Pentax Optio P70, and the Samsung TL320.
Setting Up For Testing
It is tempting to head to the nearest electronics store to try out each of the cameras that are on the final market review list to save money. While this may help remove a handful of cameras that exhibit blatant shortcomings, it will not suffice to establish which of the cameras will be the best for a telehealth program. Buying the cameras – whether through a physical store or an online retailer – will help ensure consistent tests and ample opportunity to put each camera through its paces.
Organization is crucial when the cameras first arrive at the testing location. Cables and battery chargers can quickly get mixed up or lost in the evaluation process if care is not taken to keep things organized from the very beginning. Identify and label the relevant components for each camera. At TTAC, we place each device and its accessories and manuals into a clear plastic container, though the box that the camera initially came in may be sufficient.
Establish the testing criteria before diving into full camera reviews. If multiple people are participating in the review process, it is useful to arrange some initial meetings to ensure that everyone understands what the evaluation process will entail.
Before testing begins, create standardized tools for documenting the results of the evaluation. This may be as simple as a spreadsheet with standard fields, or may include databases and computer-based tools.
Hypothetical Case Study
The nine cameras have begun arriving at the project office’s warehouse. As they come in, the cameras, batteries, chargers, cables, and manuals are all placed into clear plastic bins. These bins are labeled with which camera they contain, and can be easily stacked and identified within the test lab.
A testing team comes together to provide feedback on the cameras. After the initial requirements are discussed, a more detailed set of evaluation criteria is passed around and agreed upon. Minor changes are made as the team discusses how the tests will be completed. When everyone understands the scope of the evaluation, testing can begin.
Assessing the Cameras
The initial review should focus on the basics of the mechanics and functions of the camera. Is the camera easy to use? How does the camera feel? Do the automatic functions work well? How configurable are the various automatic and manual settings? The exact review questions may vary, and additional questions may come up as features are discovered as the cameras are used.
The next phase of the camera assessment should look specifically at factors relating to the quality of the images taken by the camera. The previous tests should have assisted in establishing familiarity with the functions of the cameras; the next tests will compare both color accuracy and image detail in a variety of settings.
Pictures should be taken in an environment that closely mimics the clinical settings intended for the camera’s eventual use. Common elements include fluorescent or mixed fluorescent/natural lighting, possible uneven or low light levels, and relatively short distances between the subject and the camera.
Similar images of a single subject should be captured by all cameras before moving on to the next series of photos. This reduces the possibility of differences in distance to the subject, lighting conditions, and subject positioning.
Settings should be consistent across all cameras. Ideally, multiple images will be captured on the same camera in different modes, such as with flash on, flash off, and flash set to auto. This allows for a more thorough evaluation of a camera’s image quality.
Many possible images can be taken for comparison, and the planned clinical use of the camera should drive which images are used in the course of assessment. Common subjects for imaging include the extremities, face profile, back, mouth, tongue, hands, feet, eyes, skin pathologies, mesh objects, and various still-life items. The five sample subjects that best test the quality of a camera’s images include the back, a facial profile, a hand, a close-up shot of the thumb or fingers, and a mole or rash.
Track the settings for each photo, and then upload the images to a computer for review. Number the photos so that they can be quickly compared across cameras. An example might be Mole – Auto Flash Auto Macro – Kodak M1063 – Image 1.jpg. Comparing images will be easier if a consistent naming scheme is used.
When assessing the color accuracy aspect of image quality, a reviewer is looking for accurate skin tones and sufficient contrast between various colors. Some cameras will produce images that are flat and cadaverous. Others may produce overly-saturated images.
Reviewing image details is important, and often requires zooming in on the original image to look for sharpness and clarity in the subject matter. Some photos will appear clear at a smaller size, but blurring may occur when the image is viewed at full size. Try to find points of detail to compare between images, such as fine lines in the hand, body hair, mole appearances, and fabric texture.
Rating these aspects of image quality should happen on a 4-point scale, as opposed to the 5-point Likert scale used in the earlier assessment subjects. This forces the reviewer to decide if an image is good or bad, and removes the possibility of many images being placed as simply “OK”.
Hypothetical Case Study
The assessment team goes over the rating system with each camera, and works to get comfortable with how the different devices operate. Evaluators try to force errors when using the cameras, predicting ways in which end users might make mistakes while taking photos.
An imaging station is set up, and all cameras are run through the paces of an image evaluation. Each camera takes a pre-planned picture of the face, eyes, mouth, human profile, back, arm, hand, finger, knee, feet, lesions, scars, wounds, and “still life” scenes. Similar settings are used for each camera, with images taken with the flash forced off, forced on, and automatically set.
The assessment process reveals an unexpected feature. Whereas previous evaluations of cameras have shown that macro mode must be manually engaged, many of the new models come with the ability to automatically detect which mode to shoot in. The test plan is updated with this new evaluation item, and all cameras are assessed based on whether or not they can shoot in auto-macro mode.
The images from all of the cameras are compared side-by-side, utilizing a custom-designed application that allows simultaneous viewing, zooming, and panning of multiple images. The photos are all evaluated for image quality, with specific attention paid to color accuracy and image detail. With all assessments complete, it is now time to start making decisions.
Making a Short List
The next step in the assessment process is to create a “short-list” of three or four cameras based on the initial test results. Look at the scores of the cameras, and, if multiple people are involved, discuss differences in how cameras were rated. Decide which ratings are most important to the intended users. Is a low ease of use score a problem, or will users be able to learn how to use a more complex camera? Are certain features not required, or are they more important than others?
Spend time as a group with the smaller number of cameras. If possible, include outside participants. Give the cameras to people who have never used them before, and see if they can perform basic tasks without extensive guidance. The extra time spent reviewing the cameras, especially if end-users are included, can help to make a final decision on which camera is best suited for a particular program. When the data is all compiled, make a final selection.
Hypothetical Case Study
In the course of comparing the results and discussing discrepancies between reviewers, a short list of preferred cameras is created. Four cameras have a mean score above 4.0 in a Likert scale for both Functionality and Ease of Use. Of those four, three clearly have superior ratings for Image Quality.
The cameras are taken in for additional review, with clinicians brought in to use the cameras and provide their feedback. They discuss the various qualities and concerns around each device.
As discussions proceed, 12 key issues are raised as being of importance to their telehealth program. Each camera is evaluated on the 12 issues, and a final decision is made based upon those results.
Conclusion
Selecting a digital camera can be a laborious process, but it is essential in order to ensure that the correct camera is selected for a telehealth program. Each program’s needs will be slightly different, and what may work for one group may not work for another.
Flexibility is critical when planning and implementing an evaluation process, as the camera features available in a given year are constantly changing. Having all of the cameras available and on-hand ensures that newly discovered features can be tested on all devices. As the market changes, and cameras are no longer available, there needs to be a standard process in place for choosing replacement equipment for current and future sites.
Epilogue to the Hypothetical Case Study
The top-rated camera is purchased and shipped to the initial deployment sites. After the first orders arrive, and as the next sites to receive an upgrade were selected, our team discovers that the camera is no longer available in large quantities. Resellers offer refurbished or used cameras, but new devices are no longer available in numbers that meet the program’s needs. A new camera has to be selected for use at the remaining sites.