The “Assessment Process” section of the video teleconferencing toolkit looks at key information driving our assessment process and decisions. We include an overview of our methodology, a description of how to perform functional and quality analyses of different video teleconferencing devices, and a list products tested.
The Testing Process
Selecting the “best” of any video teleconferencing (VTC) device for mHealth scenarios can be a daunting
task, given the variables in devices and the networking capability in different regions. It is more
important than ever to spend the time performing a thorough evaluation of the products on
the market and compare them to the patient’s needs, technical ability, and type of Internet and cellular
networks available in their area. This should include an assessment of the quality of imagery that the
devices are capable of producing, compared to the intended use. For example, some of our video results
could be beneficial in a telebehavioral health setting, though the same image might not be clear enough
for diagnostic purposes. This section will focus on how we set up our testing environment, and how the results may help the mHealth consumer.
The Goals of Testing
Determining the goals of product testing is critical to shaping the test plan and procedure. With our testing, the focus was primarily on how well the devices functioned as a videoconferencing platform. A series of tests were created within this particular area of interest. Consider similar planning for any other intended use of a mobile platform. Different tests would be created when assessing devices for accessing electronic health records, patient wellness applications, or any other mobile function.
The Tests: Video Teleconferencing (VTC) Based on Connectivity Type and Processor Strength.
Saturated Wi-Fi line with tablets, calling a standards-based VTC bridge
We wanted to determe how the tablets would function if the communication lines were completely saturated when connecting to a standards-based videoconferencing system. The plan was to test connecting via Wi-Fi to a videoconferencing bridge with four 256 Kbps links on a line with a 1 Mbps upload limit.
Saturated Wi-Fi line, using smart phones, calling a standards-based VTC bridge
In a similar test, we aimed to test smart phones on a saturated line. The existing connection with three tablets was maintained, with a smart phone taking the place of the fourth device during the tests.
4G, using smart phones, calling a standards-based VTC bridge
As many users may wish to utilize smart phones for videoconferencing over cellular connections, we set up a test plan that would link smart phones to a VTC bridge on a local 4G connection.
Tablet to Tablet, clear Wi-Fi line, calling over a non-standards-based VTC bridge
There is a significant interest in using mobile devices to perform calls over various non-standards-based videoconferencing applications. We set up tests between two tablets over an unsaturated Wi-Fi connection.
Phone to tablet, clear Wi-Fi line, calling over a non-standards-based VTC bridge
Similar tests were conducted between a phone and tablet, utilizing the consumer videoconferencing software to place a call.
Additional Tests to Capture Images
The final test planned for this evaluation was to connect each device to a laptop computer via consumer videoconferencing software. This test served three purposes. The first of these was to eliminate a variable when placing a video call, as mobile-to-mobile calls require additional guesswork when determining if call quality issues are the result of one mobile device or the other. The second was to assess how call quality would be if either one side of a video call was engaged on a laptop. The last was to support image capture, allowing a static comparison of image quality for those who could not participate in the test directly.