Toolkits

mHealth - VTC Testing Results

We tested 9 different tablets and smart phones in this evaluation. The goal of this testing serieswas to discover the efficacy of each device for video teleconferencing, given variables in connection method and video standards. We wanted to test each device over both standards-based and non-standards-based video teleconferencing technology, using both a Wi-Fi connection and the local cellular network. Standards-based video teleconferencing uses the ITU (International Telecommunications Union) standards that define the equipment and protocols needed to support videoconferencing. Non-standards-based video teleconferencing does not. We conducted our standards-based tests over the lab’s PolyCom system, and our non-standards-based tests over Skype.

Saturated Wi-Fi line with tablets, calling a standards-based VTC bridge (PolyCom)

We connected the new iPad and the Samsung Galaxy 2 Tablet to our PolyCom bridge, and were unable
to connect the other pads in this test. We realized that the pads were set to stream at 512 Kbps, which
took all of the lab’s 1 Mbps upload capacity. We reset them all to 256Kbps and were able to connect
four – the ASUS Nexus 7, the Lenovo Ideapad, the Galaxy 2 Pad, and the iPad, using exactly our allotted
1 Mbps upload ration. We anticipated degradation in the quality of both audio and video on the
saturated line. The images were pixelated on some screens and the video frames froze on others, and
the audio cut in and out when all of the bandwidth was in use. The new iPad and the Samsung Galaxy
had slightly less pixilation than the ASUS Nexus 7. The Lenovo could not stay connected to the saturated line.

Saturated Wi-Fi line, using smart phones, calling a standards-based VTC bridge (PolyCom)

We disconnected one tablet device to free up 256Kbps for mobile phone testing. We tested the
Samsung Galaxy S III, iPhone 4S, the LG Optimus, and the HTC One x using Wi-Fi to connect. All four
phones required the PolyCom TelePresence app to be downloaded before the device could call in to the
bridge. Using Wi-Fi, all four smart phones connected really well to the PolyCom system, with clear video
images that tracked motion well.

4G, using smart phones, calling a standards-based VTC bridge (PolyCom)

Alaska has a recently deployed 4G network, and with our new smart phones, we wanted to test it out
for video streaming. Unfortunately, none of the Android products could reach the PolyCom bridge to
log in to the video conference.  We reported this to our carrier, and the carrier and manufacturers are working on a solution. Unfortunately, we didn’t have resolution before the testing period ended.
 
The iPhone 4S, however, dialed directly in to the PolyCom bridge. The iPhone 4S isn’t rated for
use on 4G networks, so we believe the connection was made over the 3G network. The audio and video
worked quite well, with no freezing frames or audio distortion, though the image was not as crisp as
those we saw on the tablets.

Tablet to Tablet, clear Wi-Fi line, calling over a non-standards-based VTC bridge (Skype)

The first tablet-to-tablet VTC call we made was from the ASUS Nexus 7 to the new iPad. We noticed
immediately the superior picture clarity displayed by the Nexus 7. We suspected it had to do with the
Nexus 7’s quad-core processor vs. the iPad’s dual-core. While the iPad tracked movement without
freezing and had clear audio, the picture had a much lower resolution (VGA quality video on the iPad)
when compared to the Nexus 7.
 
The second call from the Nexus 7 was also to a dual-core processor tablet -- the Samsung Galaxy 2. Both
had a great display using the front camera, but the back camera on the Galaxy surprised us. Rather than
a lower-resolution, full-screen video display like the iPad, the Galaxy 2 showed a very clear, tight picture,
but it only used the bottom half of the screen in landscape format.
 
The back-camera half-screen issue presented itself again when we tested the Lenovo Ideapad dual-
core processor tablet calling the Galaxy 2 over Skype. The Lenovo’s front 1.3 MP camera took the image
down to a 4x3 aspect ratio. We found that the iPad was the only tablet that made consistent use of
100% of its screen real estate.
 
We finally tested the two quad-core processor machines, the ASUS Nexus 7 calling the ASUS
Transformer Infinity tablet. As we expected, the quad-to-quad call resulted in the best video clarity. The
ASUS Transformer Infinity had a great, full-screen high resolution video display, but then it arbitrarily
shrunk to an equally clear, but much smaller square display.

Phone to tablet, clear Wi-Fi line, calling over a non-standards-based VTC bridge (Skype)

We were curious to see how the quad-core phone worked calling in to a quad-core tablet over Wi-Fi. We
used the HTC One x and called the ASUS Transformer Infinity. The image from ASUS to the HTC screen
was clear and sharp. We anticipated that the quad-to-quad display would be superior on both devices,
but the ASUS tablet displayed red and green vertical lines over the caller’s image, then the call dropped
from Skype. We recreated this multiple times with various tablets and phones, and the green tone
appeared on the receiving end of all test calls we made from the HTC One x.

Additional Tests to Capture Images

We finally used each device to call into a MacBook Air laptop, on which we captured screen images of
each video stream for demonstration purposes. All of the devices were tested on the same day, over the
same network, and the images were captured in the same environment with the same lighting.
 
Three people were involved in the image capturing process. One person called the laptop from each
device, another captured images, while a third person documented relevant data.
 
The captured images were broken down into two categories: technical images and subject images. For
the technical images, we used a digital camera resolution chart. These charts are used to help assess a
camera’s ability to detect and display detail. For the subject images, we used a TTAC staff member. The
images below, and their captions, will demonstrate our findings.
 

Technical Images

Images were captured with each mobile device, with the technical imaging target consisting of a resoution chart.  The resolution chart consists of a variety of horizontal, vertical, and diagnol lines that should be clear in all images.  Note that for many of the images, individual lines are hard to discern; many devices struggles to properly capture and transmit a clear video signal.

Apple iPhone 4S - Technical
HTC One X - Technical
LG Optimus 4X HD - Technical
Samsung S3 - Technical
ASUS Nexus 7 - Technical
ASUS Transformer Infinity - Technical
Apple iPad (3rd Generation) - Technical
Lenovo Ideapad - Technical
Samsung Galaxy Tab 2 - Technical

Technical Images

Subject Images

Images were captured with each mobile device, with the subject imaging target consisting of a human face.  The face was chosen, as it represents a common subject matter for those performing neurological or psychological evaluations over videoconferencing systems.  Note that for many of the images, important facial features are hard to discern; many devices struggle to properly capture and transmit a clear video signal.

HTC One X - Subject
Apple iPhone 4S - Subject
LG Optimus 4X HD - Subject
Samsung S3 - Subject
ASUS Nexus 7 - Subject
ASUS Transformer Infinity - Subject
Apple iPad (3rd Generation) - Subject
Lenovo Ideapad - Subject
Samsung Galaxy Tab 2 - Subject

Subject Images

General Issues

We noted the display functions of each tablet were widely varied. The iPad’s Picture-in-a-Picture (PiP)
showed all three callers in a grid, plus a small view of its own user in the lower right hand corner of
the screen. The ASUS Nexus 7 had a very impressive, sharp video image for both in-motion and still
streaming, however, it sometimes displayed the PiP image upside-down, with no intuitive way to
reverse it.
 
The Samsung Galaxy 2 Tablet, as noted above, would shrink the video image to half the screen size
rather than use a full-size image when the video output was switched to the back camera. The Lenovo
Ideapad and both ASUS tablets had clear incoming video, but arbitrarily resized it during the call.
 
The new iPad, the Lenovo Ideapad, and the Galaxy 2 also had gyroscopes working during video calls,
so that regardless if the tablet was placed right side up or upside down, the camera image it displayed
stayed upright. Neither ASUS had a gyroscope function in its video system so tilting the camera also
tilted the video image.
 
After initiating each video call, we muted the devices’ microphones to avoid echo in the lab. We noticed
more than once on both the ASUS Nexus 7 and the ASUS Transformer Infinity that the screen showed
the microphone icon with a line through it (which indicates that the mics are disabled) but they were still on.
When we tapped the icon twice more (once to ‘unmute’ and once again to mute), the microphones did turn off.
 
We also noted that, when calling in to Skype from any of the Android OS devices, the first call would
always drop immediately after connecting. We consistently had to place the call a second time to
connect and stay connected. This problem didn’t present itself with the iOS outgoing video calls.

The Results

The TTAC is unable to share the numerical results from this evaluation as we are federally funded. Numerical
scores may be construed as a product endorsement or recommendation, and we must be very mindful
that we do not actively endorse any single product. The overall evaluation process is much more involved
than simply looking at a table of numbers and making a decision. The result of an evaluation needs to take
into consideration the end users and their experience, the goals of implementing the device (screening as
opposed to diagnosis), and the existing infrastructure that the equipment needs to work with.
 
We can share a variety of other resources, which may be found in this toolkit. These include product
cut sheets, which allow for a comparison of the many features of the tablets and smart phones. We are
also open to any questions about the process, or about how video teleconferencing may be used within your
organization and how to match your organizational needs with an appropriate product.