We tested 9 different tablets and smart phones in this evaluation. The goal of this testing serieswas to discover the efficacy of each device for video teleconferencing, given variables in connection method and video standards. We wanted to test each device over both standards-based and non-standards-based video teleconferencing technology, using both a Wi-Fi connection and the local cellular network. Standards-based video teleconferencing uses the ITU (International Telecommunications Union) standards that define the equipment and protocols needed to support videoconferencing. Non-standards-based video teleconferencing does not. We conducted our standards-based tests over the lab’s PolyCom system, and our non-standards-based tests over Skype.
Saturated Wi-Fi line with tablets, calling a standards-based VTC bridge (PolyCom)
We connected the new iPad and the Samsung Galaxy 2 Tablet to our PolyCom bridge, and were unable to connect the other pads in this test. We realized that the pads were set to stream at 512 Kbps, which took all of the lab’s 1 Mbps upload capacity. We reset them all to 256Kbps and were able to connect four – the ASUS Nexus 7, the Lenovo Ideapad, the Galaxy 2 Pad, and the iPad, using exactly our allotted 1 Mbps upload ration. We anticipated degradation in the quality of both audio and video on the saturated line. The images were pixelated on some screens and the video frames froze on others, and the audio cut in and out when all of the bandwidth was in use. The new iPad and the Samsung Galaxy had slightly less pixilation than the ASUS Nexus 7. The Lenovo could not stay connected to the saturated line.
Saturated Wi-Fi line, using smart phones, calling a standards-based VTC bridge (PolyCom)
We disconnected one tablet device to free up 256Kbps for mobile phone testing. We tested the Samsung Galaxy S III, iPhone 4S, the LG Optimus, and the HTC One x using Wi-Fi to connect. All four phones required the PolyCom TelePresence app to be downloaded before the device could call in to the bridge. Using Wi-Fi, all four smart phones connected really well to the PolyCom system, with clear video images that tracked motion well.
4G, using smart phones, calling a standards-based VTC bridge (PolyCom)
Alaska has a recently deployed 4G network, and with our new smart phones, we wanted to test it out for video streaming. Unfortunately, none of the Android products could reach the PolyCom bridge to log in to the video conference. We reported this to our carrier, and the carrier and manufacturers are working on a solution. Unfortunately, we didn’t have resolution before the testing period ended.
The iPhone 4S, however, dialed directly in to the PolyCom bridge. The iPhone 4S isn’t rated for use on 4G networks, so we believe the connection was made over the 3G network. The audio and video worked quite well, with no freezing frames or audio distortion, though the image was not as crisp as those we saw on the tablets.
Tablet to Tablet, clear Wi-Fi line, calling over a non-standards-based VTC bridge (Skype)
The first tablet-to-tablet VTC call we made was from the ASUS Nexus 7 to the new iPad. We noticed immediately the superior picture clarity displayed by the Nexus 7. We suspected it had to do with the Nexus 7’s quad-core processor vs. the iPad’s dual-core. While the iPad tracked movement without freezing and had clear audio, the picture had a much lower resolution (VGA quality video on the iPad) when compared to the Nexus 7.
The second call from the Nexus 7 was also to a dual-core processor tablet — the Samsung Galaxy 2. Both had a great display using the front camera, but the back camera on the Galaxy surprised us. Rather than a lower-resolution, full-screen video display like the iPad, the Galaxy 2 showed a very clear, tight picture, but it only used the bottom half of the screen in landscape format.
The back-camera half-screen issue presented itself again when we tested the Lenovo Ideapad dual-core processor tablet calling the Galaxy 2 over Skype. The Lenovo’s front 1.3 MP camera took the image down to a 4×3 aspect ratio. We found that the iPad was the only tablet that made consistent use of 100% of its screen real estate.
We finally tested the two quad-core processor machines, the ASUS Nexus 7 calling the ASUS Transformer Infinity tablet. As we expected, the quad-to-quad call resulted in the best video clarity. The ASUS Transformer Infinity had a great, full-screen high resolution video display, but then it arbitrarily shrunk to an equally clear, but much smaller square display.
Phone to tablet, clear Wi-Fi line, calling over a non-standards-based VTC bridge (Skype)
We were curious to see how the quad-core phone worked calling in to a quad-core tablet over Wi-Fi. We used the HTC One x and called the ASUS Transformer Infinity. The image from ASUS to the HTC screen was clear and sharp. We anticipated that the quad-to-quad display would be superior on both devices, but the ASUS tablet displayed red and green vertical lines over the caller’s image, then the call dropped from Skype. We recreated this multiple times with various tablets and phones, and the green tone appeared on the receiving end of all test calls we made from the HTC One x.
Additional Tests to Capture Images
We finally used each device to call into a MacBook Air laptop, on which we captured screen images of each video stream for demonstration purposes. All of the devices were tested on the same day, over the same network, and the images were captured in the same environment with the same lighting.
Three people were involved in the image capturing process. One person called the laptop from each device, another captured images, while a third person documented relevant data.
The captured images were broken down into two categories: technical images and subject images. For the technical images, we used a digital camera resolution chart. These charts are used to help assess a camera’s ability to detect and display detail. For the subject images, we used a TTAC staff member. The images below, and their captions, will demonstrate our findings.
Technical Images
Images were captured with each mobile device, with the technical imaging target consisting of a resoution chart. The resolution chart consists of a variety of horizontal, vertical, and diagnol lines that should be clear in all images. Note that for many of the images, individual lines are hard to discern; many devices struggles to properly capture and transmit a clear video signal.
Subject Images
Images were captured with each mobile device, with the subject imaging target consisting of a human face. The face was chosen, as it represents a common subject matter for those performing neurological or psychological evaluations over videoconferencing systems. Note that for many of the images, important facial features are hard to discern; many devices struggle to properly capture and transmit a clear video signal.
General Issues
We noted the display functions of each tablet were widely varied. The iPad’s Picture-in-a-Picture (PiP) showed all three callers in a grid, plus a small view of its own user in the lower right hand corner of the screen. The ASUS Nexus 7 had a very impressive, sharp video image for both in-motion and still streaming, however, it sometimes displayed the PiP image upside-down, with no intuitive way to reverse it.
The Samsung Galaxy 2 Tablet, as noted above, would shrink the video image to half the screen size rather than use a full-size image when the video output was switched to the back camera. The Lenovo Ideapad and both ASUS tablets had clear incoming video, but arbitrarily resized it during the call.
The new iPad, the Lenovo Ideapad, and the Galaxy 2 also had gyroscopes working during video calls, so that regardless if the tablet was placed right side up or upside down, the camera image it displayed stayed upright. Neither ASUS had a gyroscope function in its video system so tilting the camera also tilted the video image.
After initiating each video call, we muted the devices’ microphones to avoid echo in the lab. We noticed more than once on both the ASUS Nexus 7 and the ASUS Transformer Infinity that the screen showed the microphone icon with a line through it (which indicates that the mics are disabled) but they were still on. When we tapped the icon twice more (once to ‘unmute’ and once again to mute), the microphones did turn off.
We also noted that, when calling in to Skype from any of the Android OS devices, the first call would always drop immediately after connecting. We consistently had to place the call a second time to connect and stay connected. This problem didn’t present itself with the iOS outgoing video calls.
The Results
The TTAC is unable to share the numerical results from this evaluation as we are federally funded. Numerical scores may be construed as a product endorsement or recommendation, and we must be very mindful that we do not actively endorse any single product. The overall evaluation process is much more involved than simply looking at a table of numbers and making a decision. The result of an evaluation needs to take into consideration the end users and their experience, the goals of implementing the device (screening as opposed to diagnosis), and the existing infrastructure that the equipment needs to work with.
We can share a variety of other resources, which may be found in this toolkit. These include product cut sheets, which allow for a comparison of the many features of the tablets and smart phones. We are also open to any questions about the process, or about how video teleconferencing may be used within your organization and how to match your organizational needs with an appropriate product.