LBVC: Towards Low-bandwidth Video Chats on Smartphones Xin Qi, Qing

LBVC: Towards Low-bandwidth Video Chats on Smartphones Xin Qi, Qing Yang, David T. Nguyen, Gang Zhou, Ge Peng College of William and Mary 1

Video Chats on Smartphones 2

Video Chat Apps - Big Data Consumer E.G., Skype 500MB 1.85 h 3

Our Objective: Reduce Data Usage of Video Chats on Smartphones 4

Outline Measurements LBVC Design LBVC Implementation LBVC Evaluation Related Work Conclusion 5

Reducing Video Frame Rate (1) Bandwidth vs. Frame Rate (H.264 as codec) Good News Constraints Typical frame rates adopted by video chat apps 6 Video Quality vs. Frame (H.264 Rateas codec) TVM Score – a recent objective metric for mobile video quality [mobicom’12]

Reducing Video Frame Rate (2) Power Consumption vs. Frame Rate (H.264 as codec) Power reduction is a minor benefit. 7

Research Question How to reduce frame rate of mobile video chats without introducing obvious video quality degradation? 8

Frame Interpolation to Rescue Video Quality The sender lowers bandwidth usage through reducing frame rate The receiver interpolates “missing” frames to rescue video quality However, large scene changes between two input frames esult in strong artifacts in he interpolated frames 9

Context-aware Frame Rate Adaption At small frame rates, severe device vibrations are the main cause of large scene changes between two consecutive frames To keep scene changes small between consecutive frames, 10 We use inertial sensors to determine whether a device vibration is severe or not Video chats are only recorded with a small frame rate when a device vibration is not severe

LBVC – Low-bandwidth Video Chats LBVC Architecture - Sender New Component 11

LBVC Architecture - Receiver New Component Intermediate Frames Delay 12

LBVC Implementation Implement LBVC as an extension to Linphone Easy to add new components a. Modify the MediaStreamer2 library b. Implement the GUI with Java SDK 13 Add filters into the library to implement the cross dissolve algorithm Use Android API in NDK to collect sensor readings JNI to pass user input frame rate to the C library

Evaluation – Setup Use Shark for Root to capture network traffic and WireShark to analyze network traffic Use Monsoon Power Monitor to measure power consumption Implement a video chat recorder to record video chats and organize a user study to analyze video quality 14

Evaluation – Bandwidth Usage 10 subject pairs, each of them performs video chats while standing or sitting LBVC achieves bandwidth saving by up to 43% 39% Smaller thresholds at lower frame rates esult in larger bandwidth usage variance. 15 43% 35% 22% 13%

Evaluation – Power Consumption 10 subject pairs, each of them performs video chat while standing or sitting Power saving is not a major benefit of LBVC. 16

Walking Lowest chance to save bandwidth tting in a vehicle Percentage of Time at User Selected Frame Rate (%) Consider Different Mobility Cases Car moves fast, but user’s martphone is relatively stable; ncreased chances to save bandwidth itting or standing Highest chance to save bandwidth 17 User Selected Frame Rate (fps)

Evaluation – Objective Video Quality Record video chats and calculate TVM scores with MATLAB. LBVC is able to objectively maintain video quality. 18

Evaluation – Subjective Video Quality 21 subject pairs; each pair rates video chats at different frame rates with LBVC disabled and enabled LBVC maintains subjective video quality ( 4 fps) At low frame rates, minor interpolation artifacts and frequent frame rate switches still affect subjective video quality acceptable level 19

Related Work Video streaming bandwidth reduction Video compression methods, such as H.264 and VP8 Advanced compression methods, such as compress sensing [Meng, Infocom 2012] and context-aware image compression [Bao et al., Hotmobile 2011] LBVC reduces bandwidth of streaming video chats through frame rate adaption instead of video compression Image interpolation Has been widely used in computer graphics, such as creating animations from sets of images [KemelmacherShlizerman et al., SIGGRAPH 2011] 20 LBVC exploits image interpolation techniques to rescue the video quality of mobile video chats with decreased frame rate

Conclusion LBVC adapts the video frame rate with respect to the smartphone vibrations at the sender side interpolates the missing frames at the receiver side saves bandwidth by up to 35% under typical video chat scenarios as well as maintains acceptable video quality considering the bandwidth saving, video quality and introduced delay, we recommend 4 fps (35% bandwidth saving). 21

Questions? Thank you! 22