The race toward QoE monitoring of ABR streaming.
Cable operators, telecom carriers, satellite providers and other service providers are racing to build the infrastructure necessary for delivering any program, on-demand or live, to any device at any time and over any access network.
Being first to market with that ability could be a tremendous advantage, but the desire to be first has to be tempered with the recognition that a full-scale rollout has to be done right the first time. Everyone knows Facebook, but does anyone remember that social networking was pioneered by Friendster or that the early leader was MySpace?
Clearly, in a competitive environment, ensuring the best possible quality of experience (QoE) – one free from video and audio errors that mar the viewing experience – will be critical to keeping customers and to building significant subscription numbers.
That sounds straightforward, but achieving it is quite complex. TV Everywhere is still in its infancy and requires the use of new technologies such as adaptive bit rate (ABR) streaming, for which monitoring techniques have not been firmly established.
As it sounds, ABR enables delivery of program streams that constantly adapt to the changes in bandwidth available to end user viewing devices. This is particularly key for Wi-Fi-connected mobile devices – both inside and outside of the home – and to PCs and IP set-top boxes relying on cable modem or DSL connectivity for viewing over-the-top (OTT) streaming media services, such as Netflix. The very nature of these networks results in bandwidth fluctuations that make it virtually impossible for them to deliver a high-quality viewing experience without ABR.
Fortunately, service providers employing ABR streaming to deliver multi-screen services now have the means to implement in-depth quality assurance at the most crucial point in the distribution system while avoiding excessive spending on solutions that could become outdated as this new transmission mode matures.
Clearly, service providers must eventually be able to assess performance and identify causes when quality is subpar across all points in the end-to-end ABR streaming infrastructure. Not only will this be essential to maintaining subscriber satisfaction, but it will also be critical to convince content suppliers and their advertisers that they will benefit from funneling high-value first-run and on-demand content into TV Everywhere services over managed networks.
But ABR technology introduces complexities with the potential to impact viewers’ QoE that cannot be addressed by traditional quality assurance measures used with legacy digital TV service. Not only are there far more parameters to measure in the ABR transcoding, fragmentation and distribution process, but there are multiple points in the network where those processes can go wrong, extending from source encoders through origin servers and content distribution network (CDN) caching points to all of the different types of unmanaged IP-connected devices possessed by end users.
The ABR challenge
ABR streaming introduces a wide range of processes that pose challenges to assessing audio and video quality that are new to the premium television environment. At the same time, the fundamentals of frame-by-frame quality assessment within the video stream that characterize robust quality assurance in traditional service operations must be brought into the ABR realm, as well.
ABR employs a “pull” mode in distribution technology that is altogether different from the “push” mode of traditional digital TV. With ABR, each device receives a manifest file from the HTTP (Hypertext Transfer Protocol) server depicting what the available bit rates are. Every few seconds, the device asks the server to send a segment, or fragment – also called a chunk – of streamed content at the highest possible bit rate based on how much bandwidth is available at that moment in time.
Ideally, the constant exchanges between all devices in the field and the HTTP server produce uninterrupted streams of content delivered at the highest possible bit rates to each end user, thereby avoiding buffering delays that would otherwise occur when access bandwidth isn’t sufficient to support steady throughput at a set bit rate.
But there’s a huge price in complexity that must be paid to create this uninterrupted streaming environment for premium content. Moreover, additional complexities associated with content protection and monetization make the achievement of premium service-level quality assurance all the more daunting.
Clearly, any attempt at this point in time to provide an end-to-end quality assurance regime capable of serving the needs of an all-encompassing multi-screen premium service represents a massive undertaking that would have to be pursued without knowing the ROI potential of such a service.
Indeed, given that most service providers are taking a step-by-step approach to introducing multi-screen services by focusing on the transcoding and ABR streaming requirements of specific categories of devices, such as PCs, Xboxes and iPads, it remains to be seen how aggressive they’ll be when it comes to spending on an infrastructure capable of reaching all devices with a multi-screen service. Although it appears that early market responses to TV Everywhere offerings are pulling them in that direction.
The weakest link
Fortunately for service providers, there is already one key step in the complete ABR streaming process for which established QoE monitoring methods are in use: transcoding.
Transcoding content ingested in MPEG-2 into the H.264 codec used for multi-screen TV Everywhere services is the first of several critical steps in the entire ABR streaming process.
Service providers are employing a wide range of vendor approaches to supporting the basic requirements of ABR transcoding, where for every class of device, there is a range of bit rates that must be assigned to each content stream, whether it’s fed live from an original encoding source or from file storage. Typically, this entails generating up to a dozen combined H.264 video streams encoded at various rates using either purpose-built hardware or software running on off-the-shelf processors.
Here, the critical challenge is to perform comprehensive quality analysis on each H.264/AAC stream in real time, which is much more difficult than analysis of MPEG-2 video. Due to both complexity and the use of entropy bit stream encoding, H.264 content requires seven to 10 times the computational resource required to decode MPEG-2. This greatly complicates direct detection of impairments such as tiling or macroblocking.
Another important component to watch for in the H.264 macroblock encoding process is over-compression, which commonly happens with motion-intensive action sequences in sports and other programming. Such artifacts – resulting in blurred or soft, washed-out images – are virtually undetectable once the signals are encrypted, which is why the transcoding output must be directly monitored independent of what comes next in the ABR processing train.
Further complicating matters is the fact that, with transcoding, there are multiple streams for each piece of programming that must be monitored with respect to ancillary content feeds, such as synchronization of closed captioning.
Given that transcoding is the first step in the ABR streaming process, it is also arguably the most critical, as any errors introduced here will be carried through all the way to the end user. So while a service provider can perform the fragmentation and packaging steps flawlessly, the result could still be a flawed viewing experience.
To ensure the transcoding process is being done correctly, service providers should monitor MPEG-2 streams prior to transcoding and then monitor each of the resulting H.264 streams post-transcode. In this way, service providers can determine if the transcoder is introducing errors or if the content being ingested already contains the errors. This will help identify the underlying source of any errors, which is critical to resolving the issue and ensuring an optimal QoE.
Choosing a monitor
Fortunately, well-defined “best practices” and proven solutions exist for comprehensively monitoring both MPEG-2 and H.264 streams.
There are several key characteristics service providers should look for in their TV Everywhere QoE monitoring solutions.
Obviously, the ability to closely monitor MPEG-2 and H.264 streams is required, as is scalability to monitor all of the resulting H.264 streams created by the transcode process.
Additionally, these QoE monitors should be capable of gauging the actual severity of errors and how much they will impact subscribers. Simply put, not all errors are created equal, and one-time “blips” that are imperceptible to subscribers do not require the same attention that significant and recurring errors require. Identifying major errors – often before they impact subscribers – is a key advantage for service providers.
Finally, the ideal QoE monitor will also employ historical reporting capabilities that log errors and associated network conditions at the time of the errors. These reporting features can help service providers tremendously in diagnosing recurring errors, which in turn leads to faster fixes.
In Figure 2, the TV Everywhere QoE monitoring solution is monitoring Comedy Central pre- and post-transcode and captures a number of statistics for the VSP, including average video and audio QoE, bit rates, and program availability.
TV Everywhere is clearly the future, and gaining subscribers early on means nothing if service providers cannot retain them in a fiercely competitive environment. Ensuring the best possible QoE for TV Everywhere customers will be key to keeping them happy and subscribing.
While QoE monitoring of every point in the ABR streaming process, which is at the heart of TV Everywhere, is not currently feasible, it is both possible and vital for service providers to monitor what they can today. In particular, the critical MPEG-2-to-H.264 transcode step can be closely monitored today, enabling service providers to address potentially serious “garbage in, garbage out” situations before they can dramatically reduce viewers’ precious QoE.
Email: steve.liu@tektronix.com
