Encoding and processing media in Azure Media Services

Previously I’ve summarised the media upload process from client apps into a video on-demand service that uses Azure Media Services.

In this blog post I’ll summarise how to incorporate Media Services’ encoding and processing functionality into an app. To do this you create processing jobs that enable you to schedule and automate the encoding and processing of assets.

Encoding and processing media

Media services provides a number of media processors that enable media to be processed. Media processors handle a specific processing task, such as encoding, format conversion, encrypting, or decrypting content. Encoding video is the most common Media Services processing operation, and is performed by the Azure Media Encoder. The Media Encoder is configured using encoder preset strings, with each preset specifying a group of settings required for the encoder. For a list of all the presets see Azure Media Encoder Presets.

Media Services supports progressive download of video and streaming. When encoding for progressive download you encode to a single bitrate. To be able to stream content it must first be converted into a streaming format. There are two types of streaming offered by Media Services:

• Single bitrate streaming
• Adaptive bitrate streaming

With single bitrate streaming a video is encoded to a single bitrate stream and divided into chunks. The stream is delivered to the client one chunk at a time. The chunk is displayed and the client then requests the next chunk. When encoding for adaptive bitrate streaming you encode to an MP4 bitrate set that creates a number of different bitrate streams. These streams are also broken into chunks. However, adaptive bitrate technologies allow the client to determine network conditions and select from among several bitrates. When network conditions degrade, the client can select a lower bitrate allowing the video to continue to play at a lower quality. Once network conditions improve the client can switch back to a higher bitrate with improved quality.

Media Services supports three adaptive bitrate streaming technologies:

1. Smooth streaming, created by Microsoft
2. HTTP Live Streaming (HLS), created by Apple
3. MPEG-DASH, an ISO standard

Accessing media processors

Processing jobs involve calling a specific media process to process the job. Media Services supports the following media processors:

Media processor	Description
Azure Media Encoder	Allows you to run encoding tasks using the Media Encoder,
Azure Media Packager	Allows you to convert media assets from MP4 to Smooth Streaming format, and Smooth Streaming assets to HLS format..
Azure Media Encryptor	Allows you to encrypt media assets using PlayReady protection.
Storage Decryption	Allows you to decrypt media assets that were encrypted using storage encryption.

To use a specific media processor you should pass the name of the processor into the GetLatestMediaProcessorByName method.

private IMediaProcessor GetLatestMediaProcessorByName(string mediaProcessorName)

{

    var processor = this.context.MediaProcessors.Where(p => p.Name == mediaProcessorName)

        .ToList().OrderBy(p => new Version(p.Version)).LastOrDefault();

    if (processor == null)

    {

        throw new ArgumentException(string.Format("Unknown media processor: {0}", mediaProcessorName));

    }

    return processor;

}

This method retrieves the specified media processor and returns an instance of it. It can be invoked as follows:

IMediaProcessor mediaProcessor = this.GetLatestMediaProcessorByName(MediaProcessorNames.WindowsAzureMediaEncoder);

Creating encoding jobs

After media has been uploaded into Media Services it can be encoded into one of the formats supported by the Media Services Encoder. The Media Services Encoder supports encoding using the H.264 and VC-1 codecs, and can generate MP4 and Smooth Streaming content. However, MP4 and Smooth Streaming content can be converted to HLS v3 or MPEG-DASH by using dynamic packaging.

Encoding jobs are created and controlled using a Job. Each Job contains metadata about the processing to be performed, and contains one or more Tasks that specify a processing task, its input Assets, output Assets, and a media processor and its settings. Tasks within a Job can be chained together, where the output asset of one task is given as the input asset to the next task. By following this approach one Job can contain all of the processing required for a media presentation.

The following diagram shows a high-level overview of the media encoding process used in the Building an On-Demand Video Service with Microsoft Azure Media Services project.

The EncodingService class retrieves the asset details from the CMS database and passes the encoding job to Media Services, where it’s submitted to the Azure Media Encoder. The encoding job and video details are saved to the CMS database while the Media Encoder processes the job, retrieving the input asset from Azure Storage, and writing the output assets to Azure Storage. When encoding is complete Media Services notifies the EncodingService class, which generate locator URLS to the output assets in Azure Storage, and updates the encoding job and video details in the CMS database. For a code walkthrough of this process see Encoding process in the Contoso Azure Media Services web service.

By default, each Media Services account can have one active encoding task at a time. However, you can reserve encoding units that allow you to have multiple encoding tasks running concurrently. For more information see How to Scale a Media Service.

Accessing encoded media

Accessing content in Media Services requires a locator, which combines the URL to the media file with a set of time-based access permissions. There are two types of locators – shared access signature (SAS) locators and on-demand origin locators.

A SAS locator grants access rights to a specific media asset through a URL. By using the URL you are granting users who have the URL access to a specific resource for a period of time, in addition to specifying what operations can be performed on the resource.

On-demand origin locators are used when streaming content to a client app, and are exposed by the Media Services Origin Service which pulls the content from Azure Storage and delivers it to the client. An on-demand origin locator URL will point to a streaming manifest file in asset. For more information about the Origin Service see Origin Service.

Summary

This blog post has summarised how to use the Azure Media Encoder to encode uploaded media for delivery to client apps.To do this you create a media processing job that enables you to schedule and automate the encoding of assets. For more info see Building an On-Demand Video Service with Microsoft Azure Media Services.

In my next blog post I’ll discuss the final step in the Media Services workflow – delivering and consuming media.

Using a custom overlay in a Windows Phone QR code scanning app

Previously I’ve demonstrated how to build a simple Windows Phone app to perform QR code scanning, using the ZXing.Net.Mobile library. This library makes the scanning and decoding of bar codes effortless, leaving you to focus on other user experiences in your app.

In this blog post I’m going to extend the sample app so that it doesn’t use the default UI included with ZXing.Net.Mobile, when QR code scanning. Instead, a custom UI will be created and used during the QR code scanning process. This custom UI is referred to as an overlay.

Implementation

The first step is to define the overlay in the XAML code for the page.

<Grid Name="Overlay" Visibility="Collapsed">

    <Grid Background="Transparent">

        <Grid.RowDefinitions>

            <RowDefinition Height="*" />

            <RowDefinition Height="Auto" />

        </Grid.RowDefinitions>

        <Grid.ColumnDefinitions>

            <ColumnDefinition Width="*" />

            <ColumnDefinition Width="*" />

        </Grid.ColumnDefinitions>

        <Button Background="Black" Grid.Row="1" Grid.Column="0" Name="ButtonCancel">Cancel</Button>

        <Button Background="Black" Grid.Row="1" Grid.Column="1" Name="ButtonTorch">Torch</Button>

    </Grid>

</Grid>

This code defines a custom overlay named Overlay, which contains a Cancel button and a Torch button (for toggling on/off the flash) that will appear at the bottom of the page. The next step is pass the overlay to the ZXing.Net.Mobile library.

private UIElement _overlay = null;

private async void MainPage_Loaded(object sender, RoutedEventArgs e)

{

    _scanner = new MobileBarcodeScanner(this.Dispatcher);

    if (_overlay == null)

    {

        _overlay = this.Overlay.Children[0];

        this.Overlay.Children.RemoveAt(0);

    }

    this.ButtonCancel.Click += (s, e2) =>

        {

            _scanner.Cancel();

        };

    this.ButtonTorch.Click += (s, e2) =>

        {

            _scanner.ToggleTorch();

        };

    _scanner.CustomOverlay = _overlay;

    _scanner.UseCustomOverlay = true;

    var result = await _scanner.Scan();

    ProcessScanResult(result);

}

After creating an instance of the MobileBarcodeScanner class the Overlay is retrieved from the visual tree and stored in a UIElement instance named _overlay. Then, the Cancel and Torch button Click events are wired up to methods in the MobileBarcodeScanner class, to cancel scanning, and toggle the torch respectively. The CustomOverlay property of the MobileBarcodeScanner instance is then set to _overlay, the UseCustomOverlay property of the MoblileBarcodeScanner instance is set to true to indicate that a custom overlay will be used during the QR scanning process, before the Scan method of the MobileBarcodeScanner instance is invoked. The following screenshot shows the custom overlay being displayed during the QR code scanning process (QR code image courtesy of Google Images):

While the overlay shown here is basic, it does show the mechanism used for creating a UI for the QR code scanning process that matches the rest of the UI used in your app.

When a QR code is successfully recognized the decoded result can be passed to a method for processing, in this case the ProcessScanResult method. For an explanation of this method see my previous blog post.

Summary

This blog post has demonstrated how to replace the default UI used by ZXing.Net.Mobile, with a custom UI of your own design. This allows you to create a UI for the QR code scanning process that matches the rest of the UI used in your app.

The sample app can be downloaded here.

QR code scanning in a Windows Phone app

A recent Windows Phone app that I developed had a requirement to include QR code scanning, and to take an action based on the data contained in the QR code. A QR code is a type of machine readable barcode that contains data that can be extracted from patterns present in both horizontal and vertical components of the image. Initially this seemed like quite a big task, but with the help of a library it became an effortless task.

This blog post demos a simple Windows Phone 8 app for performing QR code scanning using ZXing.Net.Mobile. ZXing.Net.Mobile is a .NET library based on the widely used open source ZXing (Zebra Crossing) barcode library, whose goal is to make scanning barcodes as painless as possible in an app.

Implementation

The first step is to add the ZXing.Net.Mobile nuget package to your app, and import the ZXing.Mobile namespace into your class.

private MobileBarcodeScanner _scanner;

private async void MainPage_Loaded(object sender, RoutedEventArgs e)

{

    _scanner = new MobileBarcodeScanner(this.Dispatcher);

    _scanner.UseCustomOverlay = false;

    _scanner.TopText = "Hold camera up to QR code";

    _scanner.BottomText = "Camera will automatically scan QR code\r\n\rPress the 'Back' button to cancel";

    var result = await _scanner.Scan();

    ProcessScanResult(result);

}

Scanning a QR code is as simple as creating an instance of the MobileBarcodeScanner class, and invoking it’s Scan method. Optional setup includes specifying text to display at the top and bottom of the page, while scanning a QR code, and a flag that indicates whether you want to use ZXings in-built overlay, or specify your own.

The following screenshot shows the app scanning a QR code (courtesy of Google Images).

When a QR code is successfully recognized the decoded result can be passed to a method for processing, in this case the ProcessScanResult method. Note that decoding of the image is all automatically performed by the ZXing.Net.Mobile library.

private void ProcessScanResult(ZXing.Result result)

{

    string message = string.Empty;

    message = (result != null && !string.IsNullOrEmpty(result.Text)) ? "Found QR code: " + result.Text : "Scanning cancelled";

    this.Dispatcher.InvokeAsync(() =>

        {

            MessageBox.Show(message);

        });

}

This method simply extracts the text from the ZXing.Result instance and displays it in a mesage box, as shown in the following screenshot.

Obviously in a real app you may choose to log the decoded QR code result somewhere, or in the case of a URL, open it in a web browser.

Summary

This blog post has demonstrated how to construct a simple app for performing QR code scanning, using the ZXing.Net.Mobile library. This library makes the scanning and decoding of bar codes effortless, leaving you to focus on other user experiences in your app.

The sample app can be downloaded here.