Tuesday, 22 December 2015

Accessing Image Pixel Data in a Xamarin.Forms App – WinRT

Previously, I’ve described the basic architecture for accessing and manipulating pixel data from a Xamarin.Forms project. A Xamarin.Forms PCL project defines the IBitmap interface, which specifies the operations that must be implemented in each platform-specific project to access and manipulate pixel data. I then explained the operation of the Bitmap class in the Xamarin.Forms iOS project, and the Xamarin.Forms Android project, which both implement the IBitmap interface.

In this blog post I’ll explain the operation of the Bitmap class in the Xamarin.Forms WinRT project, which also implements the IBitmap interface. The DependencyService can be used to invoke a method from the Xamarin.Forms.PCL project, which in turn invokes IBitmap operations.

Note that this WinRT pixel access implementation differs from that in a previous blog post about Accessing Image Pixel Data in a C# Windows Store App.

Disclaimer: The code featured here is alpha code, so all the usual caveats apply.


The following code shows the Bitmap class implemented in WinRT:

1 public class Bitmap : IBitmap<Task<Stream>> 2 { 3 const byte bytesPerPixel = 4; 4 int width; 5 int height; 6 double dpiX, dpiY; 7 StorageFile photoFile; 8 byte[] pixelData; 9 10 public Bitmap(StorageFile file) 11 { 12 photoFile = file; 13 } 14 15 public void ToPixelArray() 16 { 17 throw new NotImplementedException(); 18 } 19 20 public async Task ToPixelArrayAsync() 21 { 22 var imageDataStream = await photoFile.OpenReadAsync(); 23 var decoder = await BitmapDecoder.CreateAsync(imageDataStream); 24 var frame = await decoder.GetFrameAsync(0); 25 var dataProvider = await frame.GetPixelDataAsync(BitmapPixelFormat.Bgra8, BitmapAlphaMode.Straight, new BitmapTransform(), ExifOrientationMode.RespectExifOrientation, ColorManagementMode.DoNotColorManage); 26 27 width = Convert.ToInt32(decoder.PixelWidth); 28 height = Convert.ToInt32(decoder.PixelHeight); 29 dpiX = decoder.DpiX; 30 dpiY = decoder.DpiY; 31 32 pixelData = new byte[width * height * bytesPerPixel]; 33 pixelData = dataProvider.DetachPixelData(); 34 35 imageDataStream.Dispose(); 36 } 37 38 public void TransformImage(Func<byte, byte, byte, double> pixelOperation) 39 { 40 byte r, g, b; 41 42 try 43 { 44 // Pixel data order is BGRA 45 for (int i = 0; i < pixelData.Length; i += bytesPerPixel) 46 { 47 b = pixelData[i]; 48 g = pixelData[i + 1]; 49 r = pixelData[i + 2]; 50 51 // Leave alpha value intact 52 pixelData[i] = pixelData[i + 1] = pixelData[i + 2] = (byte)pixelOperation(r, g, b); 53 } 54 } 55 catch (Exception ex) 56 { 57 Debug.WriteLine(ex.Message); 58 } 59 } 60 61 public async Task<Stream> ToImage() 62 { 63 var ras = new InMemoryRandomAccessStream(); 64 var encoder = await BitmapEncoder.CreateAsync(BitmapEncoder.JpegEncoderId, ras); 65 encoder.SetPixelData(BitmapPixelFormat.Bgra8, BitmapAlphaMode.Straight, (uint)width, (uint)height, dpiX, dpiY, pixelData); 66 await encoder.FlushAsync(); 67 pixelData = null; 68 return ras.AsStream(); 69 } 70 }

The ToPixelArrayAsync method is used to read and decode the photo, get the pixel data, and convert it to a byte array of raw pixel data. The image’s pixel data is formatted as 32 bits per pixel, with each pixel being composed of a Red, Green, Blue, and Alpha channel. Therefore, the pixelData array stores each pixel over four array elements (B, G, R, A).

The TransformImage method is used to perform an imaging operation on the pixel data, such as convert to greyscale. The method specifies a Func parameter that’s used to perform a per-pixel operation. The Func is passed from the Xamarin.Forms PCL project into the method, and operates on each pixel component. At the moment the TransformImage method is hard-coded to place the result of the Func into each pixel component. This is purely for my own test purposes, for implementing a ConvertToGreyscale Func. It can easily be modified to not duplicate the result of the Func across all four colour components.

The ToImage method is then used to place the transformed pixel data into an InMemoryRandomAccessStream that contains a JPEG encoded image.

The Bitmap class can then be invoked as follows:

1 public async Task<Stream> TransformPhotoAsync(Func<byte, byte, byte, double> pixelOperation) 2 { 3 ... 4 5 return await Task.Run(async () => 6 { 7 var bitmap = new Bitmap(PhotoFile); 8 await bitmap.ToPixelArrayAsync(); 9 bitmap.TransformImage(pixelOperation); 10 stream = await bitmap.ToImage(); 11 return stream; 12 }); 13 }

A new Bitmap instance is created, with a StorageFile being passed into the constructor. The photo is read and converted to an array of pixels, transformed by the pixelOperation Func, and then converted into JPEG image that’s returned as a Stream for display by a Xamarin.Forms Image control. All of this work is wrapped in a Task.Run in order for it to be executed on a background thread.


This blog post has described how to access and manipulate pixel data in a Xamarin.Forms WinRT project, by implementing the Bitmap class. The Bitmap class implements the IBitmap interface that’s provided by the Xamarin.Forms PCL project.

No comments:

Post a Comment