Wednesday, 2 December 2015

Accessing Image Pixel Data in a Xamarin.Forms App – iOS

Previously, I’ve described a basic architecture for accessing and manipulating pixel data from a Xamarin.Forms project. A Xamarin.Forms PCL project defines the IBitmap interface, which specifies the operations that must be implemented in each platform-specific project to access and manipulate pixel data.

In this blog post I’ll explain the operation of the Bitmap class in the Xamarin.Forms iOS project, which implements the IBitmap interface. The DependencyService can be used to invoke a method from the Xamarin.Forms PCL project, which in turn invokes IBitmap operations.

Disclaimer: The code featured here is alpha code, so all the usual caveats apply.

Implementation

The following code shows the Bitmap class implementation on iOS:

1 public class Bitmap : IBitmap<UIImage>
2 {
3 const byte bitsPerComponent = 8;
4 const byte bytesPerPixel = 4;
5 UIImage image;
6 readonly int width;
7 readonly int height;
8 IntPtr rawData;
9 byte[] pixelData;
10 UIImageOrientation orientation;
11
12 public Bitmap (UIImage uiImage)
13 {
14 image = uiImage;
15 orientation = image.Orientation;
16 width = (int)image.CGImage.Width;
17 height = (int)image.CGImage.Height;
18 }
19
20 public void ToPixelArray ()
21 {
22 using (var colourSpace = CGColorSpace.CreateDeviceRGB ()) {
23 rawData = Marshal.AllocHGlobal (width * height * 4);
24 using (var context = new CGBitmapContext (rawData, width, height, bitsPerComponent, bytesPerPixel * width, colourSpace, CGImageAlphaInfo.PremultipliedLast)) {
25 context.DrawImage (new CGRect (0, 0, width, height), image.CGImage);
26 pixelData = new byte[width * height * bytesPerPixel];
27 Marshal.Copy (rawData, pixelData, 0, pixelData.Length);
28 Marshal.FreeHGlobal (rawData);
29 }
30 }
31 }
32
33 public void TransformImage (Func<byte, byte, byte, double> pixelOperation)
34 {
35 byte r, g, b;
36
37 // Pixel data order is RGBA
38 try {
39 for (int i = 0; i < pixelData.Length; i += bytesPerPixel) {
40 r = pixelData [i];
41 g = pixelData [i + 1];
42 b = pixelData [i + 2];
43
44 // Leave alpha value intact
45 pixelData [i] = pixelData [i + 1] = pixelData [i + 2] = (byte)pixelOperation (r, g, b);
46 }
47 } catch (Exception ex) {
48 Console.WriteLine (ex.Message);
49 }
50 }
51
52 public UIImage ToImage ()
53 {
54 using (var dataProvider = new CGDataProvider (pixelData, 0, pixelData.Length)) {
55 using (var colourSpace = CGColorSpace.CreateDeviceRGB ()) {
56 using (var cgImage = new CGImage (width, height, bitsPerComponent, bitsPerComponent * bytesPerPixel, bytesPerPixel * width, colourSpace, CGBitmapFlags.ByteOrderDefault, dataProvider, null, false, CGColorRenderingIntent.Default)) {
57 image.Dispose ();
58 image = null;
59 pixelData = null;
60 return UIImage.FromImage (cgImage, 0, orientation);
61 }
62 }
63 }
64 }
65 }

The ToPixelArray method is used to convert the UIImage to a byte array of raw pixel data. The image’s pixel data is placed into unmanaged memory through the DrawImage method, and is referenced through the rawData IntPtr. It’s formatted as 32 bits per pixel, with each pixel being composed of a Red, Green, Blue, and Alpha channel. It’s then copied to the managed pixelData array, before the unmanaged memory is freed. Therefore, the pixelData array stores each pixel over four array elements (R, G, B, A).

The TransformImage method is used to perform an imaging operation on the pixel data, such as convert to greyscale. The method specifies a Func parameter that’s used to perform a per-pixel operation. The Func is passed from the Xamarin.Forms PCL project into the method, and operates on each pixel component. At the moment the TransformImage method is hard-coded to place the result of the Func into each pixel component. This is purely for my own test purposes, for implementing a ConvertToGreyscale Func (see previous blog post). It can easily be modified to not duplicate the result of the Func across all four colour components.

The ToImage method is then used to place the transformed pixel data in a new UIImage, before performing some cleanup to allow managed memory to be reclaimed. Note that the ToImage method creates a 32 bit image. An overload could be provided to create a 8 bit image.

The Bitmap class can then be invoked as follows:

1 public Task<Stream> TransformPhotoAsync (Func<byte, byte, byte, double> pixelOperation)
2 {
3 return Task.Run (() => {
4 var bitmap = new Bitmap (image);
5 bitmap.ToPixelArray ();
6 bitmap.TransformImage (pixelOperation);
7 return bitmap.ToImage ().AsJPEG ().AsStream ();
8 });
9 }

A new Bitmap instance is created, with a UIImage instance being passed into the constructor. The UIImage is converted to an array of pixels, transformed by the pixelOperation Func, and then converted into a new UIImage before being returned for display by a Xamarin.Forms Image control. All of this work is wrapped in a Task.Run in order for it to be executed on a background thread.

Summary

This blog post has described how to access and manipulate pixel data in a Xamarin.Forms iOS project, by implementing the Bitmap class. The Bitmap class implements the IBitmap interface that’s provided by the Xamarin.Forms PCL project. My next blog post will explain the implementation of the IBitmap interface on Android.

No comments:

Post a Comment