Revisiting Pixel Access in a Xamarin.Forms App – iOS

Previously, I’ve described a basic architecture for accessing and manipulating pixel data from a Xamarin.Forms project. A Xamarin.Forms PCL project defines the IBitmap interface, which specifies the operations that must be implemented in each platform-specific project to access and manipulate pixel data. I then explained the operation of the Bitmap class in the Xamarin.Forms iOS project, which implements the IBitmap interface.

Over Christmas I revisited my iOS implementation of the Bitmap class and discovered that I had redundant copies of pixel data from unmanaged memory to managed memory. In particular, the pixel data did not need placing into unmanaged memory in the first place. This blog post will outline the improvements I’ve made to the Bitmap class on iOS.

Disclaimer: The code featured here is alpha code, so all the usual caveats apply.

Implementation

The following code shows the revised Bitmap class implementation on iOS:

 1 public class Bitmap : IBitmap<UIImage>
 2 {
 3     const byte bitsPerComponent = 8;
 4     const byte bytesPerPixel = 4;
 5     UIImage image;
 6     readonly int width;
 7     readonly int height;
 8     byte[] pixelData;
 9     UIImageOrientation orientation;
10 
11     public Bitmap (UIImage uiImage)
12     {
13         image = uiImage;
14         orientation = image.Orientation;
15         width = (int)image.CGImage.Width;
16         height = (int)image.CGImage.Height;
17     }
18 
19     public void ToPixelArray ()
20     {
21         using (var colourSpace = CGColorSpace.CreateDeviceRGB ()) {
22             pixelData = new byte[width * height * bytesPerPixel];
23             using (var context = new CGBitmapContext (pixelData, width, height, bitsPerComponent, bytesPerPixel * width, colourSpace, CGImageAlphaInfo.PremultipliedLast)) {
24                 context.DrawImage (new CGRect (0, 0, width, height), image.CGImage);
25             }
26         }
27     }
28 
29     public void TransformImage (Func<byte, byte, byte, double> pixelOperation)
30     {
31         byte r, g, b;
32     
33         // Pixel data order is RGBA
34         try {
35             for (int i = 0; i < pixelData.Length; i += bytesPerPixel) {
36                 r = pixelData [i];
37                 g = pixelData [i + 1];
38                 b = pixelData [i + 2];
39 
40                 // Leave alpha value intact
41                 pixelData [i] = pixelData [i + 1] = pixelData [i + 2] = (byte)pixelOperation (r, g, b);
42             }
43         } catch (Exception ex) {
44             Console.WriteLine (ex.Message);
45         }
46     }
47 
48     public UIImage ToImage ()
49     {
50         using (var colourSpace = CGColorSpace.CreateDeviceRGB ()) {
51             using (var context = new CGBitmapContext (pixelData, width, height, bitsPerComponent, bytesPerPixel * width, colourSpace, CGImageAlphaInfo.PremultipliedLast)) {
52                 image.Dispose ();
53                 image = null;
54                 pixelData = null;
55                 return UIImage.FromImage (context.ToImage (), 0, orientation);
56             }
57         }
58     }
59 }

The big changes are to the ToPixelArray and ToImage methods. Previously the ToPixelArray method placed the pixel data into unmanaged memory, before copying it to managed memory and freeing the unmanaged memory. This was redundant and wasteful, and I was able to eliminate it once I discovered that one of the CGBitmapContext constructor overloads allowed the placing of pixel data direct into managed memory.

Similarly, the ToImage method also contained a redundant operation in that previously a new UIImage was created by creating a CGDataProvider and a CGImage, with the UIImage then being created from the CGImage instance. This has now been slightly streamlined to creating a new CGBitmapContext, with the UIImage then being created from the CGBitmapContext instance.

I also explored options for increasing the performance of the TransformImage method, including parallelism and discovered that the performance of the method is not presenting a bottleneck.

Summary

This blog post has revisited how to access and manipulate pixel data in a Xamarin.Forms iOS project, by more efficiently implementing the Bitmap class. A redundant copying of pixel data from unmanaged memory to managed memory has been eliminated, and in doing so has removed a potential source of memory leaks. In addition, following pixel manipulation, the creation of a new UIImage has been slightly streamlined.