Playing local audio files with .NET MAUI

Previously I wrote about playing audio with .NET MAUI on Android, iOS, Mac Catalyst, and Windows. The main problem with my implementation was that the FilePicker for picking local audio files only worked on Windows. On iOS/Android it let you browse the device, and displayed audio files, but it didn’t let you select an audio file. On Mac Catalyst it did nothing at all. I was convinced these were MAUI bugs. Turns out I was completely wrong!

The problem was this block of code:

var pickedAudio = await FilePicker.Default.PickAsync(new PickOptions
{
	PickerTitle = "Select audio file",
	FileTypes = new FilePickerFileType(
		new Dictionary<DevicePlatform, IEnumerable<string>>
		{
			{ DevicePlatform.WinUI, new [] { "*.mp3", "*.m4a" } },
			{ DevicePlatform.Android, new [] { "*.mp3", ".3gp", ".mp4", ".m4a", ".aac", ".ts", ".amr", ".flac", ".mid", ".xmf", ".mxmf", ".rtttl", ".rtx", ".ota", ".imy", ".mkv", ".ogg", ".wav" } },
			{ DevicePlatform.iOS, new[] { "*.mp3", "*.aac", "*.aifc", "*.au", "*.aiff", "*.mp2", "*.3gp", "*.ac3" } }
		})
});

The problem was ultimately caused by the order in which I wrote and tested the code on each platform. I did Windows first, followed by Android and iOS. Also, note the lack of an entry for Mac Catalyst.

So, I started off by adding code for Windows and specified *.mp3 and *.m4a as the file extensions to display in the FilePicker. This was purely because I couldn’t find any docs on which audio file formats are supported in WinUI 3. I got the MauiAudioPlayer working on Windows using these file extensions and moved onto Android.

I assumed that I could just use the same file extensions on Android and iOS. I managed to find docs from Google and Apple that mentioned the audio file formats they supported, and what their file extensions are, hence the more extensive entries for each platform. The issue was that I didn’t know that file picking on Android and iOS, using FilePicker and FilePickerFileType, doesn’t require an array of file extensions.

It was Gerald who alerted me to this, by pointing to the Xamarin.Essentials PickOptions.FileTypes API docs. Sure, it’s for Xamarin.Essentials rather than .NET MAUI but the same content applies. Specifically:

On Android and iOS the files not matching this list is only displayed
grayed out. When the array is null or empty, all file types can be
selected while picking. The contents of this array is platform
specific; every platform has its own way to specify the file types. On
Android you can specify one or more MIME types, e.g. “image/png”; also
wild card characters can be used, e.g. “image/*”. On iOS you can
specify UTType constants, e.g. UTType.Image. On UWP, specify a list of
extensions, like this: “.jpg”, “.png”.

So there’s the answer: on Android you specify the file types as MIME types, and wild cards can be used. On iOS you specify UTType constants, and on Windows you use file extensions.

I looked at common MIME types and discovered that for audio the MIME types are entries like audio/aac, audio/mpeg etc. When combined with wildcards it meant that the MIME type for all audio files supported by a platform is audio/*. So this became the string entry in the FilePickerFileType dictionary for Android.

iOS was a thornier problem. UTType constants are the recommended approach and are documented here. The problem I have with this approach is it requires iOS14+. I wanted my code to work with all versions of iOS that MAUI supports (which the docs says is iOS10+). On a random StackOverflow post I read that you could use system-defined uniform type identifiers to specify which files can be selected, which lead me to Apple’s System-declared Uniform Type Identifiers doc. A quick search of the doc revealed that the correct constant for audio files is public.audio. So this became the string entry in the FilePickerFileType dictionary for iOS, that I also duplicated for Mac Catalyst.

This lead to the following code:

var pickedAudio = await FilePicker.Default.PickAsync(new PickOptions
{
	PickerTitle = "Select audio file",
	FileTypes = new FilePickerFileType(
		new Dictionary<DevicePlatform, IEnumerable<string>>
		{
			{ DevicePlatform.WinUI, new [] { "*.mp3", "*.m4a" } },
			{ DevicePlatform.Android, new [] { "audio/*" } },
			{ DevicePlatform.iOS, new[] { "public.audio" } },
			{ DevicePlatform.MacCatalyst, new[] { "public.audio" } }
		})
});

When combined with a couple of small updates to MauiAudioPlayer on iOS/Android/Mac Catalyst, to correctly load the selected file, all platforms gained the ability to play locally stored audio files. The PR of changes to enable this can be found here.

So there we have - a .NET MAUI audio player for iOS, Android, Mac Catalyst, and Windows, that plays audio from URLs, audio embedded in your app package, and local audio files on your device that can be chosen by the user. You can find the code here.

Playing audio with .NET MAUI

Many apps, whether mobile or desktop, require the ability to play audio. That audio may be remote, stored in the app bundle, or be chosen from the user’s device. However, .NET MAUI currently doesn’t have a cross-platform control capable of playing audio. However, all of the underlying platforms that .NET MAUI supports have native controls for playing audio. Android has MediaPlayer, iOS/Mac Catalyst has AVPlayer, and WinUI has MediaPlayerElement (only available in WinAppSDK 1.2-preview).

I’ve used these native types to create a cross-platform Audio control. It’s based on the Video control I created a month or two ago (see Playing video with .NET MAUI and Playing video with .NET MAUI on Windows). It plays audio from URLs, from audio embedded in your app package (and hence embedded in your single project), and files chosen by the user on your device. As well as using the in-built transport controls to control audio playback, you can provide your own transport controls.

Are there code sharing opportunities between my Audio and Video controls? Yes. The code to play audio on iOS/Mac/Windows is 99% identical to the code to play video on iOS/Mac/Windows. Similarly, the cross-platform Audio control is a renamed version of the cross-platform Video control. The big difference is audio and video playback on Android. The Video control uses an Android VideoView, combined with a MediaController, while the Audio control uses an Android MediaPlayer, combined with a MediaController. It’ll be possible to merge the two controls into a single Media cross-platform control, capable of playing video and audio. Will I be doing this? Maybe at some point. Just not now.

You can download the control from its GitHub repo.

An alternative approach to playing audio in a .NET MAUI app is to use Plugin.Maui.Audio, created by Gerald Versluis and Shaun Lawrence. Are there any code similarities between Plugin.Maui.Audio and what I’ve done? Not really. Plugin.Maui.Audio uses different types to play audio on iOS/Mac/Windows than I’ve used, requires you to provide your own transport controls, and doesn’t use handlers.

Handler architecture

.NET MAUI has an extension mechanism, known as handlers, that you can use to customise existing .NET MAUI controls, and write your own cross-platform views (controls) whose implementations are provided by native views (controls).

Each .NET MAUI cross-platform control is known as a virtual view. Handlers map these virtual views to native views on each platform, and are responsible for creating the underlying native view, and mapping their API to the cross-platform control. Each handler typically provides a property mapper, and potentially a command mapper, that maps the cross-platform view API to the native view API.

The following diagram shows the handler architecture for the Audio view:

The Audio type represents the cross-platform control. On iOS/Mac Catalyst the AudioHandler maps the Audio view to an iOS/Mac Catalyst AVPlayer. On Android, the Audio view is mapped to a MediaPlayer, and on WinUI the Audio view is mapped to a MediaPlayerElement.

The PropertyMapper in the AudioHandler class maps the cross-platform view properties to native view APIs via mapper methods. Each platform then provides implementations of the mapper methods, which manipulate the native view API as appropriate. The overall effect is that when a property is set on the cross-platform view, the underlying native view is updated as required.

The CommandMapper in the AudioHandler class maps cross-platform view commands to native view APIs via mapper methods. Command mappers provide a way for cross-platform controls to send commands to native views on each platform. They’re similar to property mappers, but allow for additional data to be passed. Note that commands, in this context, doesn’t mean ICommand implementations. In this context, a command is just a way of invoking some functionality on a native control. For example, the ScrollView in .NET MAUI uses a command mapper so that the ScrollView asks its handler to instruct the native views to scroll to a specific location, passing along the scroll arguments (such as the position or element it wants to scroll to). The ScrollView handler on each platform unpacks the scroll arguments and invokes native view functionality to perform the desired scroll. This was analogous in Xamarin.Forms to having an event on the cross-platform view, with the renderer subscribing to the event. The advantage of the command mapper approach is that it decouples the native view from the cross-platform view, and avoids the need to unsubscribe from events. It also allows for easy customisation - the command mapper can be modified by consumers without subclassing.

Handler implementations on each platform must override the CreatePlatformView method, and optionally the ConnectHandler and DisconnectHandler methods. The CreatePlatformView method should return the native view that implements the cross-platform view. The ConnectHandler method should perform any required native view setup, and the DisconnectHandler method should perform any required native view cleanup. Note that the DisconnectHandler override is intentionally not invoked by .NET MAUI - you have to invoke it yourself from a suitable place in your app’s lifecycle.

Code

I’m not going to provide a walkthrough of the code. But you can download it, and step through it yourself, by cloning the repo. However, I’ll give you some pointers to working through the code.

The solution is structured as follows:

The important folders in the solution are:

• Controls - the cross-platform view implementation.

  

  The Audio class derives from .NET MAUI’s View class, provides the cross-platform implementation, and is a collection of BindableProperty objects and public methods.

• Handlers - the handler implementation.

  

  The AudioHandler class is a partial class, whose platform-specific implementations are in the AudioHandler.Android.cs, AudioHandler.MaciOS.cs and AudioHandler.Windows.cs files. The AudioHandler class exposes a VirtualView property that can be used to access the cross-platform view from the handler/native view layer. It also exposes a PlatformView property that can be used to access the native view that implements the Audio view.

• Platforms - the native view implementations.

  

  Rather than implement the native views directly in the handler, I’ve split them out into native view implementations named MauiAudioPlayer. On Android, MauiAudioPlayer derives from CoordinatorLayout and uses a MediaPlayer (along with a MediaController). On Android there’s also an AudioProvider class, which is a content provider that retrieves the embedded audio files from the assets folder of its bundle. On iOS/Mac Catalyst, MauiAudioPlayer derives from UIView and uses an AVPlayer (along with an AVPlayerViewController for the transport controls). On Windows, MauiAudioPlayer derives from Grid and uses a MediaPlayerElement.

• Resources/Raw - three embedded audio files.

  

  The audio files have a build action of MauiAsset.

• Views - pages that exercise the Audio view. An event handler for the Unloaded event on each page invokes the DisconnectHandler override on AudioHandler.

A handler must be registered against its cross-platform view, and this takes place in MauiProgram.cs with the ConfigureMauiHandler/AddHandler methods.

On iOS, I modified the ContentOverlayView of the AVPlayerViewController object to display an image (the idea being you could display something that represents the audio being played - any UIView-derived object):

UIImageView imageView = new UIImageView();
imageView.Image = UIImage.FromBundle("dotnet_bot.png");
imageView.ContentMode = UIViewContentMode.ScaleAspectFit;
_playerViewController.ContentOverlayView.AddSubview(imageView);

imageView.TranslatesAutoresizingMaskIntoConstraints = false;
imageView.BottomAnchor.ConstraintEqualTo(_playerViewController.ContentOverlayView.BottomAnchor).Active = true;
imageView.TopAnchor.ConstraintEqualTo(_playerViewController.ContentOverlayView.TopAnchor).Active = true;
imageView.LeadingAnchor.ConstraintEqualTo(_playerViewController.ContentOverlayView.LeadingAnchor).Active = true;
imageView.TrailingAnchor.ConstraintEqualTo(_playerViewController.ContentOverlayView.TrailingAnchor).Active = true;

This code retrieves the dotnet_bot image that’s stored in the Resources\Raw folder of the project, from the app bundle (where it’s copied to at build time), and centers it in the ContentOverLay view of the AVPlayerViewController object using constraints. This makes the audio player look less plain:

Next steps

There are a couple of issues I’m aware of in the implementation. Firstly, the FilePicker, for selecting an audio file from the device, only works on Windows. On iOS/Android it lets you browse the device, but won’t let you select a file. On Mac Catalyst, it does nothing. At the moment I’ve yet to investigate whether these are MAUI bugs, or whether I’m doing something wrong/lacking a piece of config. Secondly, I’ve encountered a random crash on Android - I’m still trying to produce a firm repro case for it.

In addition, on Windows, the MediaPlayerElement seems to be reserving rendering space for a non-existent video. I need to look into whether there’s something I can do about that, or whether it’s due to MediaPlayerElement still being in preview.

Playing video with .NET MAUI on Windows

Previously I wrote about playing video with .NET MAUI on Android, iOS, and Mac Catalyst. The problem platform was Windows because WinUI 3 lacked any media playback controls. This has been rectified with the preview release of the WinAppSDK v1.2, which contains the MediaPlayerElement and MediaTransportControls types. If you want to know how to use these types, see Media players.

I’ve updated my VideoPlayer code with Windows support, so it’s now a truly cross-platform Video control. This includes the ability to play remote videos, videos embedded in the app, and video's from the file system. As with the other platforms, the MauiVideoPlayer class provides video playback capabilities on Windows. This class derives from Microsoft.UI.Xaml.Controls.Grid and adds a MediaPlayerElement to it (all WinUI controls must be in a layout, and so this mechanism ensures this will always be the case).

I also updated the code to remove the IVideo and IVideoHandler interfaces. After chatting to Shane I realised the interfaces weren’t necessary and weren’t adding anything. While .NET MAUI decouples its handlers from its cross-platform controls via interfaces, this is to enable experimental frameworks such as Comet and Fabulous to provide their own cross-platform controls, that implement the interfaces, while still using .NET MAUI’s handlers. Therefore, creating an interface for your cross-platform control is only necessary if you need to decouple your handler from its cross-platform control for a similar purpose, or for testing purposes.

My understanding is that v1.2 of the WinAppSDK will move out of preview by the end of the year, so there may be issues with MediaPlayerElement until then.