Wednesday, 24 March 2021

Adventures in ARKit - display a 3D model

In my previous blog post I discussed how to overlay an image of the earth on a SphereNode, which derives from SCNNode, and manipulate it through touch gestures.

In this blog post I’ll discuss how to display a 3D model in a scene. Specifically, it’ll be a model of the moon that can be manipulated similarly through touch gestures. Although the end result, a rotating moon, appears to be similar to a rotating earth, they are accomplished via different techniques.

The sample this code comes from can be found on GitHub.

Display a 3D model

ARKit and SceneKit support many different 3D model formats, including .dae, .usdz, .obj and .mtl, and many more. The exact formats supported are dependent upon the version of iOS you are using. Apple currently recommends using .usdz files (and has some samples), but this format can’t be consumed by the first release of ARKit. Therefore, for maximum compatibility, I’ve used a .dae model.

free3d.com is a good source of 3D models, both free and paid. However, it’s quite likely that any 3D model you download will first need manipulating to fit your requirements. This can be accomplished in a tool such as Blender. I used Blender to convert the model I downloaded to .dae format, and to scale it to my needs. Note that there’s a learning curve in getting to grips with Blender.

Once you have a 3D model ready to use it’s worth opening it in Xcode, for two reasons. Firstly, Xcode can be used to reveal the name of the root node in the model, that you may need when adding the model to your scene. Secondly, the model will display in Xcode exactly how it will display in your scene. So you can use Xcode to discover any problems with your model, and even fix some of them. For example, my model of the moon was displaying in red only. This is because, for memory reasons when handling greyscale images assigned as the Diffuse property, SceneKit will store the greyscale data in the red channel, but will zero the blue and green channels. This can be fixed converting any greyscale images to RGB, and sometimes by manipulating the Components drop down for the Diffuse property in Xcode.

Once you have a 3D model that renders correctly in Xcode, it can be added to your ARKit app. 3D models are added to a scene as a SCNNode, which can then be positioned and manipulated as required. As always, this can be accomplished in the ViewDidAppear method in the ViewController class:

public override void ViewDidAppear(bool animated)
{
    base.ViewDidAppear(animated);

    sceneView.Session.Run(new ARWorldTrackingConfiguration
    {
        AutoFocusEnabled = true,
        LightEstimationEnabled = true,
        WorldAlignment = ARWorldAlignment.Gravity
    }, ARSessionRunOptions.ResetTracking | ARSessionRunOptions.RemoveExistingAnchors);

    SCNScene scene = SCNScene.FromFile("moon.dae");
    SCNNode node = scene.RootNode;
    node.Position = new SCNVector3(0, 0, -25f);
    sceneView.Scene.RootNode.AddChildNode(node);
    ... 
}

In this example, the 3D model of the moon is retreived using the SCNScene type, and its root node is retrieved from the scene as a SCNNode. The node is then positioned and added to the scene. In addition, gesture recognisers are added to the SCNNode that aren’t shown in the code above.

The overall effect is that when the app runs, a SCNNode that resembles the moon appears:

Tapping on the SCNNode starts it rotating, and while rotating, tapping it a second time stops it rotating. In addition, the pinch gesture will resize the SCNNode, and the rotate gesture enables the Z-axis of the SCNNode to be manipulated.

If you want to manipulate a particular node in the 3D model, you’ll need to know its name. This can be determined by opening the model in Xcode and navigating the scene graph for the model until you find the name for the required part of the model. This can then be retrieved as a SCNNode:

SCNNode node = scene.RootNode.FindChildNode("MyModelPart", true);

Once you’ve retreived the desired part of the model as a SCNNode, it can be manipulated as required. For example, you could use this technique to retrieve the arm from a model of a person, and then animate it.

No comments:

Post a comment