CLEAR

Iris Tracking

Follow this guide to create an effect that changes the appearance of someone's iris.

Download this example project to try iris tracking.

In the example, you'll see:

  • A face tracker and face mesh. We've applied an occluder material to the face mesh, which will block out whatever's behind it. This means the irises will be covered in a natural way - for example they won't be visible when someone blinks.
  • 3D objects, with textures applied to them. They're children of the face tracker.

You can adapt the example project to make your own effects.

You can either use the Patch Editor or scripting to create an effect with iris tracking. We've included instructions and a script for doing both in this guide.

If you haven't used the Patch Editor before, it's worth learning the basics.

Using the Patch Editor

Drag the face tracker from the Scene tab into the Patch Editor. You'll see a group of 3 patches: Face Finder, Face Select, and Face Follower.

The purple Face Follower patch is usually used to detect and follow the entire face. We need to replace this patch with an Eyeball patch, which can detect and follow someone's eyes. To do this:

  1. Delete the Face Follower patch.
  2. Connect the Face output port of the Face Select patch to the Face input on the Eyeball patch.

Your graph should look like this:

The Eyeball patch has 6 outputs:

  • Left and Right Iris Position: the 3D position of the the pupil on the surface of the eye, relative to the face tracker.
  • Left and Right Eyeball Center Position: the position of the center of the eyeball, relative to the face tracker.
  • Left and Right Eyeball Rotation: the 3D rotation of the eyeball. For example, if the person is looking down, this rotation will be rotated with a positive angle around the X axis.

For this effect, you'll use the Left and Right Iris Position and the Left and Right Iris Rotation output ports.

You'll need to connect these ports to patches that represent the Position and Rotation properties of the left and right iris objects:

To do this:

  1. Select the left iris and right iris objects in the Scene tab.
  2. In the Inspector, click the small circles next to both their Position and Rotation values

This should create 4 blue patches, to represent these values.

Connect:

  1. The Iris Position ports in the Eyeball patch to the 3D Position ports of their corresponding object patches.
  2. The Iris Rotation ports in the Eyeball patch to the 3D Rotation ports of their corresponding object patches.

Your finished graph should look like this:

And the objects should cover and track the eyes in your finished effect:

Using Scripting

In the Assets view, click +, then Create New Script.

You'll see a new script in the Assets panel.

Double-click the script to open it in a text editor, then paste this script:

    var FaceTracking = require("FaceTracking");
    var IrisTracking = require("IrisTracking");
    var Scene = require("Scene");
        
    var leftIris = Scene.root.find("left iris");
    var rightIris = Scene.root.find("right iris");
        
    var leftEyeballInfo = IrisTracking.leftEyeball(FaceTracking.face(0));
    var rightEyeballInfo = IrisTracking.rightEyeball(FaceTracking.face(0));
        
    leftIris.transform.position = leftEyeballInfo.iris;
    rightIris.transform.position = rightEyeballInfo.iris;
        
    leftIris.transform.rotation = leftEyeballInfo.rotation;
    rightIris.transform.rotation = rightEyeballInfo.rotation;
      

The Eyeball object stored in leftEyeballInfo and rightEyeballInfo has three properties:

  • iris, which gives the pupil's central position relative to the face tracker.
  • center, which gives the position of the center of the whole eyeball relative to the face tracker.
  • rotation, which gives the rotation of the eyeball.