CLEAR

Sample Effect: Skeleton Animation

In this effect we've used the Patch Editor to animate a 3D object with a skeleton and joints. This object in particular looks like a teddy bear. We've connected the object to the face tracker, so the teddy moves its body and face in sync with the movement of the person using the effect - like a virtual puppet.

We've also added an extra layer of interactivity, so the clothes the teddy is wearing change color and style when the screen is tapped.

Download the sample content to follow along with this tutorial. You can adapt any of the assets in this effect in your own projects.

Learn about:

  1. What's in the unfinished effect - we've already added the 3D object, and a background.
  2. Animating 3D objects with skeletons and joints.
  3. Making objects move in sync with facial movement.
  4. Using interaction to change the teddy's outfit.

In the Unfinished Effect

If you want to build this effect yourself, open the unfinished effect in the sample content folder. We've already imported the assets you need, to help you get started quickly.

We've added a simple gradient to the background of the scene, using visual shaders. You can change the color of the background by adjusting the Previous Step and New Color values in the Gradient Step patch.

We've also instantiated the 3D Object - teddy - in the scene, as a child of a null object. We've adjusted the Transformations in the null object, to position the teddy correctly.

Teddy is a 3D Object that contains a skeleton with several joints. You can see the skeleton in the viewport - it's the pink lines.

Finally, we've applied a texture and material to the teddy. The project looks like this:

The 3D Object

Before you start building the effect, it's worth taking a look at the 3D Object. It's listed as teddy in the Scene tab:



It's made up of a skeleton, which groups several other objects:

  1. model_grp - underneath this null object there are several meshes. They create the shape of the teddy. We've already applied a texture and material to some of them. You'll add a sequence of textures to the remaining meshes, to create the appearance of the outfit.
  2. skel_root - these are the joints. You'll use patches to control how these joints move, to animate the body and the face of the teddy.

Animating the 3D Object

In the Scene tab, spring_mid, jack_base and jack_neck are joints that can be manipulated to animate the body of the teddy. Follow these steps to apply procedural animation to the teddy using the Patch Editor.

Moving up and down

You'll use the Position properties in jack_base and spring_mid to make the teddy move up and down:

  1. Select jack_base and spring_mid in the Scene tab. You can select them both at the same time by holding down control on your keyboard.
  2. In the Inspector panel, click the circle next to Position.

A blue patch will be created for the Position property of both joints. We chose the position property, because we'll use a Transition patch to set the values that the position of the joints move between, and the Loop Animation patch to animate this movement.

Right-click in the Patch Editor to open the menu. Select 2 Transition patches and a Loop Animation patch. Connect the patches, so your graph looks like this:



For the object to move correctly, you'll need to adjust the properties in the Transition and Loop Animation patches.

In the Loop Animation Patch:

  • Check the box next to Mirror - this will mirror the movement of the teddy, so it moves up and then back down again.
  • Change the Duration to 0.25 - to speed the animation up.

In the Transition patch connected to jack_base:

  1. Next to Start, change Y to 8.
  2. Next to End, change Y to 10.
  3. Change the Curve to Sinusoidal In-Out - to set the pacing of the movement.

It'll now move up and down from the base. You could experiment with adjusting these values to make the movement more or less dramatic.

In the other Transition patch:

  1. Next to Start, change Y to 10.
  2. Next to End, change Y to 11.4.
  3. Change the Curve to Sinusoidal In-Out.

For both patches, make sure the other Start and End values are set to zero.



Moving side to side

You'll use the Rotation properties in sping_mid and jack_neck to make the puppet move from side to side.

  1. Select spring_mid and jack_neck in the Scene tab.
  2. In the Inspector panel, click the circle next to Rotation.

Again, create 2 Transition patches and a Loop Animation patch. Connect the patches, so your graph looks like this:



In the Loop Animation patch:

  1. Check the box next to Mirrored.
  2. Set the Duration to 0.75.

In the Transition patch connected to spring_mid:

  1. Next to Start, change Z to -10.
  2. Next to End, change Z to 10.
  3. Change the Curve to Sinusoidal In-Out.

It'll now move up and down from the base. You could experiment with adjusting these values to make the movement more or less dramatic.

In the other Transition patch:

  1. Next to Start, change Z to -15.
  2. Next to End, change Z to 15.
  3. Change the Curve to Sinusoidal **In-Out.

Again, for both patches, make sure the other Start and End values are set to zero.

The teddy is now moving from side to side too. To add extra interest and a slightly more natural movement, you can add a Delay patch to this graph:

  1. Right-click in the Patch Editor, and select a Delay patch.
  2. Break the connection between Loop Animation, and the second Transition patch.
  3. Connect to Progress output in the Loop Animation to the Delay input.
  4. Connect the Delay output to the Progress port in the Transition patch.
  5. Set the Duration in the Delay patch to 0.25.

Because the Transition patch is connected to 3D Rotation of the the joint that controls the movement of the teddy's head, the head is now moving at a slightly different time to the rest of the puppet.

Responding to Facial Movement

In this effect, we've used the face tracker to control the movement of the object. As a result, the movement of the teddy is controlled by the movement of the user - like a puppet.

Start by adding a face tacker:

  1. Click Insert.
  2. Select a Face Tracker.
  3. Drag the face tracker from the Scene tab into the Patch Editor, to create a patch.

Responding to an open mouth

This part of the graph will make the teddy's mouth open, at the same time as the mouth of the person using the effect. It uses the Rotation property in the joint called mouth_lower, which is a child of jack_neck in the Scene tab:

  1. Select mouth_lower in the Scene tab.
  2. In the Inspector panel, click the circle next to Rotation to create a patch.

From the menu in the Patch Editor, select:

  1. The Mouth Open patch.
  2. The Multiply patch - you'll need to change the Type to Point 3D, by right-clicking on the patch.

Connect the patches, so your graph looks like this:



The final step is to add values to the Multiply patch - to set how much the mouth should open. Set X on the bottom row of the Multiply patch to 15.

The teddy's mouth should open at the same time as the user:



Raising the teddy's arms

This graph will make the teddy's arms wave more dramatically when the person using the effect opens their mouth.

It uses the Rotation property in the joints called wrist_r, wrist_l, arm_l and arm_r. These joints are children of the null objects listed under jack_base in the Scene tab. To find them, click skel_root to reveal the objects underneath it. Under jack_base there's another joint called jack_neck. There's also two null objects, selected in the image below:



Expanding these two null objects will reveal a series of null objects, and the joints you'll need to manipulate for this part of the effect.

So, like we did before, start by creating patches to represent the Rotation property of wrist_r, wrist_l, arm_l and arm_r.

From the menu in the Patch Editor, select:

  1. A Loop Animation patch.
  2. A Multiply patch - this patch will multiply the signal from the Mouth Open patch, to create a more dramatic effect.
  3. A Pack patch.
  4. 2 Transition patches.

Connect them, so your graph looks like this:



Again, make a few changes to the Transition patches:

  1. In each Transition patch, set the Curve to Sinsusoidal In-Out.
  2. For the Transition patch at the top of the graph, the End position to -20 next to Z.
  3. For the Transition patch at the bottom of the graph, the End position to -30 next to Z.

In the Loop Animation patch:

  1. Check the box next to Mirrored.
  2. Set the Duration to 0.25.

In the Multiply patch, change the lower input value to 30.

To connect this part of the graph to the face tracker, connect the Mouth Openness output in the Mouth Open patch, to the input in the Multiply patch.

Making the teddy blink

Start by selecting an Eyelid patch from the menu in the Patch Editor. This patch captures data on the position of someone's eyelids - so it always needs to be connected to the Face port in the face tracker patch.

Next:

  1. Create patches to represent the Scale properties of the joints eye_l and eye_r. The Teddy's eyes will disappear - because their scale is now being controlled by the patches.
  2. Select a Subtract and Pack patch from the menu in the Patch Editor.

Connect the patches, so your graph looks like this:



You'll notice we've only used the Left Openness port in the Eyelid patch. We've then used Subtract and Pack to make this signal drive the scale of both eyes. This is because if we used both the left and right eye openness ports, the eyes would likely appear to blink at very slightly different times, in sync with the person using the effect. We felt that for this effect, making both eyes move at exactly the same time would look better.

You'll need to make some changes to the values in the patches:

  1. Set the Subtract input to 0.1. This will make the teddy's eyes completely close when the person viewing the effect blinks. If you didn't do this, they'd stay a tiny bit open.
  2. Set both Pack inputs to 1. This will maintain a uniform scale on the X and Z axis.

The teddy's eyes should blink at the same time as yours.

Moving the eyebrows

This graph will make the teddy's eyebrows move at the same time as the person using the effect.

Select an Eyebrow patch from the menu in the Patch Editor. This patch captures data on the position of someone's eyebrows.

Next:

  1. Create patches to represent the Position properties of the joints brow_r and brow_l.
  2. Select two Add patches from the menu in the Patch Editor.
  3. Change the Type of the Add patch to Point 3D - because we're working with a 3D object.

Connect the patches, so your graph looks like this:



For each Add patch, set the Y value to 7 and the Z value to 11. This sets the co-ordinates that each eyebrow moves between. You could experiment with changing these values for a more or less dramatic effect.

The teddy's eyebrows should move at the same time as yours.

Setting the head movement

The final stage is to connect the rotation of the user's head, to the rotation of the teddy's.

We'll use the 3D Rotation port in the face tracker patch, which captures data on the rotation of the user's head.

You'll also need to revisit the first graph you made - which made the bear move up or down. You'll use a Multiply patch and two Add patches to combine the user's movement with the movement of the teddy's neck and middle. This movement is driven by the spring_mid and jack_neck patches you created earlier.

Create:

  1. 2 Add patches - these patches will add the data from the face tracker to the movement of the teddy.
  2. A Multiply patch - we'll use this to control how dramatic the movement of the teddy is.

For each patch, set the Type to Point 3D.

Next, break the connections between the Transition patches and spring_mid and jack_neck. Insert an Add patch after each Transition patch, so your graph looks like this:



There won't be any change to the effect just yet.

Now create the connections to make the head move side to side:

  1. Connect the output of Multiply to the Add input in the Add patch connected to spring_mid.
  2. Connect the 3D Rotation output in the face tacker patch to the Multiply patch input.

The teddy's body should now rotate when you move your head.

In the finished effect, we set the values in the Multiply patch to 0.5 - to make the movement a bit more subtle.

Finally, connect the 3D Rotation output in the face tacker patch to Add patch connected to jack_neck. The teddy's head should now move at the same time as yours.

You graph should look like this:



We can now move on to the teddy's outfit!

Changing the Teddy's Outfit

This effect uses a patch group to change the number on the teddy's vest and hat, and the color of the outfit when the user taps the screen. You can test this out in the finished effect, by selecting Simulate Touch in the Simulator:



The different numbers that appear on the teddy's vest are textures, imported as a sequence. They're listed in the Assets panel under Animations, as inventory_tex.-animation.

The patch group is listed in the assets panel under Patches, it's called Teddy_outfit.

Drag the patch group into the Patch Editor to create a patch:



You can click the button in the right corner of the patch to expand the group.

You'll see we've used patches to change the shirt and hat decal, and the color of the outfit, based on screen taps.

If you want, you can change the colors in the Option Picker to choose different colors for the effect:



You'll notice the colors in the option picker are darker than the colors you can see in the finished effect. This is because we'll combine the texture applied to the teddy with these colors. The texture has a light area covering the teddy's shirt and hat, which makes the outfit lighter.

Click Back to Main in the top left of the Patch Editor to return to the rest of your graph:



The patch group has 2 input ports:

  • Tex Input - this is where you'll connect the texture.
  • Decal - this is where you'll connect the animation sequence. The logic in the patch group will cause this to change when the screen is tapped.

It has 2 outputs:

  • Output - you'll connect this to a patch representing the Diffuse Texture property in the material.
  • Decal Fame Output - you'll connect this to a patch representing the Current Frame in the animation.

To create the patches to you need representing the texture and animation:

  1. Drag teddy_tex and inventory_tex into the Patch Editor - 2 orange patches will be created.

To create a patch representing the material, select teddy_outfit_mat in the Assets panel. In the Inspector panel:

  1. Go to Diffuse under Shader Properties.
  2. Click the circle next to Texture.

To create a patch representing the animation, select inventory_tex.-animation in the Assets panel. In the Inspector panel:

  1. Go to Advanced.
  2. Click the circle next to Current Frame.

Connect the patches, so your graph looks like this:



If you tap the teddy in the Simulator, you'll see the outfit and color changes.

You've now finished the effect!