Text-Based AI Motion Capture Solutions
September 12, 2024
snaptchat ai
Snapchat’s Game Changing Move in AI Video and Portrait Creation
September 17, 2024
Text-Based AI Motion Capture Solutions
September 12, 2024
snaptchat ai
Snapchat’s Game Changing Move in AI Video and Portrait Creation
September 17, 2024

Beta Testing Cartwheel AI Text-to-Animation

I recently had the chance to try out the private beta of Cartwheel, an AI tool for motion capture. I decided to give it a go and see how well it could handle different animation requests.

Cartwheel is an AI text-to-animation solution where you can type in the kind of animation you want, and it will create it for a biped character. It’s a great example of how you can bring your imagination to life.

Cartwheel text-based Motion generation
Path to
Creating a Multi-Action Animation Sequence Using Cartwheel
An Example of a Multi-Action Animation Sequence Using Cartwheel and Unreal Engine

Prompting a Complex Animation

At first, I used basic prompts like "walk" and "jump," and while the results were fine, I felt the need to push the limits. So, I tried a more complex sequence where the character walks forward 10 steps, jumps, rotates, and then runs in the opposite direction. The AI understood the walk and jump parts, but the full sequence didn’t come out quite right.

The character's movements were a bit off, and sometimes one character would execute the action well while another would struggle. Occasionally, the animation results were also biased, like the character holding a rifle while walking.

Prompt "Walk forward 10 steps, then jump high and forward. After landing, turn around and run backward for 20 steps"
Prompt "Walk forward 10 steps, then jump high and forward. After landing turn around and run backwards for 20 steps."

Varitation Selection

The variation selection popup is useful, however, I wish I could preview the motions more clearly in each video without having to select them. I was initially concerned that if I chose one variation, I might lose access to the others. At first, I was very particular about selecting the one that looked best. Later, I realized I could always return to the variations without losing them.

I believe this concern stemmed from other platforms' pricing systems, where losing the variations could mean needing to re-enter the prompt, with each prompt potentially costing credits.

Break the animation sequence down into individual actions.

To fix the issue with the complex animation sequence, I decided to take a different approach and break the sequence down into individual actions: walk, jump, rotate, and run.
I tried to write each prompt in detail:

  1. Walk forward 10 steps.
  2. Jump high and forward.
  3. Turn around to face the opposite direction.
  4. Run for 20 steps.

This worked better, although step 2, "the jump," wasn't great at first. After tweaking the prompt and adding more detail, the jump improved, and the final results were closer to what I wanted.

Revised prompt for number 2:

  2. Perform a forward long jump, leading with the right leg, like in a sporting event.

Top Left: 1.Walk forward 10 steps. Top Right 3.Turn around to face the opposite direction. Bottom Left: 4.Run for 20 steps. Bottom Right: 2.Perform a forward long jump, leading with the right leg, like in a sporting event

The Issues I Encountered

While exporting the animations into the 3D software to combine, a few thoughts came to mind.

Can I access all my previous animation prompts again?

As I was prompting and exporting the animations, one concern was whether my previous prompts and animations were saved. After looking around, I found a "Favorites" button, which made me wonder: if I favorite an animation variation, does that mean I can re-access it?

However, it wasn’t clear how to access past prompts, and I wondered, what if I forget to favorite an animation? Does that mean I lose it forever?

These concerns arose with the rise of prompt engineering, where each prompt might cost users credits, adding pressure to not lose past work.

Favourite button on Cartwheel's header

Character Polygons / Vertices Are Not Connected

Cartwheel's brand characters look nice and smooth, but when I exported the animation with the mesh into the 3D software, I realized the character's polygon meshes were not connected. This resulted in a low-poly, hard-edged appearance, which is great if that's the look you're after, but it didn’t appear as smooth as it did on the Cartwheel platform. While this is something that can be easily fixed in 3D software, it might be confusing for users who expect the same smooth look they see in Cartwheel.

Left "Cartwheel platform', Right "3D software Cinema 4D".
Feature Request
Allow Combining Animations on the Cartwheel Platform

Overall, I was able to complete the animation work and combine the actions into one sequence where the character walks, jumps, rotates, and runs. While doing so, a feature request came to mind. Cartwheel's AI is impressive but struggles with complex animations. One improvement would be to guide or allow users to break down complicated animations into smaller actions directly on the platform.

It may be challenging for the AI team to improve the model for complex animations, but introducing a feature that lets users combine individual animations and export them as a sequence could be much easier. This would simplify creating a single animation without needing external 3D software, which many users may not be familiar with.

In conclusion, Cartwheel is a powerful text-to-animation tool, but it could be even better with features that make handling complex sequences easier and more flexible.

Combined animation in Unreal Engine
Combined animation with a character skeleton view in Unreal Engine

Written by

  • Pooya Deperson
By using this website, you agree to the terms.