Snapchat’s Game Changing Move in AI Video and Portrait Creation
September 17, 2024Pro Nov 2024 MC
October 30, 2024Exploring Meshcapade
Revolutionary AI-Powered Motion Capture Solution
From tips and tricks on creating multi-actor boxing matches to complex martial arts movements and athletic sports motions, it seems we're on the right track.
Meshcapade has recently launched MoCapade feature, and it’s nothing short of amazing.
Unlike traditional motion capture techniques that often rely on suits and complex calibration, or different methods like text prompt to motion capture, MoCapade allows you to transform any video of human motion into 3D animations without any calibration. This means you can simply transform any kind of video into a 3D character animation as long as the actor is within the frame, though MoCapade is smart enough to understand the motion even if the actor is sometimes partially out of frame.
The potential of this technology is powerful, though there are some limitations. For now, let’s focus on what MoCapade can achieve in the 3D motion capture industry.
Hands-On Testing with Complex and Simple Motions
My name is Pooya Deperson, and I am a product designer for 3D and SaaS solutions.
Together with my athlete friends Matteo De Luca, Alessio Strolighi, and Guglielmo Frittoli, we decided to put Meshcapade to the test.
We tried a mix of complex and simple motions to evaluate its performance. For simpler motions like walking and dancing, which are typically easy for AI-powered motion capture systems, MoCapade handled them effortlessly.
We started by recording various scenarios, including multi-actor boxing sequences, martial arts moves, and athletic activities like rope climbing.
For most of the tests, we ensured the subject remained fully in frame, providing MoCapade with a clear view. Here's a breakdown of our process and what we discovered.
Testing Complex Martial Arts Movements
Starting with Alessio Strolighi, we tried some high-kick and low-kick sequences, typical kickboxing movements. These are much more complex motions due to the speed and range of movements.
We discovered one limitation involving fast movements like kickboxing. When the motion is very rapid, it can be hard for the software to capture it perfectly.
To overcome this, we experimented with recording the video at a higher frame rate of 60 frames per second (FPS) and then slowing it down to 30 FPS before uploading it to MoCapade. This trick allowed MoCapade to capture more precise movements.
Slow motion can help enhance motion clarity.
Elevation from the ground
We also wanted to see how well MoCapade handles actor elevation from the ground, so we recorded a rope climbing scenario and the results were quite impressive.
Path to
Capturing a Multi Actor Boxing Match
Next, we recorded a boxing session with two actors, realizing that it cannot track multiple actors. However, what it does well is track one actor for the entire session.
So, we intentionally had the second actor enter the frame a bit later
Remarkably, MoCapade tracked the initial actor with ease. We saw the potential to reach our goal, so we tested the same scene but masked out the first actor for a few frames to trick MoCapade into tracking the second actor.
This means that for multi-person animations, like a boxing match, it is possible by masking out one actor in the initial frames and later combining both in 3D software to create a fully animated scene.
However, it would be great if MoCapade included a built-in feature to focus on multiple actors at once without the need for manual editing afterward. This is definitely something to request from the team.
Aligning Meshcapade Animations to Industry Standards
We found that Meshcapade animations use the Root for animation instead of the hip. While this works, following the industry standard of animating the hip would be ideal. We fixed it by shifting the animation data from the Root to the Hip, but this step could've been avoided if it had been done this way initially.
Multi-person tracking could save artists significant time in the animation process.
Combining both motions, we brought them into Cinema 4D to create a complete boxing match with both characters
MoCapade is an incredibly powerful tool for artists and animators looking to create 3D motion capture from video without the need for expensive suits or hardware.
From boxing and martial arts to rope climbing, it can handle a wide range of motions. However, some improvements, such as multi-person tracking, could make it even better.
Written by
