%20(2).png)
In discussion with Ashley Keeler, Head of StudioT3D and CTO, Target3D
What is the future of performance capture technology?
It's very, very exciting.
If you look at the performance capture tech development pipeline we're seeing avatars and physical likenesses improve on an almost week-by-week basis.
This will have a major impact across a number of different areas
A few years ago, when Epic released the Metahuman pipeline, they introduced a photo-real, free-to-access character creation tool-kit that could be used inside their free render engine.
This was a significant step forward in democratising digital human creation, giving individuals and smaller studios access to tools that were once reserved for high-budget productions. As a result, the industry saw a surge in independent creators producing high-fidelity characters that would have previously required entire teams and expensive software pipelines to achieve.
Since then, we've seen a push toward giving consumers and independent artists access to previously unobtainable levels of detail. Now, they can drive hyper-realistic characters using the technology in their pockets—apps that deliver compelling face animation and full-body data using just an iPhone or Android device.
Also, we’re seeing a trend of increased interest from bedroom producers who want to create online virtual influencers or animated social media content. They’re creating either real or hyper-stylised characters and generating meaningful conversations on social media platforms, whether through livestreams or pre-produced content.
The accessibility of tools like these has also led to an explosion of user-generated digital content, with independent artists exploring new ways to tell stories through performance capture.
Additionally, many brands and companies are leveraging this technology for marketing campaigns, product placement and demonstrations, and interactive storytelling.
And, of course, larger entertainment studios continue to develop these methods to create virtual doubles for actors, allowing for high-fidelity motion and facial replication in blockbuster productions.
The music industry has also begun experimenting with AI-assisted virtual concerts, where digital avatars of artists perform live shows synchronised with real-time motion capture. Additionally, large-scale events are beginning to integrate virtual audience members, creating hybrid digital and physical experiences where AI-driven crowd simulation enhances the realism of performances.
That’s where we are now. It’s something we’ve talked about for a long time, but it’s a very, very exciting moment.
We’re not far away from utilising markerless systems - for instance the tech in your phone - that doesn’t require performers to wear anything at all, technology wise at least!
This means actors could perform in their natural environment without bulky suits or markers, making the process more intuitive and less intrusive. This is especially relevant in industries such as sports analysis, where real-time body tracking can provide valuable insights into an athlete’s form, movement efficiency, and potential areas for improvement.
So, instead of needing to visit a motion capture studio, we will begin to see pipelines for extracting motion capture data from actors on set - subtracting backgrounds using the same core advances that sit behind AR filters and AI tools. There is now no need to do these things twice—we can go to a film set, an athletic performance, or a stunt rehearsal and capture movement without actors needing to wear anything or altering their performance.
This removes the mental barrier that comes with unfamiliar equipment. By keeping performers comfortable and environments non-invasive, we can use markerless tracking systems to extract or estimate skeletal data, which we can then use for an avatar pipeline.
Can you tell us about the importance of volumetric capture?
So the next step is the volumetric approach.
Volumetric capture is a branch of digitising performance that runs alongside traditional performance capture - a different path to digital recreations. The difference is that we’re not capturing motion and then retargeting it to a lifelike avatar. Instead, we are capturing the actual mesh and texture information of the person as the performance happens.
So, rather than being left with skeletal information, we’re left with— for want of a better expression—a ‘hologram’ of the actor performing. This is akin to markerless mocap because people remain in their regular clothing and again making the process non-invasive.
However, it does currently require intense, bright lighting to achieve an even ambient illumination across the whole subject, which can be uncomfortable. But otherwise, we're not asking anyone to wear specialised suits or be out of costume.
The nice thing about volumetric capture is that it also grabs texture. It captures clothing movement, hair movement, and removes the need for a digital avatar pipeline where costly steps like simulating hair, clothing, and muscle dynamics still exist. Instead, volumetric capture handles all of that as part of the raw capture.
Of course there are pros and cons…
With the digital avatar performance capture pipeline, it's easier to edit and tweak things since you have control over everything. On the other hand, editing volumetric capture data is trickier because what you capture is what you get.
Traditional performance capture can easily fall into the uncanny valley since everything is digital, whereas volumetric capture, by nature, avoids that because it directly captures the real performance instead of approximating it. The result is a mesh, texture, and audio performance of what actually happened on stage.
That’s where we are now with those two avenues of performance digitisation. It’s not a perfect term, but you get what I mean.
How do you help clients understand the landscape of capture technology?
Our clients are split 50/50 - some people are knowledgeable, but many aren’t.
That’s why we kicked off with a kind of "beginner’s guide" to markerless mocap—framing what it is so that it's not too simplified for those already familiar but remains accessible and democratic for those who may not know much about it.
I think there’s still value in figuring out where markerless mocap sits as performance digitisation is a broad field. There are at least ten different types of motion capture and multiple methods within just markerless capture alone - we’ll discuss these in future articles.
So, looking ahead—what’s next for volumetric capture?
In that space, there’s a lot of excitement around Gaussian splatting. This technique allows for complex textures and shapes to be rendered in game engines more efficiently. People are excited because it provides a new way to enhance realism in animations and simulations.
Right now, volumetric capture loses some intricate details, but Gaussian splatting offers an approach that retains more detail while being more efficient. That method can also integrate with machine learning to dynamically generate content based on the game or experience.
And, of course, we can’t ignore what’s happening in generative AI with tools like Sora, MidJourney, or even just this week—Google DeepMind's VO2 video.
Generative AI…where can it go?
Did you see the recent Porsche ad?
A producer, Laszlo Gaal, spent three weeks creating a fake Porsche commercial using DeepMind's VO2 text-to-video system.
It’s mind-blowing.
What he achieved would have cost thousands to produce traditionally. He would have needed a full crew, multiple locations, and there would have been a significant carbon footprint.
A conversation for a different time but—are we considering the environmental impact of generative AI? It’s a crucial topic, and one I’m keen to highlight, but later!
There are clear benefits: AI brings high-quality production capabilities to more people. But there’s also a massive ethics conversation.
We're rapidly moving toward AI being able to generate entire movie scripts, 3D assets, video, music, and assemble everything into a final cut. It’s both terrifying and exciting. And the knock-on effect? Quantity over quality?
What about, content personalisation—people becoming the main character in their favourite shows or even swapping every actor in a film for Nicolas Cage (if that’s your thing!).
Given the state of deepfakes today, we’re not far from being able to do this in real time. That raises serious concerns around safeguarding and identity theft.
For me, the Laszlo Gaal video was the first time I genuinely couldn't tell which parts were AI-generated and which were real. The second half of the video even shows how it was made, but that part is also AI-generated! That was the first moment I thought, wow, this is pretty wild.
What about the hidden costs of AI?
No one in the creative space can afford to ignore generative AI anymore. But I don’t think we fully understand its financial and environmental costs. From a sustainability perspective, it’s staggering. Just consider Nvidia’s commitment to producing AI hardware—the energy required to power those GPUs is astronomical.
And the comparison between traditional production vs. AI-generated content? We still don’t have a clear picture of whether the savings outweigh the environmental and economic costs.
One thing’s for sure: we don’t want to reach a point where AI takes over all creative roles, leaving us as mere prompt engineers.
Actors, voice artists, animators, designers—what they bring to the table comes from the human mind. I hope we hold on to the idea that their creativity will always be more valuable than what we can type into a machine.
It’s not just about job losses. The environmental impact is just as alarming.
There was a great post recently on LinkedIn from a copywriter saying:
"I drink 1.5 litres of water a day while writing content, meanwhile, every ChatGPT search consumes half a litre of water."
Think about that. Every single interaction with AI comes at an unseen cost.
And as ChatGPT moves behind a paywall it’s clear that the impact of these models is something we haven’t fully reckoned with.
AI in gaming - what are the ethical and practical challenges?
There are justifiable use cases for generative AI, particularly in gaming. AI-generated environments, dynamic character behaviours, and personalised experiences could revolutionise open-world games.
That’s part of the reason why recent industry strikes have been so complex.
Motion capture and voice actor strikes weren’t just about pay—they were about defining AI policies. Many actors see the benefits of AI-enhanced NPCs that can interact with players in a more dynamic way. But they also want fair compensation for their likenesses and performances being adapted - and control over what product (or campaign) their likenesses are promoting.
Those strikes had huge ramifications for the industry. They affected studios worldwide. But AI’s potential in gaming remains incredibly exciting if it’s managed correctly.
The key question is: how do we create policies that ensure ethical use without stifling innovation?
Final thoughts?
What started as an effort to make processes more efficient—like motion capture reducing animation time or 3D printing improving prototyping—has now shifted toward maximising data consumption at any cost.
Generative AI isn’t about efficiency; it’s about brute-force computing that has a HUGE overhead - half a litre of water per ChatGPT interaction? That’s terrifying.
[I've not even broached whether or not the source training content has been obtained with fair compensation... which, let's face it, it probably hasn't.]
I fear that sometimes we get so excited about what’s possible that we don’t always stop to ask - should we even be doing this? And that might be the most important question of all.
Let’s keep people and art at the centre of performance capture technology and spare a thought for the planet… please.
Stay tuned for further discussions with Ashley!
To see all the different technologies we house in our studio, visit www.studiot3d.com, or contact the studio team at info@studiot3d.com.
Comments