%20(2).png)
In discussion with Allan Rankin, Managing Director of Target3D
Allan, can you please explain markerless technology?
Markerless technology is the ability to capture (primarily) human movement without using any physical tracking sensors or trackables on the human or object. No retroreflective dots, no inertial sensors and no suits.
It's an incredible technology that understands the human form through video processing and represents movement in the form of a ‘skeleton’. The systems built on machine learning datasets and operating in controlled lighting environments, can identify and label human parts, head, legs, arms, torso and therefore infer a skeleton form for a participant. The skeleton is then used to derive movement of human form through processed algorithms and can be applied to an avatar to drive a virtual character.
Markerless motion capture is incredibly efficient and adaptable and crucially—unobtrusive. It can use either a single camera or multiple cameras to determine skeletal movement and then apply that data to other people or objects.
The resolution and number of cameras and software systems employed also then determine the accuracy and fidelity of capture achieved.
When did this technology first appear and really make an impact?
If we go back to the historical origins of motion capture systems (we’re not talking about rotoscoping or the earliest forms of motion capture, which were essentially pixels on a frame representing an object) the first attempt at motion capture was a team of people trying to understand the movement of a racehorse way back in the early 1900s. Étienne-Jules Marey (France) and Eadweard Muybridge (USA) were pioneers in studying human and animal motion.
Marey developed a "chronophotographic gun" to capture motion in a single image sequence, while Muybridge used sequential cameras to photograph galloping horses, providing insights into locomotion.
Even before Max Fleischer introduced rotoscoping (tracing film frames for animation), we were using cameras to better understand motion. As far back as 1878, Eadweard Muybridge captured every stage of a horse's gallop to analyse and (allegedly) settle a bet with a racehorse enthusiast as to whether all 4 of a horse's feet are off the ground at a full gallop.
Markerless technology, as we know it, originated about 25 years ago. It began when synchronised cameras were able to remove background imagery in a calibrated space and therefore introducing the human body into a negative space.
Algorithms would identify the silhouette, labelling parts as “arm", "leg" and so forth, enabling the tracking of skeletal movement.
As an organisation, we've been providing markerless technology since our inception in 2017; we’ve been evaluating and demonstrating these capture systems since day one!
While markerless technology has been around for some time, it has only become robust and widely deployable in the last five or six years. Systems like Kinect—Microsoft's early motion-tracking device—played a role in this evolution.
The Kinect used three cameras: an infrared, a time-of-flight, and a video camera to interpret human movement. Initially designed for gaming, it was also used in interactive museum exhibits.
For example, I remember an exhibit at the Barbican in London where visitors flapped their arms to trigger animations of birds flying away. This was devised by Chris Milk and called The Treachery of Sanctuary.
Today, markerless capture can support multiple people - simultaneously driving avatars in a shared space. This is a major advancement from earlier systems.
But why is it such a game-changer?
Traditional motion capture, which requires actors to wear suits with markers, has overhead in terms of setup and preparation. Markerless technology eliminates the need for suits, enabling faster and more versatile deployments.
The system can also resize and rescale skeletons based on vision processing, offering a more representative understanding of the human form of the person being tracked.
For example, in an industrial setting, you can analyse the ergonomics of an assembly line without influencing worker behaviour. With marker-based systems, people might subconsciously adjust their movements because they know they're being tracked.
Markerless systems allow for natural observation, even in challenging environments. For instance, because the technology doesn't rely on reflective markers there is no interference from shiny materials like workbenches or tools - this has historically been a problem.
The versatility of markerless capture opens up many possibilities. It reduces setup time, allows deployment in public or non-standard environments, and supports simultaneous tracking of multiple individuals.
For instance, you can integrate it into a virtual game where multiple players compete without needing specialised suits.
The reduced barrier to entry makes it a true game-changer.
You mentioned standalone markerless systems and upgrades that integrate with existing technologies. Do you think these advancements will render older systems obsolete, or will they enhance and coexist with them?
It's more about enhancement and improvement.
For example, manufacturers like OptiTrack are integrating markerless capabilities into their existing systems. With the right hardware upgrades, such as better GPUs and updated software, many users can achieve markerless capture with their current setup. They can then have a truly dynamic system and enable you to use either type of system depending on your use case. Marker based systems are fantastic for tracking almost any object in a well calibrated environment, just add a marker.
This versatility is still a huge advantage if you need more than the human form in one session.
All sounds too good to be true!
Markerless systems have limitations. They struggle with precision tracking of small objects, fingers, or facial expressions without specialised setups. Peripheral technologies like gloves or head-mounted cameras are still necessary for many performance capture use cases.
While markerless technology is advancing, it hasn’t yet fully replaced marker-based systems, which still remain robust and reliable. But they’re getting there..fast!
How do you see markerless technology impacting future outputs and digital content?
Markerless technology will enable systems to operate in more challenging environments and support a broader range of applications. For example, you could track a musician on stage and transpose their movements to a Metaverse experience. This opens up new possibilities for virtual production and real-time broadcast applications.
In the medical field, markerless systems are being explored for gait analysis and preventative medicine. Imagine a patient recovering from an injury who can be assessed without the discomfort of wearing a suit.
Markerless technology allows for natural movement while still providing valuable data for clinicians and practitioners.
Overall, it’s about increasing accessibility and versatility, enabling innovations across multiple industries and society as a whole.
What are the guiding factors when choosing a tracking technology? Can you explain the approach taken by Target3D?
At Target3D, we focus on three guiding factors to help clients adopt the right technology. First, we ask what they’re trying to do. Second, we consider the environment they’re working in. Finally, we look at their budget. Even with unlimited resources, environmental constraints or specific use cases can limit the options available.
Understanding these three elements allows us to recommend tailored solutions. For example, if someone wants high accuracy in a small volume, we might suggest magnetic tracking solutions.
For tracking multiple people in larger spaces, RFID or ultra-wideband technologies can work and for animation or VFX, we’d look at optical systems with multiple cameras.
By identifying the client’s goals, environment, and budget, we ensure they adopt the most effective system for their needs.
What are some of the top technologies available today, and how do they fit into different applications?
If you’re looking for super high accuracy in a small volume where optical occlusion is a strong possibility, magnetic tracking solutions are ideal. These systems operate at high frame rates and provide detailed positional data.
For larger spaces with multiple participants, RFID or ultra-wideband technologies can track individuals with lower accuracy, such as within a meter squared. These are often used for understanding foot traffic in office buildings.
For animation or VFX applications, optical systems, marker based and markerless with multiple cameras are essential. These setups allow for detailed motion capture of multiple people simultaneously in controlled environments.
Additionally, we offer solutions for inertial suit systems, which are portable and can be used without external cameras. These are great for capturing motion in different locations.
We also provide hand and finger tracking solutions through technologies like Manus gloves and StretchSense.
For haptic feedback, we partner with companies like HaptX and Valkyrie VR to deliver immersive tactile experiences. There’s a wide range of products available, tailored to specific needs and applications.
How will markerless impact future outputs and digital content creation?
I think it will open up the landscape for people to use systems in more challenging environments where the necessity to track props and additional peripherals may not be required.
For example, if you were looking to track a performer on stage, like a musician performing in front of a large crowd, previously, you’d need to put them in a suit to capture their movements. Now, with the right setup and conditions, you could track them directly on stage.
This capability then enables marking and reconstructing their movements less obtrusive and opens up the possibility of sharing the performance in additional mediums, through game engine technologies, for VR and MR in Metaverse experiences, for training and simulation and human ergonomic analysis.
That character could have an avatar, and the audience could experience it in VR or a WebXR platform.
Contact our sales team for more information about how to integrate markerless motion capture into your OptiTrack system, or to schedule a consultation.
Comments