Story image

Are you ready to face your digital doppelganger

14 Mar 2019

Creating your true digital twin is now possible and will provide a groundbreaking 3D future for the entertainment, gaming, tourism and telecommunication industries, says a global expert on computational photography and computer vision.

Professor Jingyi Yu, of ShanghaiTech University and DGene, uses a combination of light field acquisition and reconstruction technology to generate an exact 3D digital replica of a person. 

He has been studying and developing Augmented and Virtual Reality technology for 20 years. 

Light field technology works by capturing all light emitting from an object, such as a person, in place of 2D images.  

Then using advanced machine learning and image processing technology, Yu and his team can recover ultra-high resolution and super realistic dynamic human models from their real twins, in real time. 

While seemingly futuristic the technology is expected to become commonplace within a few years, replacing 2D imagery with 3D for most digital communications.

Yu says the potential is huge for areas such as events and venues, as well as uses for remote education and medicine, and virtual tourism.  

“We have lived in a 2D image world for decades but we are close to when 3D imaging will become normal, and bring a new dimension and more reality to our digital experience,” he says.

The light field technology’s first foray into practical use came when it was used to show products on e-commerce sites.  This enables a product to be shown in 3D, such as a bottle of perfume, with the ability to rotate it as if you were picking it up off a shelf in a shop. 

“From these small beginnings the technology has grown to encompass both medium and large scale applications,” he says.

“We can now capture all aspects of a person’s appearance automatically including skin reflection, light, depth, colour, clothing textures and aspect, to provide an ultra-realistic and finely detailed reproduction of anyone in real time. We can now all have a ‘digital twin’.

“A larger scale use would be for a live broadcast and using it for 360 degree and 3D imaging - people will never miss a single angle and even freeze any moment and replay. The personal TMO is not far away, which may solve some discussions about the referee’s decision,” he says.

All of this is possible using current broadband speeds, which will accelerate and improve once 5G arrives.  

The exact replica imaging is the result of multiple cameras, up to 80 being used, to capture constant images of every angle that are then reconstructed in 3D instantly.  The technology is powered by accelerated computer and algorithm speeds. 

AI Forum NZ director Ben Reid says this is an incredible example of visual technology enabled by and working with Artificial Intelligence.

Apple launches revamped iPad Air & iPad mini
Apple loves tinkering with its existing product lines and coming up with new ways to make things more powerful – and both the iPad Air and iPad mini seem to be no exception.
Epson innovations and Mercedes-AMG Petronas Motorsport
The world’s greatest motorsport event, the Formula One Grand Prix World Championship, descended on Melbourne’s Albert Park over the weekend for the first race of the 2019 season.
Tesla unveils the Model Y SUV
After much anticipation, Tesla unveiled the Model Y last week – a vehicle that is described as an all-electric, mid-size SUV that can seat seven adults – and the vehicle has a glass roof.
Preparation for Tokyo 2020 Olympics begins - with robots
The Tokyo 2020 Olympic Games are quickly approaching, but it won’t just be a sea of athletes and sports fans – now robots will make up a significant part of the fan experience.
NZ ISPs block internet footage of Christchurch shootings
2degrees, Spark, Vodafone and Vocus are now blocking any website that shows footage of the mosque shootings.
How AI could warn civilians before a volcanic eruption
Advance monitoring could lead to better disaster planning and evacuation warnings in the event of an eruption.
Facebook launches dedicated home for its Gaming
"All of our work on the Facebook Gaming team adds up to helping build the world's gaming community."
Spotify calls out Apple's anti-competitive behaviour
Apple's App Store rules "purposely limit choice and stifle innovation at the expense of the user experience—essentially acting as both a player and referee to deliberately disadvantage other app developers".