TOKYO--(BUSINESS WIRE)--NTT Communications Corporation, Toei Company, Ltd. and NTT QONOQ, INC. jointly announced today the immediate launch of a customer-service demonstration using a digital human to communicate with people. In the demonstration, a realistic digital human, named "CONN," created by combining human-like visuals generated by Toei Zukun Laboratory’s digital human technology1 and human-like personalities (behavior and voice) developed with NTT Human Informatics Laboratories' body motion-generation and voice-synthesis AI technology, will interact with people visiting NTT Com's OPEN HUB Park2 virtual space. The results will be used to determine the possibility of using digital humans as new points of contact for commercial services.
CONN was developed from the faces of nine real people, or "catalysts," who were scanned with the latest version of LightStage3 from Toei Zukun Laboratory, based on which a realistic human (digital human) was created using computer graphics.
For CONN's personality, including behavior and voice, the motion-capture data and voice data of the catalysts were used to train an AI engine developed with NTT Human Informatics Laboratories, which reproduced the behavior and voice characteristics of real people in CONN.
In the final step, proprietary software developed by NTT Com converted the feature points of the captured facial and personality data to 3DCG.4 The software then generated digital motion and voice based on a learned program.
When people visit the OPEN HUB Park, CONN will communicate with them in a natural style, exhibiting various facial expressions, behaviors and voice tones, as if it were actually thinking, while guiding people through the various zones of the virtual park.
Reception zone: CONN will introduce the OPEN HUB Park concept using information displayed on seven LED monitors that visitors will view while wearing AR glasses.
Presentation zone: CONN will appear on a life-size LED monitor to explain the video content displayed on the monitor, and also will conduct a brief interactive Q&A session with the visitors.
Library zone5: Animals symbolizing Smart World's initiatives will appear in the visitors' AR glasses to guide people through the library zone.
The 3DCG CONN digital catalyst will be optimized for multiple devices to be able to move seamlessly between the AR glasses, seven LED monitors and life-size monitor.
For the demonstration, each company has been assigned specific roles:
- NTT Com: Provision of the OPEN HUB Park demonstration site, preparation of the evaluation scenario, architecture and infrastructure, and management of quality control
- Toei: Supervision, production, and construction of digital catalyst system
- QONOQ: Development of software, demonstration content and future business opportunities
The three companies will use learnings from the demonstration to explore opportunities for using digital humans within their own companies and in solutions for customers in a wide range of industries and sectors, such as event/ showroom attendants, corporate receptionists, and clothing store clerks.
As the challenges of labor shortages and needs to improve productivity grow in Japan, companies are attempting to create solutions that incorporate advanced technologies such as XR, the metaverse and robotics. Chatbots and virtual characters, for example, have been widely deployed in contact centers and stores. But many people still prefer to deal with real people, or at least virtual characters that can move and communicate like real people.
Against this backdrop, the three companies created CONN, a digital human that behaves like a natural human. CONN combines the digital human generation technology of Toei Zukun Laboratory with the body motion-generation AI technology6 and voice-synthesis AI technology7 of NTT Human Informatics Laboratories. CONN will serve as a digital catalyst8 to deliver customer services in the OPEN HUB for Smart World,9 based on which possibilities will be explored for using digital humans, developed mainly for entertainment so far, as new points of contact for corporate services.
1 Digital human technology uses high-definition scanning and motion capture, which are combined with 3DCG to create photorealistic human figures.
2 OPEN HUB Park is a workplace equipped with cutting-edge technology that NTT Com opened in February 2022.
3 LightStage is a human face scanning system that can reproduce not only shapes but also textures in high definition by taking pictures while controlling a large number of lights set up in a celestial sphere.
4 Three-dimensional computer graphics is a method of representing objects as 3G objects on flat surface in 3D space.
5 The library zone will be implemented in a later stage of the demonstration.
6 NTT Human Informatics Laboratories is developing body motion-generation AI technology that uses models built from the speech and motion data of specific persons to automatically generate motions unique to that person.
7 Voice synthesis AI technology developed by NTT Human Informatics Laboratories utilizes Deep Neural Network technology and a database of many speakers' voices to synthesize clear and natural speech with a high degree of naturalness from a small amount of speech data from a specific person to reproduce their unique characteristics.
8 In the OPEN HUB, internal and external experts who co-create projects together with corporate clients are called "catalysts." As of February 2023, there were approximately 400 catalysts.
9 NTT Com launched the OPEN HUB program with clients and partners in October 2021 to co-create and eventually implement new business concepts.