More on the tech, using google translate:
The system developed by Square Enix is used for the motions of the enemies. Rather than simply letting the enemies blow away by physics, it seems that, for example, when a sword hits your hand, you move down while holding your hand and generate it on the spot.
In addition, next-generation technology is used for the bones of the characters, and if the bones are distorted, the bodies of the characters will naturally be distorted. It seems that if you use the technology you are using, 3D characters will be automatically formed with a more accurate skeleton and you will be familiar with the automatically generated motion.
You can see that the right arm of the cloud before applying is slightly distorted.
Automatically generate facial expressions by referring to lines!
So far, as Mr. Hara explained, it was formed by multiplying many existing technologies. However, it seems that they are making new efforts with regard to the facial movements and facial motions of characters.
It seems that the character's eyes do not stop in this work, and he is trying to repeat the eye movement that sways vertically and horizontally like a human being. Also, for example, when talking with someone, we adopted a movement that unconsciously diverts the gaze somewhere, similar to human psychology. This seems to occur randomly.
Blinking basically occurs randomly. It also occurs when the position of the line of sight suddenly changes or the direction of the face changes. Also, the eyelids do not always close completely, but the eyes open only slightly, and various blinks occur with motion blending technology and random elements. All of these are automatically generated from body movements and face directions.
Furthermore, what can be said to be a big focus of this conference is an automatic lip sync generation system called “HappySadFace” (lip sync is that the lips move as if they are uttering the language according to the character voice's dialogue).
This lip sync refers to audio data and text, outputs the data, and plays back lip sync animation. In this work, a cloud video was released in Japanese, English, French, and German, with the same dialogue but with a completely different lip sync.
Japanese voice lip sync
English voice lip sync. The motion looks like biting the lower lip, which is peculiar to English.
This seems to have made it possible to prepare a large number of lip-syncs for each language, without having to manually create lip-sync motions. However, "HappySadFace" is a problem, it seems that it is difficult to respond if the text data to be referenced is incorrect in the first place or the ad lib performance of the voice actors is mixed. Also, the error that the automatically generated lip sync does not match the voice can only be found visually, and the current method is.
In addition to automatically generating the volume of the voice to the animation of the eyebrows, the technology of "ST Emotion" is used to estimate the emotional parameters from the voice actors' voices and automatically blend the emotional facial expressions. That there is. Thanks to this, the characters' conversations can be expressed with a human expression in every scene, simply by uttering the lines.
Up until now, it was said that the planner had to manually enter facial expressions, and the number of scenes is huge, so it seems that there were many scenes where conversations proceeded without any designation. By combining all of the above technologies, it is possible to automatically generate except for cut scenes and to enjoy rich conversation in action and casual conversation scenes.
A more human look than before applying This "ST Emotion" is said to have been used for the game field for the first time in "FFVII Remake", but it seems that it has been used in various fields up to now, and Mr. Iwasawa says it has high reliability. It may be close to being used in various games in the future.
However, as a problem, for example, Barrett's voice is often angry, so it is easy to judge it as an angry feeling, and because the child's voice is shrill, it seems that it was often judged as a feeling of joy.
This concludes the conference. When you play "FFVII Remake" again, if you pay attention to the facial expressions such as casual motions and conversation scenes outside the cut scene, you will be able to enjoy the game from a new perspective.