一句话：集成LLM与OpenFace优化blendshape到面部动作单元映射，生成动态3D表情。
本文针对游戏中情感响应型3D角色生成困难的问题，提出一个优化的生成式AI框架。该框架将OpenAI的LLM与OpenFace结合，改进blendshape（或morph target）与FACS定义的面部动作单元之间的映射，以生成动态面部表情。系统嵌入原创游戏场景，根据玩家交互对话生成表情，并提供模块化、可扩展的AI驱动动画管线，支持自适应情感建模与游戏引擎集成。用户评估显示，系统具有较高真实感、清晰情感识别与较强参与度；同时指出需进一步优化blendshape映射和实时调整，以区分Disgust、Contempt等细微情感，提升整体表现力。
Facial expressiveness is essential for immersive gaming, yet generating emotionally responsive 3D characters remains challenging. This paper presents an optimized generative AI framework that integrates OpenAI's LLMs with OpenFace to improve the mapping between blendshapes (or morph targets) and facial action units (standardized facial muscle movements defined by Facial Action Coding System) for dynamic facial expressions. The system, embedded in an original game scene, generates facial expressions based on interactive player dialogues. From a software engineering perspective, it features modular, scalable AI-driven animation pipelines that support adaptive emotion modeling and game engine integration. User evaluations demonstrate high realism, clear emotion recognition, and strong engagement, confirming the system's potential to enhance player interaction. Further refinements in blendshape mappings and real-time adjustments can enhance the differentiation of subtle emotions like Disgust and Contempt, improving the system's overall expressiveness.
@inproceedings{paperbot8960,
title = {Enhancing Emotional Realism in Games: An Optimized Generative AI Framework for Dynamic 3D Facial Animation},
author = {Jonas de Araújo Luz and Rafael Fonseca Pessoa and Guadalupe Prado Saldanha Ribeiro and João Vitor Vieira Lira and Maria Andréia Formico Rodrigues},
booktitle = {FSE},
year = {2025},
doi = {10.1145/3696630.3730554}
}