Monash University researchers’ new project will be developing an AI (Artificial intelligence) assisted application that will aid with real-time interpretations for diplomatic talks, global business, and tourism.
DARPA (Department of Defense’s Defense Advanced Research Projects Agency) funding the US$5 million project will be developing a smartphone-based assistive dialogue system utilizing smart glasses that will be implementing machine learning, speech recognition, and also vision technology for providing multicultural communication aid in real-time.
Monash University’s VLG (Vision and Language Group) project researchers at the Faculty of IT (Information Technology) stated that the primary objective of the program lies in developing language processing technology capable of recognizing and adapting to the social, emotional, and cultural practices that will be varying across communities, societies, and languages.
This program does not limit its ability to the interpretation of the speech content but also performs the additional task of interpreting body language and facial expressions, offering cultural prompts for the prevention of a communication breakdown and assuring smooth multicultural dialogue.
The researchers explained that during this project their major focus is on negotiation-based dialogues.
The dialogue assistance system may observe an impending communication disruption during a conversation by the analysis of audio-visual prompts in real-time. The negotiation can then be secured with the help of the system sending notifications to the consumer’s smart glass offering relevant culturally accustomed prompts.
The system may provide cues to the user for amending the negotiation by giving the other user a more comfortable feeling. After that, it may advise the user of the various methods for increasing the level of comfort like addressing the other person more courteously according to their specific cultural practices.
Deputy Dean (Research) Professor Maria Garcia de la Banda, IT Faculty greeted the support for research that will pave the way for innovation in the usage of Artificial intelligence and data science for dialogue assistance technologies.
Professor Garcia de la Banda stated that existing AI-enabled systems cannot accurately analyze the various nuances of human communications or offer effective support beyond the fundamental machine translation.
In this project, researchers will work on combining experienced speech technology with modern multimedia analysis and cultural knowledge for building systems that offer a full solution.
The research will be conducted in two phases over the next three years. The first model will be released by March 2023.
Vision and Language Group from the faculty of Information Technology at Monash University will lead this research by collaborating with researchers from the following departments:
1. David Nazarian College of Business and Economics
2.Northridge and the Department of Biostatistics & Health Informatics
3.The Institute of Psychiatry and
4.Psychology & Neuroscience at King’s College London.
