Emotion Recognition and Artificial Intelligence: The Core Modules

Artificial Intelligence for Emotion Recognition

“Emotion recognition is the core component of modern Artificial Intelligence. The objective is to capture both the self-emotional traits and the others-emotional traits. ” – Sri Amit Ray, Pioneer of Compassionate Artificial Intelligence.

Emotion Recognition and Artificial Intelligence - The Core Modules

Emotion recognition within the Ray compassionate machine learning system includes many domains of social functioning for the AI-based social robots. They are vital for modelling the core modules of our machine learning based deep emotion recognition algorithms. The primary functions of these modules are:

  1. maintaining integrity in occupation
  2. positively participating in the community
  3. social competence and
  4. maintaining good peer relationships
  5. core domain performance and competence.

For emotional recognition, there is growing interest in our compassionate artificial intelligence lab about using different intrinsic functional connectivity between the machine learning modules. The objective is to capture both the self-emotional traits and the others-emotional traits.

Our Ray Deep Self-awareness algorithms are focused to capture both the self-emotional traits awareness and the others-emotional traits awareness. There are two important parts in the Ray Deep Self-awareness learning algorithms, the Reflective-AI-SELF and the Core-AI-SELF.

The reflective-AI-self is designed to top-down modulate the Core-AI-SELF so that there is bilateral modulation between lower and higher aspects of selfhood.

The Ray Deep Reflective-AI-SELF algorithms are focused on assessing the values, morality, ethics, and the behavior of the system itself. Self-reflection is the ability to witness and evaluate the system’s own cognitive, emotional, and behavioral processes.

The social robot or the AI system reflective AI algorithms questions its own strengths, weaknesses, limitations, misuses, skills, ethics, problems, achievements, and solutions. The algorithms review and update its ethics and effectiveness in the limiting situations. The algorithms tries to adapt and modify itself so that the next time the behavior will be beneficial for self and others.

The Ray Deep Reflective-AI-Self algorithms are important and flexible tools for investigating the social cognition and behavior of intelligent agents of machine learning.

In the current model, we examined the effectiveness of the algorithms with the TASIT Emotion Evaluation Task, which assesses emotion recognition ability via a sequence of short (15–60 s) videotaped vignettes featuring interactions.

The Ray Deep Core-AI-Self  Awareness algorithms are powerful for analyzing higher-order beliefs about the opponent’s intentions. According to the Ray Core-Self Reinforcement Learning models, there is an appraisal mechanism that assigns value to actions based on the likely outcomes and the amount of effort required.

The ability to attend to and interpret a variety of social phenomena such as verbal messages, paralinguistic information (e.g., intonation), nonverbal behaviors (e.g., facial expression, eye gaze, and gesture), and contextual information such as knowledge of the type of social relationships and potential conflict of goals between intelligent agents and human is part of the Core-self awareness AI algorithms.

Data and Reliability Testing of the AI based Emotion Recognition

The Awareness of Social Inference Test (TASIT) is an audiovisual tool designed for the clinical assessment of social perception with alternate forms for retesting. The TASIT datasets are very useful for our emotion recognition performance testing.

Module – I – Assess emotion recognition.

Module- II – Assess the ability to interpret conversational remarks meant literally (i.e. sincere remarks and lies) or non-literally (i.e. sarcasm) as well as the ability to make judgments about the thoughts, intentions and feelings of speakers.

The performance study is associated with sentiment analysis,  face perception, information processing speed and other factors.

Theory of mind and self-awareness

Theory of mind and self-awareness is considered pivotal component to our project for understanding social behavior in general, and appear to play a significant role in understanding non-literal language including metaphor and sarcasm.


Ray Deep Self-awareness algorithms assess a number of facets of social perception that have been identified as important to artificial intelligence based social emotion recognition competence. However, there is a need to establish more robust validity testing constructs that are likely to assess emotion recognition, social perception judgments, and other measures of human-machine interactions and social perceptions.