How is Facial Expression Recognition done?

Facial Expression Recognition is in charge of making the prediction of facial expressions and social cues. As soon as the prediction is made, the output is sent back securely to the frontend. Recognition algorithms are based on our unique assets of emotion-based technology.

Why is recognition done in the cloud?

We constantly improve our facial expressions recognition models. By offering a cloud-based service, we propose to our customers the most up-to-date version of our technology.

The tradeoff is that it requires sending data to the cloud which comes with matters of privacy, security and performance.

Which emotions are available?

Modern science of emotion gave evidence that 16 emotions are commonly expressed by people around the world, whatever their age and gender - Source: Cowen et al 2020.

We currently extract 3 of them (more are to come…) that are expressed by facial expressions: Surprise, Amusement and Confusion. Check out how business cases make good use of these emotions, and how Leading Companies are giving high value for their own customers by leveraging these emotions into their business value proposition.

Surprise

Brief emotional state experienced as the result of an unexpected significant event.

Amusement

State of experiencing humorous and entertaining events or situations.

Confusion

Brief state of being bewildered or unclear.

Which social cues are available?

Building the framework upon thousands of successful interactions in the real world, through social robots, interactive avatars and smartphone apps, we have proprietary metrics used to evaluate the engagement of the user. Our implementation comes from practical usage of the Snider et al 2005 studies and our patented methods for evaluating the interaction metrics within the course of an interaction providing : Attention, Engagement and Interaction Status.

Attention

Visual attention is the way you allocate your attention by gazing at the object of interest. You are visually attentive to an event if you are currently looking at it.

Engagement

Engagement is the process by which participants establish, maintain and terminate their connection with the device. The notion of engagement is composed of 4 phases : “engaging”, “engaged”, “disengaging”, “idle”.

Interaction Status

Estimation of the state of a user within the course of an interaction. The state is derived from the transitions of the engagement. Interaction status can be 5 differents stages :

Interaction Status	Description
idle	The user is visible, not engaged and attention low.
possible	The user is changing to an engaging status, this is a transitional status.
ready	The user is engaged, and attention is high.
in_short_interruption	The user used to be engaged, changing to disengaging for less than 3 seconds.
in_long_interruption	The user used to be in short interruption status, for a period between 3 to 8 seconds.