Automated Analysis of Emotional Expressions in Dogs based on AI-Enhanced Geometric Morphometrics

Automated analysis of facial expressions is a vibrant field in human affective computing, while research in nonhuman animals is still in its early stages. Compared to labor-intensive manual coding, automation can provide a more reliable and objective alternative, eliminating subjectivity and bias. However, using automated approaches of facial analysis in nonhuman animals ''in the wild”, i.e. outside of controlled laboratory conditions, is a challenge given the nature of noisy datasets. Here we present the first study using a fully automated analysis of facial landmarks associated with different emotional states in a morphologically diverse sample of pet dogs. We applied a novel AI-pipeline to study fear expressions of dogs in their home environment, analysing owner-provided video recordings during a real-life firework situation on New Year’s Eve in comparison to a control evening without fireworks. Using a static geometric morphometrics-inspired analysis, the pipeline allows for quantifying dog facial expressions in an extremely noisy and diverse ''in the wild” dataset, encompassing various breeds, angles and environments. We used an automated facial landmark system of 46 dog facial landmarks based on the Dog Facial Action Coding System. Due to the great variety in morphology of the included dogs, landmarks denoting the ear pinnae were excluded. Nonetheless, landmarks relating to the base of the ears differentiated most strongly between the conditions, suggesting backwards-drawn ears as the best indicator of the firework condition, which is in agreement with manually coded data. Additionally, the firework condition was associated with more mouth-opening, possibly reflecting panting in a subset of dogs. We conclude that automated analysis of dog facial expressions, based on the previously validated landmark system, is feasible in a diverse sample of pet dogs, paving the way towards automated emotion detection.