The latest Snapchat lens that gives you fox ears may be nothing more than a passing distraction to send to your friends, but that lens, colloquially known as a “filter,” utilizes surprisingly advanced face tracking technology. Looksery, the Ukrainian startup that developed Snapchat’s filter technology (which Snapchat acquired for $150 million), created software that can not only recognize a human face, but accurately map it to ensure that those fox ears are on top of your head and not floating in space.
These Snapchat filters use computer vision, the complex process by which a program can detect objects in the real world. If a computer sees an image as a running list of pixels, how can it identify a face when it sees one? At its most basic, the computer vision that allows face-tracking is based on the Viola-Jones Algorithm.This algorithm uses the contrast between light and dark on faces to identify a face when it sees one by looking for patterns, such as light on the bridge of the nose and shadow in the eye sockets.
However, this algorithm only allows a computer to recognize a face, not track its movements. To take face-tracking technology one step further, developers have to introduce machine learning to the Viola-Jones Algorithm and teach a program not only how a face can move, but also how various faces can differ. After all, you can’t track the nuances of a face’s movement without being able to map a face with accuracy.
To create that accurate face map, Looksery manually mapped out thousands of faces for its program and created an average face, with dots representing important points on the human face, such as the corners of the eyes and lips, the outline of the jaw, etc. They then trained the program to look for the contrast patterns that those dots correlate too, allowing this facial map to shift and match that user’s face, and track their expression as the user rotates and angles their face to the camera. This is the technology behind Snapchat’s filters, impressive on mobile hardware, and it points to the most promising aspect of face-tracking technology: its implications for socializing in the digital sphere.
Snapchat filters are a fun way to amuse your friends, but what happens when you take this face tracking technology and, for example, apply it to a virtual reality device? That is exactly what Veeso is working on. Veeso is building a VR headset that tracks the wearer’s eye and mouth movement. Compatible with both Google Cardboard and Daydream, Veeso is working on partnerships with social platforms in the growing VR space in the hopes of recreating the success of social platforms like Facebook and Twitter on the latest technological wave. Considering that the VR market will be worth $30 billion by 2020, Veeso has the potential to capitalize on social VR in a big way, and the startup has already found a manufacturer and a distributor for its hardware.
Despite its forward thinking, Veeso’s understanding of the human face is still limited. Other companies however are pushing their understanding of nuances in human expression even further. Realeyes, an emotion measurement firm, can track 49 different points on the human face to track emotional responses. The company specializes in marketing and helping companies gain insight on how consumers react to their brand and marketing campaigns. Realeyes recently garnered attention in the media this past summer when the company measured emotional reactions to advertisements for the summer Olympics. While Veeso simply reproduces elements of facial expressions, Realeyes measures them and understand what each means. As of now, Realeyes can measure six emotions, including happiness, sadness, fear, surprise, disgust, and confusion, and the number of emotions the company understands will only grow as research continues.
Every year, face-tracking technology improves, and in 2016, the technology has already grown by leaps and bounds, from introducing the first face-tracking VR headsets all the way to providing a B2B marketing analysis service that can bring businesses a heightened understanding of their relationship with consumers. Where do you think face-tracking technology will go towards the end of 2016? Leave a comment below!
Latest posts by Ellie Martin
- Automation Apps that Will Launch Your Startup - March 13, 2018
- The Ultimate Guide to Starting a WordPress Blog - September 25, 2017
- 3 Things CTOs Should Know About Gig Economy This Year - January 30, 2017