Giant Emoji

The app is running several algorithms to detect facial landmarks, and run sentimental analysis on these. The openFrameworks app delivers these landmark points and sentimental analysis results via JavaScript injection into a local WebView and/or over WebSockets to a remote browser, ready for HTML5 canvas animation. Within a local WebView or remote browser a Paper.js app creates the persistent emoji vector objects for animation upon initialization. A lightweight JavaScript animation loop parses the incoming JSON facial landmarks and models the corresponding X/Y coordinates, viewable in debug mode. The landmarks and confidence scores are then translated into preset emoji models (such as kissy face, cheesy smile, etc.), augmented by live expressions like mouth shape or lip geometry. Giant emoji was created specifically for and debuted at Google I/O 2016. Being a multithreaded, processor intensive application we engineered it for the Google Pixel C and can guarantee performance only on that device. This application was developed in two parts: the Android openFrameworks app that handles the face detection (written in C++), and the Web app to render the emoji (written primarily in JavaScript).