Web designers are always looking for new ways to improve the presentation of a page’s content. Sometimes, this can lead to ingenious solutions or to interact with technologies that are often kept away from the design field. In this article we will bring typography in contact with Artificial Intelligence, using machine learning to detect such things as the proximity of the user’s face in order to improve the legibility of the text.
We will experiment on how to use face recognition with Tensorflow in order to extract some information from the camera, such as the distance between the screen and user’s face or the amount of people reading the page. Then, we will pass those data to CSS in order to adapt typography and to adjust the page layout.
What Is Tensorflow?
Tensorflow is an open-source platform from Google for Machine Learning. Machine Learning is a field of Computer Science that studies algorithms that learn to recognize complex relations and recurring patterns out of images, audio tracks, time series, natural text, and data in general. These algorithms generate mathematical models (also called trained models), which are a sort of schema that can be used to make decisions based on input data. If you’d like to approach the topic, Charlie Gerard wrote about ML for frontend developers here on Smashing Mag.
Tensorflow provides a lot of tools for AI developers, data scientists, mathematicians, but don’t panic if data analysis is not your daily bread! The good news is that you have not to be an expert to use it, as long as you are using pre-built models, just as we are going to.
Tensorflow models are available to be used on the Web with their JavaScript SDK.
Setup
In order to start using face recognition algorithms, we need to follow a few steps:
- load the Tensorflow SDK.
- load the Facemesh library which contains the mathematical model.
- access the user’s camera and stream it to an HTML video element. Facemesh will analyze frames from the video tag to detect the presence of faces.
In this projects we are going to use Tensorflow via CDN, but it is also available on NPM if you prefer the bundler-way:
Tensorflow does not do the trick itself, so we need to add Facemesh, a library that is built on the top of the ML framework and provides an already trained model for face recognition: