Scalable Facial Recognition Solution with OutSystems and AWS

Written by Patricia | Sep 5, 2022 12:47:56 AM

There is no doubt that we are constantly going through technological evolution, living in a world where cars drive themselves and surgeries are performed by robots. Continuous advancements in AI are continually opening new doors for creativity and innovation in mobile apps, elevating user experience, security and application design. With these capabilities becoming more accessible than ever, the limits of what we can do in the world of software development are slowly diminishing. Bundle that with the evolving powers of Low-Code, the possibilities become endless.

Facial Recognition (FR) is one branch of AI which is gaining popularity and is now being used as an authentication mechanism for mobile applications. In this article, I will break down the key components needed to deliver a scalable FR solution, walking you through the purpose and functionality of each. I will not be diving into low-level technical details but the required AWS & OutSystems architectures needed will be highlighted and explained.

Note that this article follows the same structure as my webinar at the OutSystems Cloud Innovation Summit 2022, which you can checkout via the link at the bottom of this page.

Ok, let’s talk FR.

Why implement Facial Recognition?

Before jumping into the FR solution, it is important to consider the benefits and drawbacks of this implementation to determine if this is the correct user identification approach for your use case.

Starting off with the benefits, a FR solution is an extremely accurate method of identifying users, and thanks to the latest advancements, accuracy can easily exceed 99.9%. In addition, many FR services are built on top of machine learning networks, which means that they are continuously being trained and will keep getting more accurate over time. The second advantage of FR is its ease of integration because of how easily accessible and easy it’s to integrate into your apps.

There are a few drawbacks to consider as well. Most importantly, capturing and potentially storing images raises privacy concerns for users. Poorly managing or unnecessarily storing Personal Identifiable Information (PII) and biometric information can be risky, and consequently, additional security measures need to be put in place. Another disadvantage is that this technology can be expensive very quickly, depending on consumption rates. Constructing your scalable FR solution in the most cost-effective way possible without compromising performance and availability is important.

Assuming that a FR solution is the way to go, let’s start talking about how to do it – starting with the OutSystems piece.

The OutSystems Setup

First, let’s break down the user flow we’re targeting. It’s important to note that there isn’t only one correct way of integrating FR into your application; it depends on the use case at hand.

In this example, the FR capability is integrated as part of the login flow. The flow starts by presenting the user with the Login Screen, where they’re prompted to login. Once they successfully enter their credentials, the user would be redirected to a Camera Page where they’re asked to take a selfie. The Camera Page is essentially a view of the user’s front-facing camera with a face outline indicating where the user’s face needs to be to achieve an accurate face comparison result. Once the user aligns their face to the outline, the application automatically captures the image.

After capturing the image, the application will need to determine whether the user trying to log in has already signed up for FR or if we need to register them. Let’s start with the registration flow. The application determines that the user is using FR for the first time and stores the image in OutSystems to be used as the user’s profile picture. The image is also sent to AWS via an API and logs the registration details in Amazon RDS (more on this later). Finally, OutSystems captures specific data from the API response in an Audit Entity and seamlessly redirects the user to the application’s dashboard, commencing the session.

Now, let’s assume that the same user logs in again at a later date. After capturing their image on the Camera Page, the application determines that the user has previously signed up for FR (this could be achieved by looking up the profile image, using a specific flag in the data model or by referencing the Audit Entity) and sends the image to AWS to execute the FR capability, which compares the captured image with the stored reference image. The comparison rate is returned, and OutSystems captures that plus specific data from the API response in the Audit Entity. If the comparison was successful (in other words, the comparison rate returned is larger than the predetermined pass rate), the user is redirected to the dashboard. If the comparison was unsuccessful, the Camera Page is presented again, prompting the user to try again.

Solution Architecture

Next, let’s look at the proposed solution architecture. The number of OutSystems modules and applications needed will vary depending on the complexity of the application; however, at a minimum, your solution will need to have one application per layer in OutSystems 3 Layer Canvas, each with a particular function:

1. Mobile App (End User): Mobile application which exposes the FR functionality to users, and the application they would install on their devices.
2. FR Core Services (Core): Service application which consolidates all FR business logic, including the data model and comparison rate thresholds.
3. FR Connector (Library): Library application which consumes the AWS APIs.FR

Data Model

The last key component to talk about is the data model for the FR capability. Again, the complexity of the data model will directly correlate with the complexity of the application; however, there are a few Entities that are important to consider:

1. User Extension Entities: Typically, this would be 1 or more Entities that extend the OutSystems Users Entity, where the user’s profile image and FR-specific flags are stored.
2. Audit Entity: A simple Audit Entity to store and track AWS API responses for future analysis and visibility.

The AWS Setup

It’s time to talk about the AWS piece. There are 6 core AWS services that not only make this integration possible but also scalable:

Amazon Rekognition: This is where the FR magic happens! It provides highly accurate facial analysis, face comparison, and face search capabilities.
AWS Lambda: A compute service that lets you run code without provisioning or managing servers. This service can be viewed as the ‘glue’ which ties all the other services together.
Amazon S3: An object storage service that offers scalability, data availability, security, and performance. This is where the images used in the FR process are stored.
Amazon RDS: A collection of managed services that makes it simple to set up, operate, and scale databases in the cloud. Tables used for the FR process are defined here. These tables store information about all the historical FR runs and links to the images in Amazon S3 associated with these runs.
Amazon API Gateway: A service for creating, publishing, maintaining, monitoring, and securing REST, HTTP, and WebSocket APIs at any scale. This service exposes the APIs consumed in OutSystems, tying the two pieces together.
Amazon Cloudwatch: A monitoring service that provides data and actionable insights for AWS, hybrid, and on-premises applications and infrastructure resources. This service gives us the analytical visibility needed to monitor our AWS solution.

The AWS piece kicks into action when OutSystems invokes the API normalised in the FR Connector module we previously discussed. The API itself is defined in Amazon API Gateway, where we have control over different API versions, authentication policies and the tools needed to ensure the scalability of the API. Once successfully invoked and authorised, the request is forwarded to a Lambda function which orchestrates the entire process.

Firstly, the function checks if the user exists in Amazon RDS, where we have a table that holds all previous FR records and the users they belong to. These records also contain the location of the user’s reference image in S3, which is also retrieved as part of this process. Secondly, the image captured in OutSystems is sent and stored in Amazon S3 temporarily (more on this later). With the reference image and captured image now in hand, the Lambda function forwards the images to Amazon Rekognition, which executes the comparison and returns the comparison rate.

Before we return the comparison rate to OutSystems, the FR comparison result is stored in the Amazon RDS table previously mentioned for future reference. Only then is the comparison rate sent back to OutSystems for our application to handle.

Periodic Image Purge

One of the biggest concerns related to FR solutions is privacy and storage of PII information. These are valid concerns, and to minimise them, we should only be storing images captured for FR when necessary and delete them when they’re no longer needed.

To ensure that’s the case, an additional function in AWS Lambda can be built to purge historical FR images. This includes defining a PASS comparison rate threshold (e.g. 90%), a PASS and a FAIL image cache periods (e.g. 3 days and 28 days, respectively). The reason why you might set the FAIL image cache period larger than the PASS image cache period is to allow for more time to potentially analyse and deal with failed scenarios. If that’s not a requirement in your app, then both the PASS and FAIL image cache periods can be the same.

The Lambda function then runs on a daily basis, iterates through the records in the Amazon RDS table and runs the following rules:

If the comparison rate is above the defined PASS threshold and the PASS image cache period has passed, then access the corresponding image in S3 and delete it.
If the comparison rate is below the defined PASS threshold and the FAIL image cache period has passed, then access the corresponding image in S3 and delete it.

And that’s how you do it! Adopting a scalable FR solution in your mobile app has never been easier.

About the author

Ossama Ghanem OutSystems Tech Lead & Trainer @ PhoenixDX | OutSystems Ambassador

The AI & AWS talk never stops! From webinar to article: How to Deliver a Scalable Facial Recognition Solution in OutSystems and AWS, as presented in OutSystems’ Cloud Innovation Summit. You can watch Ossama’s talk here.

This article was originally published on LinkedIn.

View full post