Guide To FastAPI With Machine Learning Deployment

Any machine learning model’s end goal is a deployment for production purposes. Building a REST API(Application Programming Interface) is the best possible way to evaluate model performance. In python, Django and more evidently Flask frameworks are used for this purpose. For machine learning, Flask is preferred more than Django. Here comes FastAPI which is faster than Flask, providing higher performance boost, easier to code, comes with automatic documentation, provides data validation on input data provided to the server, and many more such amazing features which I’ll be covering in this article.

FastAPI is one of the fastest python frameworks available. The automatic documentation comes with free of cost. Since it is based on python so it provides nice python type hints for autocompletion and type checks. FastAPI is built upon two major python libraries – Starlette(for web handling) and Pydantic(for data handling). FastAPI is based on some standard integrations — OpenAPI, JSON Schema, OAuth2.

In this article, we will discuss the implementation of FastAPI and demonstrate how it can be used for deployment purposes.

Getting Started With FastAPI

Software Dependencies that are required for FastAPI:-

• Python 3.6+
• FastAPI
• ASGI server for production such as Uvicorn or Hypercorn.
• Pydantic

Basic Hello World 

First of all, we will do a very basic implementation to get familiar with FastAPI.

from fastapi import FastAPI
app = FastAPI()
@app.get("/")
def index():
    return {"message": "Hello World"}

Save this program in main.py then from a terminal, execute:

$uvicorn main:app --reload

The following message should be printed:

INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)
INFO:     Started reloader process [28720]
INFO:     Started server process [28722]
INFO:     Waiting for application startup.
INFO:     Application startup complete.

After redirecting to the webpage, it should appear as:

{ “message”  :  “Hello World” }

Interactive Documentation

http://127.0.0.1.8000/docs will show the interactive documentation by Swagger UI.

Alternative documentation by redoc

FastAPI for ML models

Let’s take a pre-trained NLP model to extract the entities present in an English sentence. For this I’ll be using the Spacy library’s model ‘en_core_web_sm’  which is a pre-trained statistical model for English and can be installed with the command:

python -m spacy download en_core_web_sm

Then use it in the following code:

from typing import List
import spacy
from pydantic import BaseModel
from fastapi import FastAPI
app = FastAPI()
nlp = spacy.load("en_core_web_sm")
@app.get("/")
def read_root():
    return {"Hello": "World"}
class Item(BaseModel):
    content: str
    comments: List[str] = []
@app.post("/item/")
def post_item(item: Item):
    doc = nlp(item.content)
    ents = []
    for ent in doc.ents:
        ents.append({"text":ent.text,"label":ent.label_})
    return {"message":item.content,"comments":item.comments, "ents":ents}

This code takes an English statement as input(‘item’) in realtime and extracts entities using the spacy model to show which part of the sentence falls under which class. Here class Item inherits the BaseModel from pydantic for its properties to be used.

Execute the code similarly as done above which will redirect to http://127.0.0.1.8000/docs and the following page will appear.

First, click onto ‘post’ as a post request is being made then click ‘Try it out’ to make changes to content and comments.

{
  "message": "Apple buys U.K. based startup for $1 billion ",
  "comments": [
    "Nice as expected"
  ],
  "ents": [
    {
      "text": "Apple",
      "label": "ORG"
    },
    {
      "text": "U.K.",
      "label": "GPE"
    },
    {
      "text": "$1 billion",
      "label": "MONEY"
    }
  ]
}