You are reading Part 2 of a multi-part series in which I explain the inner workings of How Tape Index Uses Modal for Its Natural Language Processing Flow.

–

As mentioned before, Modal is a powerful serverless computing platform. These means that code (functions) can be called on demand and only run for as long as the function requires. In many cases, this method of computing can be significantly cheaper and more scalable than other solutions. Since Tape Index needs to take advantage of GPU cores for certain aspects of its NLP flow, Modal is an ideal and cost-effective solution for running GPU enabled servers on demand.

While Modal code is written in Python there’s a lot of flexibility around what can be run as a function. In the case of DigitalOcean Functions or AWS Lambdas, you have very little control over the underlying “container” running your code. This is great for keeping serverless functions small and fast, but it doesn’t offer as much flexibility for the type of code that could be run. There are also limits on RAM, storage, and execution time. Modal provides flexibility at the “container” level, giving you as much control over what is loaded on the server as a Docker container might give. You also have significant greater capacity when it comes to RAM, storage and execution.

WSGI Flask Web Endpoint

There are three ways to invoke a Modal function:

Call the function directly from a Python application (recommend and most secure)
Expose the function directly as a web endpoint
Expose an endpoint through an ASGI or WSGI application that then calls a function

I ended up choosing option #3 (for now) since Tape Index is running on Node. While I might eventually want to call a Python script from Node, a web endpoint gave me a little more flexibility while I was testing out the full capabilities of Modal.

The Basics

If you’re anything like me, you want to see all the code at the beginning to understand the deeper context. At the most basic level, creating a simple Flask endpoint on Modal can be achieved with the following bit of code. It’s maybe so simple that it doesn’t need further explanation.

You’ll notice that Image is the “container” that the Flask app is running inside of and is using Debian Slim as the base with Flask being install with Pip.
The flask_app() function is a special wrapper function that Modal uses to encapsulate the Flask app. This allows the Flask app to be ephemeral and only spun up when someone calls the Modal specific endpoint.
The rest of the code is standard Flask app code.

What you’ll notice missing is everything specific to Tape Index. So let’s move on.

The Specifics

To make this a useable web endpoint, I need to add code that will handle the POST data, request authentication and spawning the internal Modal functions.

Import Modal Functions

As discussed in Part 1, the sole purpose of this web endpoint is to call two separate functions that downloads the file to be processed and to extract out the file’s metainfo. These functions are separate Modal functions, which will be discussed in later posts. As you can see here I’m importing those functions into my Flask app endpoint.

Load Environment Secrets

Secrets can be specified within the Modal dashboard interface. Those secrets can be called and used within any Modal function. These are exposed through the os.environ.get() function.

Authenticate and Get POST Data

Since the route that was called is a Flask app, we can get the POST form data the typical way. Since these endpoints are public to the world, authentication is important. For the purposes of this post, I am purposely showing a very simple implementation of authentication. The calling application needs to pass an authentication token. These tokens need to match. If there isn’t a match, the system won’t do anything. Otherwise, if everything matches it will get the rest of the necessary POST data.

Spawn the Functions

The last and final step is to spawn the Modal functions. When spawned, the Modal functions run asynchronously and the web endpoint continues with a successful response. These modal functions both get callbacks. The callbacks will eventually be used to send relevant information back to Tape Index to be stored in the database. The whole web endpoint call takes a couple seconds max.

Final Code

Putting all of the above pieces together, the final code is relatively simple and straightforward. If you have questions or comments, toss them in the comment section below, or email me at hello@teev.io.