Project Nayuki

FastCGI library

Introduction

FastCGI is a protocol that enables communication between a server-side web application and a web server or reverse proxy server. Compared to the ancient Common Gate Interface (CGI) standard, FastCGI is more efficient because the former launches an application process per HTTP request whereas the latter uses a persistently running app that can serve arbitrarily many requests.

This mini-project is a rudimentary set of code where the main program listens on a Unix domain socket to incoming connections from a web server / reverse proxy, communicates using the FastCGI protocol, and handles requests in multiple threads by calling the user-supplied WSGI-compliant server application function. The code in this project is very incomplete in supporting all FastCGI features and handling uncommon error conditions, but is sufficient to talk to the nginx server and serve real web pages.

Components

Browse the project’s source code at GitHub: https://github.com/nayuki/FastCGI-library

Or download a ZIP of all the files: https://github.com/nayuki/FastCGI-library/archive/master.zip

FastCGI record class hierarchy

The fundamental unit of data transfer in the FastCGI protocol is a record, up to about 64 KiB long. FastCGI supports request multiplexing over a connection, so records are analogous to TCP packets. There are about a dozen record types, each with different fields, and fastcgi.py implements a hierarchy of classes with appropriate factories, constructors, and getter methods to represent each record type in the protocol specification document. It also includes a few utilities for performing I/O and encoding/decoding dictionaries.

This file is feature-complete in representing all FastCGI record types, but the code merely holds the data and does not interpret them to carry out the HTTP request/response life cycle. This file is useful only for those who want to handle FastCGI at a low level by parsing/serializing/handling raw records, but is not useful for those who want to build a FastCGI-based server application.

FastCGI record test suite

fastcgi-test.py ensures that record objects are converted to/from bytes reliably, and that field values stay within their specified ranges.

FastCGI-WSGI multi-threaded server

wsgi.py contains a WSGI server which accepts a WSGI application, a private thread pool executor implementation with periodic clean-up of idle threads, and a private request life cycle management class. The WsgiServer class is the one thing that most users need from this library.

Example

Here is a minimal application that uses this library to communicate with a web server / reverse proxy of your choice (which you need to set up separately):

from typing import Callable, Iterable
import fastcgi

def _app(environ: dict[str,object],
		startresp: Callable[[str,list[tuple[str,str]]],Callable[[bytes],None]]
		) -> Iterable[bytes]:
	b: bytes = b"<h1>Hello, world!</h1>"
	startresp("200 OK", [
		("Content-Type", "text/html; charset=UTF-8"),
		("Content-Length", str(len(b)))])
	return [b]

if __name__ == "__main__":
	fastcgi.wsgi.WsgiServer(_app, "/tmp/fastcgi-demo.sock", 0o111).run()

Notes

  • For over a decade, I used Allan Saddi’s flup library to satisfy my FastCGI server needs. His library supports FastCGI, SCGI, AJP, single-threading, multi-threading, and multi-processing. In late year , I spent some time to understand and clean up the FastCGI and multi-threading part of his library in my private copy. A few months later in early , I wrote a basic FastCGI implementation from scratch, with some lessons and inspiration taken from his work.

  • By changing a few lines of wsgi.py, this codebase can be adapted to be a drop-in replacement for the old flup library. This is useful for software that depends on flup, such as Bottle.py which lists flup as one of many server choices.

  • All the Python code in this project has complete and correct static type annotations. This practice is uncommon in the Python community in general, and sets this project apart from others.

  • The record classes strictly enforce their invariants at instance construction time, are all immutable, have private fields, and have public accessor methods. Every field that the protocol specification defines as an enumeration is translated into a proper subclass of enum.Enum instead of merely being a named integer constant.

  • This bare-bones library does not support various features such as: Handling multiple concurrent requests per socket connection (it seems that no web server uses it anyway), handling the standard error stream, record types such as FCGI_{GET_VALUES, GET_VALUES_RESULT, ABORT_REQUEST, DATA}.

More info