arsd.cgi

Provides a uniform server-side API for CGI, FastCGI, SCGI, and HTTP web applications. Offers both lower- and higher- level api options among other common (optional) things like websocket and event source serving support, session management, and job scheduling.

import arsd.cgi;

// Instead of writing your own main(), you should write a function
// that takes a Cgi param, and use mixin GenericMain
// for maximum compatibility with different web servers.
void hello(Cgi cgi) {
	cgi.setResponseContentType("text/plain");

	if("name" in cgi.get)
		cgi.write("Hello, " ~ cgi.get["name"]);
	else
		cgi.write("Hello, world!");
}

mixin GenericMain!hello;

Or:

import arsd.cgi;

class MyApi : WebObject {
	@UrlName("")
	string hello(string name = null) {
		if(name is null)
			return "Hello, world!";
		else
			return "Hello, " ~ name;
	}
}
mixin DispatcherMain!(
	"/".serveApi!MyApi
);

Test on console (works in any interface mode): $ ./cgi_hello GET / name=whatever

If using http $ ./cgi_hello --port 8080 # now you can go to http://localhost:8080/?name=whatever

Please note: the default port for http is 8085 and for scgi is 4000. I recommend you set your own by the command line argument in a startup script instead of relying on any hard coded defaults. It is possible though to code your own with [RequestServer], however.

Build_Configurations:

cgi.d tries to be flexible to meet your needs. It is possible to configure it both at runtime (by writing your own main function and constructing a [RequestServer] object) or at compile time using the version switch to the compiler or a dub subConfiguration.

If you are using dub, use:

subConfiguration "arsd-official:cgi" "VALUE_HERE"

or to dub.json:

"subConfigurations": {"arsd-official:cgi": "VALUE_HERE"}

to change versions. The possible options for VALUE_HERE are:

• embedded_httpd for the embedded httpd version (built-in web server). This is the default for dub builds. You can run the program then connect directly to it from your browser.

  Note

  prior to version 11, this would be embedded_httpd_processes on Linux and embedded_httpd_threads everywhere else. It now means embedded_httpd_hybrid everywhere supported and embedded_httpd_threads everywhere else.
• cgi for traditional cgi binaries. These are run by an outside web server as-needed to handle requests.
• fastcgi for FastCGI builds. FastCGI is managed from an outside helper, there's one built into Microsoft IIS, Apache httpd, and Lighttpd, and a generic program you can use with nginx called spawn-fcgi. If you don't already know how to use it, I suggest you use one of the other modes.
• scgi for SCGI builds. SCGI is a simplified form of FastCGI, where you run the server as an application service which is proxied by your outside webserver. Please note: on nginx make sure you add scgi_param PATH_INFO $document_uri; to the config!
• stdio_http for speaking raw http over stdin and stdout. This is made for systemd services. See RequestServer.serveSingleHttpConnectionOnStdio for more information.

With dmd, use:

• Interfaces
• (mutually exclusive)

• - -version=plain_cgi

• The default building the module alone without dub - a traditional, plain CGI executable will be generated.

• - -version=embedded_httpd

• A HTTP server will be embedded in the generated executable. This is default when building with dub.

• - -version=fastcgi

• A FastCGI executable will be generated.

• - -version=scgi

• A SCGI (SimpleCGI) executable will be generated.

• - -version=embedded_httpd_hybrid

• A HTTP server that uses a combination of processes, threads, and fibers to better handle large numbers of idle connections. Recommended if you are going to serve websockets in a non-local application.

• - -version=embedded_httpd_threads

• The embedded HTTP server will use a single process with a thread pool. (use instead of plain embedded_httpd if you want this specific implementation)

• - -version=embedded_httpd_processes

• The embedded HTTP server will use a prefork style process pool. (use instead of plain embedded_httpd if you want this specific implementation)

• - -version=embedded_httpd_processes_accept_after_fork

• It will call accept() in each child process, after forking. This is currently the only option, though I am experimenting with other ideas. You probably should NOT specify this right now.

• - -version=stdio_http

• The embedded HTTP server will be spoken over stdin and stdout.

• + Tweaks

• (can be used together with others)

• - -version=cgi_with_websocket

• The CGI class has websocket server support. (This is on by default now.)

• - -version=with_openssl

• not currently used

• - -version=cgi_embedded_sessions

• The session server will be embedded in the cgi.d server process

• - -version=cgi_session_server_process

• The session will be provided in a separate process, provided by cgi.d.

For example,

For CGI, dmd yourfile.d cgi.d then put the executable in your cgi-bin directory. You should check your server's configuration for security as described here https://httpoxy.org/ if you also do client http requests.

For FastCGI: dmd yourfile.d cgi.d -version=fastcgi and run it. spawn-fcgi helps on nginx. You can put the file in the directory for Apache. On IIS, run it with a port on the command line (this causes it to call FCGX_OpenSocket, which can work on nginx too).

For SCGI: dmd yourfile.d cgi.d -version=scgi and run the executable, providing a port number on the command line.

For an embedded HTTP server, run dmd yourfile.d cgi.d -version=embedded_httpd and run the generated program. It listens on port 8085 by default. You can change this on the command line with the --port option when running your program.

Command_line_interface:

If using [GenericMain] or [DispatcherMain], an application using arsd.cgi will offer a command line interface out of the box.

See RequestServer.listenSpec for more information.

Simulating_requests:

If you are using one of the [GenericMain] or [DispatcherMain] mixins, or main with your own call to RequestServer.trySimulatedRequest, you can simulate requests from your command-ine shell. Call the program like this:

./yourprogram GET / name=adr

And it will print the result to stdout instead of running a server, regardless of build more..

CGI_Setup_tips:

On Apache, you may do SetHandler cgi-script in your .htaccess file to set a particular file to be run through the cgi program. Note that all "subdirectories" of it also run the program; if you configure /foo to be a cgi script, then going to /foo/bar will call your cgi handler function with cgi.pathInfo == "/bar".

Overview_Of_Basic_Concepts:

cgi.d offers both lower-level handler apis as well as higher-level auto-dispatcher apis. For a lower-level handler function, you'll probably want to review the following functions:

Input: Cgi.get, Cgi.post, Cgi.request, Cgi.files, Cgi.cookies, Cgi.pathInfo, Cgi.requestMethod, and HTTP headers (Cgi.headers, Cgi.userAgent, Cgi.referrer, Cgi.accept, Cgi.authorization, Cgi.lastEventId)

Output: Cgi.write, Cgi.header, Cgi.setResponseStatus, Cgi.setResponseContentType, Cgi.gzipResponse

Cookies: Cgi.setCookie, Cgi.clearCookie, Cgi.cookie, Cgi.cookies

Caching: Cgi.setResponseExpires, Cgi.updateResponseExpires, Cgi.setCache

Redirections: Cgi.setResponseLocation

Other Information: Cgi.remoteAddress, Cgi.https, Cgi.port, Cgi.scriptName, Cgi.requestUri, Cgi.getCurrentCompleteUri, Cgi.onRequestBodyDataReceived

Websockets: [Websocket], [websocketRequested], [acceptWebsocket]. For websockets, use the embedded_httpd_hybrid build mode for best results, because it is optimized for handling large numbers of idle connections compared to the other build modes.

Overriding behavior for special cases streaming input data: see the virtual functions Cgi.handleIncomingDataChunk, Cgi.prepareForIncomingDataChunks, Cgi.cleanUpPostDataState

A basic program using the lower-level api might look like:

import arsd.cgi;

// you write a request handler which always takes a Cgi object
void handler(Cgi cgi) {
	/+
		when the user goes to your site, suppose you are being hosted at http://example.com/yourapp

		If the user goes to http://example.com/yourapp/test?name=value
		then the url will be parsed out into the following pieces:

			cgi.pathInfo == "/test". This is everything after yourapp's name. (If you are doing an embedded http server, your app's name is blank, so pathInfo will be the whole path of the url.)

			cgi.scriptName == "yourapp". With an embedded http server, this will be blank.

			cgi.host == "example.com"

			cgi.https == false

			cgi.queryString == "name=value" (there's also cgi.search, which will be "?name=value", including the ?)

			The query string is further parsed into the `get` and `getArray` members, so:

			cgi.get == ["name": "value"], meaning you can do `cgi.get["name"] == "value"`

			And

			cgi.getArray == ["name": ["value"]].

			Why is there both `get` and `getArray`? The standard allows names to be repeated. This can be very useful,
			it is how http forms naturally pass multiple items like a set of checkboxes. So `getArray` is the complete data
			if you need it. But since so often you only care about one value, the `get` member provides more convenient access.

		We can use these members to process the request and build link urls. Other info from the request are in other members, we'll look at them later.
	+/
	switch(cgi.pathInfo) {
		// the home page will be a small html form that can set a cookie.
		case "/":
			cgi.write(`<!DOCTYPE html>
			<html>
			<body>
				<form method="POST" action="set-cookie">
					<label>Your name: <input type="text" name="name" /></label>
					<input type="submit" value="Submit" />
				</form>
			</body>
			</html>
			`, true); // the , true tells it that this is the one, complete response i want to send, allowing some optimizations.
		break;
		// POSTing to this will set a cookie with our submitted name
		case "/set-cookie":
			// HTTP has a number of request methods (also called "verbs") to tell
			// what you should do with the given resource.
			// The most common are GET and POST, the ones used in html forms.
			// You can check which one was used with the `cgi.requestMethod` property.
			if(cgi.requestMethod == Cgi.RequestMethod.POST) {

				// headers like redirections need to be set before we call `write`
				cgi.setResponseLocation("read-cookie");

				// just like how url params go into cgi.get/getArray, form data submitted in a POST
				// body go to cgi.post/postArray. Please note that a POST request can also have get
				// params in addition to post params.
				//
				// There's also a convenience function `cgi.request("name")` which checks post first,
				// then get if it isn't found there, and then returns a default value if it is in neither.
				if("name" in cgi.post) {
					// we can set cookies with a method too
					// again, cookies need to be set before calling `cgi.write`, since they
					// are a kind of header.
					cgi.setCookie("name" , cgi.post["name"]);
				}

				// the user will probably never see this, since the response location
				// is an automatic redirect, but it is still best to say something anyway
				cgi.write("Redirecting you to see the cookie...", true);
			} else {
				// you can write out response codes and headers
				// as well as response bodies
				//
				// But always check the cgi docs before using the generic
				// `header` method - if there is a specific method for your
				// header, use it before resorting to the generic one to avoid
				// a header value from being sent twice.
				cgi.setResponseLocation("405 Method Not Allowed");
				// there is no special accept member, so you can use the generic header function
				cgi.header("Accept: POST");
				// but content type does have a method, so prefer to use it:
				cgi.setResponseContentType("text/plain");

				// all the headers are buffered, and will be sent upon the first body
				// write. you can actually modify some of them before sending if need be.
				cgi.write("You must use the POST http verb on this resource.", true);
			}
		break;
		// and GETting this will read the cookie back out
		case "/read-cookie":
			// I did NOT pass `,true` here because this is writing a partial response.
			// It is possible to stream data to the user in chunks by writing partial
			// responses the calling `cgi.flush();` to send the partial response immediately.
			// normally, you'd only send partial chunks if you have to - it is better to build
			// a response as a whole and send it as a whole whenever possible - but here I want
			// to demo that you can.
			cgi.write("Hello, ");
			if("name" in cgi.cookies) {
				import arsd.dom; // dom.d provides a lot of helpers for html
				// since the cookie is set, we need to write it out properly to
				// avoid cross-site scripting attacks.
				//
				// Getting this stuff right automatically is a benefit of using the higher
				// level apis, but this demo is to show the fundamental building blocks, so
				// we're responsible to take care of it.
				cgi.write(htmlEntitiesEncode(cgi.cookies["name"]));
			} else {
				cgi.write("friend");
			}

			// note that I never called cgi.setResponseContentType, since the default is text/html.
			// it doesn't hurt to do it explicitly though, just remember to do it before any cgi.write
			// calls.
		break;
		default:
			// no path matched
			cgi.setResponseStatus("404 Not Found");
			cgi.write("Resource not found.", true);
	}
}

// and this adds the boilerplate to set up a server according to the
// compile version configuration and call your handler as requests come in
mixin GenericMain!handler; // the `handler` here is the name of your function

Even if you plan to always use the higher-level apis, I still recommend you at least familiarize yourself with the lower level functions, since they provide the lightest weight, most flexible options to get down to business if you ever need them.

In the lower-level api, the [Cgi] object represents your HTTP transaction. It has functions to describe the request and for you to send your response. It leaves the details of how you o it up to you. The general guideline though is to avoid depending any variables outside your handler function, since there's no guarantee they will survive to another handler. You can use global vars as a lazy initialized cache, but you should always be ready in case it is empty. (One exception: if you use -version=embedded_httpd_threads -version=cgi_no_fork, then you can rely on it more, but you should still really write things assuming your function won't have anything survive beyond its return for max scalability and compatibility.)

A basic program using the higher-level apis might look like:

/+
import arsd.cgi;

struct LoginData {
	string currentUser;
}

class AppClass : WebObject {
	string foo() {}
}

mixin DispatcherMain!(
	"/assets/.serveStaticFileDirectory("assets/", true), // serve the files in the assets subdirectory
	"/".serveApi!AppClass,
	"/thing/".serveRestObject,
);
+/

Guide_for_PHP_users: (Please note: I wrote this section in 2008. A lot of PHP hosts still ran 4.x back then, so it was common to avoid using classes - introduced in php 5 - to maintain compatibility! If you're coming from php more recently, this may not be relevant anymore, but still might help you.)

If you are coming from old-style PHP, here's a quick guide to help you get started:

$(COLUMN

<?php
	$foo = $_POST["foo"];
	$bar = $_GET["bar"];
	$baz = $_COOKIE["baz"];

	$user_ip = $_SERVER["REMOTE_ADDR"];
	$host = $_SERVER["HTTP_HOST"];
	$path = $_SERVER["PATH_INFO"];

	setcookie("baz", "some value");

	echo "hello!";
?>

import arsd.cgi;
void app(Cgi cgi) {
	string foo = cgi.post["foo"];
	string bar = cgi.get["bar"];
	string baz = cgi.cookies["baz"];

	string user_ip = cgi.remoteAddress;
	string host = cgi.host;
	string path = cgi.pathInfo;

	cgi.setCookie("baz", "some value");

	cgi.write("hello!");
}

mixin GenericMain!app

)

Array elements

In PHP, you can give a form element a name like "something[]", and then $_POST["something"] gives an array. In D, you can use whatever name you want, and access an array of values with the cgi.getArray["name"] and cgi.postArray["name"] members.

Databases

PHP has a lot of stuff in its standard library. cgi.d doesn't include most of these, but the rest of my arsd repository has much of it. For example, to access a MySQL database, download database.d and mysql.d from my github repo, and try this code (assuming, of course, your database is set up):

import arsd.cgi;
import arsd.mysql;

void app(Cgi cgi) {
	auto database = new MySql("localhost", "username", "password", "database_name");
	foreach(row; mysql.query("SELECT count(id) FROM people"))
		cgi.write(row[0] ~ " people in database");
}

mixin GenericMain!app;

Similar modules are available for PostgreSQL, Microsoft SQL Server, and SQLite databases, implementing the same basic interface.