The Digital Cat - concurrent programming

Dissecting a Web stack

2020-02-16T15:00:00+00:00

It was gross. They wanted me to dissect a frog.

(Beetlejuice, 1988)

Introduction¶

Having recently worked with young web developers who were exposed for the first time to proper production infrastructure, I received many questions about the various components that one can find in the architecture of a "Web service". These questions clearly expressed the confusion (and sometimes the frustration) of developers who understand how to create endpoints in a high-level language such as Node.js or Python, but were never introduced to the complexity of what happens between the user's browser and their framework of choice. Most of the times they don't know why the framework itself is there in the first place.

The challenge is clear if we just list (in random order), some of the words we use when we discuss (Python) Web development: HTTP, cookies, web server, Websockets, FTP, multi-threaded, reverse proxy, Django, nginx, static files, POST, certificates, framework, Flask, SSL, GET, WSGI, session management, TLS, load balancing, Apache.

In this post, I want to review all the words mentioned above (and a couple more) trying to build a production-ready web service from the ground up. I hope this might help young developers to get the whole picture and to make sense of these "obscure" names that senior developers like me tend to drop in everyday conversations (sometimes arguably out of turn).

As the focus of the post is the global architecture and the reasons behind the presence of specific components, the example service I will use will be a basic HTML web page. The reference language will be Python but the overall discussion applies to any language or framework.

My approach will be that of first stating the rationale and then implementing a possible solution. After this, I will point out missing pieces or unresolved issues and move on with the next layer. At the end of the process, the reader should have a clear picture of why each component has been added to the system.

The perfect architecture¶

A very important underlying concept of system architectures is that there is no perfect solution devised by some wiser genius, that we just need to apply. Unfortunately, often people mistake design patterns for such a "magic solution". The "Design Patterns" original book, however, states that

Your design should be specific to the problem at hand but also general enough to address future problems and requirements. You also want to avoid redesign, or at least minimize it.

And later

Design patterns make it easier to reuse successful designs and architectures. [...] Design patterns help you choose design alternatives that make a system reusable and avoid alternatives that compromise reusability.

The authors of the book are discussing Object-oriented Programming, but these sentences can be applied to any architecture. As you can see, we have a "problem at hand" and "design alternatives", which means that the most important thing to understand is the requirements, both the present and future ones. Only with clear requirements in mind, one can effectively design a solution, possibly tapping into the great number of patterns that other designers already devised.

A very last remark. A web stack is a complex beast, made of several components and software packages developed by different programmers with different goals in mind. It is perfectly understandable, then, that such components have some degree of superposition. While the division line between theoretical layers is usually very clear, in practice the separation is often blurry. Expect this a lot, and you will never be lost in a web stack anymore.

Some definitions¶

Let's briefly review some of the most important concepts involved in a Web stack, the protocols.

TCP/IP¶

TCP/IP is a network protocol, that is, a set of established rules two computers have to follow to get connected over a physical network to exchange messages. TCP/IP is composed of two different protocols covering two different layers of the OSI stack, namely the Transport (TCP) and the Network (IP) ones. TCP/IP can be implemented on top of any physical interface (Data Link and Physical OSI layers), such as Ethernet and Wireless. Actors in a TCP/IP network are identified by a socket, which is a tuple made of an IP address and a port number.

As far as we are concerned when developing a Web service, however, we need to be aware that TCP/IP is a reliable protocol, which in telecommunications means that the protocol itself takes care or retransmissions when packets get lost. In other words, while the speed of the communication is not granted, we can be sure that once a message is sent it will reach its destination without errors.

HTTP¶

TCP/IP can guarantee that the raw bytes one computer sends will reach their destination, but this leaves completely untouched the problem of how to send meaningful information. In particular, in 1989 the problem Tim Barners-Lee wanted to solve was how to uniquely name hypertext resources in a network and how to access them.

HTTP is the protocol that was devised to solve such a problem and has since greatly evolved. With the help of other protocols such as WebSocket, HTTP invaded areas of communication for which it was originally considered unsuitable such as real-time communication or gaming.

At its core, HTTP is a protocol that states the format of a text request and the possible text responses. The initial version 0.9 published in 1991 defined the concept of URL and allowed only the GET operation that requested a specific resource. HTTP 1.0 and 1.1 added crucial features such as headers, more methods, and important performance optimisations. At the time of writing the adoption of HTTP/2 is around 45% of the websites in the world, and HTTP/3 is still a draft.

The most important feature of HTTP we need to keep in mind as developers is that it is a stateless protocol. This means that the protocol doesn't require the server to keep track of the state of the communication between requests, basically leaving session management to the developer of the service itself.

Session management is crucial nowadays because you usually want to have an authentication layer in front of a service, where a user provides credentials and accesses some private data. It is, however, useful in other contexts such as visual preferences or choices made by the user and re-used in later accesses to the same website. Typical solutions to the session management problem of HTTP involve the use of cookies or session tokens.

HTTPS¶

Security has become a very important word in recent years, and with a reason. The amount of sensitive data we exchange on the Internet or store on digital devices is increasing exponentially, but unfortunately so is the number of malicious attackers and the level of damage they can cause with their actions. The HTTP protocol is inherently

HTTP is inherently insecure, being a plain text communication between two servers that usually happens on a completely untrustable network such as the Internet. While security wasn't an issue when the protocol was initially conceived, it is nowadays a problem of paramount importance, as we exchange private information, often vital for people's security or for businesses. We need to be sure we are sending information to the correct server and that the data we send cannot be intercepted.

HTTPS solves both the problem of tampering and eavesdropping, encrypting HTTP with the Transport Layer Security (TLS) protocol, that also enforces the usage of digital certificates, issued by a trusted authority. At the time of writing, approximately 80% of websites loaded by Firefox use HTTPS by default. When a server receives an HTTPS connection and transforms it into an HTTP one it is usually said that it terminates TLS (or SSL, the old name of TLS).

WebSocket¶

One great disadvantage of HTTP is that communication is always initiated by the client and that the server can send data only when this is explicitly requested. Polling can be implemented to provide an initial solution, but it cannot guarantee the performances of proper full-duplex communication, where a channel is kept open between server and client and both can send data without being requested. Such a channel is provided by the WebSocket protocol.

WebSocket is a killer technology for applications like online gaming, real-time feeds like financial tickers or sports news, or multimedia communication like conferencing or remote education.

It is important to understand that WebSocket is not HTTP, and can exist without it. It is also true that this new protocol was designed to be used on top of an existing HTTP connection, so a WebSocket communication is often found in parts of a Web page, which was originally retrieved using HTTP in the first place.

Implementing a service over HTTP¶

Let's finally start discussing bits and bytes. The starting point for our journey is a service over HTTP, which means there is an HTTP request-response exchange. As an example, let us consider a GET request, the simplest of the HTTP methods.

GET / HTTP/1.1
Host: localhost
User-Agent: curl/7.65.3
Accept: */*

As you can see, the client is sending a pure text message to the server, with the format specified by the HTTP protocol. The first line contains the method name (GET), the URL (/) and the protocol we are using, including its version (HTTP/1.1). The remaining lines are called headers and contain metadata that can help the server to manage the request. The complete value of the Host header is in this case localhost:80, but as the standard port for HTTP services is 80, we don't need to specify it.

If the server localhost is serving HTTP (i.e. running some software that understands HTTP) on port 80 the response we might get is something similar to

HTTP/1.0 200 OK
Date: Mon, 10 Feb 2020 08:41:33 GMT
Content-type: text/html
Content-Length: 26889
Last-Modified: Mon, 10 Feb 2020 08:41:27 GMT

<!DOCTYPE HTML>
<html>
...
</html>

As happened for the request, the response is a text message, formatted according to the standard. The first line mentions the protocol and the status of the request (200 in this case, that means success), while the following lines contain metadata in various headers. Finally, after an empty line, the message contains the resource the client asked for, the source code of the base URL of the website in this case. Since this HTML page probably contains references to other resources like CSS, JS, images, and so on, the browser will send several other requests to gather all the data it needs to properly show the page to the user.

So, the first problem we have is that of implementing a server that understands this protocol and sends a proper response when it receives an HTTP request. We should try to load the requested resource and return either a success (HTTP 200) if we can find it, or a failure (HTTP 404) if we can't.

1 Sockets and parsers¶

1.1 Rationale¶

TCP/IP is a network protocol that works with sockets. A socket is a tuple of an IP address (unique in the network) and a port (unique for a specific IP address) that the computer uses to communicate with others. A socket is a file-like object in an operating system, that can be thus opened and closed, and that we can read from or write to. Socket programming is a pretty low-level approach to the network, but you need to be aware that every software in your computer that provides network access has ultimately to deal with sockets (most probably through some library, though).

Since we are building things from the ground up, let's implement a small Python program that opens a socket connection, receives an HTTP request, and sends an HTTP response. As port 80 is a "low port" (a number smaller than 1024), we usually don't have permissions to open sockets there, so I will use port 8080. This is not a problem for now, as HTTP can be served on any port.

1.2 Implementation¶

Create the file server.py and type this code. Yes, type it, don't just copy and paste, you will not learn anything otherwise.

import socket

## Create a socket instance
## AF_INET: use IP protocol version 4
## SOCK_STREAM: full-duplex byte stream
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

## Allow reuse of addresses
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)

## Bind the socket to any address, port 8080, and listen
s.bind(('', 8080))
s.listen()

## Serve forever
while True:
    # Accept the connection
    conn, addr = s.accept()

    # Receive data from this socket using a buffer of 1024 bytes
    data = conn.recv(1024)

    # Print out the data
    print(data.decode('utf-8'))

    # Close the connection
    conn.close()

This little program accepts a connection on port 8080 and prints the received data on the terminal. You can test it executing it and then running curl localhost:8080 in another terminal. You should see something like

$ python3 server.py 
GET / HTTP/1.1
Host: localhost:8080
User-Agent: curl/7.65.3
Accept: */*

The server keeps running the code in the while loop, so if you want to terminate it you have to do it with Ctrl+C. So far so good, but this is not an HTTP server yet, as it sends no response; you should actually receive an error message from curl that says curl: (52) Empty reply from server.

Sending back a standard response is very simple, we just need to call conn.sendall passing the raw bytes. A minimal HTTP response contains the protocol and the status, an empty line, and the actual content, for example

HTTP/1.1 200 OK

Hi there!

Our server becomes then

import socket

## Create a socket instance
## AF_INET: use IP protocol version 4
## SOCK_STREAM: full-duplex byte stream
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

## Allow reuse of addresses
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)

## Bind the socket to any address, port 8080, and listen
s.bind(('', 8080))
s.listen()

## Serve forever
while True:
    # Accept the connection
    conn, addr = s.accept()

    # Receive data from this socket using a buffer of 1024 bytes
    data = conn.recv(1024)

    # Print out the data
    print(data.decode('utf-8'))

    conn.sendall(bytes("HTTP/1.1 200 OK\n\nHi there!\n", 'utf-8'))

    # Close the connection
    conn.close()

At this point, we are not really responding to the user's request, however. Try different curl command lines like curl localhost:8080/index.html or curl localhost:8080/main.css and you will always receive the same response. We should try to find the resource the user is asking for and send that back in the response content.

This version of the HTTP server properly extracts the resource and tries to load it from the current directory, returning either a success of a failure

import socket
import re

## Create a socket instance
## AF_INET: use IP protocol version 4
## SOCK_STREAM: full-duplex byte stream
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

## Allow reuse of addresses
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)

## Bind the socket to any address, port 8080, and listen
s.bind(('', 8080))
s.listen()

HEAD_200 = "HTTP/1.1 200 OK\n\n"
HEAD_404 = "HTTP/1.1 404 Not Found\n\n"

## Serve forever
while True:
    # Accept the connection
    conn, addr = s.accept()

    # Receive data from this socket using a buffer of 1024 bytes
    data = conn.recv(1024)

    request = data.decode('utf-8')

    # Print out the data
    print(request)

    resource = re.match(r'GET /(.*) HTTP', request).group(1)
    try:
        with open(resource, 'r') as f:
            content = HEAD_200 + f.read()
        print('Resource {} correctly served'.format(resource))
    except FileNotFoundError:
        content = HEAD_404 + "Resource /{} cannot be found\n".format(resource)
        print('Resource {} cannot be loaded'.format(resource))

    print('--------------------')

    conn.sendall(bytes(content, 'utf-8'))

    # Close the connection
    conn.close()

As you can see this implementation is extremely simple. If you create a simple local file named index.html with this content

<head>
    <title>This is my page</title>
    <link rel="stylesheet" href="main.css">
</head>
<html>
    <p>Some random content</p>
</html>

and run curl localhost:8080/index.html you will see the content of the file. At this point, you can even use your browser to open http://localhost:8080/index.html and you will see the title of the page and the content. A Web browser is a software capable of sending HTTP requests and of interpreting the content of the responses if this is HTML (and many other file types like images or videos), so it can render the content of the message. The browser is also responsible of retrieving the missing resources needed for the rendering, so when you provide links to style sheets or JS scripts with the <link> or the <script> tags in the HTML code of a page, you are instructing the browser to send an HTTP GET request for those files as well.

The output of server.py when I access http://localhost:8080/index.html is

GET /index.html HTTP/1.1
Host: localhost:8080
User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:72.0) Gecko/20100101 Firefox/72.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Accept-Language: en-GB,en;q=0.5
Accept-Encoding: gzip, deflate
Connection: keep-alive
Upgrade-Insecure-Requests: 1
Pragma: no-cache
Cache-Control: no-cache


Resource index.html correctly served
--------------------
GET /main.css HTTP/1.1
Host: localhost:8080
User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:72.0) Gecko/20100101 Firefox/72.0
Accept: text/css,*/*;q=0.1
Accept-Language: en-GB,en;q=0.5
Accept-Encoding: gzip, deflate
Connection: keep-alive
Referer: http://localhost:8080/index.html
Pragma: no-cache
Cache-Control: no-cache


Resource main.css cannot be loaded
--------------------
GET /favicon.ico HTTP/1.1
Host: localhost:8080
User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:72.0) Gecko/20100101 Firefox/72.0
Accept: image/webp,*/*
Accept-Language: en-GB,en;q=0.5
Accept-Encoding: gzip, deflate
Connection: keep-alive
Pragma: no-cache
Cache-Control: no-cache


Resource favicon.ico cannot be loaded
--------------------

As you can see the browser sends rich HTTP requests, with a lot of headers, automatically requesting the CSS file mentioned in the HTML code and automatically trying to retrieve a favicon image.

1.3 Resources¶

These resources provide more detailed information on the topics discussed in this section

Python 3 Socket Programming HOWTO
HTTP/1.1 Request format
HTTP/1.1 Response format
The source code of this example is available here

1.4 Issues¶

It gives a certain dose of satisfaction to build something from scratch and discover that it works smoothly with full-fledged software like the browser you use every day. I also think it is very interesting to discover that technologies like HTTP, that basically run the world nowadays, are at their core very simple.

That said, there are many features of HTTP that we didn't cover with our simple socket programming. For starters, HTTP/1.0 introduced other methods after GET, such as POST that is of paramount importance for today's websites, where users keep sending information to servers through forms. To implement all 9 HTTP methods we need to properly parse the incoming request and add relevant functions to our code.

At this point, however, you might notice that we are dealing a lot with low-level details of the protocol, which is usually not the core of our business. When we build a service over HTTP we believe that we have the knowledge to properly implement some code that can simplify a certain process, be it searching for other websites, shopping for books or sharing pictures with friends. We don't want to spend our time understanding the subtleties of the TCP/IP sockets and writing parsers for request-response protocols. It is nice to see how these technologies work, but on a daily basis, we need to focus on something at a higher level.

The situation of our small HTTP server is possibly worsened by the fact that HTTP is a stateless protocol. The protocol doesn't provide any way to connect two successive requests, thus keeping track of the state of the communication, which is the cornerstone of modern Internet. Every time we authenticate on a website and we want to visit other pages we need the server to remember who we are, and this implies keeping track of the state of the connection.

Long story short: to work as a proper HTTP server, our code should at this point implement all HTTP methods and cookies management. We also need to support other protocols like Websockets. These are all but trivial tasks, so we definitely need to add some component to the whole system that lets us focus on the business logic and not on the low-level details of application protocols.

2 Web framework¶

2.1 Rationale¶

Enter the Web framework!

As I discussed many times (see the book on clean architectures or the relative post) the role of the Web framework is that of converting HTTP requests into function calls, and function return values into HTTP responses. The framework's true nature is that of a layer that connects a working business logic to the Web, through HTTP and related protocols. The framework takes care of session management for us and maps URLs to functions, allowing us to focus on the application logic.

In the grand scheme of an HTTP service, this is what the framework is supposed to do. Everything the framework provides out of this scope, like layers to access DBs, template engines, and interfaces to other systems, is an addition that you, as a programmer, may find useful, but is not in principle part of the reason why we added the framework to the system. We add the framework because it acts as a layer between our business logic and HTTP.

2.2 Implementation¶

Thanks to Miguel Gringberg and his amazing Flask mega-tutorial I can set up Flask in seconds. I will not run through the tutorial here, as you can follow it on Miguel's website. I will only use the content of the first article (out of 23!) to create an extremely simple "Hello, world" application.

To run the following example you will need a virtual environment and you will have to pip install flask. Follow Miguel's tutorial if you need more details on this.

The app/__init__.py file is

from flask import Flask

application = Flask(__name__)

from app import routes

and the app/routes.py file is

from app import application


@application.route('/')
@application.route('/index')
def index():
    return "Hello, world!"

You can already see here the power of a framework in action. We defined an index function and connected it with two different URLs (/ and /index) in 3 lines of Python. This leaves us time and energy to properly work on the business logic, that in this case is a revolutionary "Hello, world!". Nobody ever did this before.

Finally, the service.py file is

from app import application

Flask comes with a so-called development web server (do these words ring any bell now?) that we can run on a terminal

$ FLASK_APP=service.py flask run
 * Serving Flask app "service.py"
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off
 * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)

You can now visit the given URL with your browser and see that everything works properly. Remember that 127.0.0.1 is the special IP address that refers to "this computer"; the name localhost is usually created by the operating system as an alias for that, so the two are interchangeable. As you can see the standard port for Flask's development server is 5000, so you have to mention it explicitly, otherwise your browser would try to access port 80 (the default HTTP one). When you connect with the browser you will see some log messages about the HTTP requests

127.0.0.1 - - [14/Feb/2020 14:54:27] "GET / HTTP/1.1" 200 -
127.0.0.1 - - [14/Feb/2020 14:54:28] "GET /favicon.ico HTTP/1.1" 404 -

You can recognise both now, as those are the same request we got with our little server in the previous part of the article.

2.3 Resources¶

These resources provide more detailed information on the topics discussed in this section

Miguel Gringberg's amazing Flask mega-tutorial
What is localhost
The source code of this example is available here

2.4 Issues¶

Apparently, we solved all our problems, and many programmers just stop here. They learn how to use the framework (which is a big achievement!), but as we will shortly discover, this is not enough for a production system. Let's have a closer look at the output of the Flask server. It clearly says, among other things

   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.

The main issue we have when we deal with any production system is represented by performances. Think about what we do with JavaScript when we minimise the code: we consciously obfuscate the code in order to make the file smaller, but this is done for the sole purpose of making the file faster to retrieve.

For HTTP servers the story is not very different. The Web framework usually provides a development Web server, as Flask does, which properly implements HTTP, but does it in a very inefficient way. For starters, this is a blocking framework, which means that if our request takes seconds to be served (for example because the endpoint retrieves data from a very slow database), any other request will have to wait to be served in a queue. That ultimately means that the user will see a spinner in the browser's tab and just shake their head thinking that we can't build a modern website. Other performances concerns might be connected with memory management or disk caches, but in general, we are safe to say that this web server cannot handle any production load (i.e. multiple users accessing the web site at the same time and expecting good quality of service).

This is hardly surprising. After all, we didn't want to deal with TCP/IP connections to focus on our business, so we delegated this to other coders who maintain the framework. The framework's authors, in turn, want to focus on things like middleware, routes, proper handling of HTTP methods, and so on. They don't want to spend time trying to optimise the performances of the "multi-user" experience. This is especially true in the Python world (and somehow less true for Node.js, for example): Python is not heavily concurrency-oriented, and both the style of programming and the performances are not favouring fast, non-blocking applications. This is changing lately, with async and improvements in the interpreter, but I leave this for another post.

So, now that we have a full-fledged HTTP service, we need to make it so fast that users won't even notice this is not running locally on their computer.

3 Concurrency and façades¶

3.1 Rationale¶

Well, whenever you have performance issues, just go for concurrency. Now you have many problems! (see here)

Yes, concurrency solves many problems and it's the source of just as much, so we need to find a way to use it in the safest and less complicated way. We basically might want to add a layer that runs the framework in some concurrent way, without requiring us to change anything in the framework itself.

And whenever you have to homogenise different things just create a layer of indirection. This solves any problem but one. (see here)

So we need to create a layer that runs our service in a concurrent way, but we also want to keep it detached from the specific implementation of the service, that is independent of the framework or library that we are using.

3.2 Implementation¶

In this case, the solution is that of giving a specification of the API that web frameworks have to expose, in order to be usable by independent third-party components. In the Python world, this set of rules has been named WSGI, the Web Server Gateway Interface, but such interfaces exist for other languages such as Java or Ruby. The "gateway" mentioned here is the part of the system outside the framework, which in this discussion is the part that deals with production performances. Through WSGI we are defining a way for frameworks to expose a common interface, leaving people interested in concurrency free to implement something independently.

If the framework is compatible with the gateway interface, we can add software that deals with concurrency and uses the framework through the compatibility layer. Such a component is a production-ready HTTP server, and two common choices in the Python world are Gunicorn and uWSGI.

Production-ready HTTP server means that the software understands HTTP as the development server already did, but at the same time pushes performances in order to sustain a bigger workload, and as we said before this is done through concurrency.

Flask is compatible with WSGI, so we can make it work with Gunicorn. To install it in our virtual environment run pip install gunicorn and set it up creating a file names wsgi.py with the following content

from app import application


if __name__ == "__main__":
    application.run()

To run Gunicorn specify the number of concurrent instances and the external port

$ gunicorn --workers 3 --bind 0.0.0.0:8000 wsgi
[2020-02-12 18:39:07 +0000] [13393] [INFO] Starting gunicorn 20.0.4
[2020-02-12 18:39:07 +0000] [13393] [INFO] Listening at: http://0.0.0.0:8000 (13393)
[2020-02-12 18:39:07 +0000] [13393] [INFO] Using worker: sync
[2020-02-12 18:39:07 +0000] [13396] [INFO] Booting worker with pid: 13396
[2020-02-12 18:39:07 +0000] [13397] [INFO] Booting worker with pid: 13397
[2020-02-12 18:39:07 +0000] [13398] [INFO] Booting worker with pid: 13398

As you can see, Gunicorn has the concept of workers which are a generic way to express concurrency. Specifically, Gunicorn implements a pre-fork worker model, which means that it (pre)creates a different Unix process for each worker. You can check this running ps

$ ps ax | grep gunicorn
14919 pts/1    S+     0:00 ~/venv3/bin/python3 ~/venv3/bin/gunicorn --workers 3 --bind 0.0.0.0:8000 wsgi
14922 pts/1    S+     0:00 ~/venv3/bin/python3 ~/venv3/bin/gunicorn --workers 3 --bind 0.0.0.0:8000 wsgi
14923 pts/1    S+     0:00 ~/venv3/bin/python3 ~/venv3/bin/gunicorn --workers 3 --bind 0.0.0.0:8000 wsgi
14924 pts/1    S+     0:00 ~/venv3/bin/python3 ~/venv3/bin/gunicorn --workers 3 --bind 0.0.0.0:8000 wsgi

Using processes is just one of the two ways to implement concurrency in a Unix system, the other being using threads. The benefits and demerits of each solution are outside the scope of this post, however. For the time being just remember that you are dealing with multiple workers that process incoming requests asynchronously, thus implementing a non-blocking server, ready to accept multiple connections.

3.3 Resources¶

These resources provide more detailed information on the topics discussed in this section

The WSGI official documentation and the Wikipedia page
The homepages of Gunicorn and uWSGI
A good entry point for your journey into the crazy world of concurrency: multithreading.
The source code of this example is available here

3.4 Issues¶

Using a Gunicorn we have now a production-ready HTTP server, and apparently implemented everything we need. There are still many considerations and missing pieces, though.

Performances (again)¶

Are 3 workers enough to sustain the load of our new killer mobile application? We expect thousands of visitors per minute, so maybe we should add some. But while we increase the amount of workers, we have to keep in mind that the machine we are using has a finite amount of CPU power and memory. So, once again, we have to focus on performances, and in particular on scalability: how can we keep adding workers without having to stop the application, replace the machine with a more powerful one, and restart the service?

Embrace change¶

This is not the only problem we have to face in production. An important aspect of technology is that it changes over time, as new and (hopefully) better solutions become widespread. We usually design systems dividing them as much as possible into communicating layers exactly because we want to be free to replace a layer with something else, be it a simpler component or a more advanced one, one with better performances or maybe just a cheaper one. So, once again, we want to be able to evolve the underlying system keeping the same interface, exactly as we did in the case of web frameworks.

HTTPS¶

Another missing part of the system is HTTPS. Gunicorn and uWSGI do not understand the HTTPS protocol, so we need something in front of them that will deal with the "S" part of the protocol, leaving the "HTTP" part to the internal layers.

Load balancers¶

In general, a load balancer is just a component in a system that distributes work among a pool of workers. Gunicorn is already distributing load among its workers, so this is not a new concept, but we generally want to do it on a bigger level, among machines or among entire systems. Load balancing can be hierarchical and be structured on many levels. We can also assign more importance to some components of the system, flagging them as ready to accept more load (for example because their hardware is better). Load balancers are extremely important in network services, and the definition of load can be extremely different from system to system: generally speaking, in a Web service the number of connections is the standard measure of the load, as we assume that on average all connections bring the same amount of work to the system.

Reverse proxies¶

Load balancers are forward proxies, as they allow a client to contact any server in a pool. At the same time, a reverse proxy allows a client to retrieve data produced by several systems through the same entry point. Reverse proxies are a perfect way to route HTTP requests to sub-systems that can be implemented with different technologies. For example, you might want to have part of the system implemented with Python, using Django and Postgres, and another part served by an AWS Lambda function written in Go and connected with a non-relational database such as DynamoDB. Usually, in HTTP services this choice is made according to the URL (for example routing every URL that begins with /api/).

Logic¶

We also want a layer that can implement a certain amount of logic, to manage simple rules that are not related to the service we implemented. A typical example is that of HTTP redirections: what happens if a user accesses the service with an http:// prefix instead of https://? The correct way to deal with this is through an HTTP 301 code, but you don't want such a request to reach your framework, wasting resources for such a simple task.

4 The Web server¶

4.1 Rationale¶

The general label of Web server is given to software that performs the tasks we discussed. Two very common choices for this part of the system are nginx and Apache, two open source projects that are currently leading the market. With different technical approaches, they both implement all the features we discussed in the previous section (and many more).

4.2 Implementation¶

To test nginx without having to fight with the OS and install too many packages we can use Docker. Docker is useful to simulate a multi-machine environment, but it might also be your technology of choice for the actual production environment (AWS ECS works with Docker containers, for example).

The base configuration that we will run is very simple. One container will contain the Flask code and run the framework with Gunicorn, while the other container will run nginx. Gunicorn will serve HTTP on the internal port 8000, not exposed by Docker and thus not reachable from our browser, while nignx will expose port 80, the traditional HTTP port.

In the same directory of the file wsgi.py, create a Dockerfile

FROM python:3.6

ADD app /app
ADD wsgi.py /

WORKDIR .
RUN pip install flask gunicorn
EXPOSE 8000

This starts from a Python Docker image, adds the app directory and the wsgi.py file, and installs Gunicorn. Now create a configuration for nginx in a file called nginx.conf in the same directory

server {
    listen 80;
    server_name localhost;

    location / {
        proxy_pass http://application:8000/;
    }
}

This defines a server that listens on port 80 and that connects all the URL starting with / with a server called application on port 8000, which is the container running Gunicorn.

Last, create a file docker-compose.yml that will describe the configuration of the containers.

version: "3.7"
services:
  application:
    build:
      context: .
      dockerfile: Dockerfile
    command: gunicorn --workers 3 --bind 0.0.0.0:8000 wsgi
    expose:
        - 8000

  nginx:
    image: nginx
    volumes:
     - ./nginx.conf:/etc/nginx/conf.d/default.conf
    ports:
      - 8080:80
    depends_on:
      - application

As you can see the name application that we mentioned in the nginx configuration file is not a magic string, but is the name we assigned to the Gunicorn container in the Docker Compose configuration. Please note that nginx listens on port 80 inside the container, but the port is published as 8080 on the host.

To create this infrastructure we need to install Docker Compose in our virtual environment through pip install docker-compose. I also created a file named .env with the name of the project

COMPOSE_PROJECT_NAME=service

At this point you can run Docker Compose with docker-compose up -d

$ docker-compose up -d
Creating network "service_default" with the default driver
Creating service_application_1 ... done
Creating service_nginx_1       ... done

If everything is working correctly, opening the browser and visiting localhost:8080 should show you the HTML page Flask is serving.

Through docker-compose logs we can check what services are doing. We can recognise the output of Gunicorn in the logs of the service named application

$ docker-compose logs application
Attaching to service_application_1
application_1  | [2020-02-14 08:35:42 +0000] [1] [INFO] Starting gunicorn 20.0.4
application_1  | [2020-02-14 08:35:42 +0000] [1] [INFO] Listening at: http://0.0.0.0:8000 (1)
application_1  | [2020-02-14 08:35:42 +0000] [1] [INFO] Using worker: sync
application_1  | [2020-02-14 08:35:42 +0000] [8] [INFO] Booting worker with pid: 8
application_1  | [2020-02-14 08:35:42 +0000] [9] [INFO] Booting worker with pid: 9
application_1  | [2020-02-14 08:35:42 +0000] [10] [INFO] Booting worker with pid: 10

but the one we are mostly interested with now is the service named nginx, so let's follow the logs in real-time with docker-compose logs -f nginx. Refresh the localhost page you visited with the browser, and the container should output something like

$ docker-compose logs -f nginx
Attaching to service_nginx_1
nginx_1        | 192.168.192.1 - - [14/Feb/2020:08:42:20 +0000] "GET / HTTP/1.1" 200 13 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:72.0) Gecko/20100101 Firefox/72.0" "-"

which is the standard log format of nginx. It shows the IP address of the client (192.168.192.1), the connection timestamp, the HTTP request and the response status code (200), plus other information on the client itself.

Let's now increase the number of services, to see the load balancing mechanism in action. To do this, first we need to change the log format of nginx to show the IP address of the machine that served the request. Change the nginx.conf file adding the log_format and access_log options

log_format upstreamlog '[$time_local] $host to: $upstream_addr: $request $status';

server {
    listen 80;
    server_name localhost;

    location / {
        proxy_pass http://application:8000;
    }

    access_log /var/log/nginx/access.log upstreamlog;
}

The $upstream_addr variable is the one that contains the IP address of the server proxied by nginx. Now run docker-compose down to stop all containers and then docker-compose up -d --scale application=3 to start them again

$ docker-compose down
Stopping service_nginx_1       ... done
Stopping service_application_1 ... done
Removing service_nginx_1       ... done
Removing service_application_1 ... done
Removing network service_default
$ docker-compose up -d --scale application=3
Creating network "service_default" with the default driver
Creating service_application_1 ... done
Creating service_application_2 ... done
Creating service_application_3 ... done
Creating service_nginx_1       ... done

As you can see, Docker Compose runs now 3 containers for the application service. If you open the logs stream and visit the page in the browser you will now see a slightly different output

$ docker-compose logs -f nginx
Attaching to service_nginx_1
nginx_1        | [14/Feb/2020:09:00:16 +0000] localhost to: 192.168.240.4:8000: GET / HTTP/1.1 200

where you can spot to: 192.168.240.4:8000 which is the IP address of one of the application containers. Please note that the IP address you see might be different, as it depends on the Docker network settings. If you now visit the page again multiple times you should notice a change in the upstream address, something like

$ docker-compose logs -f nginx
Attaching to service_nginx_1
nginx_1        | [14/Feb/2020:09:00:16 +0000] localhost to: 192.168.240.4:8000: GET / HTTP/1.1 200
nginx_1        | [14/Feb/2020:09:00:17 +0000] localhost to: 192.168.240.2:8000: GET / HTTP/1.1 200
nginx_1        | [14/Feb/2020:09:00:17 +0000] localhost to: 192.168.240.3:8000: GET / HTTP/1.1 200
nginx_1        | [14/Feb/2020:09:00:17 +0000] localhost to: 192.168.240.4:8000: GET / HTTP/1.1 200
nginx_1        | [14/Feb/2020:09:00:17 +0000] localhost to: 192.168.240.2:8000: GET / HTTP/1.1 200

This shows that nginx is performing load balancing, but to tell the truth this is happening through Docker's DNS, and not by an explicit action performed by the web server. We can verify this accessing the nginx container and running dig application (you need to run apt update and apt install dnsutils to install dig)

$ docker-compose exec nginx /bin/bash
root@99c2f348140e:/# apt update
root@99c2f348140e:/# apt install -y dnsutils
root@99c2f348140e:/# dig application

; <<>> DiG 9.11.5-P4-5.1-Debian <<>> application
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 7221
;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;application.                   IN      A

;; ANSWER SECTION:
application.            600     IN      A       192.168.240.2
application.            600     IN      A       192.168.240.4
application.            600     IN      A       192.168.240.3

;; Query time: 1 msec
;; SERVER: 127.0.0.11#53(127.0.0.11)
;; WHEN: Fri Feb 14 09:57:24 UTC 2020
;; MSG SIZE  rcvd: 110

To see load balancing performed by nginx we can explicitly define two services and assign them different weights. Run docker-compose down and change the nginx configuration to

upstream app {
    server application1:8000 weight=3;
    server application2:8000;
}

log_format upstreamlog '[$time_local] $host to: $upstream_addr: $request $status';

server {
    listen 80;
    server_name localhost;

    location / {
        proxy_pass http://app;
    }

    access_log /var/log/nginx/access.log upstreamlog;
}

We defined here an upstream structure that lists two different services, application1 and application2, giving to the first one a weight of 3. This mean that each 4 requests, 3 will be routed to the first service, and one to the second service. Now nginx is not just relying on the DNS, but consciously choosing between two different services.

Let's define the services accordingly in the Docker Compose configuration file

version: "3"
services:
  application1:
    build:
      context: .
      dockerfile: Dockerfile
    command: gunicorn --workers 6 --bind 0.0.0.0:8000 wsgi
    expose:
        - 8000

  application2:
    build:
      context: .
      dockerfile: Dockerfile
    command: gunicorn --workers 3 --bind 0.0.0.0:8000 wsgi
    expose:
        - 8000

  nginx:
    image: nginx
    volumes:
     - ./nginx.conf:/etc/nginx/conf.d/default.conf
    ports:
      - 80:80
    depends_on:
      - application1
      - application2

I basically duplicated the definition of application, but the first service is running now 6 workers, just for the sake of showing a possible difference between the two. Now run docker-compose up -d and docker-compose logs -f nginx. If you refresh the page on the browser multiple times you will see something like

$ docker-compose logs -f nginx
Attaching to service_nginx_1
nginx_1         | [14/Feb/2020:11:03:25 +0000] localhost to: 172.18.0.2:8000: GET / HTTP/1.1 200
nginx_1         | [14/Feb/2020:11:03:25 +0000] localhost to: 172.18.0.2:8000: GET /favicon.ico HTTP/1.1 404
nginx_1         | [14/Feb/2020:11:03:30 +0000] localhost to: 172.18.0.3:8000: GET / HTTP/1.1 200
nginx_1         | [14/Feb/2020:11:03:31 +0000] localhost to: 172.18.0.2:8000: GET / HTTP/1.1 200
nginx_1         | [14/Feb/2020:11:03:32 +0000] localhost to: 172.18.0.2:8000: GET / HTTP/1.1 200
nginx_1         | [14/Feb/2020:11:03:33 +0000] localhost to: 172.18.0.2:8000: GET / HTTP/1.1 200
nginx_1         | [14/Feb/2020:11:03:33 +0000] localhost to: 172.18.0.3:8000: GET / HTTP/1.1 200
nginx_1         | [14/Feb/2020:11:03:34 +0000] localhost to: 172.18.0.2:8000: GET / HTTP/1.1 200
nginx_1         | [14/Feb/2020:11:03:34 +0000] localhost to: 172.18.0.2:8000: GET / HTTP/1.1 200
nginx_1         | [14/Feb/2020:11:03:35 +0000] localhost to: 172.18.0.2:8000: GET / HTTP/1.1 200
nginx_1         | [14/Feb/2020:11:03:35 +0000] localhost to: 172.18.0.3:8000: GET / HTTP/1.1 200

where you can clearly notice the load balancing between 172.18.0.2 (application1) and 172.18.0.3 (application2) in action.

I will not show here an example of reverse proxy or HTTPS to prevent this post to become too long. You can find resources on those topics in the next section.

4.3 Resources¶

These resources provide more detailed information on the topics discussed in this section

Docker Compose official documentation
nginx documentation: in particular the sections about log_format and upstream directives
How to configure logging in nginx
How to configure load balancing in nginx
Setting up an HTTPS Server with nginx and how to created self-signed certificates
How to create a reverse proxy with nginx, the documentation of the location directive and some insights on the location choosing algorithms (one of the most complex parts of nginx)
The source code of this example is available here

4.4 Issues¶

Well, finally we can say that the job is done. Now we have a production-ready web server in front of our multi-threaded web framework and we can focus on writing Python code instead of dealing with HTTP headers.

Using a web server allows us to scale the infrastructure just adding new instances behind it, without interrupting the service. The HTTP concurrent server runs multiple instances of our framework, and the framework itself abstracts HTTP, mapping it to our high-level language.

Bonus: cloud infrastructures¶

Back in the early years of the Internet, companies used to have their own servers on-premise, and system administrators used to run the whole stack directly on the bare operating system. Needless to say, this was complicated, expensive, and failure-prone.

Nowadays "the cloud" is the way to go, so I want to briefly mention some components that can help you run such a web stack on AWS, which is the platform I know the most and the most widespread cloud provider in the world at the time of writing.

Elastic Beanstalk¶

This is the entry-level solution for simple applications, being a managed infrastructure that provides load balancing, auto-scaling, and monitoring. You can use several programming languages (among which Python and Node.js) and choose between different web servers like for example Apache or nginx. The components of an EB service are not hidden, but you don't have direct access to them, and you have to rely on configuration files to change the way they work. It's a good solution for simple services, but you will probably soon need more control.

Go to Elastic Beanstalk

Elastic Container Service (ECS)¶

With ECS you can run Docker containers grouping them in clusters and setting up auto-scale policies connected with metrics coming from CloudWatch. You have the choice of running them on EC2 instances (virtual machines) managed by you or on a serverless infrastructure called Fargate. ECS will run your Docker containers, but you still have to create DNS entries and load balancers on your own. You also have the choice of running your containers on Kubernetes using EKS (Elastic Kubernetes Service).

Go to Elastic Container Service

Elastic Compute Cloud (EC2)¶

This is the bare metal of AWS, where you spin up stand-alone virtual machines or auto-scaling group of them. You can SSH into these instances and provide scripts to install and configure software. You can install here your application, web servers, databases, whatever you want. While this used to be the way to go at the very beginning of the cloud computing age I don't think you should go for it. There is so much a cloud provider can give you in terms of associated services like logs or monitoring, and in terms of performances, that it doesn't make sense to avoid using them. EC2 is still there, anyway, and if you run ECS on top of it you need to know what you can and what you can't do.

Go to Elastic Compute Cloud

Elastic Load Balancing¶

While Network Load Balancers (NLB) manage pure TCP/IP connections, Application Load Balancers are dedicated to HTTP, and they can perform many of the services we need. They can reverse proxy through rules (that were recently improved) and they can terminate TLS, using certificates created in ACM (AWS Certificate Manager). As you can see, ALBs are a good replacement for a web server, even though they clearly lack the extreme configurability of a software. You can, however, use them as the first layer of load balancing, still using nginx or Apache behind them if you need some of the features they provide.

Go to Elastic Load Balancing

CloudFront¶

CloudFront is a Content Delivery Network, that is a geographically-distributed cache that provides faster access to your content. While CDNs are not part of the stack that I discussed in this post I think it is worth mentioning CF as it can speed-up any static content, and also terminate TLS in connection with AWS Certificate Manager.

Go to CloudFront

Conclusion¶

As you can see a web stack is a pretty rich set of components, and the reason behind them is often related to performances. There are a lot of technologies that we take for granted, and that fortunately have become easier to deploy, but I still believe a full-stack engineer should be aware not only of the existence of such layers, but also of their purpose and at least their basic configuration.

Feedback¶

Feel free to reach me on Twitter if you have questions. The GitHub issues page is the best place to submit corrections.

The Lord of the Rings: an Erlang epic

2013-06-20T17:49:00+01:00

Abstract¶

One of the first really challenging problems an Erlang novice must face is the classical process ring, which can be found around the Internet and most notably in "Erlang Programming" by Cesarini and Thompson (page 115).

Its formulation may vary, but the core of it requires the programmer to design and implement a closed ring of processes, to make them pass a given number of messages each other and then terminate gracefully.

I try here to give an in-depth analysis of the matter, to point out some of the most interesting issues of this exercise. I strongly suggest the Erlang novice to try and solve the exercise before looking at the solutions proposed here.

Linking processes¶

The aim of the exercise is to build a chain of processes that exchange messages. The latter specification is important since we are required to connect our processes with the only purpose of sending messages and we know that, in Erlang, a process may send a message to another process simply by knowing its pid.

Actual linking between processes, in Erlang, has the aim of making them exchange exit signals; links are thus a mean to control the way the system collapses (or not) when a process crashes. The processes in the ring will thus be linked to ensure that when one of them exits all other processes will be terminated, but this has nothing to do with the ring itself.

The point is to let a process know how to send a message to the next one, forming a chain. This chain, then, is closed, i.e. the last process sends messages to the first one.

There are two main strategies that can be leveraged to get the ring build and behave in a correct way: the master process approach and the recursive one. For each of the two, I am going to present and analyze the following steps: a ring of processes with no message exchange, a ring with a single message travelling through it, a ring with multiple messages and a ring that exposes a functional interface to send messages.

Debugging¶

Debugging concurrent applications is everything but simple. For this exercise, it is however enough to get simple information about which process is forwarding which message. To allow a simple removal of debug prints a very small macro has been included in the file debug.hrl, activated by the definition of debug.

-ifdef(debug).
-define(DEBUG(Format, Args),
        io:format("~p: " ++ Format, [self()] ++ Args)).
-else.
-define(DEBUG(Format, Args), true).
-endif.

This converts the line

?DEBUG("A message~n", []).

<0.44.0>: A message

where <0.44.0> is the pid of the message calling ?DEBUG (only if -define(debug, true). is in the file).

The “master process” solution¶

A good way to face the problem of building the ring is to spawn a master process, which will then spawn all other processes. This solution keeps the creation phase simple since everything happens between the master process and the process that is currently being spawned.

A small caveat: the creation of the ring must be performed backwards. When the master process spawns another process the only other process it knows is the process spawned in the previous loop. This means that the master process spawns a process that connects backwards to it, then spawns a process that connects backward to the second one, and so on.

Building the ring¶

%%% Author:  Leonardo Giordani <giordani.leonardo@gmail.com>
%%% Description: Implements a simple ring of processes without message passing
%%% Created: Jun 2013 by Leonardo Giordani

-module(ring_master_no_messages).

%% Include some debug macros
-define(debug, true).
-include("debug.hrl").

%% User interface
-export([start/1]).

%% Private exports
-export([create/1, create/2, loop/1]).

%% This version of the processes ring program uses the first process as a
%% master that spawns all other processes. It links to each process it spawns,
%% but processes are not linked each other.
%% Processes send no messages.

%% This spawns the first process. There is no need to register the process.
start(NumberProcesses) ->
    spawn(?MODULE, create, [NumberProcesses]).

%% Function create/1 needs to be defined so that the first process can call
%% create/2 passing its pid through self().
create(NumberProcesses) ->
    create(NumberProcesses, self()).

%% Function create/2 manages the single process creation; in this version the
%% master process just spawns each process and links to it. The exit clause
%% is when the proceses counter reaches 1 (that is, we completed the ring).
create(1, NextProcess) ->
    ?DEBUG("Process ~p connected with ~p~n", [self(), NextProcess]),
    loop(NextProcess);
create(NumberProcesses, NextProcess) ->
    Prev = spawn_link(?MODULE, loop, [NextProcess]),
    ?DEBUG("Process ~p created and connected with ~p~n", [Prev, NextProcess]),
    create(NumberProcesses - 1, Prev).

%% In this version function loop/1 sits down and does nothing.
loop(NextProcess) ->
    loop(NextProcess).

source code

Note that the first spawn cannot directly call create/2 passing self() since at that moment self() is the pid of the calling process (e.g. the Erlang shell) and not the pid of the first process of the ring, which is what we want. So we have to bypass this by spawning a process which executes create/1, which in turn calls create/2 with the pid the process extracted with self().

This first program spawns a set of processes that sit down and do nothing. Since they are all linked together (through the link with the master process), you can terminate the whole set by sending an exit signal to one of them. You can get the list of pid and names through the shell function i(). Otherwise, you can get and use the pid returned by the function start/1.

1> i().
[...]
2> exit(pid(0,44,0), boom).

Sending a single message¶

Now I am going to develop further the solution by making a single message travel the whole ring.

The code undergoes the following modifications:

function start() must accept the message and pass it to create().
function create() must accept the message
when the ring is ready the master process has to send the message to the second process; then it terminates
each node just forwards the incoming message and terminates

%%% Author:  Leonardo Giordani <giordani.leonardo@gmail.com>
%%% Description: Implements a simple ring of processes with a single
%%%              message traveling once through the ring.
%%% Created: Jun 2013 by Leonardo Giordani

-module(ring_master_single_message).

%% Include some debug macros
-define(debug, true).
-include("debug.hrl").

%% User interface
-export([start/2]).

%% Private exports
-export([create/2, create/3, loop/1]).

%% This version of the processes ring program uses the first process as a
%% master that spawns all other processes. It links to each process it spawns,
%% but processes are not linked each other.
%% The first process then injects a single message in the ring, sending it to
%% the second process.

%% This spawns the first process. There is no need to register the process.
start(NumberProcesses, Message) ->
    spawn(?MODULE, create, [NumberProcesses, Message]).

%% Function create/2 needs to be defined so that the first process can call
%% create/3 passing its pid through self().
create(NumberProcesses, Message) ->
    create(NumberProcesses, self(), Message).

%% Function create/3 manages the single process creation; in this version the
%% master process spawns each process and links to it. Then each process starts
%% a loop. When the ring is completed the first process injects the message and
%% terminates.
create(1, NextProcess, Message) ->
    ?DEBUG("Process ~p connected with ~p~n", [self(), NextProcess]),
    ?DEBUG("Process ~p injects message ~p~n", [self(), Message]),
    NextProcess ! Message;
create(NumberProcesses, NextProcess, Message) ->
    Prev = spawn_link(?MODULE, loop, [NextProcess]),
    ?DEBUG("Process ~p created and connected with ~p~n", [Prev, NextProcess]),
    create(NumberProcesses - 1, Prev, Message).

%% Now loop/1 blocks each process making it wait for a message to pass along.
loop(NextProcess) ->
    receive
        Msg ->
            ?DEBUG("Got message ~p, passing it to ~p~n", [Msg, NextProcess]),
            NextProcess ! Msg,
            ok
    end.

source code

Beware that a subtle mechanism conceals here: in Erlang, you can always send a message to a pid, even if it the relative process does not exist; in the latter case the message is simply discarded. This feature allows us to make the first process send the message and terminate without having the last process crash by sending a message to it. Remember that you cannot do this with registered processes alias, only with pids.

Sending multiple messages¶

The exercise requests that a message can travel more than once through the whole ring. Adding this possibility is pretty straightforward: it simply requires to add a parameter to start() and create() and to change the loop clauses.

Function loop/1 now becomes loop/2 and has the following clauses:

when the number of remaining travels is 1 the process forwards the message and then terminates.
when the number of remaining travels is more than 1 the process loops again with a decremented value of travels.

%%% Author:  Leonardo Giordani <giordani.leonardo@gmail.com>
%%% Description: Implements a simple ring of processes with a single
%%%              message traveling multiple times through the ring.
%%% Created: Jun 2013 by Leonardo Giordani

-module(ring_master_multiple_messages).

%% Include some debug macros
-define(debug, true).
-include("debug.hrl").

%% User interface
-export([start/3]).

%% Private exports
-export([create/3, create/4, loop/2]).

%% This version of the processes ring program uses the first process as a
%% master that spawns all other processes. It links to each process it spawns,
%% but processes are not linked each other.
%% The first process then injects a message in the ring, sending it to the
%% second process and when getting the message from the last process, it
%% injects it again in the ring a given number of times.

%% This spawns the first process. There is no need to register the process.
start(NumberProcesses, Message, NumberMessages) ->
    spawn(?MODULE, create, [NumberProcesses, Message, NumberMessages]).

%% Function create/3 needs to be defined so that the first process can call
%% create/4 passing its pid through self().
create(NumberProcesses, Message, NumberMessages) ->
    create(NumberProcesses, self(), Message, NumberMessages).

%% Function create/4 manages the single process creation; in this version the
%% master process spawns each process and links to it. Then each process starts
%% a loop. When the ring is completed the first process injects the message and
%% starts looping.
create(1, NextProcess, Message, NumberMessages) ->
    ?DEBUG("Process ~p connected with ~p~n", [self(), NextProcess]),
    ?DEBUG("Process ~p injects message ~p~n", [self(), Message]),
    NextProcess ! Message,
    loop(NextProcess, NumberMessages - 1);
create(NumberProcesses, NextProcess, Message, NumberMessages) ->
    Prev = spawn_link(?MODULE, loop, [NextProcess, NumberMessages]),
    ?DEBUG("Process ~p created and connected with ~p~n", [Prev, NextProcess]),
    create(NumberProcesses - 1, Prev, Message, NumberMessages).

loop(NextProcess, 1) ->
    receive
        Msg ->
            ?DEBUG("Got message ~p, passing it to ~p and terminating~n",
                   [Msg, NextProcess]),
            NextProcess ! Msg,
            ok
    end;
loop(NextProcess, NumberMessages) ->
    receive
        Msg ->
            ?DEBUG("Got message ~p, passing it to ~p (still ~p time(s))~n",
                   [Msg, NextProcess, NumberMessages]),
            NextProcess ! Msg,
            loop(NextProcess, NumberMessages - 1)
    end.

source code

As before, we are here relying on the possibility of sending a message to a non existent process.

Exposing a functional interface¶

An Erlang best practice is to expose a functional interface to the user that hides the underlying message passing. We are going to convert our ring program to expose the following functions:

start/1 - starts a ring with the given number of processes
stop/0 - terminates all processes in the ring
send_message/1 - sends a message that travels once through the ring
send_message/2 - sends a message that travels the given number of times through the ring

To expose a functional interface you need to register one or more processes, to get a global access point for your functions, so the first change to the code is that the spawned process is registered. Note that Message and NumberProcesses are no more passed to the create() function.

register(ring_master_functional, spawn(?MODULE, create, [NumberProcesses])),

This is not enough. We have to be sure that the whole ring is ready before giving control back to the user issuing start(). Until now the message was sent by the master process just after the creation of the last ring process, so there was no need to synchronize the return of the start function with the spawned processes. Now we need it, so just after registering the master process we wait for a ready message coming from the ring. To allow the ring to send the message to the initial caller we have to pass self() to create(). Pay attention that self() passed to the spawned process is the pid of the external process (e.g. the Erlang shell) while self() passed to create() is the pid of the master ring process.

%%% Author:  Leonardo Giordani <giordani.leonardo@gmail.com>
%%% Description: Implements a simple ring of processes that exposes a
%%%              functional interface to send messages.
%%% Created: Jun 2013 by Leonardo Giordani

-module(ring_master_functional).

%% Include some debug macros
%%-define(debug, true).
-include("debug.hrl").

%% User interface
-export([start/1, stop/0, send_message/1, send_message/2]).

%% Private exports
-export([create/2, loop/1]).

%% This version of the processes ring program uses the first process as a
%% master that spawns all other processes. It links to each process it spawns,
%% but processes are not linked each other.
%% The module exposes a functional interface to send messages in the ring
%% and to terminate it.

%% This spawns the first process. The process is registered to allow the
%% functional interface to work with it.
start(NumberProcesses) ->
    register(ring_master_functional, spawn(?MODULE, create,
                                           [NumberProcesses, self()])),
    receive
        ready ->
            ok
    after 5000 ->
            {error, timeout}
    end.

%% Functional interface: terminate the ring
stop() ->
    ring_master_functional ! {command, stop}.

%% Functional interface: send a message that travels through the ring once
send_message(Message) ->
    send_message(Message, 1).

%% Functional interface: send a message that travels through the ring
%% Times times
send_message(Message, Times) ->
    ring_master_functional ! {command, message, [Message, Times]}.

%% Function create/2 needs to be defined so that the first process can call
%% create/3 passing its pid through self(). The Starter process is passed to
%% allow the ring to send a ready message.
create(NumberProcesses, Starter) ->
    create(NumberProcesses, self(), Starter).

%% Function create/3 manages the single process creation; in this version the
%% master process spawns each process and links to it. Then each process starts
%% a loop. When the ring is completed the first process injects the message and
%% terminates.
create(1, NextProcess, Starter) ->
    ?DEBUG("Process ~p connected with ~p~n", [self(), NextProcess]),
    Starter ! ready,
    loop_master(NextProcess);
create(NumberProcesses, NextProcess, Starter) ->
    Prev = spawn_link(?MODULE, loop, [NextProcess]),
    ?DEBUG("Process ~p created and connected with ~p~n", [Prev, NextProcess]),
    create(NumberProcesses - 1, Prev, Starter).

%% Function loop_master/1 rules the behaviour of the master process (the first
%% of the whole ring). When it receives a stop command it forwards it and
%% terminates. When it receives a message command if Times is 0 the message is
%% thrown away, otherwise is is injected again in the ring with a decremented
%% Times value.
loop_master(NextProcess) ->
    receive
        Msg = {command, stop} ->
            NextProcess ! Msg,
            ok;
        {command, message, [_Message, 0]} ->
            loop_master(NextProcess);
        {command, message, [Message, Times]} ->
            ?DEBUG("Got message ~p, passing it to ~p (still ~p time(s)~n",
                   [Message, NextProcess, Times]),
            NextProcess ! {command, message, [Message, Times - 1]},
            loop_master(NextProcess)
    end.

%% Function loop/1 rules the behaviour of the standard processes.
%% When a process receives a stop command it forwards it and
%% terminates. When it receives a message command it just forwards it.
loop(NextProcess) ->
    receive
        Msg = {command, stop} ->
            NextProcess ! Msg,
            ok;
        Msg = {command, message, [_Message, _]} ->
            ?DEBUG("Got message ~p, passing it to ~p~n",
                   [_Message, NextProcess]),
            NextProcess ! Msg,
            loop(NextProcess)
    end.

source code

The exposed functions are very simple. As you can see they use two different messages: {command, stop} and {command, message, [Message, Times]}. One of the advantages of exposing a functional API is that you are free to format your messages according to the current status of the software and change the format if it does no more suit the application needs.

The loop() function has been splitted in two different functions now: loop/1 rules the behaviour of the standard ring process, while loop_master/1 rules the behaviour of the master process.

The standard process has to react to a stop command, forwarding it and terminating, and to a message command, simply forwarding it. The master process has to check an incoming message to decide if it shall be injected again in the ring.

A recursive solution¶

The simplest way to make a process know the pid of another process is to let the first spawn the second, and this gives us a hint about another way the process ring can be built. Like many things in Erlang, this can be solved recursively by saying:

In a process do:

spawn another process and store its pid, call recursively on the spawned process
if you are the last process just connect to the first

This solution has the advantage of being very straightforward for an Erlang programmer since it implements the standard recursion pattern. However, it forces the programmer to deal most of the time with the last node, which is a little counterintuitive.

The following programs are the recursive version of the four presented in the previous section. Once grasped the two main differences, building the ring forwards instead of backwards and dealing with the last node instead of the first, the two solutions present pretty much the same evolution steps.

The recursive ring construction.

%%% Author:  Leonardo Giordani <giordani.leonardo@gmail.com>
%%% Description: Implements a simple ring of processes without message passing
%%% Created: Jun 2013 by Leonardo Giordani

-module(ring_recursion_no_messages).

%% Include some debug macros
-define(debug, true).
-include("debug.hrl").

%% User interface
-export([start/1]).

%% Private exports
-export([create/1, loop/1]).

%% This version of the processes ring program starts the processes in a
%% recursive way. Each process spawns the next one and links to it.
%% Processes send no messages.

%% Please remember that io:format() does not always respect processes order
%% and that as a general rule printing on the standard output from concurrent
%% jobs is not very useful. Here I do this just to show that processes have been
%% spawned. Remember to define debug false if you create a ring of more than
%% 30-40 processes.

%% This spawns the first process, registering it under the name
%% 'ring_recursion_no_messages'.
%% Note that spawn() does never fail, but register may; if spawning the
%% first process fails an exception is raised and the process calling
%% start/1 is terminated.
start(NumberProcesses) ->
    register(ring_recursion_no_messages, spawn(?MODULE, create,
                                               [NumberProcesses])).

%% Function create/1 manages the single process creation; in this version the
%% only thing a process shall do is to spawn another process. The exit clause
%% is when the proceses counter reaches 1 (that is, the current process is the
%% last one): the last process uses 'ring_recursion_no_messages' as its next
%% process, thus closing the ring.
%% Here we use the only parameter(NumberProcesses) as the index for the current
%% process, so we end when the parameter is 1. If we want to think of it as the
%% number of REMAINING processes the only changes are that the exit clause is
%% create(0) instead of create(1), and that the first process is spawned in
%% start/1 with [NumberProcesses - 1] as argument. It is only a matter of taste.
%% Since all processes in the ring are meant to work together we are linking
%% them together to ensure that if one fails all others are terminated.
create(1) ->
    ?DEBUG("Process ~p created and connected with ~p (last)~n",
           [self(), whereis(ring_recursion_no_messages)]),
    loop(ring_recursion_no_messages);
create(NumberProcesses) ->
    Next = spawn_link(?MODULE, create, [NumberProcesses -  1]),
    ?DEBUG("Process ~p created and connected with ~p ~n", [self(), Next]),
    loop(Next).

%% In this version function loop/1 sits down and does nothing.
loop(NextProcess) ->
    loop(NextProcess).

source code

The recursive ring with a single message travelling.

%%% Author:  Leonardo Giordani <giordani.leonardo@gmail.com>
%%% Description: Implements a simple ring of processes with a single
%%%              message traveling once through the ring.
%%% Created: Jun 2013 by Leonardo Giordani

-module(ring_recursion_single_message).

%% Include some debug macros
-define(debug, true).
-include("debug.hrl").

%% User interface
-export([start/2]).

%% Private exports
-export([create/2, loop/1]).

%% This version of the processes ring program starts the processes in a
%% recursive way. Each process spawns the next one and links to it.
%% The last process then injects a single message in the ring, sending it to the
%% first process.

%% This spawns the first process passing the requested message to it
start(NumberProcesses, Message) ->
    register(ring_recursion_single_message, spawn(?MODULE, create,
                                                  [NumberProcesses, Message])).

%% Function create/2 manages the single process creation; in this version each
%% process just spawns another process, except the last one, which shall inject
%% the message in the ring and terminate.
create(1, Message) ->
    ?DEBUG("Process ~p created and connected with ~p (last)~n",
           [self(), whereis(ring_recursion_single_message)]),
    ?DEBUG("Process ~p injects message ~p~n", [self(), Message]),
    ring_recursion_single_message ! Message;
create(NumberProcesses, Message) ->
    Next = spawn_link(?MODULE, create, [NumberProcesses -  1, Message]),
    ?DEBUG("Process ~p created and connected with ~p ~n", [self(), Next]),
    loop(Next).

%% Now loop/1 blocks each process making it wait for a message to pass along.
loop(NextProcess) ->
    receive
        Msg ->
            ?DEBUG("Got message ~p, passing it to ~p~n", [Msg, NextProcess]),
            NextProcess ! Msg,
            ok
    end.

source code

The recursive ring with support for multiple message travels.

%%% Author:  Leonardo Giordani <giordani.leonardo@gmail.com>
%%% Description: Implements a simple ring of processes with a single
%%%              message traveling multiple times through the ring.
%%% Created: Jun 2013 by Leonardo Giordani

-module(ring_recursion_multiple_messages).

%% Include some debug macros
-define(debug, true).
-include("debug.hrl").

%% User interface
-export([start/3]).

%% Private exports
-export([create/3, loop/2]).

%% This version of the processes ring program starts the processes in a
%% recursive way. Each process spawns the next one and links to it.
%% The last process then injects a message in the ring, sending it to the
%% first process and when getting the message from the second to last, it
%% injects it again in the ring a given number of times.

%% This spawns the first process passing it the requested message and the number
%% of times the message shall travel through the ring.
start(NumberProcesses, Message, NumberMessages) ->
    register(ring_recursion_multiple_messages, spawn(?MODULE, create,
                          [NumberProcesses, Message, NumberMessages])).

%% Function create/3 manages the single process creation; in this version each
%% process spawns another process and then loops, except the last one, which
%% shall inject the message in the ring before looping. Since the last process
%% starts the message passing it loops an already decremented number of times.
create(1, Message, NumberMessages) ->
    ?DEBUG("Process ~p created and connected with ~p (last)~n",
           [self(), whereis(ring_recursion_multiple_messages)]),
    ?DEBUG("Process ~p injects message ~p~n", [self(), Message]),
    ring_recursion_multiple_messages ! Message,
    loop(ring_recursion_multiple_messages, NumberMessages - 1);
create(NumberProcesses, Message, NumberMessages) ->
    Next = spawn_link(?MODULE, create,
                      [NumberProcesses -  1, Message, NumberMessages]),
    ?DEBUG("Process ~p created and connected with ~p ~n", [self(), Next]),
    loop(Next, NumberMessages).

%% Function loop/2 needs two clauses now. The first one is the exit condition,
%% when the forward counter reaches 1: the process forwards the message and
%% terminates. The second clause is the standard behaviour: the process forwards
%% the incoming message and loops again with a decremented counter.
loop(NextProcess, 1) ->
    receive
        Msg ->
            ?DEBUG("Got message ~p, passing it to ~p and terminating~n",
                   [Msg, NextProcess]),
            NextProcess ! Msg,
            ok
    end;
loop(NextProcess, NumberMessages) ->
    receive
        Msg ->
            ?DEBUG("Got message ~p, passing it to ~p (still ~p time(s))~n",
                   [Msg, NextProcess, NumberMessages]),
            NextProcess ! Msg,
            loop(NextProcess, NumberMessages - 1)
    end.

source code

The recursive ring exposing the functional API.

%%% Author:  Leonardo Giordani <giordani.leonardo@gmail.com>
%%% Description: Implements a simple ring of processes that exposes a
%%%              functional interface to send messages.
%%% Created: Jun 2013 by Leonardo Giordani

-module(ring_recursion_functional).

%% Include some debug macros
-define(debug, true).
-include("debug.hrl").

%% User interface
-export([start/1, stop/0, send_message/1, send_message/2]).

%% Private exports
-export([create/2, loop/1]).

%% This version of the processes ring program starts the processes in a
%% recursive way. Each process spawns the next one and links to it.
%% The module exposes a functional interface to send messages in the ring
%% and to terminate it.

%% This spawns the first process. The process is registered as in the previous
%% recursive versions to allow the last process to connect with the first.
%% Moreover, exposing a functional interface needs a global entry point, that is
%% the name of the first process.
start(NumberProcesses) ->
    register(ring_recursion_functional, spawn(?MODULE, create,
                                              [NumberProcesses, self()])),
    receive
        ready ->
            ok
    after 5000 ->
            {error, timeout}
    end.

%% Functional interface: terminate the ring
stop() ->
    ring_master_functional ! {command, stop}.

%% Functional interface: send a message that travels through the ring once
send_message(Message) ->
    send_message(Message, 1).

%% Functional interface: send a message that travels through the ring
%% Times times
send_message(Message, Times) ->
    ring_recursion_functional ! {command, message, [Message, Times]}.

%% Function create/2 manages the single process creation; in this version each
%% process just spawns another process, except the last one, which connects with
%% the first one. The Starter process is passed to allow the ring to send a
%% ready message.
create(1, Starter) ->
    ?DEBUG("Process ~p created and connected with ~p (last)~n",
           [self(), whereis(ring_recursion_functional)]),
    Starter ! ready,
    loop_last(ring_recursion_functional);
create(NumberProcesses, Starter) ->
    Next = spawn_link(?MODULE, create, [NumberProcesses -  1, Starter]),
    ?DEBUG("Process ~p created and connected with ~p ~n", [self(), Next]),
    loop(Next).

%% Function loop_last/1 rules the behaviour of the last process of the whole
%% ring). When it receives a stop command it just terminates since the message
%% has already been received by all processes in the ring. When it receives a
%% message command if Times is 1 the message is thrown away (it just travelled
%% through the ring for the last time), otherwise is is injected again in the
%% ring with a decremented Times value.
loop_last(NextProcess) ->
    receive
        {command, stop} ->
            ok;
        {command, message, [_Message, 1]} ->
            loop_last(NextProcess);
        {command, message, [Message, Times]} ->
            ?DEBUG("Got message ~p, passing it to ~p (still ~p time(s)~n",
                   [Message, NextProcess, Times]),
            NextProcess ! {command, message, [Message, Times - 1]},
            loop_last(NextProcess)
    end.

%% Function loop/1 rules the behaviour of the standard processes.
%% When a process receives a stop command it forwards it and
%% terminates. When it receives a message command it just forwards it.
loop(NextProcess) ->
    receive
        Msg = {command, stop} ->
            NextProcess ! Msg,
            ok;
        Msg = {command, message, [_Message, _]} ->
            ?DEBUG("Got message ~p, passing it to ~p~n",
                   [_Message, NextProcess]),
            NextProcess ! Msg,
            loop(NextProcess)
    end.

source code

Conclusions¶

The process ring is an exercise that can be solved in many ways (I just presented the two more straightforward ones) but makes the programmer face problems that may later rise in real-world applications. For this reason, it is an invaluable sandbox where the Erlang programmer can try different approaches to solve both the concurrency and the topology problems.

Keep in touch for other Erlang articles on this page.

Error handling in Erlang - a primer

2013-05-30T11:41:00+01:00

Abstract¶

This article aims to summarize Erlang error handling both in sequential and in concurrent environments. The targets of this article are novices that, like me, make their first steps into the beautiful world of Erlang. Moreover, I always find that trying to explain things makes me understand them better.

Disclaimer (for Erlang gurus): I'm a complete Erlang novice so please be indulgent with me while you read my thoughts. Corrections and suggestions are welcome!

Introduction¶

Recently I started studying Erlang; coming from a pure imperative background (C, C++, and Python) I have to say that Erlang has been a surprise and a joy. I did not find something so innovative since long, even if the pure functional part of the language was not totally new since it is available in Python too.

The concept of runtime system, with a support for concurrency built in the language itself, the pattern matching idea and the recursion as a way to implement loops are all very intriguing, so learning them is fun (pun intended, if you do not get it review Erlang anonymous functions).

One of the innovative concepts that ploughed through my imperative mind was that of defensive programming under its formulation in the Erlang tenet "Let it crash". This was something new, partly because I rarely found advice on system organization while learning the foundations of a programming language and partly because about 80% of the code I write has the task of avoiding programs to crash.

I found this very interesting post of Mazen Harake on the subject. Basically, he clarifies that the Erlang philosophy is not that of just let errors happen and propagate: the point is that the programmer should deal only with errors that are documented. This means that the code specification includes that error as a possibility. Well, it is not my intention to state something already well explained: go and read Mazen's post.

Anyway, before discussing the Erlang way of dealing with code errors, it is necessary to firmly grasp syntax and structures that the language provides.

Exceptions in Erlang¶

The simplest way to make something go wrong when dealing with a computer is to treat it like a sentient being. Joking apart, a good way to crash a program is to execute a division by zero.

1> 1/0.
** exception error: bad argument in an arithmetic expression
     in operator  '/'/2
        called as 1 / 0
2>

As you can see Erlang does not get mad at your provocation and simply raises and exception, i.e. signals that something went wrong, giving some details about why and where it happened. This is not different from what other languages, like Python, do.

>>> 1/0
Traceback  (most recent call last):
  File "<stdin>", line 1, in <module>
ZeroDivisionError: integer division or modulo by zero
>>>

So exceptions are in Erlang, as in other languages, a "reserved channel" the language uses to propagate errors. If you code in C, you cannot leverage something like exceptions and must rely on return values. This means that you have to format the results of your functions so that they can host a wrong result too and end up returning an int with the error code while functions results are managed by reference.

Back to Erlang. Exceptions crash you program, i.e. they make your program immediately stop, reporting the error to the system process that executed it (usually the OS GUI or a textual shell). Indeed the Erlang shell crashed when you tried to reach for the infinity.

1> self().
<0.32.0>
2> 1/0.
** exception error: bad argument in an arithmetic expression
     in operator  '/'/2
        called as 1 / 0
3> self().
<0.35.0>
4>

Remember that in Erlang self() gives you the Erlang PID of the process (not the operating system's one). As you can see the Eshell process crashed and was restarted by some magic behind the scenes.

I hear you mumble that, well, world is not perfect, and errors happen. So how do we deal with this? Shall we crash every time something wrong occurs in our code or is there some way to unravel the knot and happily continue running the code?

This seems to be a question that even Erlang creators wanted to answer, and their answer was: you can, but you shouldn't always. By now, let us drop the "you shouldn't always" part and learn the basics; you can stop an exception before it crashes your pretty program. Before diving into stopping exceptions techniques, let me review the types of exception you may encounter.

Exception types¶

There are three types (or classes) of exceptions in Erlang: throw, error, and exit.

The first, throw, identifies an exception that a called function voluntarily raises (throwing it at you); such exceptions shall be documented, i.e. the documentation of the function you are calling shall state that this exception may be raised and specify under what conditions this may happen. "Shall" here means that if the programmer does not document the exception all sorts of curses will be casted on his or her code forever. As a Python programmer I strongly advice you to read "shall" as "must".

The second exception type, error, signals that something very bad happened in the system, something that was unexpected to the author of the code raising the exception. Even if this type of exception can be raised explicitly, it is usually raised by the Erlang run-time system. If you recall the first example of this post, the division by zero, you can now understand why the shell printed "exception error"; that exception has not been raised by an instruction in the code of the Erlang shell, but from the run-time system itself. This type of exception also contains a stack trace, but I will not cover it in this article.

The third and last exception type, exit, means that your code is being told to stop immediately.

As you can see the real difference between the three types is the communication intent, not a special behaviour. So from the pure theoretical point of view an error exception can be replaced by a throw exception without any side effect. Obviously, the communication intent is not negligible: indeed, as explained before, throw exceptions are usually documented while errors are not intended for being formalized.

In addition to a class, exceptions encompass a reason, that is a valid Erlang item (an atom, an integer, a pid, ...). The reason carries the explanation of the exception, i.e. a detailed insight in what really happened.

Dealing with exceptions¶

Now that we got acquainted with Erlang exception types we may step further into exception management structures and learn how to stop exceptions from crashing our programs. The way of managing exceptions raised by a function called in our code should be familiar to Python, C++, java and Ruby developers (and to many others, probably); the basic Erlang syntax is

try <expressions> of
  <result_pattern_matching>
catch
  <exception_pattern_matching>
after
  <after_expressions>
end

The large part of this structure is well known. Here, <expressions> is a series of Erlang expressions, comma-separated as usual; the <result_pattern_matching> part is a classical Erlang pattern matching structure, just like that you write in a case construct; last, the <after_expressions> is a series of Erlang expressions. The <exception_pattern_matching> part has a slightly new syntax we will cover in a moment.

The structure works like in other languages: the <expressions> code is evaluated and the result is pattern matched against <result_pattern_matching> and the result is returned by the whole try statement. If an exception is raised when evaluating <expressions>, it is pattern matched against the code listed in <exception_pattern_matching> and the relative code is executed. Regardless of what happens in the try/catch part the code in <after_expressions> is executed, and its result is not returned.

The exception matching code has a syntax that is very similar to that of the usual pattern matching, but exceptions are listed in the new form ExceptionType:Reason, where type and reason have been already described in the previous section. When the exception is a run-time error the reason is one of the values listed here.

So the complete form of a try/catch statement in Erlang is the following, where Expressions is always a comma-separated list of Erlang expressions.

try Expression1,...,ExpressionN of
  Pattern1 [when Guard1] -> PatternExpressions1;
  Pattern2 [when Guard2] -> PatternExpressions2;
  ...
  PatternN [when GuardN] -> PatternExpressionN
catch
  ExceptionType:Reason1 [when ExceptionGuard1] -> ExceptionExpressions1;
  ExceptionType:Reason2 [when ExceptionGuard2] -> ExceptionExpressions2;
  ...
  ExceptionType:ReasonN [when ExceptionGuardN] -> ExceptionExpressionsN
after
  AfterExpressions
end.

Exceptions pattern matching allows the use of the do-not-care variable _ not only for reasons but for also for types. So the following syntax catches all exceptions of type ExceptionType

...
catch
  ExceptionType: _ -> ExceptionExpressions1;
end

while the following catches all exceptions

...
catch
  _:_ -> ExceptionExpressions1;
end

I will not cover here the old-style error handling mechanism with catch; the interested reader can find it documented here.

Returning values from try/catch statements¶

Try/catch statements return the value of the last expression executed, that is one of PatternExpression1,...,PatternExpressionN if no exception is raised, or one of ExceptionExpressions1,...,ExceptionExpressionsN. This means that we can assign the value of the whole expression to a variable

Result = try Expression of ... end.

Remember that AfterExpressions are always executed, but their final value is not returned by the statement.

Since a lot of times you want to return the result of the expression after the try keyword, you can omit the of part

try Expressions
catch
 ...
end.

Raising exceptions¶

Erlang provides three different BIFs to raise exceptions, profitably called like the exception type they raise: throw/1, erlang:error/1, and exit/1. As you see, error/1 is not automatically imported by the system and must be called in its full form. This is a hint for us programmers: error/1 is there and can be used, but it is not something you should need often; otherwise, you misunderstood what an error exception is in Erlang.

So in most cases, if your code encounters an error condition and you need to raise an exception, you end up using throw/1 or exit/1. The argument of these functions is the reason of the exception: remember that you can format the reason you attach to your exception to match your needs, you only need to document it.

my_function(Somebadvalue) -> throw({badvalue, Somebadvalue}).

Pay attention, however, that exceptions are a double-edged sword; the fact that they return values through a reserved channel is powerful, but can lead to subtle bugs and to long debug sessions. The advice in Erlang is to spare throw() for some special cases and to try always to communicate the failure through standard function results. This is, however, part of a coding philosophy that cannot be examined in depth here.

Exceptions and exit signals¶

Erlang is a run-time system, not just a language; as such, it has built-in structures and concepts that are usually provided by libraries in other languages. One of these concepts is the dependency between processes, which may be realized through links (and monitors, but I am not going to introduce them in this article). Process linking in Erlang means a very simple thing: when two processes are linked they die together, i.e. when one of the two terminates abnormally the other one is terminated too. A process can link to more than one other process, and the dying behaviour is propagated among all them.

What is the point of this structure? In Erlang, you are encouraged to spawn processes to accomplish tasks, even the simplest ones. Thus, you can easily end up with a multitude of processes working together to perform some action, and if one of them crashes it is likely that others should exit too, being them dependent from it. This is not mandatory; it is all up to you to decide what processes have to be linked, but if they are they must die together.

How is this accomplished? Linked processes are connected by a hidden communication channel, which carries information about their termination with so-called exit signals. Exit signals are invisible to the programmer, and when a process receives one of them it simply terminates, spreading the news under the form of other exit signals.

Process termination and reasons¶

Exit signals are always sent when a process dies, but they carry a reason for its termination, just like exceptions. This reason is very important for the subsequent events concerning processes linked to the dying one.

First of all let us look at process termination. A process in Erlang can terminate normally or abnormally: the former happens when it has no more code to execute or when it raises an exception passing as reason the atom normal; the latter occurs when a raised exception has a reason different from the atom normal.

So an exit signal process contains either the atom normal or another reason, and it travels from the terminating process to each linked process. When it hits one of them, if the reason is not normal the process is terminated and sends its own exit signals to its linked processes with the same reason of the incoming one. The result is that the entire network of linked processes terminates automatically.

The best way to terminate a process with a reason is to execute the BIF exit(Reason). This BIF has also the form of arity 2 where you pass the pid of a process exit(Pid, Reason): the addressed process will get an exit signal with the given reason. A caveat: when using exit/1 the exit signal will contain the pid of the terminating process, when using exit/2 the exit signal will contain the pid of the target process.

Stopping exit signals¶

Having a way to stop an entire group of processes when one of them crashes is a big benefit, but it could be a good thing to be able to stop the propagation somewhere. Obviously, if processes are not linked they do not influence each other when terminating, but this also means that no one notices that a process terminated, which in turn means that no one will be restarting it.

The whole point of linking processes is indeed the control over terminating processes. If a process runs there is a reason and if it crashes the system should investigate why it crashed and possibly restart it.

Erlang gives a process the chance to receive an abnormal exit signal from a process it is linked to, without forcing it to terminate: in Erlang speech this is called trapping exit signals or trapping exits. When a process traps exits the incoming exit signals coming from linked processes are converted by the run-time system into messages that the process can fetch with a receive construct. Thus, a process trapping exits can be notified that a linked process died without being affected by this.

A process can start trapping exits by executing the BIF process_flag(trap_exit, true). It is a best practice to call it at the beginning of the process and to avoid turning it off during the execution since it makes the system difficult to debug.

Once the BIF has been executed, an exit signal with the reason Reason coming from another process is converted in an incoming message under the form {'EXIT', Pid, Reason}, where Pid is the pid of the terminated process.

Unstoppable exit signals¶

Now we can convert a process so that it does not terminate with its linked processes. This has a downside: if the process contains errors such as infinite loops or if for some reason we need to stop the entire system, the processes that trap exits cannot be stopped. For this reason, Erlang provides the special atom kill as a reason for an exit signal.

An exit signal containing the reason kill cannot be trapped; thus the exit signal is unstoppable. Unconditionally terminating the entire network is however something dangerous, so when a process terminates because of an incoming kill exit signal it will send to its linked processes a killed exit signal, which can be possibly trapped.

Conclusions¶

The aim of the article was to give an overview of error handling in Erlang: not everything has been covered, but a novice should find here almost everything he or she needs to step into this part of the language.

There is obviously much more than this in the Erlang treasure chest. If the whole link and exit signals stuff thrilled you like it did with me, I promise you that OTP behavious (sorry, behaviors) will take your breath away.

Keep in touch for other Erlang articles on this page.

Concurrent programming - 6

2013-04-23T08:29:00+02:00

Abstract¶

Issue 5 of this series ended with a small program where two processes exchanged ten numbers through a message queue, thus being a synchronized producer-consumer couple. This time I am going to show and comment the code of a very simple communication simulator written in C. The code leverages IPC queues to allow multiple processes to talk each other while running concurrently.

The simulator¶

The program simulates a messaging switch, where multiple users (child processes) connect to a central switch (parent process) and send text messages (SMS-like) to other users through it. The switch receives messages and route them if the recipient is reachable (process is running). Moreover, the switch can perform a timing operation on a user, checking how much time the user needs to answer a message, or select it for termination. Last, the switch counts how many times a user sends a message to an unreachable user and when a threshold is reached terminates it. Switch and user decisions are taken extracting pseudo-random numbers and comparing them with thresholds given on the command line. Remember that the simulation wants to demonstrate a concrete use of message queues and does not claim to be a complete communication system between processes.

10,000 feet overview¶

I splitted the simulator in 3 parts, namely two stacked layers of function and the main application code. The first layer implements the message structure, provides functions to interact with it (getters and setters), exports basic functions to manage queues and to send and receive messages. The second layer exports functions that implement the protocol, i.e. the actions users and switch can perform. Last, the main function contains the actual logic of the whole simulation. Due to the instructional purpose of the simulation some of the solution implemented are far from optimal or even correct (e.g. random numbers management); I suggest you to try to evolve the simulation, adding new services or messages between users.

Debugging nightmares¶

Be ready to spend some time debugging your multiprocessing applications! If you ever debugged some code, you know that the most friendly bugs are by far those which are reproducible: you only need to add some debugging output or step through the code with the debugger and the bug is spotted. Multiprocessing and network application are at the very opposite corner: most of the time the bugs in that sort of applications are very difficult to reproduce, when not impossible. The concurrent execution of multiple processes, indeed, makes every execution unique, since the actual execution sequence is random (random means that it is the result of so many factors that the result is unpredictable and thus irreproducible). Network programs suffers from the same problems (concurrent execution), worsened by network lags that add a factor of randomness.

Code and analysis¶

I am not going to include the whole code of the layers directly in the post text as they are a straightforward implementation of what was presented in the previous articles; the code of the main application is, on the contrary, analyzed block by block. The links to the source files are at the end of the article, together with compilation instructions.

Layer 1¶

Message structure¶

The structure of the message is

typedef struct
{
 int sender;
 int recipient;
 char text[160];
 int service;
 int service_data;
} message_t;

typedef struct
{
 long mtype;
 message_t mtext;
} messagebuf_t;

Here, sender and recipient are numbers that identify the switch and the users (0 is the switch, then users are numbered increasingly when they connect to the switch); text is the content of a message a user sends to another user, and is 160 characters long to mimic SMS behaviour. Last service is the identifier of some system operation, like a request the switch sends to the users; service_data carries the optional data the service needs to communicate. Actual services are implemented in layer 2 and I am going to describe them later.

Other functions¶

Layer 1 exposes many functions: some are simple set and get functions to deal with the message structure while five simplify access to IPC structures; build_key() makes an IPC key from a given character, create_queue() and remove_queue() manage IPC queues and last send_message() and receive_message() give a simple way to route messages.

A very simple error management code has been introduced here: C language does not allow to use exceptions, so errors have to be managed by functions or returned through the return statement. A solid error management in C is outside the scope of this article, so here you will find the bare minimum.

The error management of receive_message() needs a little explanation.

int receive_message(int qid, long type, messagebuf_t *qbuf){
  int result, length;
  length = sizeof(messagebuf_t) - sizeof(long);

  if((result = msgrcv(qid, (struct msgbuf *)qbuf, length, type, IPC_NOWAIT)) == -1){
    if(errno == ENOMSG){
      return 0;
    }
    else{
      perror("msgrcv");
      exit(1);
    }
  }

  return result;
}

Here, msgrcv() fails even when there are no messages of the given type, producing a ENOMSG error. This is the reason why that case has been ignored through the if(errno == ENOMSG) construct.

Layer 2¶

The protocol¶

Layer2 implements the actual communication protocol between the switch and the users. Users send text messages to other users, and those messages are routed by the switch. Users and switch can also send service messages; those are messages encompassing information used to manage the system.

Every user, when spawned, initializes a queue, connects to the switch and communicates the queue id. Since the queues can contain old unread messages, it is necessary to empty them before the use. Each user enters a loop where it sleeps a while, receives service messages, sends messages and last receives text messages. At the same time, the switch collects queue ids, routes text messages from the sender to the recipient and sometimes sends a service request to a user.

The services the switch can request from users are to terminate and to test the answering time. Both can be requested on a probability base, but the first is also forced when the user sends too many messages to unreachable recipients.

The answering time test is performed in a simple way: the switch requests the service and records the time of the request. The user answers with a message that contains the time at which it received the request and the switch computes the difference between the two. Pay attention that both the timing service and the random number extraction used to request services on a probability base are not exact; the code works but is good only for a demonstration system.

I defined some useful constants in the header file: MAX_SLEEP is the maximum number of seconds a user waits before performing an action while TYPE_SERVICE and TYPE_TEXT identify the type of message. The defines which name starts with SERVICE_ list all the possible services: SERVICE_TERMINATE forces a user to quit; SERVICE_TIME makes it perform a timing operation; SERVICE_CONNECT and SERVICE_DISCONNECT tell the switch a new user has connected or disconnected; SERVICE_QID bears the identifier of the user queue to the switch; SERVICE_UNREACHABLE_DESTINATION communicates a user that the recipient of a message is no more online.

Queues¶

Two functions are dedicated to queues, init_queue() and close_queue(). The first builds an IPC key from a given number (previously converting it to a char) and runs the create_queue() function from layer 1. Because of the char conversion the range of integers it can accept is 0-255, and for simplicity's sake there is no check in the whole code that a queue key has not yet been assigned. For this example, I am simply leveraging that different numbers return different keys and, thus, queues.

The second function closes the queue running the remove_queue() function from layer 1. The underlying system calls are not used directly to allow the future introduction of checks on the assigned queues.

User functions¶

Users have five functions that implement their part of the protocol. Two functions communicate to the switch that the user connected or disconnected, namely user_send_connect() and user_send_disconnect(); both carry the sender id and the switch id sw, which however in the main program is always 0. The user_send_qid() function communicates to the switch the queue id of the user; user_send_text_message() sends a string of text to another user and user_send_time() answers the timing service.

Switch functions¶

The switch can execute switch_send_text_message() to deliver a text message sent by a user; switch_send_terminate() to ask a user to terminate; switch_send_time() to ask a user to perform a timing service.

Code analysis¶

Now I will briefly review the whole application code to better explain the different parts of the system. Please remember that this is a demonstration system so many choices have been made for simplicity's sake.

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <linux/ipc.h>
#include <linux/msg.h>
#include <time.h>
#include <signal.h>
#include <wait.h>
#include "layer1.h"
#include "layer2.h"

#define MINCHILDS 1
#define MAXCHILDS 15

#define MAXFAILS 10

After the list of includes, you will find three defines that rule the number of child processes the system can spawn and the maximum number of messages a user can send to unreachable recipients before the switch asks it to terminate.

int random_number(int max)
{
  double r,x;
  r = (double) random();
  x = r * (double) max / RAND_MAX;
  return((int) x);
}

void usage(char *argv[])
{
  printf("Telephone switch simulator\n");
  printf("%s <number of users> <service probability> <text message probability>\n", argv[0]);
  printf("\n");
  printf("     <number of users> - Number of users alive in the system (%d - %d)\n", MINCHILDS, MAXCHILDS);
  printf("     <service probability> - The probability that the switch requires a service from the user (0-100)\n");
  printf("     <text message probability> - The probability the a user sends a message to another user (0-100)\n\n");
}

The random_number() function is used to extract a random number between 0 and a maximum max; RAND_MAX is a define of the standard library and represents the maximum number the random() function can return and in the GNU C library it is 2^31 (2147483647); here it is used to calculate a proportion with the maximum value given by the caller. The usage() function helps the user remembering the command line arguments; as you can see, the program receives 3 mandatory input values: the number of users that the switch can spawn; the probability that the switch requests a service to a user when the latter sends a message; the probability that a user sends a text message to another user.

int main(int argc, char *argv[])
{
  pid_t pid;
  int i;

  int users_number;
  int service_probability;
  int text_message_probability;

  int status;
  int deadproc = 0; /* A counter of the already terminated user processes */
  int qid;
  int sw; /* Qid of the switch */
  int dest; /* Destination of the message */
  int olddest; /* Destination of the previous message */

  int queues[MAXCHILDS + 1]; /* Queue identifiers - 0 is the qid of the switch */

  int msg_sender;
  int msg_recipient;
  char msg_text[160];
  int msg_service;
  int msg_service_data;

  int t;
  int timing[MAXCHILDS + 1][2];

  int unreachable_destinations[MAXCHILDS + 1];

  char *padding = "                                                                      ";
  char text[160];

  messagebuf_t msg, in;

Many of these variables are just helpers that simplify the code; queues holds the queue identifier of each process; timing holds the information about user timing service results; unreachable_destinations contains how many times each user sent a message to an unreachable recipient; dest and olddest are used to avoid a user to send a message to the recipient of the previous one. The padding variable is a quick and dirty way to create two columns, the left one filled by switch messages and the right one by user ones.

Remember that in C you have to declare variables at the beginning of the program while in C++ you can declare them anywhere. This means that child processes, a copy of the parent, carry in memory some variables such as queues and timings that are used only by the switch. This is both a waste of resources and a dangerous situation, so remember that in general statically allocated variables are not a good choice for concurrent programs.

  /* Command line argument parsing */
  if(argc != 4){
    usage(argv);
    exit(0);
  }

  users_number = strtol(argv[1], NULL, 10);  
  service_probability = strtol(argv[2], NULL, 10);
  text_message_probability = strtol(argv[3], NULL, 10);


  if((users_number < MINCHILDS) || (users_number > MAXCHILDS)){
    usage(argv);
    exit(1);
  }

  if((service_probability < 0) || (service_probability > 100)){
    usage(argv);
    exit(0);
  }

  if((text_message_probability < 0) || (text_message_probability > 100)){
    usage(argv);
    exit(0);
  }

  printf("Number of users: %d\n", users_number);
  printf("Probability of a service request: %d%%\n", service_probability);
  printf("Probability of a text message: %d%%\n", text_message_probability);
  printf("\n");

  /* Initialize the random number generator */
  srandom(time(NULL));

All these lines contain initialization code and checks for the values passed on the command line.

  /* Switch queue initialization */
  sw = init_queue(255);

  /* Read the last messages we have in the queue */
  while(receive_message(sw, TYPE_TEXT, &in)){
    printf("%d -- S -- Receiving old text messages\n", (int) time(NULL), i);
  }

  /* Read the last messages we have in the queue */
  while(receive_message(sw, TYPE_SERVICE, &in)){
    printf("%d -- S -- Receiving old service messge\n", (int) time(NULL), i);
  }

  /* All queues are "uninitialized" (set equal to switch queue) */
  for(i = 0; i <= users_number; i++){
    queues[i] = sw;
    unreachable_destinations[i] = 0;
  }

The switch initializes its queue; this has to be done before spawning users since the sw variable will be copied in each child process and used to communicate with the switch. The queue is initialized with the number 255 just to be sure that no child process initializes the same queue. As explained, IPC queues provide no mechanism to ensure uniqueness of the instanced queues, so we have to establish our own system; in this simple example we spawn a maximum of 15 users, so we could also use 16 or MAXCHILDS+1 to initialize the switch queue. Since queues are shared structures and nothing prevents the system to assign to a process a previously used queue we must ensure that the queue is empty, so the switch reads and discards all text and service messages found in its queue. Last, the queues and unreachable_destinations arrays are initialized.

  /* Create users */
  for(i = 1; i <= users_number; i++){
    pid = fork();

    if (pid == 0){
      srandom(time(NULL) + 1000*i);

      /* Initialize queue  */
      qid = init_queue(i);

      /* Read the last messages we have in the queue */
      while(receive_message(qid, TYPE_TEXT, &in)){
        printf("%s%d -- U %02d -- Receiving old text messages\n", padding, (int) time(NULL), i);
      }

      /* Read the last messages we have in the queue */
      while(receive_message(qid, TYPE_SERVICE, &in)){
        printf("%s%d -- U %02d -- Receiving old service messge\n", padding, (int) time(NULL), i);
      }

      /* Let the switch know we are alive */
      user_send_connect(i, sw);

      /* Let the switch know how to reach us */
      user_send_qid(i, qid, sw);

This code spawns the users and from here the code splits in two, and the code of the user processes is inside the if construct. As you can see, the user acts like the switch at the very beginning, initializing its own queue and flushing the possible messages it contains. After this, the user sends the switch a message to communicate that it is alive and sends its qid to allow the user to communicate.

      /* Enter the main loop */
      while(1){
        sleep(rand()%MAX_SLEEP);

        /* Check if the switch requested a service */
        if(receive_message(qid, TYPE_SERVICE, &in)){
          msg_service = get_service(&in);

          switch(msg_service){

          case SERVICE_TERMINATE:
            /* Send an acknowledgement to the switch */
            user_send_disconnect(i, getpid(), sw);

            /* Read the last messages we have in the queue */
            while(receive_message(qid, TYPE_TEXT, &in)){
              msg_sender = get_sender(&in);
              get_text(&in, msg_text);
              printf("%s%d -- U %02d -- Message received\n", padding, (int) time(NULL), i);
              printf("%s                      Sender: %d\n", padding, msg_sender);
              printf("%s                      Text: %s\n", padding, msg_text);
            }

            /* Remove the queue */
            close_queue(qid);
            printf("%s%d -- U %02d -- Termination\n", padding, (int) time(NULL), i);
            exit(0);
            break;

          case SERVICE_TIME:
            user_send_time(i, sw);
            printf("%s%d -- U %02d -- Timing\n", padding, (int) time(NULL), i);
            break;
          }
        }

The user loops infinitely and at each loop sleeps a random number of seconds. After that, it checks its message queue for service messages (receive_message() with the parameter TYPE_SERVICE); if the requested service is SERVICE_TERMINATE, an acknowledgement is sent to the switch (user_send_disconnect()) so that the user is marked as offline and no more messages are sent with it as recipient. The user then reads all remaining text messages in its queue and closes the queue. Last with exit()the user process terminates. If the requested service is SERVICE_TIME, the user simply sends the current time back to the switch through user_send_time().

        /* Send a message */
        if(random_number(100) < text_message_probability){
          dest = random_number(users_number + 1);

          /* Do not send a message to the switch, to yourself and to the previous recipient */
          while((dest == 0) || (dest == i) || (dest == olddest)){
            dest = random_number(users_number + 1);
          }
          olddest = dest;

          printf("%s%d -- U %02d -- Message to user %d\n", padding, (int) time(NULL), i, dest);
          sprintf(text, "A message from me (%d) to you (%d)", i, dest);
          user_send_text_message(i, dest, text, sw);
        }

        /* Check the incoming box for simple messages */
        if(receive_message(qid, TYPE_TEXT, &in)){
          msg_sender = get_sender(&in);
          get_text(&in, msg_text);
          printf("%s%d -- U %02d -- Message received\n", padding, (int) time(NULL), i);
          printf("%s                      Sender: %d\n", padding, msg_sender);
          printf("%s                      Text: %s\n", padding, msg_text);
        }
      }
    }
  }

The user extracts a random number and tests it against the probability given on the command line; if the test gives a positive result the user extracts a random user, avoiding the switch (dest == 0), itself (dest == i) and the recipient of the previous message it sent (dest == olddest). The new recipient is saved in olddest, and the message is sent through user_send_text_message(). This part ends the user code.

  /* Switch (parent process) */ 
  while(1){
    /* Check if some user is answering to service messages */
    if(receive_message(sw, TYPE_SERVICE, &in)){
      msg_service = get_service(&in);
      msg_sender = get_sender(&in);

      switch(msg_service){
      case SERVICE_CONNECT:
        /* A new user has connected */
        printf("%d -- S -- Service: connection\n", (int) time(NULL));
        printf("                   User: %d\n", msg_sender);
        break;

      case SERVICE_DISCONNECT:
        /* The user is terminating */
        printf("%d -- S -- Service: disconnection\n", (int) time(NULL));
        printf("                   User: %d\n", msg_sender);

        deadproc++;
        break;

      case SERVICE_QID:
        /* The user is sending us its queue id */
        msg_service_data = get_service_data(&in);
        printf("%d -- S -- Service: queue\n", (int) time(NULL));
        printf("                   User: %d\n", msg_sender);
        printf("                   Qid: %d\n", msg_service_data);
        queues[msg_sender] = msg_service_data;
        break;

      case SERVICE_TIME:
        msg_service_data = get_service_data(&in);

        /* Timing informations */
        timing[msg_sender][1] = msg_service_data - timing[msg_sender][1];

        printf("%d -- S -- Service: timing\n", (int) time(NULL));
        printf("                   User: %d\n", msg_sender);
        printf("                   Timing: %d\n", timing[msg_sender][1]);

        /* The user is no more blocked by a timing operation */
        timing[msg_sender][0] = 0;
        break;
      }
    }

The switch code has a structure similar to that of the user code. It loops infinitely with a while(1) statement, and inside this it checks for incoming messages and acts accordingly. The first thing it does is to check for service messages. If the service is SERVICE_CONNECT a user is communicating that it is alive, and the switch simply print out a log of this event; when a user disconnects it sends a SERVICE_DISCONNECT message and the switch increases the number of dead processes to monitor the number of active user processes; if the service is a SERVICE_QID a user is sending the switch its queue identifier and this is stored in the queues array; last, the SERVICE_TIME messages mean that a user is answering a timing request; its answer is compared with the recorded send time and the result is printed.

    /* Check if some user has connected */
    if(receive_message(sw, TYPE_TEXT, &in)){

      msg_recipient = get_recipient(&in);
      msg_sender = get_sender(&in);
      get_text(&in, msg_text);

      /* If the destination is connected */
      if(queues[msg_recipient] != sw){
        /* Send the message (forward it) */
        switch_send_text_message(msg_sender, msg_text, queues[msg_recipient]);

        printf("%d -- S -- Routing message\n", (int) time(NULL));
        printf("                   Sender: %d -- Destination: %d\n", msg_sender, msg_recipient);
        printf("                   Text: %s\n", msg_text);
      }
      else{
        unreachable_destinations[msg_sender] += 1;

        if (unreachable_destinations[msg_sender] > MAXFAILS) {
          continue;
        }

        printf("%d -- S -- Unreachable destination\n", (int) time(NULL));
        printf("                   Sender: %d -- Destination: %d\n", msg_sender, msg_recipient);
        printf("                   Text: %s\n", msg_text);
        printf("                   Threshold: %d/%d\n", unreachable_destinations[msg_sender], MAXFAILS);

        if (unreachable_destinations[msg_sender] == MAXFAILS) {
          printf("%d -- S -- User %d reached max unreachable destinations\n", (int) time(NULL), msg_sender);

          switch_send_terminate(queues[msg_sender]);

          /* Remove its queue from the list */
          queues[msg_sender] = sw;
        }
      }

Here, the switch checks the queue for incoming text messages users send to other users. If the recipient is connected (reachable) the message is routed through switch_send_text_message(); otherwise the counter of failed dispatches unreachable_destinations is incremented for that user; when the maximum number of failures MAXFAILS is reached the switch sends that user a termination request.

      /* Randomly request a service to the sender of the last message */
      if((random_number(100)  < service_probability) && (queues[msg_sender] != sw)){
        if (random_number(100) < 40){
          /* The user must terminate */
          printf("%d -- S -- User %d chosen for termination\n", (int) time(NULL), msg_sender);

          switch_send_terminate(queues[msg_sender]);

          /* Remove its queue from the list */
          queues[msg_sender] = sw;
        }
        else {
          /* Check if we are already timing that user */
          if(!timing[msg_sender][0]){
            timing[msg_sender][0] = 1;
            timing[msg_sender][1] = (int) time(NULL);
            printf("%d -- S -- User %d chosen for timing...\n", timing[msg_sender][1], msg_sender);
            switch_send_time(queues[msg_sender]);
          }
        }
      }
    }

Each user that sends a message through the switch can be selected as the destination of a service request. Here, the switch tests a random number against the probability given on the command line, ensuring that the users is not already unreachable. The probability of a termination service is hardcoded (40%), the other case being the timing service; when the latter is requested, the switch sends a message to the user recording the dispatch time so that it can be later compared with the user answer.

    else{
      if(deadproc == users_number){
        /* All childs have been terminated, just wait for the last to complete its jobs */
        waitpid(pid, &status, 0);

        /* Remove the switch queue */
        remove_queue(sw);

        printf("\n");
        printf("No more active users. Switch turns off.\n");

        /* Terminate the program */
        exit(0);
      }
    }
  }
}

When there are no incoming messages the switch checks if the number of dead processes is equal to the number of users; when this happens the switch waits for each child process (to avoid zombie processes), removes its queue and terminates.

Files and compilation¶

The 5 files of this small program can be downloaded here:

and can be compiled with the following command line

gcc -o ipc_demo main.c layer1.c layer2.c

A typical execution can be obtained running the program with the following parameters

./ipc_demo 8 30 80

Remember that the output lines of the processes are mixed and generally not in order; indeed, you can find the answer of a user printed before the switch request. Timestamps can help you find the right order, but the resolution of the time() function is a second, and in such a time span many messages can be sent.

Conclusions¶

This article ends by now this little series on concurrent programming in C and IPC structures. As you can see C is not the best language to implement concurrent programming concepts, due to its very low level nature. However, since many OSs are written in C (and/or C++) knowledge of the way this language can provide concurrent execution is useful.

There is much left to say about concurrent programming structures: atomicity, mutual exclusions, threads, monitors are just some of the most important. I will post new articles on such topics in the future, perhaps showing their implementation in other languages.

Concurrent programming - 5

2013-02-28T17:49:00+02:00

Abstract¶

In the past articles we introduced the concept of concurrent programming and studied a first solution to the problem of intercommunication: semaphores. As explained, the use of semaphores allows us to manage access to shared resources, and this is a first and very simple form of synchronization. In this issue, we will go one step further introducing a richer tool for concurrent programming: message queues.

Limits of semaphores¶

Ruling access to a resource through semaphores is a good form of synchronization, even if it is very limited. Indeed it prevents two or more processes simultaneously modifying shared data but does not allow them to exchange information.

Moreover, the basic use of semaphores does not guarantee the absence of dangerous situations. As an example remember that, in a multitasking environment, we do not know in advance which process will be executed and when. This means that a resource could be inaccessible for a process since it is always blocked by other processes.

Thus, while being a good tool in the multitasking programmer's bag, semaphores are not the final solution to his or her problems. So we now introduce a new tool which plays a big role in multitasking and distributed systems called message queues.

Message queues theory¶

Since message queues are one of the most important structures in computer science, let us dive into them at a slow pace. So first of all, what is a message in this context? Simply stated, a message is some data with a given format, which is a complex form to say everything. Indeed in the context of communication between processes, or computers, a message is a stream of bits which must be formatted according to a previously agreed format, where format means a set of rules the stream of bit must obey.

For example, we could simply state that a message is a raw extended ASCII string: this means that the stream must contain a multiple of 8 bits (since each extended ASCII character is represented by 8 bits). Our format is very simple, in this case, but it is enough to state that a message made of 377 bit is invalid. We can also make a message encompass entire files, and there is in general no limit to the size.

Now we have a definition of message. What is a queue? A queue is a list structure, where a given number of homogeneous objects can be stored. An object can be inserted in the list (pushed) and extracted from it (popped). Usually a queue in the messaging environment is a FIFO (First In First Out) one, meaning that the first object pushed is the first object popped. There is a plenty of mathematical and computer science theory about queues, buffers, FIFO, LIFO and other list types, and we will not review such topics here. Just a notice, to open further research by the reader: the stack, one of the most important concepts in programming, is also a memory buffer, or a queue, where the computer stores function calls and local variables.

So message queues are buffers where arbitrary (but homogeneous) objects called messages can be stored for a later retrieval.

Every process can create one or more queues and every process can send a message to one of them. The only prerequisite is that the unique identifier of the queue must be known. Since we are now working with processes spawned by a fork operation, each process can know the identifier of a queue if the queue has been created before executing fork().

The knowledge of a queue identifier makes it is also possible to read messages from it. Messages can be accessed sequentially, reading the messages in chronological order (from the oldest, the first, to the most recent, the last arrived), but selectively, that is considering only the messages of a certain type: this last feature give us a sort of control on the priority of the messages we read.

Queues, therefore, can be used to implement several communication scenarios, from the simplest ones where all processes send messages to and read messages from the same queue, to more complex solutions such as a full mail system between processes.

Irrespective of the communication framework we set for our processes, synchronization can now be performed through a richer tool. Processes needing synchronization can exchange messages to establish the correct timings schedule when performing actions. Messages, thus, do not replace semaphores, but, as already seen, cover functionalities that they cannot provide.

Before we can switch to implement message queues in C language, it is necessary to speak about another problem related to messages: the need of a communication protocol.

Creating a protocol¶

A protocol is a set of rules which control the interaction of elements in a set. Every time you must regulate the exchange of information between two or more actors in a scenario you need a protocol, even when dealing with humans and not computers.

In the past article we implemented a very simple protocol when instructing processes to access a resource according to the status of a semaphore. The latter is also modified as part of the protocol itself.

As already stated in a past article, there is no difference between interprocess communication on the same machine or in a distributed environment (multiple machines): indeed, every network protocol (TCP/IP, DNS, SMTP, just to cite some of the most famous) is built on a message exchange architecture.

This is a simple example of a protocol based on message exchange: two processes, A and B, are executing concurrently and processing different data. Once they end the processing they have to merge their results. Here, synchronization problems arise: what process actually does the merging? If the process in charge of the merging operation is the process A, for example, the merging protocol can be described by these time charts, one for each process

PROCESS B¶

Work with your data
When you finish send a message to A
When A answers, begin sending it your results

PROCESS A¶

Work with your data
Wait for a message from B
Answer the message
Receive data and merge them with yours

Choosing the merger process is in this case totally arbitrary. This, however, can be a very important part of a protocol.

This protocol is extensible ina simple way to the case of n processes: every process but A works with its own data and then send a message to A. When A answers, the other process sends its results: the only process modified in this generalization is A since it has to receive messages from multiple processes instead of just one.

System V Message Queues¶

Now it is the time to speak about implementing these concepts in a Linux operating system environment. As already said we have a set of primitives that allow us to manage the structures related to message queues, and they are very similar to those dedicated to semaphore management, already described in the past article.

The structure used to describe a message is msgbuf and is declared in linux/msg.h

/* message buffer for msgsnd and msgrcv calls */
struct msgbuf {
   long mtype;         /* type of message */
   char mtext[1];      /* message text */
};

The field mtype represents the type of the message and is a strictly positive number: the correspondence between numbers and message types has to be set in advance and is part of the protocol definition.

The second field represents the content of the message. With the standard definition, no real message can be held inside it because the space is too little (just one byte). This is just a placeholder since the real structure used to describe messages can be redefined in order to contain complex data. An example of redefinition is the following

struct message {
   long mtype;         /* message type */
   long sender;        /* sender id */
   long receiver;        /* receiver id */        
   struct info data;   /* message content */
};

As you can see this is not a redefinition in the sense of object inheritance, being C a pure procedural language. The redefined structure must obey two simple rules: the first field must be a long mtype and the maximum size dimension shall be 8192 bytes.

This last limit is hard coded in the Linux kernel, i.e. there is a C #define directive in linux/msg.h which states

#define MSGMAX  8192   /* <= INT_MAX */   /* max size of message (bytes) */

In the same file, there are other interesting limits such as MSGMNB, the maximum total size of a queue, which in Linux is set by default to 16384. That means that a queue containing 2 messages is already full.

A short notice about kernel parameters such as MSGMAX, MSGMNB and others: kernel parameters can obviously be changed by modifying the source code and recompiling the kernel, that is generating a custom Linux kernel. Varying some parameters, however, can be dangerous for the portability of your programs. Indeed, if you increase the maximum size of a message to, say, 16384 bytes, in order to host big messages, your application will crash on a kernel without that modification, where the maximum size is still 8192. Kernel parameters can also be changed at runtime through the /proc filesystem and the sysctl interface: these topics are very rich and important and thus deserve a larger space. For the time being, I let the reader research about them.

Create a queue¶

To create a new queue a process should call the msgget() function

int msgget(key_t key, int msgflg)

which receives as arguments an IPC key (see issue 4) and some flags, which by now can be set to IPC_CREAT | 0660 (create the queue if it does not exist and grant access to the owner and group users). The returned integer is called queue identifier and is unique in the system.

Send messages¶

To send a message to a queue we call the msgsnd() primitive

int msgsnd(int msqid, struct msgbuf *msgp, int msgsz, int msgflg)

where msqid is the identifier of the queue, msgp is a pointer to the message we have to send, msgz the size of the message in bytes (excluding the length of the mtype field, which is 4 bytes) and msgflg a flag related to the waiting policy.

Here, msgp is a pointer to a struct msgbuf. However, since that structure has been redefined, our actual call will contain a pointer to a variable of the redefined type.

The length of the message in bytes can be easily be found as

length = sizeof(struct message) - sizeof(long);

where struct message is our redefinition of struct msgbuf.

Waiting policies are similar to those introduced for semaphores. This time the policy is used when the queue is full. If msgflg is set to IPC_NOWAIT, the sender process will not wait for some available space and will exit with an error code.

Read messages¶

To read messages contained in a queue we use the msgrcv() system call

int msgrcv(int msqid, struct msgbuf *msgp, int msgsz, long mtype, int msgflg)

where the msgp pointer identifies the variable where we will copy the message read from the queue and mtype identifies the subset of messages we want to consider. As before, msgp should be a pointer to a variable of our redefined type and both msgz and msgflg have the same meaning of their counterparts in msgsnd().

Delete a queue¶

Last, a queue can be removed calling the msgctl() primitive

int msgctl(int msqid, int cmd, struct msqid_ds *buf);

where the command cmd to remove the queue is IPC_RMID. Other two commands, IPC_STAT and IPC_SET, are available through msgctl(), but they are not interesting now.

Putting it all together¶

Let's test all these concepts with a simple program which creates a message queue, sends a message to it, reads the message and cancels the queue. After the message has been read, a comparison between the original values and the ones in the message is performed to check that the system is working.

#include <stdio.h>
#include <stdlib.h>
#include <linux/ipc.h>
#include <linux/msg.h>

/* Redefines the struct msgbuf */
typedef struct mymsgbuf
{
  long mtype;
  int int_num;
  float float_num;
  char ch;
} message_t;

int main()
{
  int qid;
  key_t msgkey;

  message_t sent;
  message_t received;

  int length;

  /* Initialize the seed of the pseudo-random number generator */
  srand (time (0));

  /* Length of the message */
  length = sizeof(message_t) - sizeof(long);

  msgkey = ftok(".", getpid());

  /* Create the queue*/
  qid = msgget(msgkey, IPC_CREAT | 0660);

  printf("QID = %d\n", qid);

  /* Build a message */
  sent.mtype = 1;
  sent.int_num = rand();
  sent.float_num = (float)(rand())/3;
  sent.ch = abs(rand())%26 + 97;

  /* Send the message */
  msgsnd(qid, &sent, length, 0);
  printf("MESSAGE SENT\n");

  /* Receive the message */
  msgrcv(qid, &received, length, sent.mtype, 0);
  printf("MESSAGE RECEIVED\n");

  /* Control that received and sent messages are equal */
  printf("Interger number = %d (sent %d) -- ", received.int_num, sent.int_num);
  if(received.int_num == sent.int_num) {
    printf(" OK\n");
  }
  else {
    printf("ERROR\n");
  }

  printf("Float number = %f (sent %f) -- ", received.float_num, sent.float_num);
  if(received.float_num == sent.float_num) {
    printf(" OK\n");
  }
  else {
    printf("ERROR\n");
  }

  printf("Char = '%c' (sent '%c') -- ", received.ch, sent.ch);
  if(received.ch == sent.ch) {
    printf(" OK\n");
  }
  else {
    printf("ERROR\n");
  }

  /* Destroy the queue */
  msgctl(qid, IPC_RMID, 0);
}

source code

Now we can create two processes and let them communicate through a message queue. Remembering forking concepts explained in the issue 2 you can recall that the child process, when created, receives a copy of the memory of its parent. This means that creating the queue before the fork operation results in both the parent and the child knowing the right queue identifier and thus capable of access it.

The following code creates a queue, then forks the execution. The child generates a random number, prints it on the standard output and sends them to the parent, which in turn prints it on the screen.

#include <stdio.h>
#include <stdlib.h>
#include <linux/ipc.h>
#include <linux/msg.h>
#include <sys/types.h>

/* Redefines the message structure */
typedef struct mymsgbuf
{
  long mtype;
  int num;
} message_t;

int main()
{
  int qid;
  key_t msgkey;
  pid_t pid;

  message_t buf;

  int length;
  int i;

  length = sizeof(message_t) - sizeof(long);

  msgkey = ftok(".", getpid());

  qid = msgget(msgkey, IPC_CREAT | 0660);

  if(!(pid = fork())){
    printf("CHILD - Queue ID = %d\n", qid);

    srand (time (0));

    for(i = 0; i < 10; i++){
      sleep (rand()%4);
      buf.mtype = 1;
      buf.num = rand()%100;
      msgsnd(qid, &buf, length, 0);      
      printf("CHILD - Sent message number %d: %d\n", i+1, buf.num);
    }

    return 0;
  }

  printf("PARENT - Queue ID = %d\n", qid);

  for(i = 0; i < 10; i++){
    sleep (rand()%4);
    msgrcv(qid, &buf, length, 1, 0);    
    printf("PARENT - Received message number %d: %d\n", i+1, buf.num);
  }

  printf("PARENT - Waiting for the child to terminate...\n"); 
  waitpid (pid, NULL, 0);
  printf("PARENT - Child ended\n");

  msgctl(qid, IPC_RMID, 0);

  return 0;
}

source code

Compiling and running the program you can check the correct working of the shared message queue.

Conclusions¶

In this article, we introduced a new IPC structure called message queue and the related concepts of message and protocol. These concepts are very important and are going to be the foundation stone of a project we will realize step by step in the future articles: a simple telephone switch simulator.

Concurrent programming - 4

2013-02-13T10:50:00+02:00

Abstract¶

In the past issue, of this series of articles about concurrent programming we started to concern ourselves with the problems of synchronization between processes. In this installment, we will investigate further the subject introducing some structures and functions collectively known as Unix System V IPC.

IPC: InterProcess Communication¶

Communication between processes, either running on the same machine or over a network, is one of the most interesting topics in computer science and, despite its age, new solutions to this problem keep arising. With the widespread availability of Internet access, the subject is now a little shifted towards pure network communication, which represents just a part of the techniques known as IPC: InterProcess Communication.

This abbreviation encompasses several different scopes in the field of multiprocessing, the most important ones being synchronization, shared data management and messaging. In this article, we will focus on local IPC, i.e. communication between processes running on the same machine. The underlying concepts and techniques, however, can be easily applied to a network environment too, but they require a layer to manage network communication, which is out of the scope of this series.

More on synchronization¶

As briefly shown in the last article synchronization problems are not only among the most complex issues to solve in computer science, but can also lead to severe malfunctions of our software.

Let's reconsider and formalize the example given in the last post. Say we create two different processes working on the same data. If the two processes just read the same data from memory (or disk) no problems can arise: data are static, and accessing does not change them. So the execution of the two processes is called consistent, which highlights that repeated executions or executions spanning a long time will always result in the same system behaviour.

If now one of the processes wants to modify data different situations can come up, depending on the actual timing of the operations. Say we have two processes A and B and an element d; process A increases element d by 1 while process B prints the element value on the screen. In a sort of metalanguage, we could express this situation with the following syntax

A { d->d+1 } & B { d->output }

where & stands for a concurrent execution; in other words the two processes are not synchronized. One of the possible resulting executions is

(-) d = 5
(A) d = 6
(B) output = 6

while the other is

(-) d = 5
(B) output = 5
(A) d = 6

As you can see, different timings can lead to different results; let's pretend that element d is the amount of money in your bank account, and you will suddenly realize the importance of the matter. Such a reliance of data on timing or system conditions is known as data inconsistency.

As we already described, waiting is a trivial form of synchronization that can solve this problem. However it is as simple as inefficient as it blocks the execution of one process while the other is operating and this happens even if the blocked process is no more operating on shared data. The block is indeed based on the simple assumption that one process could lead to data inconsistency.

It is thus necessary to increase the granularity of this block. Granularity is the scope of an action: the higher is the granularity the narrowest is the scope. So we have to restrict the scope of the block, which by now covers the whole process.

System V keys¶

A part of the IPC structures introduced by Unix System V is explicitly dedicated to resource identification, the capability of labelling arbitrary objects and limiting access management to them. This is done through the use of the so-called SysV IPC Keys: a key, in this context, is a number used to identify univocally an IPC structure. An IPC key can be generated by ftok()

key_t ftok(const char *pathname, int proj_id);

where pathname is an existing file name with the full path and proj_id is a number without any restriction. It is guaranteed that the returned key is the same for subsequent calls with the same parameters, but it is not guaranteed that different parameters result in different keys: in other words keys are not guaranteed to be unique. Since IPC functions require a key created with ftok() the best solution is to write a small library that keeps track of assigned keys and avoids duplicates.

Semaphores - part 1¶

The idea of traffic light can be profitably borrowed to control access to resources. A semaphore, in the IPC world, is a structure capable of containing a positive or null integer and which manages a queue of processes waiting for a particular condition of the semaphore itself.

Despite their simplicity, the power of semaphores is big, thus their correct use is not trivial. We will therefore start writing code without error control, just to focus on the real subject. Please remember that in real world applications, error control code can vary from 40% up to 80% of the total amount, so bear in mind that code without error control is good only for the purpose of understanding concepts.

The first possible use of a semaphore is that of access controller: the value of the semaphore represents the number of processes that can concurrently access the resource. Every time a process access the resource it decrements the value of the semaphore and every time a process releases the resource it increments the value. If the resource is exclusive (that is only one process can access the resource at a time), the maximum value of the semaphore will be 1.

A second use of the semaphore is that of resource counter. The value in this case is the number of resources available to processes, e.g. memory cells, network connections, and so on. As you can easily understand, using a resource is equivalent to locking its access.

Let's consider a practical case where both the uses of semaphores will be useful.

We build a buffer of length L where n processes W1,..., Wn can write to, but from which just one process R can read. Say also that just one process can access the buffer at a given time T. As you can easily understand W processes can write at any time except when the buffer is full while the process R can read at any time except when the buffer is empty.

This buffer could be easily managed in a standard single task environment by declaring 2 flags, the first to signal overflow and the second to signal underflow. We are in a multitasking environment now, so flags declared before the fork operation are not shared between processes (each process has its own copy of the flags, and they are not linked). So we have to use a shared structure, and this is where IPC semaphores take the field.

We can declare three semaphores: the first one will manage access to the buffer while the second and third will keep track of how many elements are in the buffer (later it will become clear why two semaphores are not sufficient).

Given that access to the buffer is exclusive the first semaphore will be a binary one (which value thus will be 0 or 1) while the second and the third will have values linked to the length of the buffer.

In C language the SysV primitive to create a semaphore is

int semget(key_t key, int nsems, int semflg);

where key is an IPC key created with ftok(), nsems is the number of semaphores we want to create and semflg a set of flags controlling access to the semaphores. Access to IPC structures is ruled by a 12 bit system which is almost identical to the Unix file system access control, but we will not dive into it. As you can also notice semget() manages sets of semaphores, which helps us to keep the code compact.

Let's review the full code to create a semaphore

#include <stdio.h>
#include <stdlib.h>
#include <linux/types.h>
#include <linux/ipc.h>
#include <linux/sem.h>

int main(void)
{
  key_t key;
  int semid;

  key = ftok("/etc/fstab", getpid());

  /* create a semaphore set with only 1 semaphore: */
  semid = semget(key, 1, 0666 | IPC_CREAT);

  return 0;
}

Do not compile and execute it by now. As you can see from the #include statements, IPC structures are provided by the operating system itself and not by the standard C library. The key is created by passing to ftok() a file that certainly exists in the system and the PID of the current process. To know more about permission flags check the manual page for semget().

Let's go on learning to manage and remove semaphores; the primitive used to interact with a semaphore is

int semctl(int semid, int semnum, int cmd, ...)

where semid is the semaphore set identifier returned from semget(), semnum the index of the semaphore inside the set and cmd the command you want to run. The value semnum is optional for some commands. Some commands need also an additional argument which type is union semun, defined as

union semun {
 int val;                  /* value for SETVAL */
 struct semid_ds *buf;     /* buffer for IPC_STAT, IPC_SET */
 unsigned short *array;    /* array for GETALL, SETALL */
                           /* Linux specific part: */
 struct seminfo *__buf;    /* buffer for IPC_INFO */
};

To set the value of a semaphore the command is SETVAL and the value is specified through a union semun variable. To set the value of the first semaphore to 1 we modify the program this way

#include <stdio.h>
#include <stdlib.h>
#include <linux/types.h>
#include <linux/ipc.h>
#include <linux/sem.h>

int main(void)
{
  key_t key;
  int semid;
  union semun arg;

  key = ftok("/etc/fstab", getpid());

  /* create a semaphore set with only 1 semaphore: */
  semid = semget(key, 1, 0666 | IPC_CREAT);

  arg.val = 1;
  semctl(semid, 0, SETVAL, arg);

  return 0;
}

Last we have to cancel the semaphore releasing the structures needed for its management; this is done by the command IPC_RMID, which removes the semaphore and notifies all processes queued to access it that it has been removed. Let this notification aside for the moment and change the code to remove the semaphore

#include <stdio.h>
#include <stdlib.h>
#include <linux/types.h>
#include <linux/ipc.h>
#include <linux/sem.h>

int main(void)
{
  key_t key;
  int semid;
  union semun arg;

  key = ftok("/etc/fstab", getpid());

  /* create a semaphore set with only 1 semaphore: */
  semid = semget(key, 1, 0666 | IPC_CREAT);

  arg.val = 1;
  semctl(semid, 0, SETVAL, arg);

  /* deallocate semaphore */
  semctl(semid, 0, IPC_RMID);

  return 0;
}

Now you can compile and run it. As you can imagine the program seems to do nothing: it just silently creates a semaphore, sets its value and removes it.

Semaphores - part 2¶

Time to use effectively the semaphore: the function that allows to performs operations on a semaphore is

int semop(int semid, struct sembuf *sops, unsigned nsops);

where semid is the set identifier, sops an array of operations and nsops the length of the latter. Each operation is declared through a struct sembuf

struct sembuf {
    ushort  sem_num;        /* semaphore index in array */
             short   sem_op;         /* semaphore operation */
             short   sem_flg;        /* operation flags */
};

For the time being we will always set sem_flg to 0. Operations are always integers and obey the following rules

sem_op < 0 This signals that you want to decrease the semaphore value, or in other words lock one of the resources controlled by it. If the value of the semaphore can be decremented without becoming negative it will be and the process running semop() continues. Otherwise, the calling process falls in a sleep status until the semaphore has that number of resources available.
sem_op = 0 This signals that you are waiting for the semaphore to reach value zero; that is the condition where no resources are available. The calling process falls in a sleep status until that moment.
sem_op > 0 This signals that you want increase the semaphore value, or in other words release the given number of resources.

Now we can write some code to implement the buffer example previously given. We are going to create 5 processes called W (writers) and a process called R (reader). Each W process tries to lock the resource (the buffer) through a semaphore, and if the buffer is not full, adds an element and releases the lock. The R process tries to lock the buffer, removes an element if available (buffer not empty) and releases the lock.

Since we did not yet talk about shared memory we have to simulate read and write operation, i.e. pretend the operation has been performed. This is necessary since each process has its own memory space and cannot access that of another process. So each process has its own copy of the buffer, and the copies are not linked each other. Later in this series we will talk about techniques that allow sharing memory regions between processes.

As already stated, we need 3 semaphores: the first acts as access controller and its maximum value is 1 while the other two manage overflow and underflow conditions. A single semaphore could not handle both conditions, since semop() acts one-way only.

Let's clarify the latter statement before looking at the code. Say we have a single semaphore to manage over- and underflow conditions, with a value equal to the number of empty spaces in the buffer. Each time a W process fills the buffer it can decrease the semaphore value by one unit until the value reaches 0, which represents the condition of full buffer. This way however the empty buffer condition cannot be managed since the R process can increment the value of the semaphore without any limit. Semaphores, indeed, just control the lower boundary and not the upper one.

#include <stdio.h>
#include <stdlib.h>
#include <errno.h>
#include <linux/types.h>
#include <linux/ipc.h>
#include <linux/sem.h>

int main(int argc, char *argv[])
{
  /* IPC structures */
  pid_t pid;
  key_t key;
  int semid;
  union semun arg;
  struct sembuf lock_res = {0, -1, 0};
  struct sembuf rel_res = {0, 1, 0};
  struct sembuf push[2] = {1, -1, IPC_NOWAIT, 2, 1, IPC_NOWAIT};
  struct sembuf pop[2] = {1, 1, IPC_NOWAIT, 2, -1, IPC_NOWAIT};

  int i, j;
  int len;
  int num_proc = 5;
  int num_write_actions = 20;
  int num_read_actions = 100;

  if(argc < 2){
    printf("Usage: %s <size>\n", argv[0]);
    exit(0);
  }

  len = strtol(argv[1], NULL, 10);

  key = ftok("/etc/fstab", getpid());

  /* Create a set with 3 semaphores */
  semid = semget(key, 3, 0666 | IPC_CREAT);

  /* Initialize semaphore #0 to 1 - Resource controller */
  arg.val = 1;
  semctl(semid, 0, SETVAL, arg);

  /* Initialize semaphore #1 to buf_length - Overflow controller */
  /* Sem value represents free space in buffer */
  arg.val = len;
  semctl(semid, 1, SETVAL, arg);

  /* Initialize semaphore #2 to buf_length - Underflow controller */
  /* Sem value represents the number of elements in buffer */
  arg.val = 0;
  semctl(semid, 2, SETVAL, arg);

  /* Fork */
  for (i = 0; i < num_proc; i++){
    pid = fork();
    if (!pid){
      /* Child process code*/
      for (j = 0; j < num_write_actions; j++){
    sleep(rand()%6);

    /* Try to lock the buffer - sem #0 */
    if (semop(semid, &lock_res, 1) == -1){
      perror("semop:lock_res (write)");
    }

    /* Lock a free cell - sem #1 */
    /* Push an element - sem #2 */
    if (semop(semid, &push, 2) != -1){
      printf("---> Child process %d: Element written\n", getpid());
    }
    else{
      printf("---> Child process %d: BUFFER FULL\n", getpid());
    }

    /* Release the buffer */
    semop(semid, &rel_res, 1);
      }

      exit(0);
    }
  }

  for (i = 0;i < num_read_actions; i++){
    sleep(rand()%3);

    /* Try to lock the buffer - sem #0 */
    if (semop(semid, &lock_res, 1) == -1){
      perror("semop:lock_res (read)");
    }

    /* Unlock a free cell - sem #1 */
    /* Pop an element - sem #2 */
    if (semop(semid, &pop, 2) != -1){
      printf("<--- Parent process %d: Element read\n", getpid());
    }
    else {
      printf("<--- Parent process %d: BUFFER EMPTY\n", getpid());
    }

    /* Release the buffer */
    semop(semid, &rel_res, 1);
  }

  /* Destroy semaphores */
  semctl(semid, 0, IPC_RMID);

  return 0;
}

Let us review the most interesting parts of the code.

  struct sembuf lock_res = {0, -1, 0};
  struct sembuf rel_res = {0, 1, 0};
  struct sembuf push[2] = {1, -1, IPC_NOWAIT, 2, 1, IPC_NOWAIT};
  struct sembuf pop[2] = {1, 1, IPC_NOWAIT, 2, -1, IPC_NOWAIT};

These four lines of code declare the actions we will use later on the semaphore set. The first two are single actions while the second two are double ones. The first action, lock_res, is used to lock the buffer: as you can see the semaphore number 0 is decremented by 1 and in case of busy resource the strategy is waiting (the last 0 in the action). The second action releases the buffer and is the mirror image of the first one.

The third and fourth actions are specular too; each of them is an array of two actions, the first on the semaphore number 1 and the second on the semaphore number 2; while the first one is incremented the second is decremented and vice versa. The policy is changed from the first two actions: IPC_NOWAIT forces the process to continue execution, in contrast with the previous case, when the process was put on hold.

  /* Initialize semaphore #0 to 1 - Resource controller */
  arg.val = 1;
  semctl(semid, 0, SETVAL, arg);

  /* Initialize semaphore #1 to buf_length - Overflow controller */
  /* Sem value represents free space in buffer */
  arg.val = len;
  semctl(semid, 1, SETVAL, arg);

  /* Initialize semaphore #2 to buf_length - Underflow controller */
  /* Sem value represents the number of elements in buffer */
  arg.val = 0;
  semctl(semid, 2, SETVAL, arg);

Next, semaphores must be initialized: as already stated the first one is set to 1, being a binary access controller, the second to the length of the buffer as overflow controller and the third to 0 as underflow controller.

    /* Try to lock the buffer - sem #0 */
    if (semop(semid, &lock_res, 1) == -1){
      perror("semop:lock_res (write)");
    }

    /* Lock a free cell - sem #1 */
    /* Push an element - sem #2 */
    if (semop(semid, &push, 2) != -1){
      printf("---> Child process %d: Element written\n", getpid());
    }
    else{
      printf("---> Child process %d: BUFFER FULL\n", getpid());
    }

    /* Release the buffer */
    semop(semid, &rel_res, 1);

Child process code (a writer) tries first of all to lock the buffer using lock_res; once the control over the resource has been acquired it performs the push action and writes on the standard output a meaningful message. Otherwise, it signals that the buffer is full. The last action it performs is to release the resource through rel_res.

    /* Try to lock the buffer - sem #0 */
    if (semop(semid, &lock_res, 1) == -1){
      perror("semop:lock_res (read)");
    }

    /* Unlock a free cell - sem #1 */
    /* Pop an element - sem #2 */
    if (semop(semid, &pop, 2) != -1){
      printf("<--- Parent process %d: Element read\n", getpid());
    }
    else {
      printf("<--- Parent process %d: BUFFER EMPTY\n", getpid());
    }

    /* Release the buffer */
    semop(semid, &rel_res, 1);

As you can immediately notice parent process (reader) mimics the behaviour of the child. It lock the buffer, reads some data with the pop action and release the resource.

Try to change the parameters such as the buffer length (on the command line) or the number of cycles performed by parent and child processes to see what happens.

Conclusions¶

In the next article, we will introduce and deal with atomicity, which is a very important concept in concurrent programming and database systems. We will also introduce a new IPC structure, which has a broad use in distributed systems: message queues.

Concurrent programming - 3

2013-02-06T12:25:00+02:00

Abstract¶

In the first article, we stepped into the world of multitasking, going over its meaning and the reasons behind its existence. In the second article, we met the fundamental fork operation and wrote our first multitasking code. In this article, we will go ahead and introduce ourselves in the topic of synchronization: the problem we face now indeed is to release the full power of multitasking, that is to share the work between processes.

In the first part of the article, I want to make a step back to the basics of multitasking to clarify the single/multiple CPU theme. Then we will start talking about shared data between processes, first looking at the problems arising and then writing some code.

Simultaneous execution¶

We should never forget that the main reason behind multitasking on a single CPU is to give the impression of simultaneous execution, not to speed up execution.

Let's clarify this statement. A single CPU can execute only one operation at a time, so running two processes means to run the first one for a little time, then switch to execute the second one and so on. As already stated, this interlacing operation is what we call multitasking, and it is clear that such a technique slows down the interlaced processes: since each of the two processes runs for half the time (the other half being spent by the other process), the result is a process that runs half the speed.

If you add to this that a switch operation does not come for free, because some time is spent by the operating system replacing the current process with the other one, you suddenly realize that multitasking is a double-edged sword: on one side, it gives us the chance to run multiple programs at the same time, on the other side, it slows down the execution of the whole system.

This issue can only be solved by using multiple CPUs or multiple cores (the two are not exactly the same), either on the same physical machine or on different ones.

So, if you are writing an algorithm to process an image in foreground, that is for a user that is waiting in front of the screen, you could split it profitably in many parts, each of which will process a tile of the whole image. This way the user could have a glance at the whole result while it is being computed. The same technique counts if, for example, you are computing statistics about a set of data: since some computations last longer than others you could split them among different processes so that the user can immediately obtain the faster ones.

If, on the contrary, you are writing code that runs in background without the user being there waiting for the results, you should never split it in different processes if you are working on a single CPU. That way you are just tangling up the code and wasting the CPU cycles to switch between processes without any real benefit.

Working together¶

Being our processes running at the very same time on different CPUs or just interlacing on a single one, a completely new set of problems arises if the two have to work on the same data.

Just imagine this very common scenario: two processes share a common storage area in memory where partial results of their computations can be temporarily saved. So each process writes data on the storage area, then runs computing other data and last loads previously saved data to finalize its work. What happens if, in the meantime, the second process wrote some of its temporary data on the same area? The first process has no way to figure out that the loaded data is not correct, and this can lead to catastrophic results. The worst thing about such a bug, as quite always in the multitasking environment, is that such a problem can arise or not, depending on input, times, status of the running machine and so on. This class of problems is usually known as race condition and is one of the most subtle and dangerous bug forms since it is difficult to spot, to reproduce and sometimes to test automatically.

A little search about the Therac-25 accident (1987) can clearly exemplify the extent of the problem that such a bug can cause.

Basic synchronization¶

The simplest form of synchronization is waiting; that is, a process sits down until another process finishes running, then runs its own code. The system call that allows a process to wait for another to end is

pid_t waitpid (pid_t PID, int *STATUS_PTR, int OPTIONS)

where PID is the PID of the process we are waiting for, STATUS_PTR a pointer to an integer which will contain the status of the child process (NULL if the information is not needed) and OPTIONS a set of options we have not to care about for now.

This is an example of a program in which the parent creates a child process and waits until it ends up

#include <unistd.h>
#include <sys/types.h>
#include <stdio.h>

int main()
{
  pid_t pid;
  int i;

  pid = fork();

  if (pid == 0){
    for (i = 0; i < 8; i++){
      printf("-child-\n");
    }
    return 0;
  }

  printf("+parent+ Waiting for the child to terminate...\n"); 
  waitpid (pid, NULL, 0);
  printf("+parent+ ...ended\n");

  return 0;
}

Let's save the code as fork_demo3.c, compile it and execute. The output I get from this program is the following

$ ./fork_demo3
+parent+ Waiting for the child to terminate...
-child-
-child-
-child-
-child-
-child-
-child-
-child-
-child-
+parent+ ...ended

Looking at it you can realize a very important thing about waitpid(): it is a so-called blocking system call; that is something that keeps running until a certain condition is met. In this case, waitpid() keeps doing nothing until the child ends. So waiting is a blocking synchronization technique.

Relationships¶

Speaking about parents and children there are two more type of processes we can find in the system: orphans and zombies. A process is orphaned when the parent terminates while it is still running, according to the real meaning of the word orphan; in such a case the orphan process is adopted by init, the first process in the system, thus becoming one of its children.

Zombie processes, on the contrary, are already terminated, but they have not been waited by the parent. In other words, they are not running, but the operating system still holds their data to allow the parent to collect the exit status.

A simple demonstration of the zombie status can be done with the following code

#include <unistd.h>
#include <sys/types.h>
#include <stdio.h>

int main()
{
  pid_t pid;
  int i;

  pid = fork();

  if (pid == 0){
    printf("-child-\n");
    return 0;
  }

  for (i = 0; i < 14; i++){
    printf("+parent+\n");
    sleep(2);
  }

  return 0;
}

Save it as zombie_demo.c, compile and run. Just after the child has printed its string open another terminal and run ps xa. The last lines of your output will be something like

 4399 pts/1    S+     0:00 ./zombie_demo
 4400 pts/1    Z+     0:00 [zombie_demo] <defunct>

As you can see the first process listed is the parent, while the second, the child, is marked as defunct and its status is zombie (Z).

Conclusion¶

Multitasking is a fundamental technique, which must be however used with care; as any other technology, it cannot solve every problem, neither its introduction comes completely for free.

Synchronization between processes is something which must be carefully considered and implemented since errors in this field can lead to devious bugs.

Last you met two new types of processes, orphans and zombies, and learned why they can be found in the system.

In the next article, we will study some classical synchronization problems and introduce the basic InterProcess Communication (IPC) structures and techniques Unix systems make available to programmers.

Concurrent programming - 2

2013-02-04T10:35:00+02:00

Abstract¶

In the past article I introduced the concept of process and how important it is for our operating system: in this new issue of the Concurrent Programming series we will go on and begin to write multitasking code; we will start from one of the basic concepts of the whole picture, the forking operation. Afterwards, we will start to introduce the problem of process synchronization and communication, which we will deeply face in a later instalment.

The C standard library¶

The C standard library, nicknamed libc, is a collection of functions for the C language which has been standardized by ANSI and later by ISO. An implementation of it can be found in any major compiler or operating system, especially Unix-like ones.

In the GNU ecosystem, which Linux distributions are grounded on, the library is implemented by glibc (GNU libc). The library reached version 2.17 in December 2012 and is free software released under LGPL, a less restrictive version of the GPL which most notably allows the use of the library by non-GPL software, even proprietary one.

You can find and navigate glibc sources here

C language and PIDs¶

Processes, like every resource in a computer, are managed by the kernel. This is a part of the operating system that runs continuously in background, exposing functions to other programs that allow them to obtain, manage and release resources such as memory, disk space and CPU time. One of the things the kernel exposes to the hosted programs is the process management, which stands for structures that represent the current status of processes in the system. Using the libc the programmer can easily access those structures in a read-only way.

Let us start with the simple concept of PID, the Process IDentifier we described in the past issue of this series. In the libc, the pid_t type is defined as an integer capable of containing a pid. To show this we will now take a little tour of the glibc source code: we will cover the whole path this time only, just to highlight both the chances given us by the source code and the complexity of a project like glibc.

We can find the initial definition of pid_t here

#ifndef __pid_t_defined
typedef __pid_t pid_t;
#define __pid_t_defined
#endif

As you can see the definition of the type is protected by multiple header inclusion, as usual in the C world. The underlying __pid_t type is then defined here

#define __STD_TYPE typedef
#define __S32_TYPE int
__STD_TYPE __PID_T_TYPE __pid_t; /* Type of process identifications.  */

Notice that even the typedef keyword is defined by a macro (remember that the C preprocessor is a little more than a text macro replacement tool); this definition in particular avoids compiler warning when using the long long type which is not supported by all versions of the C standard. S32 stands for “signed 32 bits” as you can read in the code documentation.

The last macro __PID_T_TYPE can be found here

#define __PID_T_TYPE            __S32_TYPE

The heavy use of defines and compiler features made by glibc code allows the library to be compiled on very different architectures.

Telling the PID¶

Let us now discover the function which give us the knowledge of the pid of the process containing our program

pid_t getpid (void);

which is defined with pid_t in unistd.h and sys/types.h. The libc simply uses the system call called getpid and its real implementation can be found in the kernel source

SYSCALL_DEFINE0(getpid)
{
    return task_tgid_vnr(current);
}

We can now write a program whose aim is to print on the standard output its own pid. With an editor of your choice write the following code

#include <unistd.h>
#include <sys/types.h>
#include <stdio.h>
int main() {
  pid_t pid;

  pid = getpid();
  printf("The pid assigned to the process is %d\n", pid);

  return 0;
}

source code

Save the program as print_pid.c and compile it

gcc -o print_pid print_pid.c

If this operation returns errors make sure you installed gcc and libc headers; refer to your distribution knowledge base to solve this task.

The compiler will build an executable named print_pid (since the current directory is probably not in the system path you can run it with ./print_pid). Executing the program we will have no great surprises: it prints out a positive number and if you executes it more than once this number will likely increase progressively; this is not mandatory, however, because another process could be created between two executions of print_pid. Try for example to execute print_pid, then ps and then print_pid again.

Forking¶

Now it is time to meet one of the most important concepts in the field of operating systems, that is the concept of forking.

Forking in general means that one thing splits in two and each part acts like an independent being. At the very moment of the split the two entities are the same, exactly the same, except for the fact that they live in two different spaces. This computer science concept can be retrieved in nature when a simple organism generates a clone of itself. After the cloning the two organisms are the exact copy, but they can be told apart since they are two things (the right one and the left one for example). Back to the operating system world we find the very same problem when a process forks; after the fork operation we have two processes with two different PIDs, but the two contain the same program at the same execution point.

So, when a program (running as process A) creates another process (B) the two are identical, that is they have the same code, the memory full of the same data (not the same memory, since the memory regions are different) and the same processor status. From this point on the two can continue in two different ways, just like our cloning organisms, for example depending on the user's input or some other data. The process A is called the parent process while B is the child process; now you can begin to understand better the name parent of all the processes given to init. The function which creates a new process is

pid_t fork(void)

The number returned is a pid, but it deserves a particular attention. We said that the first process duplicates itself splitting in a parent and a child, which will then execute interlacing themselves with the other running processes, doing different works; but immediately after the duplication which process will be executed, the parent or the child?

Well, the answer is as simple as unuseful: one of the two. The decision about which process has to be executed is taken by a part of the operating system called scheduler, and it pays no attention if a process is the parent or the child, following an algorithm based on other parameters.

If we are writing processes that implement a service (such as serving network connections) we need nothing more: the two processes are the same and they behave the same way, so by cloning the process we just incremented the number of service providers. This is usually done by Web servers such as Apache, which spawns a copy of itself to serve incoming connections.

Parent and child¶

Sometimes, anyway, it is fundamental to know what process is in execution: after the fork both processes are at the same execution point (that is just after the fork() call) so how can them tell if they are the parent or the child one?

In order to clarify this question let us look at the following algorithm:

fork
if you are the child then execute ...
if you are the parent then execute ...

which could represent in a sort of metalanguage the code of our program. The question can be expressed this way: “You have been cloned. Who are you, the original one or the clone?”

The answer is once again very simple: the fork() function returns 0 to the child process and the child's PID to the parent. So it is enough to test if the returned PID is zero and we will know what process is executing the current code. Putting it in C language we obtain

int main() {
    pid_t pid;
    pid = fork();

    if (pid == 0)
    {
        code of the child process
    }
    else {
        code of the parent process
    }
}

It's time to write the first real example of multitasking code: you can save it in a fork_demo.c file and compile it as we did before.

The program will fork itself, and both the parent and the child will write something on the screen; the final output will be the interlaced output of the two (if all goes right).

#include <unistd.h>
#include <sys/types.h>
#include <stdio.h>

int main() {
    pid_t pid;
    int i;

    pid = fork();

    if (pid == 0){
        for (i = 0; i < 8; i++){
            printf("-child-\n");
        }
        return 0;
    }

    for (i = 0; i < 8; i++){
        printf("+parent+\n");
    }

    return 0;
}

source code

Lines number 01-03 contain the includes for the necessary libraries (standard I/O, multitasking).

The main function returns an integer (as always in GNU), which normally is zero if the program reached the end without errors or an error code if something goes wrong; let us state this time all will run without errors (we can later add some error control code, when the basic concepts will be clear).

Then we define the data type containing a pid (line 06) and an integer working as counter for loops (line 07). At line 09 we call fork() and it will return zero to the program executed in the child process and the pid of the child process to the parent; the test is at line 11. Now the code at lines 12-15 will be executed in the child process while the rest (lines 18-22) will be executed in the parent. Note that we omitted the else statement since the child code ends with a return; thus there is no way for the child to execute the code of the parent.

The two parts simply write 8 times on the standard output the string -child- or +parent+, depending on which process executes it, and then end returning 0. Executing the program will perhaps let you unsatisfied: the result is likely not to be a real mix between the two strings, and this due to the speed of execution of such a short loop. The scheduler runs the parent or the child for a quantum of time which is enough for the process to end the loop; thus probably your output will be a list of +parent+ strings followed by a list of -child- ones, or the contrary.

To highlight the interlacing effect we can insert a random delay before each printf() call: we can do this with sleep() and rand() functions

sleep(rand()%4)

this make the program sleep for a random number of seconds between 0 and 3. Now the child code looks like

for (i = 0; i < 8; i++){
    sleep (rand()%4);
    printf("-child-\n");
}

and the same for the parent's code. Save it as fork_demo2.c, compile and execute. It is slower now, and we notice a difference in the output string order, and more differences between an execution and the previous one. For example my output is

$ ./fork_demo2
-child-
+parent+
+parent+
-child-
-child-
+parent+
+parent+
-child-
-child-
+parent+
+parent+
-child-
-child-
-child-
+parent+
+parent+

As always when dealing with multitasking code, your output is probably different from mine and will also be different between consecutive executions. Try also to change the parameters of the program, such as the sleep time or the number of loops.

Why forking¶

Thinking carefully about the parent and child example a question arises: why should I write code that forks, with all the intricacies it entails, when I could simply write two different programs and run them, letting the operating system to deal with their simultaneous execution?

This is a very interesting question, and the answer is very simple, though the underlying concept is rather complex. When you run a program the shell or the GUI follows a pattern called fork-and-exec: first it forks, then the child process executes the desired program. The reason of this behaviour is that the exec family of system calls replace the running program with a new program but do not change the PID. So the tasks are divided between system calls: fork creates a new process while exec loads the given program into the newly created process.

DOS and Windows¶

Operating systems of the DOS and Windows family do not provide a fork mechanism but a spawn family of system calls, which directly implement the fork-and-exec technique; this is a good example of the different implementation OSs make of common computer science concepts: while both implement multitasking and let many processes run concurrently, the way this target is reached can vary significantly.

Conclusion¶

Starting from the knowledge of processes you stepped into the fundamental Unix concept of forking, learning to tell apart parent and child processes and running your first multitasking-based program. You also started to understand how complex can some operating system issues be, and that real world OSs can provide very different solutions to them.

In the next article we will start to look after processes synchronization and data sharing between processes.

Concurrent programming - 1

2013-01-31T16:47:00+02:00

Abstract¶

I already published this series of articles on concurrent programming on LinuxFocus from November 2002 and now I carefully reviewed them. Feel free to notify me any error you will find.

The purpose of this article (and the following ones in this series) is to introduce the reader to the concept of multitasking and to its implementation in the Linux operating system. Thus, we will start from the theoretical concepts at the base of multitasking and from those we will move towards advanced concept like multithreading and interprocess communication. We will practice some code and some Linux internals along this path, and I will try constantly to exemplify how real world operating system realize the concepts I am introducing.

Prerequisites¶

Prerequisites for the understanding of the article are:

Minimal knowledge of the text shell use (bash)
Basic knowledge of C language (syntax, loops, libraries)

Even if I wrote the articles using a Linux machine, any Unix-like operating system, most notably Mac OS X, shares the concepts I will present. However, pay attention to the fact that while any Unix-like system behaves the same way (more or less), its internal structures and code can be significantly different. On the other hand OSs of the DOS and Windows family share some concepts with Unix-like ones but behave very differently.

One of the biggest positive aspects of dealing with an open source operating system like Linux is the possibility to look at the code: this allow us to uncover the internal working of a system that runs on the most important data center of the planet.

Introduction¶

One of the most decisive turning points in the history of operating systems was the concept of multiprogramming, a technique for interlacing the execution of several programs in order to gain a more constant use of the system's resources. Let us think about a simple workstation, where a user can execute at the same time a word processor, an audio player, a print queue, a web browser and more, and we can understand the importance of this technique. As we will discover this little list is only a minimal part of the set of programs that are currently executing on the machine, even though they are the most visible ones.

The concept of process¶

In order to interlace programs, a remarkable change of the operating system is necessary; in order to avoid conflicts between running programs an unavoidable choice is to encapsulate each of them with all the information needed for their execution.

Before we explore what happens in the operating system let us give some technical nomenclature: given a running program, at a given time the code is the set of instructions which it is made of while the memory space is the part of machine memory taken up by its code and data. The processor status is the value of the microprocessor's parameters, such as the registers, the flags or the Program Counter (the address of the next instruction to be executed).

Given a tern of objects made of code, memory space and processor status we can define a running program. If at a given time during the operation of the machine we save this information and replace them with the same set taken from another running program, the flow of the latter will continue from the point at which it was stopped: doing this once with the first program and once with the second realizes the interlacing previously described. We name process (or task) the tern of objects we used, a concept which has to do with that of program, yet being an extension of it.

Let us explain what was happening to the workstation we spoke about in the introduction: only one task is in execution at each moment (there is only one microprocessor with just one core and it cannot do two things at the same time), and the machine executes part of its code. After a certain amount of time named quantum the running process is suspended, its tern of objects is saved and replaced by those of another waiting process, whose code will be executed for a quantum of time, and so on. This mechanism is what we call multitasking (or multiprocessing).

As stated before the introduction of multitasking causes a set of problems, most of which are not trivial, such as the management of the waiting processes queues (called scheduling); nevertheless those problems have to do with the architecture of each operating system, and are not the topic of the current article.

Processes in Linux and Unix¶

Let us discover something about the processes running on the machine. The command which gives us such information is ps which is an acronym for Process Status. Opening a normal text shell and typing the ps command we will obtain an output such as

$ ps
 PID TTY          TIME CMD
2241 ttyp4    00:00:00 bash
2346 ttyp4    00:00:00 ps

I state in advance that this list is not complete, but let us concentrate ourself on this for the moment: ps has given us the list of each process running on the current terminal. We recognize in the last column the name the process is started with (such as "firefox" for Mozilla Firefox and "gcc" for the GNU Compiler Collection). Obviously "ps" appears in the list because it was running when it compiled the list of running processes. The other listed process is the Bourne Again Shell, the shell running on the terminal.

Let us leave out for the moment the information about TIME and TTY and let us look at PID, the Process IDentifier. The pid is a strictly positive number (not zero) the operating system assigns univocally to each running process; once the process terminated the pid can be reused, but the OS guarantees that the pid of a process remains the same during its execution and that it is unique. All this implies that the output each of you will obtain from the ps command will probably be different from that in the above example.

Just to check the latter sentence, let us open another shell without closing the first one and type the ps command: this time the output gives the same list of processes but with different pid numbers, showing that they are two different processes even if the program they executes is the same.

We can also obtain a list of all processes running on a Linux box: the ps command man page says that the switch -e means "select all processes". Let us type "ps -e" in a terminal and ps will print out a long list formatted as previously seen. In order to analyze in a comfortable way this list we can redirect the output of ps in the ps.log file:

$ ps -e > ps.log

Now we can read this file editing it with our preferred editor (or simply with the less command); as stated at the beginning of this article the number of running processes is higher than we would expect; on the machine I'm currently writing on they are around 170.

That list contains not only processes started by us (through the command line or our graphical environment of choice), but also a set of processes, some of which with strange names: the number and the identity of the listed processes depends on the configuration of your system, but there are some common things.

First of all, no matter how you configured the system, the process with pid equal to 1 is always "init", the so called father of all the processes. Wait, wait, I hear you screaming “What about fatherhood among processes?”. This concept will become clear later, when we will talk about forking and the way the system spawns new processes. By now just note that the fact that init owns the pid number 1 tells us that it is the first process executed by a Unix-like operating system.

Another thing we can easily note is the presence of many processes, whose name ends with a "d": they are the so called "daemons" and are some of the most important processes of the system. Since this is just an introductory article we will not diffusely talk about deamons now: just say they are programs that sit waiting for specific events and when one of those events occur the daemons waiting for it perform some actions. A typical example of a daemon is sshd, which establishes or refuses SSH connections upon external requests.

The existence of daemons makes even clearer why multitasking is vital for a modern operating system: since there are many peripherals and connections to manage, through multitasking we can run many small specialized programs (call them deamons, services, components), each of which knows only how to manage that specific object. This is a typical concept of the Unix world, summarized by the sentence “do one thing and do it well”.

Conclusion¶

This article introduced you to the meaning of multitasking and to the reason of its implementation. You started to familiarize with the concept of process and the related jargon and discovered a little about the internal working of the operating system by looking at all the processes running in it.

Next article will dig into the fundamental concept of process forking, both from the theoretical point of view and with real C examples. We will also later talk about interprocess communication, multithreading, multiple CPUs and cores and scheduling.

Feel free to ask questions or point errors through comments or social networks.