Safe signal idioms

Signals on Unix-like systems are notoriously difficult to handle correctly. Many functions are unsafe to call from C signal handlers. Worse, techniques that are reliable in multithreaded programs can cause deadlocks in signal handlers. And because signals are asynchronous and concurrent, signal handling is susceptible to race conditions.

The safest way to handle signals is to not attempt to handle them at all. You don't need to handle signals if you just want SIGTERM to terminate your program, as the kernel will do that by default. But you may need custom signal handling if you want to:

• Do something before shutting down (e.g., send a goodbye message on a socket, or remove a temporary directory).
• Take some action other than terminating (e.g., reload a config file upon SIGHUP).

This post covers four idioms for handling signals safely:

• Raise an exception
• Set a flag
• Use sigtimedwait
• Use epoll and the self-pipe trick

The key to safe signal handling is to avoid doing real work inside the signal handler, and instead defer to a later point when the program is in a consistent state. Each of these idioms accomplishes this in a different way.

Our motivating example is a network server that listens to a UDP socket and echoes whatever it receives. On receipt of SIGTERM, we want to close the socket and call a cleanup function.

sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.bind(("localhost", 4444))
while True:
    msg, addr = sock.recvfrom(4096)
    sock.sendto(msg, addr)

Raise an exception

Raise an exception in the signal handler, and use try/except/finally statements to clean up. The exception will be raised from whatever line your program was executing when the signal was delivered, so you must wrap your entire loop in the try statement.

def sighandler(_signo, _frame):
    raise SystemExit

signal.signal(signal.SIGTERM, sighandler)
print(os.getpid())

sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.bind(("localhost", 4444))
try:
    while True:
        msg, addr = sock.recvfrom(4096)
        sock.sendto(msg, addr)
finally:
    sock.close()
    cleanup()

This works well with SIGTERM, but the control flow is awkward for SIGHUP.

Set a flag

Set a global flag inside the signal handler and check it in the main loop.

shutdown_requested = False

def sighandler(_signo, _frame):
    global shutdown_requested
    shutdown_requested = True

signal.signal(signal.SIGTERM, sighandler)
print(os.getpid())

sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.bind(("localhost", 4444))
while True:
    msg, addr = sock.recvfrom(4096)
    sock.sendto(msg, addr)
    if shutdown_requested:
        sock.close()
        cleanup()
        break

• Con: The process will remain blocked inside of recvfrom until the next time a client sends a message, so signal handling may be delayed indefinitely.

sigtimedwait

Mask the signal at start-up, and call sigtimedwait with a time-out of 0 to check if the signal was delivered.

sigset = {signal.SIGTERM}
signal.pthread_sigmask(signal.SIG_BLOCK, sigset)
print(os.getpid())

sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.bind(("localhost", 4444))
while True:
    msg, addr = sock.recvfrom(4096)
    sock.sendto(msg, addr)
    if signal.sigtimedwait(sigset, 0) is not None:
        sock.close()
        cleanup()
        break

• Con: As with "Set the flag", the signal will not be handled until after the next client message.

epoll and the self-pipe trick

If you are already doing asynchronous I/O using epoll, you can write to a pipe inside the signal handler and select on the pipe in the epoll loop. This is the self-pipe trick.

shutdown_reader, shutdown_writer = os.pipe()

def sighandler(signo, _frame):
    os.write(shutdown_writer, bytes([signo]))

signal.signal(signal.SIGTERM, sighandler)
print(os.getpid())

epoll = select.epoll()
epoll.register(shutdown_reader, select.EPOLLIN)

sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.bind(("localhost", 4444))
epoll.register(sock.fileno(), select.EPOLLIN)
while True:
    events = epoll.poll()
    for fd, _ in events:
        if fd == sock.fileno():
            msg, addr = sock.recvfrom(4096)
            sock.sendto(msg, addr)
        else:
            # could read from pipe to discern signo if necessary
            sock.close()
            cleanup()
            sys.exit(0)  # can't break because of inner loop

The signal will be delivered promptly, and SIGTERM and SIGHUP can be handled separately without difficulty. But it does require you to write your whole program to use epoll or an equivalent asynchronous API.