Since the vast majority (something like 94% on boot) are sequential small
reads, and since network latency is a major factor in determining how fast the
exposed device appears to the client, it makes sense for us to try to minimise
the number of network requests where we safely can.
This patch implements the simplest possible read cache in flexnbd-proxy. When
it receives a read request, if it's a small request then flexnbd-proxy will
double the length of data requested. On receiving the data from the upstream
server, flexnbd-proxy will return the first half to the downstream as normal,
and stash the second half in a buffer. If the very next request is a read, and
the offset and length match those of what we have stored, that second request
will be satisfied from the buffer without going out over the network.
The cache is invalidated by any non-read request, or by a disconnection.
It's safe to terminate the proxy at any point in its lifecycle, so
there's no point using signalfd() (and the associated select() +
non-blocking I/O gubbins) in it. We might want to use non-blocking
I/O in the future for other reasons, of course, at which point it
might become sensible to use signalfd() again. For now, this makes
us reliably responsive to TERM,INT and QUIT in a way that we weren't
previously.
In the event of a fiemap ioctl failing (when the file is on a tmpfs,
for instance), we would free() serve->allocation_map, but it would
remain not NULL, leading to segfaults in client.c when responding to
write requests.
Keeping the free() behaviour is more hassle than it's worth, as there
are synchronization problems with setting serve->allocation_map to
NULL, so we just omit the free() instead to avoid the segfault. This
is safe because we never consult the map until allocation_map_built is
set to true, and we never do that when the builder thread fails.
This makes it easier for the tests (and supervisor) to guarantee to be
able to connect to the server socket.
Also this patch moves freeing the mirror supervisor into the server
thread.
server startup (sparse write avoidance doesn't happen until it is finished).
Added mutex to bitset functions, which were already being called from
multiple threads. Rewrote allocation map builder to request file
information in multiple chunks, to avoid uninterruptible wait and dynamic
memory allocation.
