bytemark
/
flexnbd-c
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
flexnbd-c/README.txt

404 lines
12 KiB
Plaintext
Raw Permalink Blame History

NAME
flexnbd - A fast NBD server
SYNOPSIS
flexnbd MODE [ ARGS ]
flexnbd serve --addr ADDR --port PORT --file FILE [--sock SOCK]
[--default-deny] [--killswitch] [global_option]* [acl_entry]*
flexnbd listen --addr ADDR --port PORT --file FILE [--sock SOCK]
[--default-deny] [global_option]* [acl_entry]*
flexnbd mirror --addr ADDR --port PORT --sock SOCK [--unlink]
[--bind BIND_ADDR] [global_option]*
flexnbd acl --sock SOCK [acl_entry]+ [global_option]*
flexnbd break --sock SOCK [global_option]*
flexnbd status --sock SOCK [global_option]*
flexnbd read --addr ADDR --port PORT --from OFFSET --size SIZE
[--bind BIND_ADDR] [global_option]*
flexnbd write --addr ADDR --port PORT --from OFFSET --size SIZE
[--bind BIND_ADDR] [global_option]*
flexnbd help [mode] [global_option]*
DESCRIPTION
Flexnbd is a fast NBD server which supports live migration. Live
migration is performed by writing the data to a new server. A failed
migration will be invisible to any connected clients.
Flexnbd tries quite hard to preserve sparsity of files it is serving,
even across migrations.
SERVE MODE
Serve a file.
$ flexnbd serve --addr <ADDR> --port <PORT> --file <FILE>
[--sock <SOCK>] [--default-deny] [-k] [global_option]*
[acl_entry]*
If any ACL entries are given (which should be IP
addresses), only those clients listed will be permitted to connect.
flexnbd will continue to serve until a SIGINT, SIGQUIT, or a successful
migration.
OPTIONS
--addr, -l ADDR
The address to listen on. Required.
--port, -p PORT
The port to listen on. Required.
--file, -f FILE
The file to serve. Must already exist. Required.
--sock, -s SOCK
Path to a control socket to open. You will need this if you want to
migrate, get the current status, or manipulate the access control
list.
--default-deny, -d
How to interpret an empty ACL. If --default-deny is given, an
empty ACL will let no clients connect. If it is not given, an
empty ACL will let any client connect.
--killswitch, -k
If set, we implement a 2-minute timeout on NBD requests and
responses. If a request takes longer than that to complete,
the client is disconnected. This is useful to keep broken
clients from breaking migrations, among other things.
LISTEN MODE
Listen for an inbound migration, and quit with a status of 0 on
completion.
$ flexnbd listen --addr ADDR --port PORT --file FILE
[--sock SOCK] [--default-deny] [global_option]*
[acl_entry]*
flexnbd will wait for a successful migration, and then quit. The file
to write the inbound migration data to must already exist before you
run 'flexnbd listen'.
Only one sender may connect to send data, and if the sender
disconnects part-way through the migration, the destination will
expect it to reconnect and retry the whole migration. It isn't safe
to assume that a partial migration can be resumed because the
destination has no knowledge of whether a client has made a write to
the source in the interim.
If the migration fails for a reason which the 'flexnbd listen' process
can't fix (say, a failed local write), it will exit with an error
status. In this case, the sender will continually retry the migration
until it succeeds, and you will need to restart the 'flexnbd listen'
process to allow that to happen.
OPTIONS
As for serve.
MIRROR MODE
Start a migration from the server with control socket SOCK to the server
listening at ADDR:PORT.
$ flexnbd mirror --addr ADDR --port PORT --sock SOCK [--unlink]
[--bind BIND_ADDR] [global_option]*
Migration can be a slow process. Rather than block the 'flexnbd mirror'
process until it completes, it will exit with a message of "Migration
started" once it has confirmation that the local server was able to
connect to ADDR:PORT and got an NBD header back. To check on the
progress of a running migration, use 'flexnbd status'.
If the destination unexpectedly disconnects part-way through the
migration, the source will attempt to reconnect and start the migration
again. It is not safe to resume the migration from where it left off
because the source can't see that the backing store behind the
destination is intact, or even on the same machine.
If the --unlink option is given, the local file will be deleted
immediately before the mirror connection is terminated. This allows
an otherwise-ambiguous situation to be resolved: if you don't unlink
the file and the flexnbd process at either end is terminated, it's not
possible to tell which copy of the data is canonical. Since the
unlink happens as soon as the sender knows that it has transmitted all
the data, there can be no ambiguity.
Note: files smaller than 4096 bytes cannot be mirrored.
OPTIONS
--addr, -l ADDR
The address of the remote server to migrate to. Required.
--port, -p PORT
The port of the remote server to migrate to. Required.
--sock, -s SOCK
The control socket of the local server to migrate from. Required.
--unlink, -u
Unlink the served file from the local filesystem after
successfully mirroring.
--bind, -b BIND_ADDR
The local address to bind to. You may need this if the remote
server is using an access control list.
BREAK MODE
Stop a running migration.
$ flexnbd break --sock SOCK [global_option]*
OPTIONS
--sock, -s SOCK
The control socket of the local server whose migration to stop.
Required.
ACL MODE
Set the access control list of the server with the control socket SOCK
to the given access control list entries.
$ flexnbd acl --sock SOCK [acl_entry]+ [global_option]*
ACL entries are given as IP addresses.
OPTIONS
--sock, -s SOCK
The control socket of the server whose ACL to replace. Required
STATUS MODE
Get the current status of the server with control socket SOCK.
$ flexnbd status --sock SOCK [global_option]*
The status will be printed to STDOUT. It is a space-separated list of
key=value pairs. The space character will never appear in a key or
value. Currently reported values are:
pid
The process id of the server listening on SOCK.
is_mirroring
'true' if this server is sending migration data, 'false' otherwise.
has_control
'false' if this server was started in 'listen' mode. 'true' otherwise.
OPTIONS
--sock, -s SOCK
The control socket of the server of interest. Required.
READ MODE
Connect to the server at ADDR:PORT, and read SIZE bytes starting at
OFFSET in a single NBD query.
$ flexnbd read --addr ADDR --port PORT --from OFFSET --size SIZE
[--bind BIND_ADDR] [global_option]*
The returned data will be echoed to STDOUT. In case of a remote ACL,
set the local source address to BIND_ADDR.
OPTIONS
--addr, -l ADDR
The address of the remote server. Required.
--port, -p PORT
The port of the remote server. Required.
--from, -F OFFSET
The byte offset to start reading from. Required. Maximum 2^62.
--size, -S SIZE
The number of bytes to read. Required. Maximum 2^30.
--bind, -b BIND_ADDR
The local address to bind to. You may need this if the remote
server is using an access control list.
WRITE MODE
Connect to the server at ADDR:PORT, and write SIZE bytes from STDIN
starting at OFFSET in a single NBD query.
$ cat ... | flexnbd write --addr ADDR --port PORT --from OFFSET
--size SIZE [--bind BIND_ADDR] [global_option]*
In case of a remote ACL, set the local source address to BIND_ADDR.
OPTIONS
--addr, -l ADDR
The address of the remote server. Required.
--port, -p PORT
The port of the remote server. Required.
--from, -F OFFSET
The byte offset to start writing from. Required. Maximum 2^62.
--size, -S SIZE
The number of bytes to write. Required. Maximum 2^30.
--bind, -b BIND_ADDR
The local address to bind to. You may need this if the remote
server is using an access control list.
HELP MODE
$ flexnbd help [mode] [global_option]*
Without mode, show the list of available modes. With mode, show help for that mode.
GLOBAL OPTIONS
--help, -h Show mode or global help.
--verbose, -v Output all available log information to STDERR.
--quiet, -q Output as little log information as possible to STDERR.
LOGGING
Log output is sent to STDERR. If --quiet is set, no output will be
seen unless the program termintes abnormally. If neither --quiet nor
--verbose are set, no output will be seen unless something goes wrong
with a specific request. If --verbose is given, every available log
message will be seen (which, for a debug build, is many). It is not an
error to set both --verbose and --quiet. The last one wins.
The log line format is:
<TIMESTAMP>:<LEVEL>:<PID> <THREAD> <SOURCEFILE:SOURCELINE>: <MSG>
<TIMESTAMP>
Time the log entry was made. This is expressed in terms of monotonic
ms.
<LEVEL>
This will be one of 'D', 'I', 'W', 'E', 'F' in increasing order of
severity. If flexnbd is started with the --quiet flag, only 'F'
will be seen. If it is started with the --verbose flag, any from 'I'
upwards will be seen. Only if you have a debug build and start it
with --verbose will you see 'D' entries.
<PID>
This is the process ID.
<THREAD>
There are several pthreads per flexnbd process: a main thread, a
serve thread, a thread per client, and possibly a pair of mirror
threads and a control thread. This field identifies which thread was
responsible for the log line.
<SOURCEFILE:SOURCELINE>
Identifies where in the source code this log line can be found.
<MSG>
A short message describing what's happening, how it's being done, or
if you're very lucky why it's going on.
EXAMPLES
SERVING A FILE
The simplest case is serving a file on the default nbd port:
$ cp /etc/passwd /tmp
$ flexnbd serve --file /tmp/passwd --addr 0.0.0.0 --port 4777 &
$ flexnbd read --addr 127.0.0.1 --port 4777 --from 0 --size 7
root:x:
$
READING SERVER STATUS
In order to read a server's status, we need it to open a control socket.
$ flexnbd serve --file /tmp/passwd --addr 0.0.0.0 --port 4777 \
--sock /tmp/flexnbd.sock
$ flexnbd status --sock /tmp/flexnbd.sock
pid=9635 is_mirroring=false has_control=true
$
Note that the status output is newline-terminated.
MIGRATING
To migrate, we need to provide a destination file of the right size.
$ dd if=/dev/urandom of=/tmp/data bs=1024 count=1K
$ truncate -s 1M /tmp/data.copy
$ flexnbd serve --file /tmp/data --addr 0.0.0.0 --port 4778 \
--sock /tmp/flex-source.sock &
$ flexnbd listen --file /tmp/data.copy --addr 0.0.0.0 --port 4779 \
--sock /tmp/flex-dest.sock &
$
Now we check the status of each server, to check that they are both in
the right state:
$ flexnbd status --sock /tmp/flex-source.sock
pid=9648 is_mirroring=false has_control=true
$ flexnbd status --sock /tmp/flex-dest.sock
pid=9651 is_mirroring=false has_control=false
$
With this knowledge in hand, we can start the migration:
$ flexnbd mirror --addr 127.0.0.1 --port 4779 \
--sock /tmp/flex-source.sock
Migration started
[1] + 9648 done flexnbd serve --addr 0.0.0.0 --port 4778
[2] + 9651 done flexnbd listen --addr 0.0.0.0 --port 4779
$
Note that because the file is so small in this case, we see the source
server quit soon after we start the migration, and the destination
exited at roughly the same time.
BUGS
Should be reported on GitHub at
* https://github.com/BytemarkHosting/flexnbd-c/issues
AUTHOR
Originally written by Alex Young <alex@blackkettle.org>.
Original concept and core code by Matthew Bloch <matthew@bytemark.co.uk>.
Proxy mode written by Nick Thomas <me@ur.gs>.
The full commit history is available on GitHub.
SEE ALSO
flexnbd-proxy(1), nbd-client(8), xnbd-server(8), xnbd-client(8)
COPYRIGHT
Copyright (c) 2012-2016 Bytemark Hosting Ltd. Free use of this
software is granted under the terms of the GNU General Public License
version 3 or later.
Reference in New Issue View Git Blame Copy Permalink