gcc 12.1.x (e.g. current OpenSUSE Tumbleweed, x86_64 only,
gcc-12-1.4.x86_64) reports:
tcp.c: In function ‘tcp_send_flag’:
tcp.c:1014:9: warning: writing 16 bytes into a region of size 0 [-Wstringop-overflow=]
1014 | memcpy(low_rtt_dst + hole++, &conn->a.a6, sizeof(conn->a.a6));
| ^
tcp.c:559:24: note: at offset -16 into destination object ‘low_rtt_dst’ of size 128
559 | static struct in6_addr low_rtt_dst[LOW_RTT_TABLE_SIZE];
|
but 'hole' can't be -1, because the low_rtt_dst table is guaranteed
to have a hole: if we happened to write to the last entry, we'll go
back to index 0 and clear that one.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
This feature is available in slirp4netns but was missing in passt and
pasta.
Given that we don't do dynamic memory allocation, we need to bind
sockets while parsing port configuration. This means we need to
process all other options first, as they might affect addressing and
IP version support. It also implies a minor rework of how TCP and UDP
implementations bind sockets.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Reported by Coverity: it doesn't see that tcp{4,6}_l2_buf_used are
set to zero by tcp_l2_data_buf_flush(), repeat that explicitly here.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
All instances were harmless, but it might be useful to have some
debug messages here and there. Reported by Coverity.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Field doff in struct tcp_hdr is 4 bits wide, so optlen in
tcp_tap_handler() is already bound, but make that explicit.
Reported by Coverity.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
The existing sizes provide no measurable differences in throughput
and packet rates at this point. They were probably needed as batched
implementations were not complete, but they can be decreased quite a
bit now.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
...we don't really need two extra bits, but it's easier to organise
things differently than to silence this.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Implement a packet abstraction providing boundary and size checks
based on packet descriptors: packets stored in a buffer can be queued
into a pool (without storage of its own), and data can be retrieved
referring to an index in the pool, specifying offset and length.
Checks ensure data is not read outside the boundaries of buffer and
descriptors, and that packets added to a pool are within the buffer
range with valid offset and indices.
This implies a wider rework: usage of the "queueing" part of the
abstraction mostly affects tap_handler_{passt,pasta}() functions and
their callees, while the "fetching" part affects all the guest or tap
facing implementations: TCP, UDP, ICMP, ARP, NDP, DHCP and DHCPv6
handlers.
Suggested-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
...by:
- storing the chained-hash next connection pointer as numeric
reference rather than as pointer
- storing the MSS as 14-bit value, and rounding it
- using only the effective amount of bits needed to store the hash
bucket number
- explicitly limiting window scaling factors to 4-bit values
(maximum factor is 14, from RFC 7323)
- scaling SO_SNDBUF values, and using a 8-bit representation for
the duplicate ACK sequence
- keeping window values unscaled, as received and sent
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
We can't take for granted that the hard limit for open files is
big enough as to allow to delay closing sockets to a timer.
Store the value of RTLIMIT_NOFILE we set at start, and use it to
understand if we're approaching the limit with pending, spliced
TCP connections. If that's the case, close sockets right away as
soon as they're not needed, instead of deferring this task to a
timer.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
With a lot of concurrent connections, the bitmap scan approach is
not really sustainable.
Switch to per-connection timerfd timers, set based on events and on
two new flags, ACK_FROM_TAP_DUE and ACK_TO_TAP_DUE. Timers are added
to the common epoll list, and implement the existing timeouts.
While at it, drop the CONN_ prefix from flag names, otherwise they
get quite long, and fix the logic to decide if a connection has a
local, possibly unreachable endpoint: we shouldn't go through the
rest of tcp_conn_from_tap() if we reset the connection due to a
successful bind(2), and we'll get EACCES if the port number is low.
Suggested by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
This should never happen, but there are no formal guarantees: ensure
socket numbers are below SOCKET_MAX.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Using events and flags instead of states makes the implementation
much more straightforward: actions are mostly centered on events
that occurred on the connection rather than states.
An example is given by the ESTABLISHED_SOCK_FIN_SENT and
FIN_WAIT_1_SOCK_FIN abominations: we don't actually care about
which side started closing the connection to handle closing of
connection halves.
Split out the spliced implementation, as it has very little in
common with the "regular" TCP path.
Refactor things here and there to improve clarity. Add helpers
to trace where resets and flag settings come from.
No functional changes intended.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
In pasta mode, when we get data from sockets and write it as single
frames to the tap device, we batch receive operations considerably,
and then (conceptually) split the data in many smaller writes.
It looked like an obvious choice, but performance is actually better
if we receive data in many small frame-sized recvmsg()/recvmmsg().
The syscall overhead with the previous behaviour, observed by perf,
comes predominantly from write operations, but receiving data in
shorter chunks probably improves cache locality by a considerable
amount.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
To reach (at least) a conceptually equivalent security level as
implemented by --enable-sandbox in slirp4netns, we need to create a
new mount namespace and pivot_root() into a new (empty) mountpoint, so
that passt and pasta can't access any filesystem resource after
initialisation.
While at it, also detach IPC, PID (only for passt, to prevent
vulnerabilities based on the knowledge of a target PID), and UTS
namespaces.
With this approach, if we apply the seccomp filters right after the
configuration step, the number of allowed syscalls grows further. To
prevent this, defer the application of seccomp policies after the
initialisation phase, before the main loop, that's where we expect bad
things to happen, potentially. This way, we get back to 22 allowed
syscalls for passt and 34 for pasta, on x86_64.
While at it, move #syscalls notes to specific code paths wherever it
conceptually makes sense.
We have to open all the file handles we'll ever need before
sandboxing:
- the packet capture file can only be opened once, drop instance
numbers from the default path and use the (pre-sandbox) PID instead
- /proc/net/tcp{,v6} and /proc/net/udp{,v6}, for automatic detection
of bound ports in pasta mode, are now opened only once, before
sandboxing, and their handles are stored in the execution context
- the UNIX domain socket for passt is also bound only once, before
sandboxing: to reject clients after the first one, instead of
closing the listening socket, keep it open, accept and immediately
discard new connection if we already have a valid one
Clarify the (unchanged) behaviour for --netns-only in the man page.
To actually make passt and pasta processes run in a separate PID
namespace, we need to unshare(CLONE_NEWPID) before forking to
background (if configured to do so). Introduce a small daemon()
implementation, __daemon(), that additionally saves the PID file
before forking. While running in foreground, the process itself can't
move to a new PID namespace (a process can't change the notion of its
own PID): mention that in the man page.
For some reason, fork() in a detached PID namespace causes SIGTERM
and SIGQUIT to be ignored, even if the handler is still reported as
SIG_DFL: add a signal handler that just exits.
We can now drop most of the pasta_child_handler() implementation,
that took care of terminating all processes running in the same
namespace, if pasta started a shell: the shell itself is now the
init process in that namespace, and all children will terminate
once the init process exits.
Issuing 'echo $$' in a detached PID namespace won't return the
actual namespace PID as seen from the init namespace: adapt
demo and test setup scripts to reflect that.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
clang-tidy from LLVM 13.0.1 reports some new warnings from these
checkers:
- altera-unroll-loops, altera-id-dependent-backward-branch: ignore
for the moment being, add a TODO item
- bugprone-easily-swappable-parameters: ignore, nothing to do about
those
- readability-function-cognitive-complexity: ignore for the moment
being, add a TODO item
- altera-struct-pack-align: ignore, alignment is forced in protocol
headers
- concurrency-mt-unsafe: ignore for the moment being, add a TODO
item
Fix bugprone-implicit-widening-of-multiplication-result warnings,
though, that's doable and they seem to make sense.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
The existing behaviour is not really practical: an automated agent in
charge of starting both qemu and passt would need to fork itself to
start passt, because passt won't fork to background until qemu
connects, and the agent needs to unblock to start qemu.
Instead of waiting for a connection to daemonise, do it right away as
soon as a socket is available: that can be considered an initialised
state already.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Depending on the C library, but not necessarily in all the
functions we use, statx() might be used instead of stat(),
getdents() instead of getdents64(), readlinkat() instead of
readlink(), openat() instead of open().
On aarch64, it's clone() and not fork(), and dup3() instead of
dup2() -- just allow the existing alternative instead of dealing
with per-arch selections.
Since glibc commit 9a7565403758 ("posix: Consolidate fork
implementation"), we need to allow set_robust_list() for
fork()/clone(), even in a single-threaded context.
On some architectures, epoll_pwait() is provided instead of
epoll_wait(), but never both. Same with newfstat() and
fstat(), sigreturn() and rt_sigreturn(), getdents64() and
getdents(), readlink() and readlinkat(), unlink() and
unlinkat(), whereas pipe() might not be available, but
pipe2() always is, exclusively or not.
Seen on Fedora 34: newfstatat() is used on top of fstat().
syslog() is an actual system call on some glibc/arch combinations,
instead of a connect()/send() implementation.
On ppc64 and ppc64le, _llseek(), recv(), send() and getuid()
are used. For ppc64 only: ugetrlimit() for the getrlimit()
implementation, plus sigreturn() and fcntl64().
On s390x, additionally, we need to allow socketcall() (on top
of socket()), and sigreturn() also for passt (not just for
pasta).
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Bitmap manipulating functions would otherwise refer to inconsistent
sets of bits on big-endian architectures. While at it, fix up a
couple of casts.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
tcpi_bytes_acked and tcpi_min_rtt are only available on recent
kernel versions: provide fall-back paths (incurring some grade of
performance penalty).
Support for getrandom() was introduced in Linux 3.17 and glibc 2.25:
provide an alternate mechanism for that as well, reading from
/dev/random.
Also check if NETLINK_GET_STRICT_CHK is defined before using it:
it's not strictly needed, we'll filter out irrelevant results from
netlink anyway.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
This is the only remaining Linux-specific include -- drop it to avoid
clang-tidy warnings and to make code more portable.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
This was actually fine "on the wire", but it's inconsistent with the
way we hash other addresses/protocols and also ends up with a wrong
endianness in captures in case we replace the address with our
default gateway.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
...mostly false positives, but a number of very relevant ones too,
in tcp_get_sndbuf(), tcp_conn_from_tap(), and siphash PREAMBLE().
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Unions and structs, you all have names now.
Take the chance to enable bugprone-reserved-identifier,
cert-dcl37-c, and cert-dcl51-cpp checkers in clang-tidy.
Provide a ffsl() weak declaration using gcc built-in.
Start reordering includes, but that's not enough for the
llvm-include-order checker yet.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Detect missing tcpi_snd_wnd in struct tcp_info at build time,
otherwise build fails with a pre-5.3 linux/tcp.h header.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
This shouldn't happen on any sane configuration, but I just met an
example of that: the default IPv6 gateway on the host is configured
with a global unicast address, we use that as source for RA, DHCPv6
replies, and the guest ignores it. Same later on if we talk TCP or
UDP and the guest has no idea where that address comes from.
Use our link-local address in case the gateway address is global.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
A mix of unchecked return values, a missing permission mask for
open(2) with O_CREAT, and some false positives from
-Wstringop-overflow and -Wmaybe-uninitialized.
Reported-by: Martin Hauke <mardnh@gmx.de>
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
For some reason, on 4.19, splice() doesn't honour SOCK_NONBLOCK from
accept4() while reading from a TCP socket. Pass SPLICE_F_NONBLOCK
explicitly in all cases.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
If the window isn't updated on !c->tcp.kernel_snd_wnd, we still
have to send ACKs if the ACK sequence was updated, or if an error
occurred while querying TCP_INFO.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
...and while at it, reverse the operands in the window equality
comparison to detect the need for fast re-transmit: it's easier
to read this way.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
I thought I'd get away with it, but no, after some clean-ups, I
finally got a socket with number 0. Fix up all the convenient,
yet botched assumptions.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Caching iov_len for messages from socket doesn't actually decrease
overhead by the tiniest bit, and added a lot of complexity. Drop
that.
Also drop the sendmmsg(), we don't need to send multiple messages
with TCP, as long as we make sure no messages with a length
descriptor are sent partially, qemu is fine with it.
Just like it's done for segments without data (flags), also defer
the sendmsg() for sending data segments, to improve batching.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Otherwise, tcp4_l2_flags_buf_t is not consistent with tcp4_l2_buf_t and
header fields get all mixed up in tcp_l2_buf_fill_headers().
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
List of allowed syscalls comes from comments in the form:
#syscalls <list>
for syscalls needed both in passt and pasta mode, and:
#syscalls:pasta <list>
#syscalls:passt <list>
for syscalls specifically needed in pasta or passt mode only.
seccomp.sh builds a list of BPF statements from those comments,
prefixed by a binary search tree to keep lookup fast.
While at it, clean up a bit the Makefile using wildcards.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Move netlink routines to their own file, and use netlink to configure
or fetch all the information we need, except for the TUNSETIFF ioctl.
Move pasta-specific functions to their own file as well, add
parameters and calls to configure the tap interface in the namespace.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Based on a patch from Giuseppe Scrivano, this adds the ability to:
- specify paths and names of target namespaces to join, instead of
a PID, also for user namespaces, with --userns
- request to join or create a network namespace only, without
entering or creating a user namespace, with --netns-only
- specify the base directory for netns mountpoints, with --nsrun-dir
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
[sbrivio: reworked logic to actually join the given namespaces when
they're not created, implemented --netns-only and --nsrun-dir,
updated pasta demo script and man page]
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
That's the first thing we have to do, before sending SYN, ACK:
if tcp_send_to_tap() fails, we'll get a lot of useless events
otherwise.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Now that we have a proper function checking when and how to send
ACKs and window updates, we don't need to duplicate this logic in
tcp_data_from_tap().
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
...using pre-cooked buffers, just like we do with other segments.
While at it, remove some code duplication by having separate
functions for updating ACK sequence and window, and for filling in
buffer headers.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Now that we fixed the issue with small receiving buffers, we can
safely increase this back and get slightly lower syscall overhead.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
If net.core.rmem_max and net.core.wmem_max sysctls have low values,
we can get bigger buffers by not trying to set them high -- the
kernel would lock their values to what we get.
Try, instead, to get bigger buffers by queueing as much as possible,
and if maximum values in tcp_wmem and tcp_rmem are bigger than this,
that will work.
While at it, drop QUICKACK option for non-spliced sockets, I set
that earlier by mistake.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
If the connection is local or the RTT was comparable to the time it
takes to queue a batch of messages, we can safely use a large MSS
regardless of the sending buffer, but otherwise not.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
If we start with a very small sending buffer, we can make the kernel
expand it if we cause the congestion window to get bigger, but this
won't reliably happen if we use just half (other half is accounted
as overhead).
Scale usage depending on its own size, we might eventually get some
retransmissions because we can't queue messages the sender sends us
in-window, but it's better than keeping that small buffer forever.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
...and from the sending socket only if the MTU is not configured.
Otherwise, a connection to a host from a local guest, with a
non-loopback destination address, will get its MSS from the MTU of the
outbound interface with that address, which is unnecessary as we know
the guest can send us larger segments.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Detecting bound ports at start-up time isn't terribly useful: do this
periodically instead, if configured.
This is only implemented for TCP at the moment, UDP is somewhat more
complicated: leave a TODO there.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
This introduces a number of fundamental changes that would be quite
messy to split. Summary:
- advertised window scaling can be as big as we want, we just need
to clamp window sizes to avoid exceeding the size of our "discard"
buffer for unacknowledged data from socket
- add macros to compare sequence numbers
- force sending ACK to guest/tap on PSH segments, always in pasta
mode, whenever we see an overlapping segment, or when we reach a
given threshold compared to our window
- we don't actually use recvmmsg() here, fix comments and label
- introduce pools for pre-opened sockets and pipes, to decrease
latency on new connections
- set receiving and sending buffer sizes to the maximum allowed,
kernel will clamp and round appropriately
- defer clean-up of spliced and non-spliced connection to timer
- in tcp_send_to_tap(), there's no need anymore to keep a large
buffer, shrink it down to what we actually need
- introduce SO_RCVLOWAT setting and activity tracking for spliced
connections, to coalesce data moved by splice() calls as much as
possible
- as we now have a compacted connection table, there's no need to
keep sparse bitmaps tracking connection activity -- simply go
through active connections with a loop in the timer handler
- always clamp the advertised window to half our sending buffer,
too, to minimise retransmissions from the guest/tap
- set TCP_QUICKACK for originating socket in spliced connections,
there's no need to delay them
- fix up timeout for unacknowledged data from socket
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Until now, messages would be passed to protocol handlers in a single
batch only if they happened to be dequeued in a row. Packets
interleaved between different connections would result in multiple
calls to the same protocol handler for a single connection.
Instead, keep track of incoming packet descriptors, arrange them in
sequences, and call protocol handlers only as we completely sorted
input messages in batches.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
That might just mean we shut down the socket -- but we still have to
go through the other states to ensure a orderly shutdown guest-side.
While at it, drop the EPOLLHUP check for unhandled states: we should
never hit that, but if we do, resetting the connection at that point
is probably the wrong thing to do.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Now that we dropped EPOLLET, we'll keep getting EPOLLRDHUP, and
possibly EPOLLIN, even if there's nothing to read anymore.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
EPOLLHUP just means we shut down one side of the connection on
*one* socket: remember, we have two sockets here.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
...throughput isn't everything: this leads (of course) to horrible
latency with small, sparse messages. As a consequence, there's no
need to set TCP_NODELAY either.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
If we're at the first message in a batch, it's safe to get the
window value from it, and there's no need to subtract anything for
a comparison on that's not even done -- we'll override it later in
any case if we find messages with a higher ACK sequence number.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
...tcp_handler_splice() doesn't guarantee we read all the available
data, the sending buffer might be full.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Checking it only when the cached value is smaller than the current
window of the receiver is not enough: it might shrink further while
the receiver window is growing.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
If we couldn't write the whole batch of received packets to the socket,
and we have missing segments, we still need to request their
retransmission right away, otherwise it will take ages for the guest to
figure out we're missing them.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
...instead of waiting for the remote peer to do that -- it's
especially important in case we request retransmissions from the
guest, but it also helps speeding up slow start. This should
probably be a configurable behaviour in the future.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Seen with iperf3: a control connection is established, no data flows
for a while, all segments are acknowledged. The socket starts closing
it, and we immediately time out because the last ACK from tap was one
minute before that.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Seen with iperf3: the first packet from socket (data connection) is
65520 bytes and doesn't fit in the window.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
This fixes a number of issues found with some heavier testing with
uperf and neper:
- in most closing states, we can still accept data, check for EPOLLIN
when appropriate
- introduce a new state, ESTABLISHED_SOCK_FIN_SENT, to track the fact
we already sent a FIN segment to the tap device, for proper sequence
number bookkeeping
- for pasta mode only: spliced connections also need tracking of
(inferred) FIN segments and clean half-pipe shutdowns
- streamline resetting epoll_wait bitmaps with a new function,
tcp_tap_epoll_mask(), instead of repeating the logic all over the
place
- set EPOLLET for tap connections too, whenever we are waiting for
EPOLLRDHUP or an event from the tap to proceed with data transfer,
to avoid useless loops with EPOLLIN set
- impose an additional limit on the sending window advertised to the
guest, given by SO_SNDBUF: it makes no sense to completely fill
the sending buffer and send a zero window: stop a bit before we
hit that
- handle *all* interrupted system calls as needed
- simplify the logic for reordering of out-of-order segments received
from tap: it's not a corner case, and the previous logic allowed
for deadloops
- fix comparison of seen IPv4 address when we get a new connection
from a socket directed to the configured guest address
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
This went lost in a recent rework: if the guest wants to connect
directly to the host, it can use the address of the default gateway.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
As data from socket is forwarded to the guest, sendmmsg() might send
fewer bytes than requested in three different ways:
- failing altogether with a negative error code -- ignore that,
we'll get an error on the UNIX domain socket later if there's
really an issue with it and reset the connection to the guest
- sending less than 'vlen' messages -- instead of assuming success
in that case and waiting for the guest to send a duplicate ACK
indicating missing data, update the sequence number according to
what was actually sent and spare some retransmissions
- somewhat unexpectedly to me, sending 'vlen' or less than 'vlen'
messages, returning up to 'vlen', with the last message being
partially sent, and no further indication of errors other than
the returned msg_len for the last partially sent message being
less than iov_len.
In this case, we would assume success and proceed as nothing
happened. However, qemu would fail to parse any further message,
having received a partial descriptor, and eventually close the
connection, logging:
serious error: oversized packet received,connection terminated.
as the length descriptor for the next message would be sourced
from the middle of the next successfully sent message, not from
its header.
Handle this by checking the msg_len returned for the last (even
partially) sent message, and force re-sending the missing bytes,
if any, with a blocking sendmsg() -- qemu must not receive
anything else than that anyway.
While at it, allow to send up to 64KiB for each message, the
previous 32KiB limit isn't actually required, and just switch to a
new message at each iteration on sending buffers, they are already
MSS-sized anyway, so the check in the loop isn't really needed.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
With a kernel older than 5.3 (no_snd_wnd set), ack_pending in
tcp_send_to_tap() might be true at the beginning of a new connection
initiated by a socket. This means we send the first SYN segment to the
tap together with ACK set, which is clearly invalid and triggers the
receiver to reply with an RST segment right away.
Set ack_pending to 0 whenever we're sending a SYN segment. In case of a
SYN, ACK segment sent by the caller, the caller passes the ACK flag
explicitly.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Socket-facing functions don't guarantee that all data is handled before
they return: stick to level-triggered mode for TCP sockets.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
...and while at it, fix an issue in the calculation of the last IOV
buffer size: if we can't receive enough data to fill up the window,
the last buffer can be filled completely.
Also streamline the code setting iovec lengths if cached values are
not matching.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
We won't necessarily have another choice to ACK in a timely fashion
if we skip ACKs from a number of states (including ESTABLISHED) when
there's enough window left. Check for ACKed bytes as soon as it makes
sense.
If the sending window is not reported by the kernel, ACK as soon as
we queue onto the socket, given that we're forced to use a rather
small window.
In FIN_WAIT_1_SOCK_FIN, we also have to account for the FIN flag sent
by the peer in the sequence.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Sending 64 frames in a batch looks quite bad when a duplicate ACK
comes right at the beginning of it. Lowering this to 32 doesn't
affect performance noticeably, with 16 the impact is more apparent.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Similar to UDP, but using a simple sendmsg() on iovec-style buffers
from tap instead, as we don't need to preserve message boundaries.
A quick test in PASTA mode, from namespace to init via tap:
# ip link set dev pasta0 mtu 16384
# iperf3 -c 192.168.1.222 -t 60
[...]
[ ID] Interval Transfer Bitrate
[ 5] 0.00-60.00 sec 80.4 GBytes 11.5 Gbits/sec receiver
# iperf3 -c 2a02:6d40:3cfc:3a01:2b20:4a6a:c25a:3056 -t 60
[...]
[ ID] Interval Transfer Bitrate
[ 5] 0.00-60.01 sec 39.9 GBytes 5.71 Gbits/sec receiver
# ip link set dev pasta0 mtu 65520
# iperf3 -c 192.168.1.222 -t 60
[...]
[ ID] Interval Transfer Bitrate
[ 5] 0.00-60.01 sec 88.7 GBytes 12.7 Gbits/sec receiver
# iperf3 -c 2a02:6d40:3cfc:3a01:2b20:4a6a:c25a:3056 -t 60
[...]
[ ID] Interval Transfer Bitrate
[ 5] 0.00-60.00 sec 79.5 GBytes 11.4 Gbits/sec receiver
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
There's no need to constantly query the socket for number of
acknowledged bytes if we're far from exhausting the sending window,
just do it if we're at least down to 90% of it.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Traffic with loopback source address will be forwarded to the direct
loopback connection in the namespace, and the tap interface is used
for the rest.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
This isn't optional: TCP streams must carry a unique, hard-to-guess,
non-zero label for each direction. Linux, probably among others,
will otherwise refuse to associate packets in a given stream to the
same connection.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Allow to bind IPv4 and IPv6 ports to tap, namespace or init separately.
Port numbers of TCP ports that are bound in a namespace are also bound
for UDP for convenience (e.g. iperf3), and IPv4 ports are always bound
if the corresponding IPv6 port is bound (socket might not have the
IPV6_V6ONLY option set). This will also be configurable later.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
...not just for loopback addresses, with the address of the default
gateway. Otherwise, the guest might receive packets with source and
destination set to the same address.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
This is actually reasonable in terms of memory consumption and
allows for better performance with local services.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
PASTA (Pack A Subtle Tap Abstraction) provides quasi-native host
connectivity to an otherwise disconnected, unprivileged network
and user namespace, similarly to slirp4netns. Given that the
implementation is largely overlapping with PASST, no separate binary
is built: 'pasta' (and 'passt4netns' for clarity) both link to
'passt', and the mode of operation is selected depending on how the
binary is invoked. Usage example:
$ unshare -rUn
# echo $$
1871759
$ ./pasta 1871759 # From another terminal
# udhcpc -i pasta0 2>/dev/null
# ping -c1 pasta.pizza
PING pasta.pizza (64.190.62.111) 56(84) bytes of data.
64 bytes from 64.190.62.111 (64.190.62.111): icmp_seq=1 ttl=255 time=34.6 ms
--- pasta.pizza ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 34.575/34.575/34.575/0.000 ms
# ping -c1 spaghetti.pizza
PING spaghetti.pizza(2606:4700:3034::6815:147a (2606:4700:3034::6815:147a)) 56 data bytes
64 bytes from 2606:4700:3034::6815:147a (2606:4700:3034::6815:147a): icmp_seq=1 ttl=255 time=29.0 ms
--- spaghetti.pizza ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 28.967/28.967/28.967/0.000 ms
This entails a major rework, especially with regard to the storage of
tracked connections and to the semantics of epoll(7) references.
Indexing TCP and UDP bindings merely by socket proved to be
inflexible and unsuitable to handle different connection flows: pasta
also provides Layer-2 to Layer-2 socket mapping between init and a
separate namespace for local connections, using a pair of splice()
system calls for TCP, and a recvmmsg()/sendmmsg() pair for UDP local
bindings. For instance, building on the previous example:
# ip link set dev lo up
# iperf3 -s
$ iperf3 -c ::1 -Z -w 32M -l 1024k -P2 | tail -n4
[SUM] 0.00-10.00 sec 52.3 GBytes 44.9 Gbits/sec 283 sender
[SUM] 0.00-10.43 sec 52.3 GBytes 43.1 Gbits/sec receiver
iperf Done.
epoll(7) references now include a generic part in order to
demultiplex data to the relevant protocol handler, using 24
bits for the socket number, and an opaque portion reserved for
usage by the single protocol handlers, in order to track sockets
back to corresponding connections and bindings.
A number of fixes pertaining to TCP state machine and congestion
window handling are also included here.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Before commit 8f7baad7f035 ("tcp: Add snd_wnd to TCP_INFO"), the
kernel didn't export tcpi_snd_wnd via TCP_INFO, which means we don't
know what's the window size of the receiver, socket-side.
To get TCP connections working in that case, ignore this value if
it's zero during handshake, and use the initial window value as
suggested by RFC 6928 (14 600 bytes, instead of 4 380 bytes), to
keep network performance usable.
To make the TCP dynamic responsive enough in this case, also check
the socket for available data whenever we get an ACK segment from
tap, instead of waiting until all the data from the tap is dequeued.
While at it, fix the window scaling value sent for SYN and SYN, ACK
segments: we want to increase the data pointer after writing the
option, not the value itself.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
During handshake, the initial SYN, ACK segment to the guest, send as
a response to the SYN segment, needs to report the unscaled value for
the window, given that the handshake hasn't completed yet.
While at it, fix the endianness for the window value in case TCP
parameters can't be queried via TCP_INFO and we need to use the
default value.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
and, given that the connection table is indexed by socket number,
we also need to increase MAX_CONNS now as the ICMP implementation
needs 2^17 sockets, that will be opened before TCP connections are
accepted.
This needs to be changed later: the connection table should be
indexed by a translated number -- we're wasting 2^17 table entries
otherwise. Move initialisation of TCP listening sockets as last
per-protocol initialisation, this will make it easier.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
This is in preparation for scatter-gather IO on the UDP receive path:
save a getsockname() syscall by setting a flag if we get the numbering
of all bound sockets in a strict sequence (expected, in practice) and
repurpose the tap buffer to be also a socket receive buffer, passing
it down to protocol handlers.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
As we support UDP forwarding for packets that are sent to local
ports, we actually need some kind of connection tracking for UDP.
While at it, this commit introduces a number of vaguely related fixes
for issues observed while trying this out. In detail:
- implement an explicit, albeit minimalistic, connection tracking
for UDP, to allow usage of ephemeral ports by the guest and by
the host at the same time, by binding them dynamically as needed,
and to allow mapping address changes for packets with a loopback
address as destination
- set the guest MAC address whenever we receive a packet from tap
instead of waiting for an ARP request, and set it to broadcast on
start, otherwise DHCPv6 might not work if all DHCPv6 requests time
out before the guest starts talking IPv4
- split context IPv6 address into address we assign, global or site
address seen on tap, and link-local address seen on tap, and make
sure we use the addresses we've seen as destination (link-local
choice depends on source address). Similarly, for IPv4, split into
address we assign and address we observe, and use the address we
observe as destination
- introduce a clock_gettime() syscall right after epoll_wait() wakes
up, so that we can remove all the other ones and pass the current
timestamp to tap and socket handlers -- this is additionally needed
by UDP to time out bindings to ephemeral ports and mappings between
loopback address and a local address
- rename sock_l4_add() to sock_l4(), no semantic changes intended
- include <arpa/inet.h> in passt.c before kernel headers so that we
can use <netinet/in.h> macros to check IPv6 address types, and
remove a duplicate <linux/ip.h> inclusion
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Seen with iperf3 server on tap side: connection state is SOCK_SYN_SENT,
we haven't got an ACK from the tap yet (that's why we're not in
ESTABLISHED), but a data packet comes. Don't read this data until we
reach the ESTABLISHED state, by keeping EPOLLIN disabled until that
point.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Avoid a bunch of syscalls on forwarding paths by:
- storing minimum and maximum file descriptor numbers for each
protocol, fall back to SO_PROTOCOL query only on overlaps
- allocating a larger receive buffer -- this can result in more
coalesced packets than sendmmsg() can take (UIO_MAXIOV, i.e. 1024),
so make sure we don't exceed that within a single call to protocol
tap handlers
- nesting the handling loop in tap_handler() in the receive loop,
so that we have better chances of filling our receive buffer in
fewer calls
- skipping the recvfrom() in the UDP handler on EPOLLERR -- there's
nothing to be done in that case
and while at it:
- restore the 20ms timer interval for periodic (TCP) events, I
accidentally changed that to 100ms in an earlier commit
- attempt using SO_ZEROCOPY for UDP -- if it's not available,
sendmmsg() will succeed anyway
- fix the handling of the status code from sendmmsg(), if it fails,
we'll try to discard the first message, hence return 1 from the
UDP handler
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
This is symmetric with tap operation and addressing model, and
allows again to reach the guest behind the tap interface by
contacting the local address.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Receive packets in batches from AF_UNIX, check if they can be sent
with a single syscall, and batch them up with sendmmsg() in case.
A bit rudimentary, currently only implemented for UDP, but it seems
to work.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
We don't need to keep small data as static variables, move the only
small variable we have so far to the new struct.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Replace the dummy, full array scan implementation, by a hash table
based on SipHash, with chained hashing for collisions.
This table is also statically allocated, and it's simply an array
of socket numbers. Connection entries are chained by pointers in
the connection entry itself, which now also contains socket number
and hash bucket index to keep removal reasonably fast.
New entries are inserted at the head of the chain, that is, the most
recently inserted entry is directly mapped from the bucket.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
We might receive out-of-order ACK packets from the tap device, just
like any other packet.
I guess I've been overcautious and regarded it as a condition we
can't recover from, but all that happens is that we have already seen
a higher ACK sequence number, which means that data has been already
received and discarded from the buffer. We have to ignore the lower
sequence number we receive later, though, because that would force
the buffer bookkeeping into throwing away more data than expected.
Drop the ACK sequence assignment from tcp_tap_handler(), which was
redundant, and let tcp_sock_consume() take exclusive care of that.
Now that tcp_sock_consume() can never fail, make it a void, and
drop checks from callers.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Implement siphash routines for initial TCP sequence numbers (12 bytes
input for IPv4, 36 bytes input for IPv6), and while at it, also
functions we'll use later on for hash table indices and TCP timestamp
offsets (with 8, 20, 32 bytes of input).
Use these to set the initial sequence number, according to RFC 6528,
for connections originating either from the tap device or from
sockets.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
A bunch of fixes not worth single commits at this stage, notably:
- make buffer, length parameter ordering consistent in ARP, DHCP,
NDP handlers
- strict checking of buffer, message and option length in DHCP
handler (a malicious client could have easily crashed it)
- set up forwarding for IPv4 and IPv6, and masquerading with nft for
IPv4, from demo script
- get rid of separate slow and fast timers, we don't save any
overhead that way
- stricter checking of buffer lengths as passed to tap handlers
- proper dequeuing from qemu socket back-end: I accidentally trashed
messages that were bundled up together in a single tap read
operation -- the length header tells us what's the size of the next
frame, but there's no apparent limit to the number of messages we
get with one single receive
- rework some bits of the TCP state machine, now passive and active
connection closes appear to be robust -- introduce a new
FIN_WAIT_1_SOCK_FIN state indicating a FIN_WAIT_1 with a FIN flag
from socket
- streamline TCP option parsing routine
- track TCP state changes to stderr (this is temporary, proper
debugging and syslogging support pending)
- observe that multiplying a number by four might very well change
its value, and this happens to be the case for the data offset
from the TCP header as we check if it's the same as the total
length to find out if it's a duplicated ACK segment
- recent estimates suggest that the duration of a millisecond is
closer to a million nanoseconds than a thousand of them, this
trend is now reflected into the timespec_diff_ms() convenience
routine
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
This is a reimplementation, partially building on the earlier draft,
that uses L4 sockets (SOCK_DGRAM, SOCK_STREAM) instead of SOCK_RAW,
providing L4-L2 translation functionality without requiring any
security capability.
Conceptually, this follows the design presented at:
https://gitlab.com/abologna/kubevirt-and-kvm/-/blob/master/Networking.md
The most significant novelty here comes from TCP and UDP translation
layers. In particular, the TCP state and translation logic follows
the intent of being minimalistic, without reimplementing a full TCP
stack in either direction, and synchronising as much as possible the
TCP dynamic and flows between guest and host kernel.
Another important introduction concerns addressing, port translation
and forwarding. The Layer 4 implementations now attempt to bind on
all unbound ports, in order to forward connections in a transparent
way.
While at it:
- the qemu 'tap' back-end can't be used as-is by qrap anymore,
because of explicit checks now introduced in qemu to ensure that
the corresponding file descriptor is actually a tap device. For
this reason, qrap now operates on a 'socket' back-end type,
accounting for and building the additional header reporting
frame length
- provide a demo script that sets up namespaces, addresses and
routes, and starts the daemon. A virtual machine started in the
network namespace, wrapped by qrap, will now directly interface
with passt and communicate using Layer 4 sockets provided by the
host kernel.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
