1
0
mirror of https://passt.top/passt synced 2024-11-05 20:31:11 +00:00
Commit Graph

21 Commits

Author SHA1 Message Date
David Gibson
e4ecb6d795 test: Move passt_test_log_pipe to state directory
We use this fifo to send messages to the information pane.  Put it in the
state directory so it doesn't need its own cleanup.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2022-09-13 11:12:41 +02:00
David Gibson
8cb7d5b461 test: Create common state directories for temporary files
The test scripts create a bunch of temporary files to keep track of
internal state.  Some are made in /tmp with individual mktemp calls, some
go in the passt source directory, and some go in $LOGDIR.  This can
sometimes make it messy to clean up after failed test runs.

Start cleaning this up by creating a single "state" directory ($STATEBASE)
in /tmp for all the state or temporary files used by a single test run.
Clean it up automatically in cleanup() - except when DEBUG==1, because
those files can be useful for debugging test script failures.

We create subdirectories under $STATEBASE for each setup function, exposed
as $STATESETUP.  We also create subdirectories for each test script and
expose those to the scripts as __STATEDIR__.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2022-09-13 11:12:41 +02:00
David Gibson
1e4a15398f test: Actually run cleanup function
We install a cleanup() function with 'trap' in order to clean up temporary
files we generate during the tests.  However, we deinstall it after
run_term, which means it won't run in most of the cases where it would be
useful.  Even if "run from_term" exits with an error, that error will be
hidden from the run_term wrapper because it's within a tmux session, so we
will return from run_term normally, uninstall the trap and never clean up.

In fact there's no reason to uninstall the trap at all, it works just as
well on the success exit path as an error exit path.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2022-09-13 11:12:41 +02:00
David Gibson
594c2f9371 test: Group tests by mode then protocol, rather than the reverse
For example, passt/dhcp rather than dhcp/passt.  This is more
consistent with the two_guests and other test groups, and makes some
other cleanups simpler.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2022-09-13 11:12:41 +02:00
David Gibson
c2f248588b test: Integration of old-style pane execution and new context execution
We're creating a system for tests to more reliably execute commands in
various contexts (e.g. host, guest, namespace).  That transition is going
to happen over a number of steps though, so in the meantime we need to deal
with both the old-style issuing of commands via typing into and screen
scraping tmux panels, and the new-style system for executing commands in
context.

Introduce some transitional helpers which will issue a command via context
if the requested context is initialized, but will otherwise fall back to
the old style tmux panel based method.  Re-implement the various test DSL
commands in terms of these new helpers.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2022-09-13 05:32:00 +02:00
David Gibson
3fdb0747f3 test: Log debugging output from test script
The test scripts run with sh -e, which means they will stop if any commands
return an error.  That's generally desirable, because we won't continue
after things are hopeless due to an earlier step failing.

Unfortunately, the tmux setup we run the script in means it's not obvious
where any error messages related to such a failure will go.  Depending on
exactly where the error occurs they might go to the original terminal
hidden behind tmux, or they might go to a tmux panel that's not visible in
the normal layouts.

To make it easier to find such error message, redirect direct output and
errors from the test script itself to a 'script.log' file in the logs
directory.  When in DEBUG=1 mode, additionaly 'set -x' so we log all the
commands we execute to that file.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2022-08-20 19:07:12 +02:00
David Gibson
69126d4d48 test: Use shutdown test for pasta
For the passt and passt_in_ns tests we have a "shutdown" testcase that
checks for any errors from the passt process we were using (including
valgrind warnings).  Do the same for pasta tests, so that we catch any
error codes from the pasta process.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2022-08-20 19:07:12 +02:00
David Gibson
9224af1494 test: Rename slightly misleading "valgrind" tests
The "valgrind" test cases are designed to pick up errors reported when
passt is running under valgrind.  But what it actually does is just kill
the passt process, then see if it had a non-zero exit code.  That means it
will equally well pick up any other problems which caused passt to exit
with an error status: either something detected within passt or as a result
of passt being killed by an unexpected signal.

The fact that the "valgrind" test is actually responsible for shutting down
the passt process is non-obvious and can lead to problems when selectively
running tests during debugging.

Rename the "valgrind" tests to "shutdown" tests and run it regardless of
whether we're using valgrind or not.  This allows us to remove an ugly
speacial case in the passt_in_ns teardown code.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2022-08-20 19:07:12 +02:00
David Gibson
2fa308ac6e test: Split setup/teardown functions for build and distro tests
Currently the build tests and distro tests share a common setup function.
That works for now, but changes we want to make will mean they need
slightly different setup, so split the setup functions in preparation.

Currently, neither build nor distro tests have any teardown function.
Again, future changes are going to mean we need to do some teardown, so
create some empty for now teardown functions in preparation.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2022-08-20 19:07:12 +02:00
David Gibson
c8756034b7 test: Split cppcheck and clang-tidy tests into different files
Both clang-tidy and cppcheck linting are handled by the same test file,
test/build/static_checkers.  The two linters are independent of each other
though, and each one takes quite a long time.  Split them into separate
files to make it easier to control which are executed from the top level
test script.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2022-08-20 19:07:12 +02:00
David Gibson
a832a44e67 tests: Explicitly list test files in test/run, remove "onlyfor" support
Currently test/run uses wildcards to run all of the tests in a directory.
However, that wildcard list is filtered down by the "onlyfor" directives
in the test files... usually to a single file.

Therefore, just explicitly list the files we *really* want to run for this
test mode.  This makes it easier to see at the top level what tests will
be executed, and to change that list temporarily while debugging specific
failures.

This means the "onlyfor" directive no longer has any purpose, and we can
remove it.  "onlyfor" was also the only used of the $MODE variable, so we
can remove that too.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2022-07-14 01:32:42 +02:00
David Gibson
544f790bf8 tests: Don't automatically traverse directories of test files
The top level listing control of which tests to run is in test/run, however
it uses the test() function which runs an entire directory of test files,
filtered by some criteria.  This makes it awkward to narrow down to a
subset of tests when debugging a specific failure.

To make this easier, have test() take an explicit list of test files to
run, and have the caller in test/run handle the directory traversal.  The
construct we use for this is pretty awkward to handle the fact that we're
in the source tree root directory rather than test/ at this point in
test/run.  Later cleanups will improve that.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2022-07-14 01:32:42 +02:00
Stefano Brivio
b4d41ac4b6 test/run: Return 0 from run(), exit value already reflects failures
There's no need to return non-zero if there have been failures in
run(), because the exit value is already determined from the number
of failures reported in the log file.

Return zero, so that this doesn't cause the script to fail, given we
now run it with -e.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-05-19 15:24:15 +02:00
David Gibson
3e0641f91f tests: Don't ignore errors during script
Most commands issued during the testing scripts aren't explicitly checked
for errors.  Therefore, if they fail, the shell will just keep on
executing.  This makes it difficult to figure out where things started
going wrong if things fall over.

Run the whole script with the set -e mode so that it will exit in the case
of any (unchecked) failing command.  To make this work we do need to add
explicit checks / fallbacks for some commands which we expect to fail.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
[sbrivio: use sh -e instead of setting -e later, so that we don't miss
 anything before set -e is issued]
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-05-19 15:24:15 +02:00
David Gibson
81c50acb21 tests: Remove unused XVFB variable
The XVFB variable is initialized at the beginning of test/run then never
used again.  I'm assuming it's a leftover from some ealier iteration.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2022-05-19 15:24:15 +02:00
Stefano Brivio
66a95e331e test, seccomp, Makefile: Switch to valgrind runs for passt functional tests
Pass to seccomp.sh a list of additional syscalls valgrind needs as
EXTRA_SYSCALLS in a new 'valgrind' make target, and add corresponding
support in seccomp.sh itself.

In test setup functions, start passt with valgrind, but not for
performance tests.

Add tests checking that valgrind exits without errors after all the
other tests in the group are done.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-03-29 15:35:38 +02:00
Stefano Brivio
337f55166f demo, ci: Switch to asciinema(1) for terminal recordings
For demos, cool-retro-term(1) looked fancier, but several threads of
that and ffmpeg(1) are just messing up with performance testing.

The CI videos started getting really big as well, and they were
difficult to read.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-02-22 18:36:24 +01:00
Stefano Brivio
be2a7898e9 test: Add demo for Podman with pasta
...showing setup steps, some peculiarities as --net option, and a
general side-to-side comparison with slirp4netns(1), including
"quick" TCP and UDP throughput and latency benchmarks.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-02-22 18:34:44 +01:00
Stefano Brivio
6e61b4040a test: Add distribution tests for several architectures and kernel versions
The new tests check build and a simple case with pasta sending a
short message in both directions (namespace to init, init to
namespace).

Tests cover a mix of Debian, Fedora, OpenSUSE and Ubuntu combinations
on aarch64, i386, ppc64, ppc64le, s390x, x86_64.

Builds tested starting from approximately glibc 2.19, gcc 4.7, and
actual functionality approximately from 4.4 kernels, glibc 2.25,
gcc 4.8, all the way up to current glibc/gcc/kernel versions.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-01-28 18:51:50 +01:00
Stefano Brivio
4e5129719d test: Record CI and demo videos in Xvfb by default, fix demo setup sequence
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-09-29 16:45:26 +02:00
Stefano Brivio
061519b562 test: Add CI/demo scripts
Not really quick, definitely dirty.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-09-27 15:10:35 +02:00