cf81c85bf5
Right now, the logic that takes care of deciding whether expensive tests should be run or not is not working correctly: more specifically, it's not possible to use something like $ VIR_TEST_EXPENSIVE=1 ninja test to override the default choice, because in meson.build we always pass an explicit value that overrides whatever is present in the environment. We could implement logic to make this work properly, but that would require some refactoring of our test infrastructure and is arguably of little value given that running $ meson build -Dexpensive_tests=enabled is very fast, so let's just stop telling users about the variable instead and call it a day. Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
6.4 KiB
Advanced test suite usage
The basic requirement before submitting changes to libvirt is that
$ ninja testsucceed after each commit.
The libvirt test suite, however, support additional features: for example, it's possible to look for memory leaks and similar issues by running
$ meson test --setup valgrindValgrind is a test that checks for memory management issues, such as leaks or use of uninitialized variables.
Some tests are skipped by default in a development environment, based on the time they take in comparison to the likelihood that those tests will turn up problems during incremental builds. These tests default to being run when building from a tarball or with the configure option -Dexpensive_tests=enabled.
If you encounter any failing tests, the VIR_TEST_DEBUG environment variable may provide extra information to debug the failures. Larger values of VIR_TEST_DEBUG may provide larger amounts of information:
$ VIR_TEST_DEBUG=1 ninja test (or)
$ VIR_TEST_DEBUG=2 ninja testWhen debugging failures during development, it is possible to focus in on just the failing subtests by using VIR_TEST_RANGE. I.e. to run all tests from 3 to 20 with the exception of tests 6 and 16, use:
$ VIR_TEST_DEBUG=1 VIR_TEST_RANGE=3-5,7-20,^16 ./run tests/qemuxml2argvtestAlso, individual tests can be run from inside the tests/
directory, like:
$ ./qemuxml2xmltestIf you are adding new test cases, or making changes that alter existing test output, you can use the environment variable VIR_TEST_REGENERATE_OUTPUT to quickly update the saved test data. Of course you still need to review the changes VERY CAREFULLY to ensure they are correct.
$ VIR_TEST_REGENERATE_OUTPUT=1 ./qemuxml2argvtestThere is also a ./run script at the top level, to make
it easier to run programs that have not yet been installed, as well as
to wrap invocations of various tests under gdb or Valgrind.
When running our test suite it may happen that the test result is
nondeterministic because of the test suite relying on a particular file
in the system being accessible or having some specific value. To catch
this kind of errors, the test suite has a module for that prints any
path touched that fulfils constraints described above into a file. To
enable it just set VIR_TEST_FILE_ACCESS environment
variable. Then VIR_TEST_FILE_ACCESS_OUTPUT environment
variable can alter location where the file is stored.
$ VIR_TEST_FILE_ACCESS=1 VIR_TEST_FILE_ACCESS_OUTPUT="/tmp/file_access.txt" ./qemuxml2argvtest#. The Valgrind test should produce similar output to
ninja test. If the output has traces within libvirt API's,
then investigation is required in order to determine the cause of the
issue. Output such as the following indicates some sort of leak:
==5414== 4 bytes in 1 blocks are definitely lost in loss record 3 of 89
==5414== at 0x4A0881C: malloc (vg_replace_malloc.c:270)
==5414== by 0x34DE0AAB85: xmlStrndup (in /usr/lib64/libxml2.so.2.7.8)
==5414== by 0x4CC97A6: virDomainVideoDefParseXML (domain_conf.c:7410)
==5414== by 0x4CD581D: virDomainDefParseXML (domain_conf.c:10188)
==5414== by 0x4CD8C73: virDomainDefParseNode (domain_conf.c:10640)
==5414== by 0x4CD8DDB: virDomainDefParse (domain_conf.c:10590)
==5414== by 0x41CB1D: testCompareXMLToArgvHelper (qemuxml2argvtest.c:100)
==5414== by 0x41E20F: virtTestRun (testutils.c:161)
==5414== by 0x41C7CB: mymain (qemuxml2argvtest.c:866)
==5414== by 0x41E84A: virtTestMain (testutils.c:723)
==5414== by 0x34D9021734: (below main) (in /usr/lib64/libc-2.15.so)In this example, the virDomainDefParseXML() had an error
path where the virDomainVideoDefPtr video pointer was not
properly disposed. By simply adding a
virDomainVideoDefFree(video); in the error path, the issue
was resolved.
Another common mistake is calling a printing function, such as
VIR_DEBUG() without initializing a variable to be printed.
The following example involved a call which could return an error, but
not set variables passed by reference to the call. The solution was to
initialize the variables prior to the call.
==4749== Use of uninitialised value of size 8
==4749== at 0x34D904650B: _itoa_word (in /usr/lib64/libc-2.15.so)
==4749== by 0x34D9049118: vfprintf (in /usr/lib64/libc-2.15.so)
==4749== by 0x34D9108F60: __vasprintf_chk (in /usr/lib64/libc-2.15.so)
==4749== by 0x4CAEEF7: virVasprintf (stdio2.h:199)
==4749== by 0x4C8A55E: virLogVMessage (virlog.c:814)
==4749== by 0x4C8AA96: virLogMessage (virlog.c:751)
==4749== by 0x4DA0056: virNetTLSContextCheckCertKeyUsage (virnettlscontext.c:225)
==4749== by 0x4DA06DB: virNetTLSContextCheckCert (virnettlscontext.c:439)
==4749== by 0x4DA1620: virNetTLSContextNew (virnettlscontext.c:562)
==4749== by 0x4DA26FC: virNetTLSContextNewServer (virnettlscontext.c:927)
==4749== by 0x409C39: testTLSContextInit (virnettlscontexttest.c:467)
==4749== by 0x40AB8F: virtTestRun (testutils.c:161)Valgrind will also find some false positives or code paths which cannot be resolved by making changes to the libvirt code. For these paths, it is possible to add a filter to avoid the errors. For example:
==4643== 7 bytes in 1 blocks are possibly lost in loss record 4 of 20
==4643== at 0x4A0881C: malloc (vg_replace_malloc.c:270)
==4643== by 0x34D90853F1: strdup (in /usr/lib64/libc-2.15.so)
==4643== by 0x34EEC2C08A: ??? (in /usr/lib64/libnl.so.1.1)
==4643== by 0x34EEC15B81: ??? (in /usr/lib64/libnl.so.1.1)
==4643== by 0x34D8C0EE15: call_init.part.0 (in /usr/lib64/ld-2.15.so)
==4643== by 0x34D8C0EECF: _dl_init (in /usr/lib64/ld-2.15.so)
==4643== by 0x34D8C01569: ??? (in /usr/lib64/ld-2.15.so)In this instance, it is acceptable to modify the
tests/.valgrind.supp file in order to add a suppression
filter. The filter should be unique enough to not suppress real leaks,
but it should be generic enough to cover multiple code paths. The format
of the entry can be found in the documentation found at the Valgrind home page. The following trace
was added to tests/.valgrind.supp in order to suppress the
warning:
{
dlInitMemoryLeak1
Memcheck:Leak
fun:?alloc
...
fun:call_init.part.0
fun:_dl_init
...
obj:*/lib*/ld-2.*so*
}