The previous OOM testing support would re-run the entire "main"
method each iteration, failing a different malloc each time.
When a test suite has 'n' allocations, the number of repeats
requires is (n * (n + 1) ) / 2. This gets very large, very
quickly.
This new OOM testing support instead integrates at the
virtTestRun level, so each individual test case gets repeated,
instead of the entire test suite. This means the values of
'n' are orders of magnitude smaller.
The simple usage is
$ VIR_TEST_OOM=1 ./qemuxml2argvtest
...
29) QEMU XML-2-ARGV clock-utc ... OK
Test OOM for nalloc=36 .................................... OK
30) QEMU XML-2-ARGV clock-localtime ... OK
Test OOM for nalloc=36 .................................... OK
31) QEMU XML-2-ARGV clock-france ... OK
Test OOM for nalloc=38 ...................................... OK
...
the second lines reports how many mallocs have to be failed, and thus
how many repeats of the test will be run.
If it crashes, then running under valgrind will often show the problem
$ VIR_TEST_OOM=1 ../run valgrind ./qemuxml2argvtest
When debugging problems it is also helpful to select an individual
test case
$ VIR_TEST_RANGE=30 VIR_TEST_OOM=1 ../run valgrind ./qemuxml2argvtest
When things get really tricky, it is possible to request that just
specific allocs are failed. eg to fail allocs 5 -> 12, use
$ VIR_TEST_RANGE=30 VIR_TEST_OOM=1:5-12 ../run valgrind ./qemuxml2argvtest
In the worse case, you might want to know the stack trace of the
alloc which was failed then VIR_TEST_OOM_TRACE can be set. If it
is set to 1 then it will only print if it thinks a mistake happened.
This is often not reliable, so setting it to 2 will make it print
the stack trace for every alloc that is failed.
$ VIR_TEST_OOM_TRACE=2 VIR_TEST_RANGE=30 VIR_TEST_OOM=1:5-5 ../run valgrind ./qemuxml2argvtest
30) QEMU XML-2-ARGV clock-localtime ... OK
Test OOM for nalloc=36 !virAllocN
/home/berrange/src/virt/libvirt/src/util/viralloc.c:180
virHashCreateFull
/home/berrange/src/virt/libvirt/src/util/virhash.c:144
virDomainDefParseXML
/home/berrange/src/virt/libvirt/src/conf/domain_conf.c:11745
virDomainDefParseNode
/home/berrange/src/virt/libvirt/src/conf/domain_conf.c:12646
virDomainDefParse
/home/berrange/src/virt/libvirt/src/conf/domain_conf.c:12590
testCompareXMLToArgvFiles
/home/berrange/src/virt/libvirt/tests/qemuxml2argvtest.c:106
virtTestRun
/home/berrange/src/virt/libvirt/tests/testutils.c:250
mymain
/home/berrange/src/virt/libvirt/tests/qemuxml2argvtest.c:418 (discriminator 2)
virtTestMain
/home/berrange/src/virt/libvirt/tests/testutils.c:750
??
??:0
_start
??:?
FAILED
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Use the new storage driver APIs to delete snapshot backing files in case
of failure instead of directly relying on "unlink". This will help us in
the future when we will be adding network based storage without local
representation in the host.
Introduce Wireshark dissector plugin which adds support to Wireshark
for dissecting libvirt RPC protocol.
Added following files to build Wireshark dissector from libvirt source
tree.
* tools/wireshark/*: Source tree of Wireshark dissector plugin.
Added followings to configure.ac or Makefile.am.
configure.ac
* --with-wireshark-dissector: Enable support for building Wireshark
dissector.
* --with-ws-plugindir: Specify wireshark plugin directory that dissector
will installed.
* Added tools/wireshark/{Makefile,src/Makefile} to AC_CONFIG_FILES.
Makefile.am
* Added tools/wireshark/ to SUBDIR.
Finish the cleanup of libvirt.c; all uses of virLib*Error have
now been converted to more canonical conventions.
* src/libvirt.c: Use virReportError in remaining errors.
(virLibConnError, virLibDomainError): Delete unused macros.
* cfg.mk (msg_gen_function): Drop unused names.
Signed-off-by: Eric Blake <eblake@redhat.com>
This partially reverts 5eb4b04211 and 62774afb6b.
Rewrite the domsuspend example from scratch. This time do it right.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
The domain events demo program isn't really tied to domain
events anymore, so rename it to object events.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Kill the use of atoi() and introduce syntax check to forbid it and it's
friends (atol, atoll, atof, atoq).
Also fix a typo in variable name holding the cylinders count of a disk
pool (apparently unused).
examples/domsuspend/suspend.c will need a larger scale refactor as the
whole example file is broken thus it will be exempted from the syntax
check for now.
The python binding now lives in
http://libvirt.org/git/?p=libvirt-python.git
that repo also provides an RPM which is upgrade compatible
with the old libvirt-python sub-RPM.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
This addition, however, requires some refactoring to be done. First of
all, to match the best practice we should detach the device prior
resetting it. That's why testVirPCIDeviceDetach is detaching all devices
within 0000:00:01.0 and 0000:00:03.0 range. Then, the brand new test
will reset the 0000:00:02.0 device, so the last testVirPCIDeviceReattach
can reattach all the devices back.
In order to perform a PCI device reset, the dummy config file is not
sufficient anymore and must be replaced with real PCI config (binary
mess). Such config files are to be stored under tests/virpcitestdata/
and ought to have '.config' suffix.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
This commit introduces yet another test under virpcitest:
virPCIDeviceDetach. However, in order to be able to do this, the
virpcimock needs to be extended to model the kernel behavior on PCI
device binding and unbinding (create 'driver' symlinks under the device
tree, check for device ID in driver's ID table, etc.)
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Among with this test introduce virpcimock as we need to mock some
syscalls, e.g. redirect open() of a file under /sys/bus/pci to a
stub sysfs tree.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
To make it easier to forbid future attempts at a confusing typedef
name ending in Ptr that isn't actually a pointer, insist that we
follow our preferred style of 'typedef foo *fooPtr'.
* cfg.mk (sc_forbid_const_pointer_typedef): Enforce consistent
style, to prevent issue fixed in previous storage patch.
* src/conf/capabilities.h (virCapsPtr): Fix offender.
* src/security/security_stack.c (virSecurityStackItemPtr):
Likewise.
* tests/qemucapabilitiestest.c (testQemuDataPtr): Likewise.
Signed-off-by: Eric Blake <eblake@redhat.com>
On RHEL 5, make syntax-check was failing because even strings like
'int isTempChain' matched the 'int i' rule. To be honest, I haven't
found the root cause, but the change added makes it work as expected
and keeps the proper behavior on newer systems as well.
The use of getenv is typically insecure, and we want people
to use our wrappers, to force them to think about setuid
needs.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
'const fooPtr' is the same as 'foo * const' (the pointer won't
change, but it's contents can). But in general, if an interface
is trying to be const-correct, it should be using 'const foo *'
(the pointer is to data that can't be changed).
Now that the code base has been cleaned, enforce it with a syntax
checker.
* cfg.mk (sc_forbid_const_pointer_typedef): New rule.
Signed-off-by: Eric Blake <eblake@redhat.com>
Start a test case for the virNetServerClient object, which
initially checks the creation of a virIdentityPtr object.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
We have been adding new .x files without keeping the list of
*-structs files up-to-date. This adds the support for the
recent additions.
In the process of testing this, I also noticed that Fedora 19's
use of dwarves-1.10 (providing pdwtags version 1.9) was producing
a single line on stderr but still giving enough useful info on
stdout that we could check structs; the real goal of checking
stderr separately from stdout was to avoid the bug in dwarves-1.9
where stdout was empty (see bug http://bugzilla.redhat.com/772358).
* src/Makefile.am (struct_prefix, PROTOCOL_STRUCTS): Add missing
struct tests.
(PDWTAGS): Work with Fedora 19 pdwtags.
(lxc_monitor_protocol-struct, lock_protocol-struct): New rules.
* src/lxc_monitor_protocol-structs: New file.
* src/lock_protocol-structs): Likewise.
* cfg.mk (generated_files): Enlarge list.
Signed-off-by: Eric Blake <eblake@redhat.com>
Automake has builtin support to prevent botched conditional nesting,
but only if you use:
if FOO
else !FOO
endif !FOO
An example error message when using the wrong name:
daemon/Makefile.am:378: error: else reminder (LIBVIRT_INIT_SCRIPT_SYSTEMD_TRUE) incompatible with current conditional: LIBVIRT_INIT_SCRIPT_SYSTEMD_FALSE
daemon/Makefile.am:381: error: endif reminder (LIBVIRT_INIT_SCRIPT_SYSTEMD_TRUE) incompatible with current conditional: LIBVIRT_INIT_SCRIPT_SYSTEMD_FALSE
As our makefiles tend to have quite a bit of nested conditionals,
it's better to take advantage of the benefits of the build system
double-checking that our conditionals are well-nested, but that
requires a syntax check to enforce our usage style.
Alas, unlike C preprocessor and spec files, we can't use indentation
to make it easier to see how deeply nesting goes.
* cfg.mk (sc_makefile_conditionals): New rule.
* daemon/Makefile.am: Enforce the style.
* gnulib/tests/Makefile.am: Likewise.
* python/Makefile.am: Likewise.
* src/Makefile.am: Likewise.
* tests/Makefile.am: Likewise.
* tools/Makefile.am: Likewise.
Signed-off-by: Eric Blake <eblake@redhat.com>
The use of <> is a security issue for RPC parameters, since a
malicious client can set a huge array length causing arbitrary
memory allocation in the daemon.
It is also a robustness issue for RPC return values, because if
the stream is corrupted, it can cause the client to also allocate
arbitrary memory.
Use a syntax-check rule to prohibit any use of <>
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
The virnettlscontexttest.c tests both virNetTLSContext
and virNetTLSSession functionality. Split into two
separate tests, to make the code size more manageable
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Commit 3d0e3c1 reintroduced a problem previously squelched in
commit 7e5aa78. Add a syntax check this time around.
util/virutil.c: In function 'virGetGroupList':
util/virutil.c:1015: error: 'for' loop initial declaration used outside C99 mode
* cfg.mk (sc_prohibit_loop_var_decl): New rule.
* src/util/virutil.c (virGetGroupList): Fix offender.
Signed-off-by: Eric Blake <eblake@redhat.com>
Makefiles are another easy file to enforce line limits.
Mostly straightforward; interesting tricks worth noting:
src/Makefile.am: $(confdir) was already defined, use it in more places
tests/Makefile.am: path_add and VG required some interesting compression
* cfg.mk (sc_prohibit_long_lines): Add another test.
* Makefile.am: Fix offenders.
* daemon/Makefile.am: Likewise.
* docs/Makefile.am: Likewise.
* python/Makefile.am: Likewise.
* src/Makefile.am: Likewise.
* tests/Makefile.am: Likewise.
Signed-off-by: Eric Blake <eblake@redhat.com>
Long lines are harder to read and harder to diff; in fact, if lines get
too long (> 1000 bytes), it starts causing issues where git send-email
refuses to send patches for the file. I've cleaned up the tests
directory in the past (see commits bd6c46f, 3b750d1), but new long
lines have been introduced in the meantime.
Why 90 instead of 80? Because there were too many tests on the fringe
edge, and I didn't want to edit that many files.
Add a syntax check to prevent future long lines.
* cfg.mk (sc_prohibit_long_lines): New rule.
* tests/qemuxml2argvdata/qemuxml2argv-*.args: Split lines of any
file with content longer than 90 columns.
* tests/storagevolxml2argvdata/*.argv: Likewise.
Signed-off-by: Eric Blake <eblake@redhat.com>
Add two syntax-check rules
- sc_prohibit_int_ijk - block use of 'int' as a data type
for any variables named 'i', 'j', 'k'
- sc_prohibit_int_iijjkk - block use of 'ii', 'jj', 'kk'
for any variable names
Actually, I'm turning this function into a macro as filename,
function name and line number needs to be passed. The new
function virAsprintfInternal is introduced with the extended set
of arguments.
Based on a report by Chandrashekar Shastri, at
https://bugzilla.redhat.com/show_bug.cgi?id=979360
On systems where git cannot access the outside world, a developer
can instead arrange to get a copy of gnulib at the right commit
via side channels (such as NFS share drives), set GNULIB_SRCDIR,
then use ./autogen.sh --no-git. In this setup, we will now
avoid direct use of git. Of course, this means no automatic
gnulib updates when libvirt.git updates its submodule, but it
is expected that any developer in such a situation is already
prepared to deal with the fallout.
* .gnulib: Update to latest, for bootstrap.
* bootstrap: Synchronize from gnulib.
* autogen.sh (no_git): Avoid git when requested.
* cfg.mk (_update_required): Skip automatic rerun of bootstrap if
we can't use git.
* docs/compiling.html.in: Document this setup.
* docs/hacking.html.in: Mention this.
* HACKING: Regenerate.
Signed-off-by: Eric Blake <eblake@redhat.com>
Osier Yang pointed out that I introduced a syntax error in my
syntax check (I really shouldn't make last-minute changes without
testing them....).
/bin/sh: -c: line 2: syntax error near unexpected token `;'
/bin/sh: -c: line 2: ` { echo 'maint.mk: incorrect whitespace, see HACKING for rules' 2>&; \'
make: *** [bracket-spacing-check] Error 1
* cfg.mk (bracket-spacing-check): Fix copy-and-paste error.
Signed-off-by: Eric Blake <eblake@redhat.com>
To ensure we don't regress and cause the need for further
cleanups, add a 'make syntax-check' rule that ensures new
files have proper copyright contents.
* cfg.mk (sc_copyright_address): Rename...
(sc_copyright_usage): ...and enhance.
Signed-off-by: Eric Blake <eblake@redhat.com>
https://www.gnu.org/licenses/gpl-howto.html states:
You should also include a copy of the license itself somewhere in the
distribution of your program. All programs, whether they are released
under the GPL or LGPL, should include the text version of the GPL. In
GNU programs the license is usually in a file called COPYING.
If you are releasing your program under the LGPL, you should also
include the text version of the LGPL, usually in a file called
COPYING.LESSER. Please note that, since the LGPL is a set of
additional permissions on top of the GPL, it's important to include
both licenses so users have all the materials they need to understand
their rights.
* configure.ac (COPYING): No more games with non-git file.
* COPYING: New file, copied from gnulib.
* COPYING.LIB: Rename...
* COPYING.LESSER: ...to this.
* .gitignore: Track licenses in git.
* cfg.mk (exclude_file_name_regexp--sc_copyright_address): Tweak
rule.
* libvirt.spec.in (daemon, client, python): Reflect rename.
Signed-off-by: Eric Blake <eblake@redhat.com>
Use of the select() system call is inherantly dangerous since
applications will hit a buffer overrun if any FD number exceeds
the size of the select set size (typically 1024). Replace the
two uses of select() with poll() and use cfg.mk to ban any
future use of select().
NB: This changes the phyp driver so that it uses an infinite
timeout, instead of busy-waiting for 1ms at a time.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
These all existed before virfile.c was created, and for some reason
weren't moved.
This is mostly straightfoward, although the syntax rule prohibiting
write() had to be changed to have an exception for virfile.c instead
of virutil.c.
This movement pointed out that there is a function called
virBuildPath(), and another almost identical function called
virFileBuildPath(). They really should be a single function, which
I'll take care of as soon as I figure out what the arglist should look
like.
The source code base needs to be adapted as well. Some files
include virutil.h just for the string related functions (here,
the include is substituted to match the new file), some include
virutil.h without any need (here, the include is removed), and
some require both.
POSIX says that both basename() and dirname() may return static
storage (aka they need not be thread-safe); and that they may but
not must modify their input argument. Furthermore, <libgen.h>
is not available on all platforms. For these reasons, you should
never use these functions in a multi-threaded library.
Gnulib instead recommends a way to avoid the portability nightmare:
gnulib's "dirname.h" provides useful thread-safe counterparts. The
obvious dir_name() and base_name() are GPL (because they malloc(),
but call exit() on failure) so we can't use them; but the LGPL
variants mdir_name() (malloc's or returns NULL) and last_component
(always points into the incoming string without modifying it,
differing from basename semantics only on corner cases like the
empty string that we shouldn't be hitting in the first place) are
already in use in libvirt. This finishes the swap over to the safe
functions.
* cfg.mk (sc_prohibit_libgen): New rule.
* src/util/vircgroup.c: Fix offenders.
* src/parallels/parallels_storage.c (parallelsPoolAddByDomain):
Likewise.
* src/parallels/parallels_network.c (parallelsGetBridgedNetInfo):
Likewise.
* src/node_device/node_device_udev.c (udevProcessSCSIHost)
(udevProcessSCSIDevice): Likewise.
* src/storage/storage_backend_disk.c
(virStorageBackendDiskDeleteVol): Likewise.
* src/util/virpci.c (virPCIGetDeviceAddressFromSysfsLink):
Likewise.
* src/util/virstoragefile.h (_virStorageFileMetadata): Avoid false
positive.
Signed-off-by: Eric Blake <eblake@redhat.com>
With this patch, include public headers in "" form is only allowed
for "internal.h". And only the external tools (examples|tools|python
|include/libvirt) can include the public headers in <> form.
Directories python/tools/examples should include them in <> form,
though this patch allows "" form in these directories by excluding
them, a later patch will do the cleanup.
Some aspects of the cgroups setup / detection code are quite subtle
and easy to break. It would greatly benefit from unit testing, but
this is difficult because the test suite won't have privileges to
play around with cgroups. The solution is to use monkey patching
via LD_PRELOAD to override the fopen, open, mkdir, access functions
to redirect access of cgroups files to some magic stubs in the
test suite.
Using this we provide custom content for the /proc/cgroup and
/proc/self/mounts files which report a fixed cgroup setup. We
then override open/mkdir/access so that access to the cgroups
filesystem gets redirected into files in a temporary directory
tree in the test suite build dir.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
We've already scrubbed for comparisons of 'uid_t == -1' (which fail
on platforms where uid_t is a u16), but another one snuck in.
* src/util/virutil.c (virSetUIDGIDWithCaps): Correct uid comparison.
* cfg.mk (sc_prohibit_risky_id_promotion): New rule.
testutils.c likes to print summaries after a test completes,
including if it failed. But if the test outright exit()s,
this summary is skipped. Enforce that we return instead of exit.
* cfg.mk (sc_prohibit_exit_in_tests): New syntax check.
* tests/commandhelper.c (main): Fix offenders.
* tests/qemumonitorjsontest.c (mymain): Likewise.
* tests/seclabeltest.c (main): Likewise.
* tests/securityselinuxlabeltest.c (mymain): Likewise.
* tests/securityselinuxtest.c (mymain): Likewise.
* tests/testutils.h (VIRT_TEST_MAIN_PRELOAD): Likewise.
* tests/testutils.c (virtTestMain): Likewise.
(virtTestCaptureProgramOutput): Use symbolic name.
