More bug extermination in the category of:
Error: CHECKED_RETURN:
/libvirt/src/conf/network_conf.c:595:
check_return: Calling function "virAsprintf" without checking return value (as is done elsewhere 515 out of 543 times).
/libvirt/src/qemu/qemu_process.c:2780:
unchecked_value: No check of the return value of "virAsprintf(&msg, "was paused (%s)", virDomainPausedReasonTypeToString(reason))".
/libvirt/tests/commandtest.c:809:
check_return: Calling function "setsid" without checking return value (as is done elsewhere 4 out of 5 times).
/libvirt/tests/commandtest.c:830:
unchecked_value: No check of the return value of "virTestGetDebug()".
/libvirt/tests/commandtest.c:831:
check_return: Calling function "virTestGetVerbose" without checking return value (as is done elsewhere 41 out of 42 times).
/libvirt/tests/commandtest.c:833:
check_return: Calling function "virInitialize" without checking return value (as is done elsewhere 18 out of 21 times).
One note about the error in commandtest line 809: setsid() seems to fail when running the test -- could be removed ?
Return statements with parameter enclosed in parentheses were modified
and parentheses were removed. The whole change was scripted, here is how:
List of files was obtained using this command:
git grep -l -e '\<return\s*([^()]*\(([^()]*)[^()]*\)*)\s*;' | \
grep -e '\.[ch]$' -e '\.py$'
Found files were modified with this command:
sed -i -e \
's_^\(.*\<return\)\s*(\(\([^()]*([^()]*)[^()]*\)*\))\s*\(;.*$\)_\1 \2\4_' \
-e 's_^\(.*\<return\)\s*(\([^()]*\))\s*\(;.*$\)_\1 \2\3_'
Then checked for nonsense.
The whole command looks like this:
git grep -l -e '\<return\s*([^()]*\(([^()]*)[^()]*\)*)\s*;' | \
grep -e '\.[ch]$' -e '\.py$' | xargs sed -i -e \
's_^\(.*\<return\)\s*(\(\([^()]*([^()]*)[^()]*\)*\))\s*\(;.*$\)_\1 \2\4_' \
-e 's_^\(.*\<return\)\s*(\([^()]*\))\s*\(;.*$\)_\1 \2\3_'
Sometimes, its easier to run children with 2>&1 in shell notation,
and just deal with stdout and stderr interleaved. This was already
possible for fd handling; extend it to also work when doing string
capture of a child process.
* docs/internals/command.html.in: Document this.
* src/util/command.c (virCommandSetErrorBuffer): Likewise.
(virCommandRun, virExecWithHook): Implement it.
* tests/commandtest.c (test14): Test it.
* daemon/remote.c (remoteDispatchAuthPolkit): Use new command
feature.
I hit a VERY weird testsuite failure on rawhide, which included
_binary_ output to stderr, followed by a hang waiting for me
to type something! (Here, using ^@ for NUL):
$ ./commandtest
TEST: commandtest
WARNING: gnome-keyring:: couldn't send data: Bad file descriptor
.WARNING: gnome-keyring:: couldn't send data: Bad file descriptor
.WARNING: gnome-keyring:: couldn't send data: Bad file descriptor
WARNING: gnome-keyring:: couldn't send data: Bad file descriptor
.8^@^@^@8^@^@^@^A^@^@^@^Bay^A^@^@^@)PRIVATE-GNOME-KEYRING-PKCS11-PROTOCOL-V-1
I finally traced it to the fact that gnome-keyring, called via
gnutls_global_init which is turn called by virNetTLSInit, opens
an internal fd that it expects to communicate to via a
pthread_atfork handler (never mind that it violates POSIX by
using non-async-signal-safe functions in that handler:
https://bugzilla.redhat.com/show_bug.cgi?id=772320).
Our problem stems from the fact that we pulled the rug out from
under the library's expectations by closing an fd that it had
just opened. While we aren't responsible for fixing the bugs
in that pthread_atfork handler, we can at least avoid the bugs
by not closing the fd in the first place.
* tests/commandtest.c (mymain): Avoid closing fds that were opened
by virInitialize.
virBufferContentAndReset (intentionally) returns NULL for a buffer
with no content, but it is feasible to invoke a command with an
explicit empty string.
* src/util/command.c (virCommandAddEnvBuffer): Reject empty string.
(virCommandAddArgBuffer): Allow explicit empty argument.
* tests/commandtest.c (test9): Test it.
* tests/commanddata/test9.log: Adjust.
The VIR_TEST_DEBUG and VIR_TEST_VERBOSE env vars did not work
because we replaced 'environ' with 'newenv'. Simply calling
virTestGetDebug/Verbose() before replacing the 'environ' ensures
we have processed the env variables.
The gnutls initialization code opens /dev/urandom and keeps that
FD around for later use. We have code which kills off FDs 3-5
to avoid interfereing with our test case. Move the virInitialize
call before this point, so it kills off the gnutls /dev/urandom
FD which is irrelevant for testing purposes
* tests/commandtest.c: Fix test debugging & make it robust against
opened FDs
With gcc 4.5.1:
util/virpidfile.c: In function 'virPidFileAcquirePath':
util/virpidfile.c:308:66: error: nested extern declaration of '_gl_verify_function2' [-Wnested-externs]
Then in tests/commandtest.c, the new virPidFile APIs need to be used.
* src/util/virpidfile.c (virPidFileAcquirePath): Move verify to
top level.
* tests/commandtest.c: Use new pid APIs.
Although most functions in libvirt return 0 on success and < 0 on
failure, there are a few functions lingering around that return errno
(a positive value) on failure, and sometimes code calling those
functions incorrectly assumes the <0 standard. I noticed one of these
the other day when auditing networkStartDhcpDaemon after Guido Gunther
found a place where success was improperly returned on failure (that
patch has been acked and is pending a push). The problem was that it
expected the return value from virFileReadPid to be < 0 on failure,
but it was actually positive (it was also neglected to set the return
code in this case, similar to the bug found by Guido).
This all led to the fact that *all* of the virFile*Pid functions in
util.c are returning errno on failure. This patch remedies that
problem by changing them all to return -errno on failure, and makes
any necessary changes to callers of the functions. (In the meantime, I
also properly set the return code on failure of virFileReadPid in
networkStartDhcpDaemon).
Running ./autobuild.sh failed when gcov is installed, because
commandtest ended up crashing during gcov's getenv() call after
exit() had already started. I traced this nasty bug back to
a scoping issue present since the test introduction.
* tests/commandtest.c (mymain): Move newenv...
(newenv): ...to a scope that is still useful during exit().
A few of the tests were missing basic sanity checks, while most
of them were doing copy-and-paste initialization (in fact, some
of them pasted the argc > 1 check more than once!). It's much
nicer to do things in one common place, and minimizes the size of
the next patch that fixes getcwd usage.
* tests/testutils.h (EXIT_AM_HARDFAIL): New define.
(progname, abs_srcdir): Define for all tests.
(VIRT_TEST_MAIN): Change callback signature.
* tests/testutils.c (virtTestMain): Do more common init.
* tests/commandtest.c (mymain): Simplify.
* tests/cputest.c (mymain): Likewise.
* tests/esxutilstest.c (mymain): Likewise.
* tests/eventtest.c (mymain): Likewise.
* tests/hashtest.c (mymain): Likewise.
* tests/networkxml2xmltest.c (mymain): Likewise.
* tests/nodedevxml2xmltest.c (myname): Likewise.
* tests/nodeinfotest.c (mymain): Likewise.
* tests/nwfilterxml2xmltest.c (mymain): Likewise.
* tests/qemuargv2xmltest.c (mymain): Likewise.
* tests/qemuhelptest.c (mymain): Likewise.
* tests/qemuxml2argvtest.c (mymain): Likewise.
* tests/qemuxml2xmltest.c (mymain): Likewise.
* tests/qparamtest.c (mymain): Likewise.
* tests/sexpr2xmltest.c (mymain): Likewise.
* tests/sockettest.c (mymain): Likewise.
* tests/statstest.c (mymain): Likewise.
* tests/storagepoolxml2xmltest.c (mymain): Likewise.
* tests/storagevolxml2xmltest.c (mymain): Likewise.
* tests/virbuftest.c (mymain): Likewise.
* tests/virshtest.c (mymain): Likewise.
* tests/vmx2xmltest.c (mymain): Likewise.
* tests/xencapstest.c (mymain): Likewise.
* tests/xmconfigtest.c (mymain): Likewise.
* tests/xml2sexprtest.c (mymain): Likewise.
* tests/xml2vmxtest.c (mymain): Likewise.
Sometimes, an asynchronous helper is started (such as a compressor
or iohelper program), but a later error means that we want to
abort that child. Make this easier.
Note that since daemons and virCommandRunAsync can't mix, the only
time virCommandFree can reap a process is if someone did
virCommandRunAsync for a non-daemon and didn't stash the pid.
* src/util/command.h (virCommandAbort): New prototype.
* src/util/command.c (_virCommand): Add new field.
(virCommandRunAsync, virCommandWait): Track whether pid was used.
(virCommandFree): Reap child if caller did not request pid.
(virCommandAbort): New function.
* src/libvirt_private.syms (command.h): Export it.
* tests/commandtest.c (test19): New test.
When executed from cron, commandtest would fail to correctly
identify daemon processes. Set session ID and process group
IDs at startup to ensure we have a consistent environment to
run in.
* tests/commandtest.c: Call setsid() and setpgid()
* src/fdstream.c (virFDStreamOpenFile, virFDStreamCreateFile):
Use VIR_FORCE_CLOSE instead of close.
* tests/commandtest.c (mymain): Likewise.
* tools/virsh.c (editFile): Use virCommand instead of system.
* src/util/util.c (__virExec): Special case preservation of std
file descriptors to child.
Guarantee that outbuf/errbuf are allocated on success, even if to the
empty string. Caller always has to free the result, and empty output
check requires checking if *outbuf=='\0'. Makes the API easier to use
safely. Failure is best effort allocation (some paths, like
out-of-memory, cannot allocate a buffer, but most do), so caller must
free buffer on failure.
* docs/internals/command.html.in: Update documentation.
* src/util/command.c (virCommandSetOutputBuffer)
(virCommandSetErrorBuffer, virCommandProcessIO) Guarantee empty
string on no output.
* tests/commandtest.c (test17): New test.
This introduces a new set of APIs in src/util/command.h
to use for invoking commands. This is intended to replace
all current usage of virRun and virExec variants, with a
more flexible and less error prone API.
* src/util/command.c: New file.
* src/util/command.h: New header.
* src/Makefile.am (UTIL_SOURCES): Build it.
* src/libvirt_private.syms: Export symbols internally.
* tests/commandtest.c: New test.
* tests/Makefile.am (check_PROGRAMS): Run it.
* tests/commandhelper.c: Auxiliary program.
* tests/commanddata/test2.log - test15.log: New expected outputs.
* cfg.mk (useless_free_options): Add virCommandFree.
(msg_gen_function): Add virCommandError.
* po/POTFILES.in: New translation.
* .x-sc_avoid_write: Add exemption.
* tests/.gitignore: Ignore new built file.
