Recent discussions have illustrated the potential for DOS attacks
with the hash table implementations used by most languages and
libraries.
https://lwn.net/Articles/474912/
libvirt has an internal hash table impl, and uses hash tables for
a variety of purposes. The hash key generation code is pretty
simple and thus not strongly collision resistant.
This patch replaces the current libvirt hash key generator with
the (public domain) Murmurhash3 code. In addition every hash
table now gets a random seed value which is used to perturb the
hashing code. This should make it impossible to mount any
practical attack against libvirt hashing code.
* bootstrap.conf: Import bitrotate module
* src/Makefile.am: Add virhashcode.[ch]
* src/util/util.c: Make virRandom() return a fixed 32 bit
integer value.
* src/util/hash.c, src/util/hash.h, src/util/cgroup.c: Replace
hash code generation with a call to virHashCodeGen()
* src/util/virhashcode.h, src/util/virhashcode.c: Add a new
virHashCodeGen() API using the Murmurhash3 algorithm.
Jiri Denemark reported an instance of bootstrapping libvirt
failing when run inside a sandbox, traced to rpm trying to
access /var/ which was not permitted by the sandbox.
Alex Jia reported that 0.9.8-rc1 failed to bootstrap if patch(1)
is not installed.
* bootstrap.conf (buildreq): Avoid rpm call if python-config
exists. Also, require patch, in case we have gnulib-local diffs.
We want our tarballs to be complete - this means that any
generated file that gets shipped as part of the tarball so that
ordinary users don't have to rebuild it must be something
that the maintainer can generate. There have been various
reports of random build failures when using libvirt.git
instead of a tarball, and often it is due to missing a
maintainer-specific tool to produce one of these generated
files. This patch raises the bar for what you must have
installed to build libvirt.git, but does not impact what
you can get away with for building tarballs.
Note: It still remains possible to do a successful 'make dist'
without these tools, when starting from a release tarball.
* bootstrap.conf (buildreq): Add tools that maintainers need for a
successful 'make dist' from a fresh git checkout.
Commit f7bd00c12 pulled in a gnulib module that fails to compile
on mingw. While it would be nice to pull in a newer version of
.gnulib that fixes this, it is difficult to backport any .gnulib
update to older releases. So, it makes sense to take advantage
of gnulib-tool's ability to support local diffs, where we can
apply specific diffs in our use of gnulib without waiting for
upstream gnulib to pick up those changes, as well as avoiding
a wholesale .gnulib update. The existence of local diffs will
also make it easier to backport fixes against a tarball (as long
as a tarball and libvirt.git share the same .gnulib commit, then
the tarball can be patched by applying the same local diffs as
a post-release libvirt.git commit, without having to rerun an
entire gnulib-tool bootstrap).
This patch introduces the framework for supporting local diffs,
without actually introducing any.
* bootstrap.conf (local_gl_dir): New variable.
* autogen.sh (bootstrap_hash): Hash any local diffs, to force a
re-bootstrap if just diffs change.
* cfg.mk (_update_required): Likewise.
MacOS lacks ptsname_r, and gnulib doesn't (yet) provide it.
But we can avoid it altogether, by using gnulib openpty()
instead. Note that we do _not_ want the pt_chown module;
gnulib uses it only to implement a replacement openpty() if
the system lacks both openpty() and granpt(), but all
systems that we currently port to either have at least one of
openpty() and/or grantpt(), or lack ptys altogether. That is,
we aren't porting to any system that requires us to deal with
the hassle of installing a setuid pt_chown helper just to use
gnulib's ability to provide openpty() on obscure platforms.
* .gnulib: Update to latest, for openpty fixes
* bootstrap.conf (gnulib_modules): Add openpty, ttyname_r.
(gnulib_tool_option_extras): Exclude pt_chown module.
* src/util/util.c (virFileOpenTty): Rewrite in terms of openpty
and ttyname_r.
* src/util/util.h (virFileOpenTtyAt): Delete dead prototype.
Commit 1726a73 hacked around MacOS' lack of fdatasync, since
gnulib did not have it at the time. But now that we use newer
gnulib, we can avoid the hack.
* bootstrap.conf (gnulib_modules): Add fdatasync.
* configure.ac (AC_CHECK_FUNCS_ONCE): Drop our own check.
Mingw lacks fsync, but gnulib provides that. Meanwhile, gnulib does
not (yet) provide fdatasync, so this is a quick hack to fake that
function on MacOS X; we can revert this configure change once gnulib
gives us a real module.
We have been implicitly relying on gnulib's largefile module being
pulled in by other modules, but it's better to make that explicit.
* bootstrap.conf (gnulib_modules): Add fsync. Make largefile use
explicit.
* configure.ac (AC_CHECK_FUNCS_ONCE): Check for fdatasync, and
fake it with fsync when not present.
Gettext annoyingly modifies CPPFLAGS in-place, putting
-I/usr/local/include into the search patch if libintl headers
must be used from that location. But since we must support
automake 1.9.6 which lacks AM_CPPFLAGS, and since CPPFLAGS is used
prior to INCLUDES, this means that the build picks up the _old_
installed libvirt.h in priority to the in-tree version, leading
to all sorts of weird build failures on FreeBSD.
Fix this by teaching configure to undo gettext's actions, but
to keep any changes required by gettext at the end of INCLUDES
after all in-tree locations are used first. Also requires
adding a wrapper Makefile.am and making gnulib-tool create
just gnulib.mk files during the bootstrap process.
Signed-off-by: Eric Blake <eblake@redhat.com>
Without this, a configure built by autoconf 2.59 was broken when
trying to detect which compiler warning flags were supported.
* .gnulib: Update to latest, for warnings.m4 fix.
* bootstrap.conf: Add fclose explicitly, to match recent gnulib
implicit dependency changes.
* src/qemu/qemu_conf.c (includes): Drop unused include.
* src/uml/uml_conf.c (include): Likewise.
Reported by Daniel P. Berrange.
No need to repeat common code.
* bootstrap.conf (gnulib_modules): Import calloc-posix.
* src/util/bridge.c (brInit): Use virSetCloseExec.
(brSetInterfaceUp): Adjust flags name.
* src/uml/uml_driver.c (umlSetCloseExec): Delete.
(umlStartVMDaemon): Use util version instead.
log2() is heavy when ffs() can do the same thing. But ffs()
requires gnulib support for mingw.
This patch solves this linker error on Fedora 14.
/usr/bin/ld: libvirt_lxc-domain_conf.o: undefined reference to symbol 'log2@@GLIBC_2.2.5'
/usr/bin/ld: note: 'log2@@GLIBC_2.2.5' is defined in DSO /lib64/libm.so.6 so try adding it to the linker command line
/lib64/libm.so.6: could not read symbols: Invalid operation
collect2: ld returned 1 exit status
* .gnulib: Update to latest, for ffs.
* bootstrap.conf (gnulib_modules): Import ffs.
* src/conf/domain_conf.c (virDomainDefParseXML): Use ffs instead
of log2.
Reported by Dave Allan.
Gnulib documents that mingw vsnprintf is broken (it returns -1
on out-of-space, instead of the count of what would have been
printed); but while we were using the snprintf wrapper, we had
not yet been using the vsnprintf wrapper.
Meanwhile, mingw (but not mingw64) has a replacement snprintf
that fixes return values, but still lacks %1$s support; so in
that case, gnulib didn't replace snprintf, but libintl then
went ahead and installed a version that supported %1$s but not
return values. Gnulib has since been fixed to guarantee that
the snprintf module will always guarantee the constraints needed
by libintl.
Also, we want to guarantee that strdup sets errno on failure.
* .gnulib: Update to latest, for vsnprintf fix.
* bootstrap.conf (gnulib_modules): Add vsnprintf, strdup-posix.
Reported by Matthias Bolte.
./autobuild.sh died on several messages resembling:
../../src/rpc/virnetsocket.c: In function 'virNetSocketNewListenTCP':
../../src/rpc/virnetsocket.c:231:9: error: implicit declaration of function 'bind_used_without_requesting_gnulib_module_bind' [-Wimplicit-function-declaration]
../../src/rpc/virnetsocket.c:231:9: error: nested extern declaration of 'bind_used_without_requesting_gnulib_module_bind' [-Wnested-externs]
Basically, gnulib socket fds are not safe to pass to mingw socket
functions unless we pull in those gnulib modules.
* bootstrap.conf (gnulib_modules): Add modules to handle socket
functions on mingw.
Done as a separate commit to make backporting the next patch easier.
We are already using "intprops.h", but this makes it explicit.
* .gnulib: Update, for syntax-check fix.
* bootstrap.conf (gnulib_modules): Make intprops use explicit.
* src/locking/domain_lock.c (includes): Drop unused header.
* src/nwfilter/nwfilter_learnipaddr.c (includes): Use "", not <>,
for gnulib.
Now that RHEL 5.6 ships with gettext 0.17, we can get out of the
stone age and use the newer gettext capabilities and improved
macros for certain configure tests.
* configure.ac (AM_GNU_GETTEXT_VERSION): Bump to 0.17.
(MKINSTALLDIRS): Delete hack that is no longer needed.
* bootstrap.conf (buildreq): Check for minimum gettext version.
Based on a report by Wen Congyang.
The libvirtd daemon uses fnmatch. Although we don't yet build
it on Win32, we should use gnulib's fnmatch module to ensure
portability to all platforms.
* bootstrap.conf: Add fnmatch
Always generate the rpc files, and require rpcgen during bootstrap.
* daemon/Makefile.am: Removed generated files with
maintainer-clean target
* src/Makefile.am: Removed generated files with
maintainer-clean target. Always run 'rpcgen' if
generated files are missing
Match the fact that we have virAsprintf and virVasprintf.
* src/util/buf.h (virBufferVasprintf): New prototype.
* src/util/buf.c (virBufferAsprintf): Move guts...
(virBufferVasprintf): ...to new function.
* src/libvirt_private.syms (buf.h): Export it.
* bootstrap.conf (gnulib_modules): Add stdarg, for va_copy.
This patch enables the migration of Qemu VMs between hosts of different endianess. I tested this by migrating a i686 VM between a x86 and ppc64 host.
I am converting the 'int's in the VM's state header to uint32_t assuming this doesn't break compatibility with existing deployments other than Linux.
* .gnulib: Update to latest, for pipe2.
* bootstrap.conf (gnulib_modules): Add pipe2.
* src/util/event_poll.c (virEventPollInit): Use it, to avoid
problematic virSetCloseExec on mingw.
Remove custom code for checking compiler warnings, using
gl_WARN_ADD instead. Don't list all flags ourselves, use
gnulib's gl_MANYWARN_ALL_GCC to get all possible GCC flags,
then turn off the ones we don't want yet.
* acinclude.m4: Rewrite to use gl_WARN_ADD and gl_MANYWARN_ALL_GCC
* bootstrap.conf: Add warnings & manywarnings
* configure.ac: Switch to gl_WARN_ADD
* m4/compiler-flags.m4: Obsoleted by gl_WARN_ADD
strcase{cmp/str} have the drawback of being sensitive to the global
locale; this is unacceptable in a library setting. Prefer a
hard-coded C locale alternative for all but virsh, which is user
facing and where the global locale isn't changing externally.
* .gnulib: Update to latest, for c-strcasestr change.
* bootstrap.conf (gnulib_modules): Drop strcasestr, add c-strcase
and c-strcasestr.
* cfg.mk (sc_avoid_strcase): New rule.
(exclude_file_name_regexp--sc_avoid_strcase): New exception.
* src/internal.h (STRCASEEQ, STRCASENEQ, STRCASEEQLEN)
(STRCASENEQLEN): Adjust offenders.
* src/qemu/qemu_monitor_text.c (qemuMonitorTextEjectMedia):
Likewise.
* tools/virsh.c (namesorter): Document exception.
Not every day you see a patch that nukes 27 files!
* .gnulib: Update to latest, for maint.mk improvements
* bootstrap: Resync to gnulib.
* bootstrap.conf (ACLOCAL): Swap the secondary aclocal include
directory, now that bootstrap picks up gnulib/m4 instead of m4.
* Makefile.am (syntax_check_exceptions, EXTRA_DIST): No longer
worry about nuked files.
* cfg.mk (sc_x_sc_dist_check): Delete dead rule.
(VC_LIST_ALWAYS_EXCLUDE_REGEX): Add HACKING.
(exclude_file_name_regexp--sc_*): Inline and simplify contents...
* .x-sc_*: ...from here, then delete the files.
A diff of 'make dist' from in-tree vs. a VPATH build showed
that we were missing docs/api_extension/*.patch files, but
shipping other files that we didn't need.
* bootstrap.conf (gnulib_extra_files): Don't distribute files we
don't care about.
* docs/Makefile.am (patches): Perform wildcard correctly.
Running 'make check' can sometimes fail in the gnulib/tests
subdirectory, when doing an incremental build, because
./bootstrap generates a Makefile.am that tries to refer to
../../.. instead of ../.., and gets lost.
This may be an upstream gnulib bug, where a more elegant
solution will present itself in the future:
http://thread.gmane.org/gmane.comp.lib.gnulib.bugs/24898
But in the meantime, I was able to reproduce both the issue,
and this solution to work around it.
* bootstrap.conf (bootstrap_epilogue): Ensure that no stray
../../.. components remain in gnulib/tests/Makefile.in.
Reported by Serge Hallyn.
* .gnulib: Update to latest, for sigpipe and sigaction modules.
* bootstrap.conf (gnulib_modules): Add siaction, sigpipe, strerror_r.
* tools/virsh.c (vshSetupSignals) [!SIGPIPE]: Delete, now that
gnulib guarantees it.
(SA_SIGINFO): Define for mingw fallback.
* src/util/virterror.c (virStrerror): Simplify, now that gnulib
guarantees the POSIX interface.
* configure.ac (AC_CHECK_FUNCS_ONCE): Drop redundant check.
(AM_PROG_CC_STDC): Move earlier, to keep autoconf happy.
* .gnulib: Update to latest.
* bootstrap.conf (gnulib_modules): Import pipe-posix and waitpid
for mingw.
* src/remote/remote_driver.c (pipe) [WIN32]: Drop dead macro.
* daemon/event.c (pipe) [WIN32]: Drop dead function.
* .gnulib: Update to latest, for bootstrap fixes.
* bootstrap: Synchronize with upstream.
* bootstrap.conf: Add pkg-config pre-requisite.
* autogen.sh: Tweak wording message.
Reported by Justin Clift, and with feedback from Bruno Haible.
Partial reversion of commit 76d87a59, now that bootstrap is smarter.
* .gnulib: Update to latest, for poll and bootstrap fixes.
* bootstrap: Resync from gnulib.
* autogen.sh: Drop redundant tool checks; bootstrap does them
better, by honoring environment variables.
* examples/domain-events/events-c/Makefile.am (INCLUDES)
(event_test_LDADD): Use gnulib library during build.
* bootstrap.conf (gnulib_tool_option_extras): Revert --libtool
addition, now that updated bootstrap does it for us.
Reported by Justin Clift.
snprintf is currently implicitly picked up by getaddrinfo, but we
might as well make it explicit so that mingw doesn't break if
getaddrinfo changes to drop the dependency.
func doesn't matter for gcc compilation, but may help other compilers
cope with our use of __func__.
* bootstrap.conf (gnulib_modules): Add snprintf and func.
With the previous storage pool UUID source not all storage pools
had a proper UUID, especially GSX storage pools. The mount path
is unique per host and cannot change during the lifetime of the
datastore. Therefore, it's MD5 sum can be used as UUID.
Use gnulib's crypto/md5 module to generate the MD5 sum.
On Fedora 13 with sufficient mingw32-* packages installed, running
./autobuild.sh failed to cross-compile to mingw because
mingw32-pthreads installed a broken <pthread.h>. With that
issue fixed, the build still failed due to use of O_SYNC.
Meanwhile, recent .spec.in changes got out of sync.
* bootstrap.conf (gnulib_modules): Add fcntl-h, for O_SYNC.
* .gnulib: Update to latest, to work around buggy pthreads-win32
library.
* bootstrap: Import latest from gnulib.
* mingw32-libvirt.spec.in: Distribute new file.
