Similar to the libvirt.pot, .po files contain line numbers and file
names identifying where in the source a translatable string comes from.
The source locations in the .po files are thrown away and replaced with
content from the libvirt.pot whenever msgmerge is run, so this is not
precious information that needs to be stored in git.
When msgmerge processes a .po file, it will add in any msgids from the
libvirt.pot that were not already present. Thus, if a particular msgid
currently has no translation, it can be considered redundant and again
does not need storing in git.
When msgmerge processes a .po file and can't find an exact existing
translation match, it will try todo fuzzy matching instead, marking such
entries with a "# fuzzy" comment to alert the translator to take a
look and either discard, edit or accept the match. Looking at the
existing fuzzy matches in .po files shows that the quality is awful,
with many having a completely different set of printf format specifiers
between the msgid and fuzzy msgstr entry. Fortunately when msgfmt
generates the .gmo, the fuzzy entries are all ignored anyway. The fuzzy
entries could be useful to translators if they were working on the .po
files directly from git, but Libvirt outsourced translation to the
Fedora Zanata system, so keeping fuzzy matches in git is not much help.
Finally, by default msgids are sorted based on source location. Thus, if
a bit of code with translatable text is moved from one file to another,
it may shift around in the .po file, despite the msgid not itself changing.
If the msgids were sorted alphabetically, the .po files would have
stable ordering when code is refactored.
This patch takes advantage of the above observations to canonicalize
and minimize the content stored for .po files in git. Instead of storing
the real .po files, we now store .mini.po files.
The .mini.po files are the same file format as .po files, but have no
source location comments, are sorted alphabetically, and all fuzzy
msgstrs and msgids with no translation are discarded. This cuts the size
of content in the po directory from 109MB to 19MB.
Users working from a libvirt git checkout who need the full .po files
can run "make update-po", which merges the libvirt.pot and .mini.po
file to create a .po file containing all the content previously stored
in git.
Conversely if a full .po file has been modified, for example, by
downloading new content from Zanata, the .mini.po files can be updated
by running "make update-mini-po". The resulting diffs of the .mini.po
file will clearly show the changed translations without any of the noise
that previously obscured content. Being able to see content changes
clearly actually identified a bug in the zanata python client where it
was adding bogus "fuzzy" annotations to many messages:
https://bugzilla.redhat.com/show_bug.cgi?id=1564497
Users working from libvirt releases should not see any difference in
behaviour, since the tarballs only contain the full .po files, not the
.mini.po files.
As an added benefit, generating tarballs with "make dist", will no
longer cause creation of dirty files in git, since it won't touch the
.mini.po files, only the .po files which are no longer kept in git.
To avoid creating a single commit 100+MB in size, each language is
minimized separately in a following commit.
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
This is particularly useful on operating systems that don't ship
Perl as part of the base system (eg. FreeBSD) while still working
just as well as it did before on Linux.
In one case (src/rpc/genprotocol.pl) the interpreter path was
missing altogether.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
This reverts commit e4b980c853.
When a binary links against a .a archive (as opposed to a shared library),
any symbols which are marked as 'weak' get silently dropped. As a result
when the binary later runs, those 'weak' functions have an address of
0x0 and thus crash when run.
This happened with virtlogd and virtlockd because they don't link to
libvirt.so, but instead just libvirt_util.a and libvirt_rpc.a. The
virRandomBits symbols was weak and so left out of the virtlogd &
virtlockd binaries, despite being required by virHashTable functions.
Various other binaries like libvirt_lxc, libvirt_iohelper, etc also
link directly to .a files instead of libvirt.so, so are potentially
at risk of dropping symbols leading to a later runtime crash.
This is normal linker behaviour because a weak symbol is not treated
as undefined, so nothing forces it to be pulled in from the .a You
have to force the linker to pull in weak symbols using -u$SYMNAME
which is not a practical approach.
This risk is silent bad linkage that affects runtime behaviour is
not acceptable for a fix that was merely trying to fix the test
suite. So stop using __weak__ again.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Currently all mockable functions are annotated with the 'noinline'
attribute. This is insufficient to guarantee that a function can
be reliably mocked with an LD_PRELOAD. The C language spec allows
the compiler to assume there is only a single implementation of
each function. It can thus do things like propagating constant
return values into the caller at compile time, or creating
multiple specialized copies of the function body each optimized
for a different caller. To prevent these optimizations we must
also set the 'noclone' and 'weak' attributes.
This fixes the test suite when libvirt.so is built with CLang
with optimization enabled.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
CLang's optimizer is more aggressive at inlining functions than
gcc and so will often inline functions that our tests want to
mock-override. This causes the test to fail in bizarre ways.
We don't want to disable inlining completely, but we must at
least prevent inlining of mocked functions. Fortunately there
is a 'noinline' attribute that lets us control this per function.
A syntax check rule is added that parses tests/*mock.c to extract
the list of functions that are mocked (restricted to names starting
with 'vir' prefix). It then checks that src/*.h header file to
ensure it has a 'ATTRIBUTE_NOINLINE' annotation. This should prevent
use from bit-rotting in future.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
We do not need a separate check forbidding whitespace
after the opening parenthesis after a keyword -
we forbid it after all of them.
The only allowed whitespace after an opening parenthesis
is a newline, tune the regex to reflect that.
Instead of matching multiple characters before the parenthesis,
only check for a single whitespace, which is much less cpu-intensive.
This only matches a few dozen of places where they are on an separate
line, filter out those with a separate regex.
We're looking for three consecutive lines, first one is a if/for/while
with a condition and start of body, second one is a body with one and
only semicolon and third is end of the body by itself.
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
Change while () { smth; last; } to if () { smth; } as 'last' in perl is
analogous to 'break' in C. These are probably copy-paste leftovers from
creating new syntax-check rules.
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
In bracket-spacing.pl, the current $line is being modified in $data.
That, however, spoils that $data for another check. Introduce new
$tmpdata variable that can be used for temporary modifications. The
difference between $data and $line is that $data are as much cleaned as
possible from non-code blocks and these changes must be kept.
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
Commit a1cbe4b5 added a check for spaces around assignments and this
patch extends it to checks for spaces around '=='. One exception is
virAssertCmpInt where comma after '==' is acceptable (since it is a
macro and '==' is its argument).
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
Enforce and document the style set up by the previous patches.
* build-aux/bracket-spacing.pl: Add comma checks.
* docs/hacking.html.in: Document the rules.
* HACKING: Regenerate.
Signed-off-by: Eric Blake <eblake@redhat.com>
To register virtual machines and containers with systemd-machined,
and thus have cgroups auto-created, we need to talk over DBus.
This is somewhat tedious code, so introduce a dedicated function
to isolate the DBus call in one place.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Only a few cases are allowed:
1) The expression is empty for "for" loop, E.g.
for (i = 0; ; i++)
2) An empty statement
while (write(statuswrite, &status, 1) == -1 &&
errno == EINTR)
; /* empty */
3) ";" is inside double-quote, I.e, as part of const string. E.g.
vshPrint(ctl, "a ; b ; cd;\n");
The "for" loop in src/rpc/virnettlscontext.c is the special case,
1) applies for it, so change it together in this patch.
This documents the following whitespace rules
if(foo) // Bad
if (foo) // Good
int foo (int wizz) // Bad
int foo(int wizz) // Good
bar = foo (wizz); // Bad
bar = foo(wizz); // Good
typedef int (*foo) (int wizz); // Bad
typedef int (*foo)(int wizz); // Good
int foo( int wizz ); // Bad
int foo(int wizz); // Good
There is a syntax-check rule extension to validate all these rules.
Checking for 'function (...args...)' is quite difficult since it
needs to ignore valid usage with keywords like 'if (...test...)'
and while/for/switch. It must also ignore source comments and
quoted strings.
It is not possible todo this with a simple regex in the normal
syntax-check style. So a short Perl script is created instead
to analyse the source. In practice this works well enough. The
only thing it can't cope with is multi-line quoted strings of
the form
"start of string\
more lines\
more line\
the end"
but this can and should be written as
"start of string"
"more lines"
"more line"
"the end"
with this simple change, the bracket checking script does not
have any false positives across libvirt source, provided it
is only run against .c files. It is not practical to run it
against .h files, since those use whitespace extensively to
get alignment (though this is somewhat inconsistent and could
arguably be fixed).
The only limitation is that it cannot detect a violation where
the first arg starts with a '*', eg
foo(*wizz);
since this generates too many false positives on function
typedefs which can't be supressed efficiently.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
https://www.gnu.org/licenses/gpl-howto.html recommends that
the 'If not, see <url>.' phrase be a separate sentence.
* tests/securityselinuxhelper.c: Remove doubled line.
* tests/securityselinuxtest.c: Likewise.
* globally: s/; If/. If/
The cfg.mk file rule to check for tab characters was not
applied to perl files. Much of our Perl code is full of
tabs as a result. Kill them, kill them all !
When adding new config file parameters, the corresponding
additions to the augeas lens' are constantly forgotten.
Also there are augeas test cases, these don't catch the
error, since they too are never updated.
To address this, the augeas test cases need to be auto-generated
from the example config files.
* build-aux/augeas-gentest.pl: Helper to generate an
augeas test file, substituting in elements from the
example config files
* src/Makefile.am, daemon/Makefile.am: Switch to
auto-generated augeas test cases
* daemon/test_libvirtd.aug, daemon/test_libvirtd.aug.in,
src/locking/test_libvirt_sanlock.aug,
src/locking/test_libvirt_sanlock.aug.in,
src/lxc/test_libvirtd_lxc.aug,
src/lxc/test_libvirtd_lxc.aug.in,
src/qemu/test_libvirtd_qemu.aug,
src/qemu/test_libvirtd_qemu.aug.in: Remove example
config file data, replacing with a ::CONFIG:: placeholder
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Allows bootstrap to work on FreeBSD, where gzip doesn't have a '.'
in its version; and silences false positives in the new
'make syntax-check' rule.
* .gnulib: Update to latest.
* bootstrap: Synchronize to upstream.
* .x-sc_bindtextdomain: New exemptions.
* Makefile.am (syntax_check_exceptions): Ship new file.
* .gitignore: Regenerate per latest bootstrap, anchor entries that
are only in the root directory, and consolidate entries from other
generated .gitignore files.
* build-aux/.gitignore, m4/.gitignore, po/.gitignore: Remove from
version control, since bootstrap generates them.
Picks up fixes for gethostname compilation problems on mingw.
* .gnulib: Update to latest.
* build-aux/.gitignore: Regenerate.
* cfg.mk (local-checks-to-skip): Avoid new test not relevent to
libvirt.
GNUTLS uses gcrypt for its crypto functions. gcrypt requires
that the app/library initializes threading before using it.
We don't want to force apps using libvirt to know about
gcrypt, so we make virInitialize init threading on their
behalf. This location also ensures libvirtd has initialized
it correctly. This initialization is required even if libvirt
itself were only using one thread, since another non-libvirt
library (eg GTK-VNC) could also be using gcrypt from another
thread
* src/libvirt.c: Register thread functions for gcrypt
* configure.in: Add -lgcrypt to linker flags
No longer maintain a version-controlled ChangeLog file, but do
continue to include a ChangeLog file in distribution tarball.
* Makefile.am (gen-ChangeLog): New rule.
(dist-hook): Depend on it.
(EXTRA_DIST): Add ChangeLog-old.
* bootstrap (modules): Add gitlog-to-changelog.
* ChangeLog: Remove file. Renamed to...
* ChangeLog-old: ...this. New file.
* autogen.sh: Touch ChangeLog, to ensure it exists. For automake.
This makes it so we record (via a git submodule)
a snapshot of whatever version of gnulib we're using,
and none of gnulib sources are in the libvirt repository.
The result is that we have as much reproducibility as when
we version-controlled imported copies of the gnulib sources,
but without the hassle of the manual process we used when
syncing with upstream.
Note that when you clone libvirt, you get only the libvirt
repository, but when you first run ./bootstrap, it clones
gnulib (at the SHA1 recorded via the submodule), creating
the .gnulib/ hierarchy. Then, the bootstrap script runs
gnulib-tool to populate gnulib/ with the files that make
up the selected modules.
Put the following in your ~/.gitconfig file.
[alias]
syncsub = submodule foreach git pull origin master
The update procedure is simple:
git syncsub
...build & test...
git commit -m 'gnulib: sync submodule to latest' .gnulib
* .gitmodules: New file.
* .gnulib: Initialize.
* bootstrap: Set up to use the new submodule.
Stop using --no-vc-files.
Don't remove .gitignore files.
Don't use or create .cvsignore.
Diagnose an invalid --gnulib-srcdir=DIR argument.
* build-aux/vc-list-files: Delete file, now pulled from gnulib.
* build-aux/useless-if-before-free: Likewise.
* po/POTFILES.in: Remove gnulib/lib/gai_strerror.c, since
it no longer contains translatable strings.
* gnulib/*: Remove gnulib/ hierarchy.
* Makefile.maint (sync-vcs-ignore-files): New target.
Prompted by a patch from James Morris.
http://thread.gmane.org/gmane.comp.emulators.libvirt/8619/focus=8773
Add all (now-generated) .gitignore files.
* .gitignore: New file.
* build-aux/.gitignore: New file.
* docs/.gitignore: New file.
* docs/devhelp/.gitignore: New file.
* docs/examples/.gitignore: New file.
* docs/examples/python/.gitignore: New file.
* gnulib/lib/.gitignore: New file.
* gnulib/lib/arpa/.gitignore: New file.
* gnulib/lib/netinet/.gitignore: New file.
* gnulib/lib/sys/.gitignore: New file.
* gnulib/tests/.gitignore: New file.
* include/.gitignore: New file.
* include/libvirt/.gitignore: New file.
* po/.gitignore: New file.
* proxy/.gitignore: New file.
* python/.gitignore: New file.
* python/tests/.gitignore: New file.
* qemud/.gitignore: New file.
* src/.gitignore: New file.
* tests/.gitignore: New file.
* tests/confdata/.gitignore: New file.
* tests/sexpr2xmldata/.gitignore: New file.
* tests/virshdata/.gitignore: New file.
* tests/xencapsdata/.gitignore: New file.
* tests/xmconfigdata/.gitignore: New file.
* tests/xml2sexprdata/.gitignore: New file.
* tests/Makefile.am (test_scripts): Add vcpupin.
(EXTRA_DIST): Add test-lib.sh.
* tests/test-lib.sh: Testing framework, from coreutils.
* tests/vcpupin: New file.
* build-aux/mktempd: New file, from gnulib.
* bootstrap: Add posix-shell and mktempd to the list of imported modules.
* gnulib/m4/posix-shell.m4: New file, from gnulib.
* bootstrap (gnulib_tool): Add useless-if-before-free to
the list of modules.
* build-aux/useless-if-before-free: Update from gnulib.
* gnulib/lib/vasnprintf.c: Work around a bug in HPUX 10.20.
* gnulib/m4/vasnprintf.m4: Likewise.
