As part of an goal to eliminate Perl from libvirt build tools,
rewrite the minimize-po.pl tool in Python.
This was a straight conversion, manually going line-by-line to
change the syntax from Perl to Python. Thus the overall structure
of the file and approach is the same.
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
As part of an goal to eliminate Perl from libvirt build tools,
rewrite the augeas-gentest.pl tool in Python.
This was a straight conversion, manually going line-by-line to
change the syntax from Perl to Python. Thus the overall structure
of the file and approach is the same.
The use of $(AUG_GENTEST) as a dependancy in the makefiles needed
to be fixed, because this was assumed to be the filename of the
script, but is in fact a full shell command line.
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Also define the macro for building with GLib older than 2.60
Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Instead of enumerating every single macro variant, just emit
a more generic error.
Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
We started using G_GNUC macros instead of ATTRIBUTE for some attributes.
Adjust this syntax-check accordingly.
Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
To facilitate porting over to glib, this rewrites the auto cleanup
macros to use glib's equivalent.
As a result it is now possible to use g_autoptr/VIR_AUTOPTR, and
g_auto/VIR_AUTOCLEAN, g_autofree/VIR_AUTOFREE interchangably, regardless
of which macros were used to declare the cleanup types.
Within the scope of any single method, code must remain consistent
using either GLib or Libvirt macros, never mixing both. New code
must preferentially use the GLib macros, and old code will be
converted incrementally.
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
The gnulib syntax-check rules are spread across GNUmakefile, cfg.mk and
maint.mk. This made sense when we were getting two of the files from the
gnulib submodule. Now that we own all files though, we can at least
merge maint.mk and cfg.mk together. GNUmakefile can be eliminated when
we switch to meson.
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
If we've marked rules as skipped, there's no sense keeping them in the
maint.mk file.
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
The standard maint.mk from gnulib provides alot more than just the
'syntax-check' target. This can all be purged to give a more minimal
file.
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
The syntax-check rules are the one bit of make usage that will
stay around for a while after the meson conversion. Move them
into the build-aux directory in preparation for refactoring
to make them independent from automake.
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
We're going to be eliminating autotools and gnulib, but we still wish to
have the 'make syntax-check' functionality.
This imports the minimal set of gnulib files required to keep this
working.
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Using @VARNAME@ is a normal style of automake, so lets match that.
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
The augeas-gentest.pl program merges a config file into a augeas
file, saving the output to a new file. It is going to be useful
to further process the output file, and it would be easier if this can
be done with a pipeline, so change augeas-gentest.pl to write to stdout
instead of a file.
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Internal headers should use #pragma once instead of the standard #ifndef
guard. Public headers still require the existing header guard.
Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Ján Tomko <jtomko@redhat.com>
Quite a few of the tests have a need to mock the stat() / lstat()
functions and they are taking somewhat different & inconsistent
approaches none of which are actually fully correct. This is shown
by fact that 'make check' fails on 32-bit hosts. Investigation
revealed that the code was calling into the native C library impl,
not getting intercepted by our mocks.
The POSIX stat() function might resolve to any number of different
symbols in the C library.
The may be an additional stat64() function exposed by the headers
too.
On 64-bit hosts the stat & stat64 functions are identical, always
refering to the 64-bit ABI.
On 32-bit hosts they refer to the 32-bit & 64-bit ABIs respectively.
Libvirt uses _FILE_OFFSET_BITS=64 on 32-bit hosts, which causes the
C library to transparently rewrite stat() calls to be stat64() calls.
Libvirt will never see the 32-bit ABI from the traditional stat()
call. We cannot assume this rewriting is done using a macro. It might
be, but on GLibC it is done with a magic __asm__ statement to apply
the rewrite at link time instead of at preprocessing.
In GLibC there may be two additional functions exposed by the headers,
__xstat() and __xstat64(). When these exist, stat() and stat64() are
transparently rewritten to call __xstat() and __xstat64() respectively.
The former symbols will not actally exist in the library at all, only
the header. The leading "__" indicates the symbols are a private impl
detail of the C library that applications should not care about.
Unfortunately, because we are trying to mock replace the C library,
we need to know about this internal impl detail.
With all this in mind the list of functions we have to mock will depend
on several factors
- If _FILE_OFFSET_BITS is set, then we are on a 32-bit host, and we
only need to mock stat64 and __xstat64. The other stat / __xstat
functions exist, but we'll never call them so they can be ignored
for mocking.
- If _FILE_OFFSET_BITS is not set, then we are on a 64-bit host and
we should mock stat, stat64, __xstat & __xstat64. Either may be
called by app code.
- If __xstat & __xstat64 exist, then stat & stat64 will not exist
as symbols in the library, so the latter should not be mocked.
The same all applies to lstat()
These rules are complex enough that we don't want to duplicate them
across every mock file, so this centralizes all the logic in a helper
file virmockstathelper.c that should be #included when needed. The
code merely need to provide a filename rewriting callback called
virMockStatRedirect(). Optionally VIR_MOCK_STAT_HOOK can be defined
as a macro if further processing is needed inline.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
We must do a substring match, not an exact match since
there can be an arbitrary virtual path prepended.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Require that all headers are guarded by a symbol named
LIBVIRT_$FILENAME
where $FILENAME is the uppercased filename, with all characters
outside a-z changed into '_'.
Note we do not use a leading __ because that is technically a
namespace reserved for the toolchain.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
This introduces a syntax-check script that validates header files use a
common layout:
/*
...copyright header...
*/
<one blank line>
#ifndef SYMBOL
# define SYMBOL
....content....
#endif /* SYMBOL */
For any file ending priv.h, before the #ifndef, we will require a
guard to prevent bogus imports:
#ifndef SYMBOL_ALLOW
# error ....
#endif /* SYMBOL_ALLOW */
<one blank line>
The many mistakes this script identifies are then fixed.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
In many files there are header comments that contain an Author:
statement, supposedly reflecting who originally wrote the code.
In a large collaborative project like libvirt, any non-trivial
file will have been modified by a large number of different
contributors. IOW, the Author: comments are quickly out of date,
omitting people who have made significant contribitions.
In some places Author: lines have been added despite the person
merely being responsible for creating the file by moving existing
code out of another file. IOW, the Author: lines give an incorrect
record of authorship.
With this all in mind, the comments are useless as a means to identify
who to talk to about code in a particular file. Contributors will always
be better off using 'git log' and 'git blame' if they need to find the
author of a particular bit of code.
This commit thus deletes all Author: comments from the source and adds
a rule to prevent them reappearing.
The Copyright headers are similarly misleading and inaccurate, however,
we cannot delete these as they have legal meaning, despite being largely
inaccurate. In addition only the copyright holder is permitted to change
their respective copyright statement.
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Our HACKING guide forbids these.
There's no point in exempting these from the spacing check
if their existence is against our coding style.
Note that the non-usage of these comments itself is not enforced
by syntax check, probably because of the need to implement a C parser.
Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Recent patches added indentation checks that discovered some cosmetic
issues at the cost of making this check last as long as the rest of
syntax-check combined on my system. Also, they're moving closer
to us implementing yet another C parser (docs/apibuild.py being the
other one).
Revert the following commits:
commit 11e1f11dd3
syntax-check: Check for incorrect indentation in function body
commit 2585a79e32
build-aux:check-spacing: Introduce a new rule to check misaligned stuff in parenthesises
commit a033182f04
build-aux:check-spacing: Add wrapper function of CheckCurlyBrackets
commit 6225626b6f
build-aux:check-spacing: Add wrapper function of CheckWhiteSpaces
commit c3875129d9
build-aux:check-spacing: Add wrapper function of KillComments
commit e995904c56
build-aux:check-spacing: Add wrapper function of CheckFunctionBody
commit 11e1f11dd3
syntax-check: Check for incorrect indentation in function body
This brings the speed of the script to a tolerable level and lets it
focus on the more visible issues.
Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
This patch introduces a new rule to check misaligned stuff in parenthesis:
1. For misaligned arguments of function
2. For misaligned conditions of [if|while|switch|...]
There're too much misalignment, so it adds a temporary filter which
permits 'src/util' now. It _should_ be removed as soon as fixing all.
Signed-off-by: Shi Lei <shi_lei@massclouds.com>
This patch add syntax-check rule for incorrect indentation and blank
first line in function body by changing check-spacing.pl.
For efficiency, it only checks the first line of function body.
But it's enough for most cases.
Signed-off-by: Shi Lei <shi_lei@massclouds.com>
Similar to the libvirt.pot, .po files contain line numbers and file
names identifying where in the source a translatable string comes from.
The source locations in the .po files are thrown away and replaced with
content from the libvirt.pot whenever msgmerge is run, so this is not
precious information that needs to be stored in git.
When msgmerge processes a .po file, it will add in any msgids from the
libvirt.pot that were not already present. Thus, if a particular msgid
currently has no translation, it can be considered redundant and again
does not need storing in git.
When msgmerge processes a .po file and can't find an exact existing
translation match, it will try todo fuzzy matching instead, marking such
entries with a "# fuzzy" comment to alert the translator to take a
look and either discard, edit or accept the match. Looking at the
existing fuzzy matches in .po files shows that the quality is awful,
with many having a completely different set of printf format specifiers
between the msgid and fuzzy msgstr entry. Fortunately when msgfmt
generates the .gmo, the fuzzy entries are all ignored anyway. The fuzzy
entries could be useful to translators if they were working on the .po
files directly from git, but Libvirt outsourced translation to the
Fedora Zanata system, so keeping fuzzy matches in git is not much help.
Finally, by default msgids are sorted based on source location. Thus, if
a bit of code with translatable text is moved from one file to another,
it may shift around in the .po file, despite the msgid not itself changing.
If the msgids were sorted alphabetically, the .po files would have
stable ordering when code is refactored.
This patch takes advantage of the above observations to canonicalize
and minimize the content stored for .po files in git. Instead of storing
the real .po files, we now store .mini.po files.
The .mini.po files are the same file format as .po files, but have no
source location comments, are sorted alphabetically, and all fuzzy
msgstrs and msgids with no translation are discarded. This cuts the size
of content in the po directory from 109MB to 19MB.
Users working from a libvirt git checkout who need the full .po files
can run "make update-po", which merges the libvirt.pot and .mini.po
file to create a .po file containing all the content previously stored
in git.
Conversely if a full .po file has been modified, for example, by
downloading new content from Zanata, the .mini.po files can be updated
by running "make update-mini-po". The resulting diffs of the .mini.po
file will clearly show the changed translations without any of the noise
that previously obscured content. Being able to see content changes
clearly actually identified a bug in the zanata python client where it
was adding bogus "fuzzy" annotations to many messages:
https://bugzilla.redhat.com/show_bug.cgi?id=1564497
Users working from libvirt releases should not see any difference in
behaviour, since the tarballs only contain the full .po files, not the
.mini.po files.
As an added benefit, generating tarballs with "make dist", will no
longer cause creation of dirty files in git, since it won't touch the
.mini.po files, only the .po files which are no longer kept in git.
To avoid creating a single commit 100+MB in size, each language is
minimized separately in a following commit.
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
This is particularly useful on operating systems that don't ship
Perl as part of the base system (eg. FreeBSD) while still working
just as well as it did before on Linux.
In one case (src/rpc/genprotocol.pl) the interpreter path was
missing altogether.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
This reverts commit e4b980c853.
When a binary links against a .a archive (as opposed to a shared library),
any symbols which are marked as 'weak' get silently dropped. As a result
when the binary later runs, those 'weak' functions have an address of
0x0 and thus crash when run.
This happened with virtlogd and virtlockd because they don't link to
libvirt.so, but instead just libvirt_util.a and libvirt_rpc.a. The
virRandomBits symbols was weak and so left out of the virtlogd &
virtlockd binaries, despite being required by virHashTable functions.
Various other binaries like libvirt_lxc, libvirt_iohelper, etc also
link directly to .a files instead of libvirt.so, so are potentially
at risk of dropping symbols leading to a later runtime crash.
This is normal linker behaviour because a weak symbol is not treated
as undefined, so nothing forces it to be pulled in from the .a You
have to force the linker to pull in weak symbols using -u$SYMNAME
which is not a practical approach.
This risk is silent bad linkage that affects runtime behaviour is
not acceptable for a fix that was merely trying to fix the test
suite. So stop using __weak__ again.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Currently all mockable functions are annotated with the 'noinline'
attribute. This is insufficient to guarantee that a function can
be reliably mocked with an LD_PRELOAD. The C language spec allows
the compiler to assume there is only a single implementation of
each function. It can thus do things like propagating constant
return values into the caller at compile time, or creating
multiple specialized copies of the function body each optimized
for a different caller. To prevent these optimizations we must
also set the 'noclone' and 'weak' attributes.
This fixes the test suite when libvirt.so is built with CLang
with optimization enabled.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
CLang's optimizer is more aggressive at inlining functions than
gcc and so will often inline functions that our tests want to
mock-override. This causes the test to fail in bizarre ways.
We don't want to disable inlining completely, but we must at
least prevent inlining of mocked functions. Fortunately there
is a 'noinline' attribute that lets us control this per function.
A syntax check rule is added that parses tests/*mock.c to extract
the list of functions that are mocked (restricted to names starting
with 'vir' prefix). It then checks that src/*.h header file to
ensure it has a 'ATTRIBUTE_NOINLINE' annotation. This should prevent
use from bit-rotting in future.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
We do not need a separate check forbidding whitespace
after the opening parenthesis after a keyword -
we forbid it after all of them.
The only allowed whitespace after an opening parenthesis
is a newline, tune the regex to reflect that.
Instead of matching multiple characters before the parenthesis,
only check for a single whitespace, which is much less cpu-intensive.
This only matches a few dozen of places where they are on an separate
line, filter out those with a separate regex.
We're looking for three consecutive lines, first one is a if/for/while
with a condition and start of body, second one is a body with one and
only semicolon and third is end of the body by itself.
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
