We need to be able to create event loop watches using the
GSource API for sockets. GIOChannel is able todo this, but
we don't want to use the GIOChannel APIs for reading/writing,
and testing shows just using its GSource APIs is unreliable
on Windows.
This patch thus creates a standalone helper API for creating
a GSource for a socket file descriptor. This impl is derived
from code in QEMU's io/channel-watch.c file that was written
by myself & Paolo Bonzini & thus under Red Hat copyright.
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
The net/if.h is not portable so we must check for its
existance and avoid using it when missing. Some use
of net/if.h was redundant and could be removed.
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Similarly to other libvirt object freeing APIs the function resets the
libvirt error when called and doesn't take NULL gracefully. Install the
workaround and g_autoptr handlers similarly to the 'virshDomain' type.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
The previous "QEMU shim" proof of concept was taking an approach of only
caring about initial spawning of the QEMU process. It was then
registered with the libvirtd daemon who took over management of it. The
intent was that later libvirtd would be refactored so that the shim
retained control over the QEMU monitor and libvirt just forwarded APIs
to each shim as needed. This forwarding of APIs would require quite alot
of significant refactoring of libvirtd to achieve.
This impl thus takes a quite different approach, explicitly deciding to
keep the VMs completely separate from those seen & managed by libvirtd.
Instead it uses the new "qemu:///embed" URI scheme to embed the entire
QEMU driver in the shim, running with a custom root directory.
Once the driver is initialization, the shim starts a VM and then waits
to shutdown automatically when QEMU shuts down, or should kill QEMU if
it is terminated itself. This ought to use the AUTO_DESTROY feature but
that is not yet available in embedded mode, so we rely on installing a
few signal handlers to gracefully kill QEMU. This isn't reliable if
we crash of course, but you can restart with the same root dir.
Note this program does not expose any way to manage the QEMU process,
since there's no RPC interface enabled. It merely starts the VM and
cleans up when the guest shuts down at the end. This program is
installed to /usr/bin/virt-qemu-run enabling direct use by end users.
Most use cases will probably want to integrate the concept directly
into their respective application codebases. This standalone binary
serves as a nice demo though, and also provides a way to measure
performance of the startup process quite simply.
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
G_STATIC_ASSERT() is a drop-in functional equivalent of
the GNULIB verify() macro.
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
We don't need all the platforms gnulib deals with, so
this is a cut down version of GNULIB's physmem.c
code. This also allows us to integrate libvirt's
error reporting functions closer to the error cause.
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
All UNIX platforms we care about have openpty() in the libutil
library. Use of pty.h must also be made conditional, excluding
Win32.
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Some syntax check rules validate usage of headers provided
by gnulib. We want to validate these only against the gnulib
modules we've chosen to use, not all modules, since we're
trying to eliminate them.
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
The GLib g_size_checked_mul() function is not quite the
same signature, and gives compiler warnings due to not
correctly casting from gsize to guint64/32. Implementing
a replacement for INT_MULTIPLY_OVERFLOW is easy enough
to do ourselves.
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
A few places were importing dirname.h without actually using it.
Reviewed-by: Fabiano Fidêncio <fidencio@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
When using GNULIB with Winsock, libvirt will never see the normal HANDLE
objects, instead GNULIB guarantees that libvirt gets a C runtime file
descriptor. The GNULIB poll impl also expects to get C runtime file
descriptors rather than HANDLE objects. Document this behaviour so that
it is clear to applications providing event loop implementations if they
need Windows portability.
Reviewed-by: Fabiano Fidêncio <fidencio@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
As part of a goal to eliminate Perl from libvirt build tools,
rewrite the group-qemu-caps.pl tool in Python.
This was a straight conversion, manually going line-by-line to
change the syntax from Perl to Python. Thus the overall structure
of the file and approach is the same.
Tested-by: Cole Robinson <crobinso@redhat.com>
Reviewed-by: Cole Robinson <crobinso@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
As part of a goal to eliminate Perl from libvirt build tools,
rewrite the test-wrap-argv.pl tool in Python.
This was a straight conversion, manually going line-by-line to
change the syntax from Perl to Python. Thus the overall structure
of the file and approach is the same.
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
As part of a goal to eliminate Perl from libvirt build tools,
rewrite the pdwtags processing script in Python.
The original inline shell and perl code was completely
unintelligible. The new python code is a manual conversion
that attempts todo basically the same thing.
Tested-by: Cole Robinson <crobinso@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
As part of a goal to eliminate Perl from libvirt build tools,
rewrite the gensystemtap.pl tool in Python.
This was a straight conversion, manually going line-by-line to
change the syntax from Perl to Python. Thus the overall structure
of the file and approach is the same.
Tested-by: Cole Robinson <crobinso@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
As part of a goal to eliminate Perl from libvirt build tools,
rewrite the header-ifdef.pl tool in Python.
This was a straight conversion, manually going line-by-line to
change the syntax from Perl to Python. Thus the overall structure
of the file and approach is the same.
Tested-by: Cole Robinson <crobinso@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
As part of a goal to eliminate Perl from libvirt build tools,
rewrite the mock-noinline.pl tool in Python.
This was a straight conversion, manually going line-by-line to
change the syntax from Perl to Python. Thus the overall structure
of the file and approach is the same.
Tested-by: Cole Robinson <crobinso@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
As part of an goal to eliminate Perl from libvirt build tools,
rewrite the prohibit-duplicate-header.pl tool in Python.
This was a straight conversion, manually going line-by-line to
change the syntax from Perl to Python. Thus the overall structure
of the file and approach is the same.
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Instead of vsnprintf from gnulib, use g_vsnprintf from GLib.
Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
Now that function is no longer used, it can be dropped.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com>
Simplify the list of ignored warnings now that we only have two left,
and document why we're not honouring them.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Line continuations should be 4 space indented unless a previous opening
brace required different alignment.
docs/apibuild.py:2014:24: E126 continuation line over-indented for hanging indent
token[0], token[1]))
^
docs/apibuild.py:74:3: E121 continuation line under-indented for hanging indent
"ATTRIBUTE_UNUSED": (0, "macro keyword"),
^
...more...
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
There should be a single space either side of operators. Inline
comments should have two spaces before the '#'
src/hyperv/hyperv_wmi_generator.py:130:45: E261 at least two spaces before inline comment
source += ' { "", "", 0 },\n' # null terminated
^
src/esx/esx_vi_generator.py:417:25: E221 multiple spaces before operator
FEATURE__DESERIALIZE = (1 << 6)
^
tests/cputestdata/cpu-cpuid.py:187:78: E225 missing whitespace around operator
f.write(" <msr index='0x%x' edx='0x%08x' eax='0x%08x'/>\n" %(
^
docs/apibuild.py:524:47: E226 missing whitespace around arithmetic operator
self.line = line[i+2:]
^
...more...
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Coding style expects 1 blank line between each method and 2 blank lines
before each class.
docs/apibuild.py:171:5: E303 too many blank lines (2)
def set_header(self, header):
^
docs/apibuild.py:230:1: E302 expected 2 blank lines, found 1
class index:
^
docs/apibuild.py:175:5: E301 expected 1 blank line, found 0
def set_module(self, module):
^
...more...
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Exception catching statements should always match on a class name, the
most specific one possible. Rather than analyse the code to look at what
the most specific one is, this just uses the base Exception class.
docs/apibuild.py:255:9: E722 do not use bare 'except'
except:
^
docs/apibuild.py:279:9: E722 do not use bare 'except'
except:
^
...more...
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Python code style recommends avoiding a variable named 'l' as it is
visually similar to '1'.
docs/apibuild.py:482:13: E741 ambiguous variable name 'l'
l = len(line)
^
docs/apibuild.py:503:21: E741 ambiguous variable name 'l'
l = len(line)
^
...more...
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
When writing regexes special regex matches like "\d" can get
misinterpreted as normal string escape sequences:
docs/apibuild.py:1359:51: W605 invalid escape sequence '\d'
value = value + re.sub("^(\d+)U$", "\\1", token[1])
^
docs/apibuild.py:2134:31: W605 invalid escape sequence '\('
m = re.match("\(?1<<(\d+)\)?", info[0])
^
docs/apibuild.py:2134:38: W605 invalid escape sequence '\d'
m = re.match("\(?1<<(\d+)\)?", info[0])
^
docs/apibuild.py:2134:42: W605 invalid escape sequence '\)'
m = re.match("\(?1<<(\d+)\)?", info[0])
^
To avoid this probem all regexes should use the r"...." syntax for their
strings, which disables normal string escape sequences.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
The current flake8 check only looks at one item (semicolons at end of
line). This means that our code quality will continue to get worse,
violating an increasing number of checks.
Switching to a whitelist means that we freeze the badness at its
current level & can incrementally fix things up.
We are excluding the following...
Indentation:
E114 indentation is not a multiple of four (comment)
E115 expected an indented block (comment)
E116 unexpected indentation (comment)
E121 continuation line under-indented for hanging indent
E125 continuation line with same indent as next logical line
E126 continuation line over-indented for hanging indent
E127 continuation line over-indented for visual indent
E128 continuation line under-indented for visual indent
E129 visually indented line with same indent as next logical line
E131 continuation line unaligned for hanging indent
Whitespace:
E211 whitespace before ‘(‘
E221 multiple spaces before operator
E222 multiple spaces after operator
E225 missing whitespace around operator
E226 missing whitespace around arithmetic operator
E231 missing whitespace after ‘,’, ‘;’, or ‘:’
E261 at least two spaces before inline comment
Blank lines
E301 expected 1 blank line, found 0
E302 expected 2 blank lines, found 0
E303 too many blank lines (3)
E305 expected 2 blank lines after end of function or class
Line length
E501 line too long (82 > 79 characters)
Statements
E722 do not use bare except, specify exception instead
E741 do not use variables named ‘l’, ‘O’, or ‘I’
Errors:
F821 undefined name 'name'
Warnings:
W504 line break after binary operator
W605 invalid escape sequence ‘x’
Later commits will enable most of these exclusions.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
This affects more than src/Makefile.am as the rule to generate source
files for protocols is generic for all sub-directories.
Affected files are:
src/admin/admin_protocol.{h,c}
src/locking/lock_protocol.{h,c}
src/logging/log_protocol.{h,c}
src/lxc/lxc_monitor_protocol.{h,c}
src/remote/{lxc,qemu,remote}_protocol.{h,c}
src/rpc/{virkeepalive,virnet}protocol.{h,c}
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Our naming was not consistent. Use the protocol name as prefix for all
generated files.
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
There was no need to handle files for translation from build directory
but that will change with following patches where we will stop
generating source files into source directory.
In order to have them included for translation we have to prefix each
file with SRCDIR or BUILDDIR.
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Move generated_files variable closer to the sc_po_check rule and
remove non-existent gnulib internal path.
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Introduce new rule 'generated-sources' as a helper for PO files check
to make sure that all generated files are prepared and to not duplicate
the list on different places. This will be used as a dependency for
sc_po_check rule instead of duplicated list of generated files.
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Commit <22d8e27ccd5faf48ee2bf288a1b9059aa7ffd28b> introduced our
syntax-check.mk file based on gnulib rules. However, the rule was
completely ignored as we don't have POTFILES.in file.
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
As part of an goal to eliminate Perl from libvirt build tools,
rewrite the minimize-po.pl tool in Python.
This was a straight conversion, manually going line-by-line to
change the syntax from Perl to Python. Thus the overall structure
of the file and approach is the same.
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
As part of an goal to eliminate Perl from libvirt build tools,
rewrite the augeas-gentest.pl tool in Python.
This was a straight conversion, manually going line-by-line to
change the syntax from Perl to Python. Thus the overall structure
of the file and approach is the same.
The use of $(AUG_GENTEST) as a dependancy in the makefiles needed
to be fixed, because this was assumed to be the filename of the
script, but is in fact a full shell command line.
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Also define the macro for building with GLib older than 2.60
Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Instead of enumerating every single macro variant, just emit
a more generic error.
Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
We started using G_GNUC macros instead of ATTRIBUTE for some attributes.
Adjust this syntax-check accordingly.
Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
To facilitate porting over to glib, this rewrites the auto cleanup
macros to use glib's equivalent.
As a result it is now possible to use g_autoptr/VIR_AUTOPTR, and
g_auto/VIR_AUTOCLEAN, g_autofree/VIR_AUTOFREE interchangably, regardless
of which macros were used to declare the cleanup types.
Within the scope of any single method, code must remain consistent
using either GLib or Libvirt macros, never mixing both. New code
must preferentially use the GLib macros, and old code will be
converted incrementally.
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
The gnulib syntax-check rules are spread across GNUmakefile, cfg.mk and
maint.mk. This made sense when we were getting two of the files from the
gnulib submodule. Now that we own all files though, we can at least
merge maint.mk and cfg.mk together. GNUmakefile can be eliminated when
we switch to meson.
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
If we've marked rules as skipped, there's no sense keeping them in the
maint.mk file.
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
The standard maint.mk from gnulib provides alot more than just the
'syntax-check' target. This can all be purged to give a more minimal
file.
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
The syntax-check rules are the one bit of make usage that will
stay around for a while after the meson conversion. Move them
into the build-aux directory in preparation for refactoring
to make them independent from automake.
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
We're going to be eliminating autotools and gnulib, but we still wish to
have the 'make syntax-check' functionality.
This imports the minimal set of gnulib files required to keep this
working.
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Using @VARNAME@ is a normal style of automake, so lets match that.
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
The augeas-gentest.pl program merges a config file into a augeas
file, saving the output to a new file. It is going to be useful
to further process the output file, and it would be easier if this can
be done with a pipeline, so change augeas-gentest.pl to write to stdout
instead of a file.
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Internal headers should use #pragma once instead of the standard #ifndef
guard. Public headers still require the existing header guard.
Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Ján Tomko <jtomko@redhat.com>
Quite a few of the tests have a need to mock the stat() / lstat()
functions and they are taking somewhat different & inconsistent
approaches none of which are actually fully correct. This is shown
by fact that 'make check' fails on 32-bit hosts. Investigation
revealed that the code was calling into the native C library impl,
not getting intercepted by our mocks.
The POSIX stat() function might resolve to any number of different
symbols in the C library.
The may be an additional stat64() function exposed by the headers
too.
On 64-bit hosts the stat & stat64 functions are identical, always
refering to the 64-bit ABI.
On 32-bit hosts they refer to the 32-bit & 64-bit ABIs respectively.
Libvirt uses _FILE_OFFSET_BITS=64 on 32-bit hosts, which causes the
C library to transparently rewrite stat() calls to be stat64() calls.
Libvirt will never see the 32-bit ABI from the traditional stat()
call. We cannot assume this rewriting is done using a macro. It might
be, but on GLibC it is done with a magic __asm__ statement to apply
the rewrite at link time instead of at preprocessing.
In GLibC there may be two additional functions exposed by the headers,
__xstat() and __xstat64(). When these exist, stat() and stat64() are
transparently rewritten to call __xstat() and __xstat64() respectively.
The former symbols will not actally exist in the library at all, only
the header. The leading "__" indicates the symbols are a private impl
detail of the C library that applications should not care about.
Unfortunately, because we are trying to mock replace the C library,
we need to know about this internal impl detail.
With all this in mind the list of functions we have to mock will depend
on several factors
- If _FILE_OFFSET_BITS is set, then we are on a 32-bit host, and we
only need to mock stat64 and __xstat64. The other stat / __xstat
functions exist, but we'll never call them so they can be ignored
for mocking.
- If _FILE_OFFSET_BITS is not set, then we are on a 64-bit host and
we should mock stat, stat64, __xstat & __xstat64. Either may be
called by app code.
- If __xstat & __xstat64 exist, then stat & stat64 will not exist
as symbols in the library, so the latter should not be mocked.
The same all applies to lstat()
These rules are complex enough that we don't want to duplicate them
across every mock file, so this centralizes all the logic in a helper
file virmockstathelper.c that should be #included when needed. The
code merely need to provide a filename rewriting callback called
virMockStatRedirect(). Optionally VIR_MOCK_STAT_HOOK can be defined
as a macro if further processing is needed inline.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
We must do a substring match, not an exact match since
there can be an arbitrary virtual path prepended.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Require that all headers are guarded by a symbol named
LIBVIRT_$FILENAME
where $FILENAME is the uppercased filename, with all characters
outside a-z changed into '_'.
Note we do not use a leading __ because that is technically a
namespace reserved for the toolchain.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
This introduces a syntax-check script that validates header files use a
common layout:
/*
...copyright header...
*/
<one blank line>
#ifndef SYMBOL
# define SYMBOL
....content....
#endif /* SYMBOL */
For any file ending priv.h, before the #ifndef, we will require a
guard to prevent bogus imports:
#ifndef SYMBOL_ALLOW
# error ....
#endif /* SYMBOL_ALLOW */
<one blank line>
The many mistakes this script identifies are then fixed.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
In many files there are header comments that contain an Author:
statement, supposedly reflecting who originally wrote the code.
In a large collaborative project like libvirt, any non-trivial
file will have been modified by a large number of different
contributors. IOW, the Author: comments are quickly out of date,
omitting people who have made significant contribitions.
In some places Author: lines have been added despite the person
merely being responsible for creating the file by moving existing
code out of another file. IOW, the Author: lines give an incorrect
record of authorship.
With this all in mind, the comments are useless as a means to identify
who to talk to about code in a particular file. Contributors will always
be better off using 'git log' and 'git blame' if they need to find the
author of a particular bit of code.
This commit thus deletes all Author: comments from the source and adds
a rule to prevent them reappearing.
The Copyright headers are similarly misleading and inaccurate, however,
we cannot delete these as they have legal meaning, despite being largely
inaccurate. In addition only the copyright holder is permitted to change
their respective copyright statement.
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Our HACKING guide forbids these.
There's no point in exempting these from the spacing check
if their existence is against our coding style.
Note that the non-usage of these comments itself is not enforced
by syntax check, probably because of the need to implement a C parser.
Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Recent patches added indentation checks that discovered some cosmetic
issues at the cost of making this check last as long as the rest of
syntax-check combined on my system. Also, they're moving closer
to us implementing yet another C parser (docs/apibuild.py being the
other one).
Revert the following commits:
commit 11e1f11dd3
syntax-check: Check for incorrect indentation in function body
commit 2585a79e32
build-aux:check-spacing: Introduce a new rule to check misaligned stuff in parenthesises
commit a033182f04
build-aux:check-spacing: Add wrapper function of CheckCurlyBrackets
commit 6225626b6f
build-aux:check-spacing: Add wrapper function of CheckWhiteSpaces
commit c3875129d9
build-aux:check-spacing: Add wrapper function of KillComments
commit e995904c56
build-aux:check-spacing: Add wrapper function of CheckFunctionBody
commit 11e1f11dd3
syntax-check: Check for incorrect indentation in function body
This brings the speed of the script to a tolerable level and lets it
focus on the more visible issues.
Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
This patch introduces a new rule to check misaligned stuff in parenthesis:
1. For misaligned arguments of function
2. For misaligned conditions of [if|while|switch|...]
There're too much misalignment, so it adds a temporary filter which
permits 'src/util' now. It _should_ be removed as soon as fixing all.
Signed-off-by: Shi Lei <shi_lei@massclouds.com>
This patch add syntax-check rule for incorrect indentation and blank
first line in function body by changing check-spacing.pl.
For efficiency, it only checks the first line of function body.
But it's enough for most cases.
Signed-off-by: Shi Lei <shi_lei@massclouds.com>
Similar to the libvirt.pot, .po files contain line numbers and file
names identifying where in the source a translatable string comes from.
The source locations in the .po files are thrown away and replaced with
content from the libvirt.pot whenever msgmerge is run, so this is not
precious information that needs to be stored in git.
When msgmerge processes a .po file, it will add in any msgids from the
libvirt.pot that were not already present. Thus, if a particular msgid
currently has no translation, it can be considered redundant and again
does not need storing in git.
When msgmerge processes a .po file and can't find an exact existing
translation match, it will try todo fuzzy matching instead, marking such
entries with a "# fuzzy" comment to alert the translator to take a
look and either discard, edit or accept the match. Looking at the
existing fuzzy matches in .po files shows that the quality is awful,
with many having a completely different set of printf format specifiers
between the msgid and fuzzy msgstr entry. Fortunately when msgfmt
generates the .gmo, the fuzzy entries are all ignored anyway. The fuzzy
entries could be useful to translators if they were working on the .po
files directly from git, but Libvirt outsourced translation to the
Fedora Zanata system, so keeping fuzzy matches in git is not much help.
Finally, by default msgids are sorted based on source location. Thus, if
a bit of code with translatable text is moved from one file to another,
it may shift around in the .po file, despite the msgid not itself changing.
If the msgids were sorted alphabetically, the .po files would have
stable ordering when code is refactored.
This patch takes advantage of the above observations to canonicalize
and minimize the content stored for .po files in git. Instead of storing
the real .po files, we now store .mini.po files.
The .mini.po files are the same file format as .po files, but have no
source location comments, are sorted alphabetically, and all fuzzy
msgstrs and msgids with no translation are discarded. This cuts the size
of content in the po directory from 109MB to 19MB.
Users working from a libvirt git checkout who need the full .po files
can run "make update-po", which merges the libvirt.pot and .mini.po
file to create a .po file containing all the content previously stored
in git.
Conversely if a full .po file has been modified, for example, by
downloading new content from Zanata, the .mini.po files can be updated
by running "make update-mini-po". The resulting diffs of the .mini.po
file will clearly show the changed translations without any of the noise
that previously obscured content. Being able to see content changes
clearly actually identified a bug in the zanata python client where it
was adding bogus "fuzzy" annotations to many messages:
https://bugzilla.redhat.com/show_bug.cgi?id=1564497
Users working from libvirt releases should not see any difference in
behaviour, since the tarballs only contain the full .po files, not the
.mini.po files.
As an added benefit, generating tarballs with "make dist", will no
longer cause creation of dirty files in git, since it won't touch the
.mini.po files, only the .po files which are no longer kept in git.
To avoid creating a single commit 100+MB in size, each language is
minimized separately in a following commit.
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
This is particularly useful on operating systems that don't ship
Perl as part of the base system (eg. FreeBSD) while still working
just as well as it did before on Linux.
In one case (src/rpc/genprotocol.pl) the interpreter path was
missing altogether.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
This reverts commit e4b980c853.
When a binary links against a .a archive (as opposed to a shared library),
any symbols which are marked as 'weak' get silently dropped. As a result
when the binary later runs, those 'weak' functions have an address of
0x0 and thus crash when run.
This happened with virtlogd and virtlockd because they don't link to
libvirt.so, but instead just libvirt_util.a and libvirt_rpc.a. The
virRandomBits symbols was weak and so left out of the virtlogd &
virtlockd binaries, despite being required by virHashTable functions.
Various other binaries like libvirt_lxc, libvirt_iohelper, etc also
link directly to .a files instead of libvirt.so, so are potentially
at risk of dropping symbols leading to a later runtime crash.
This is normal linker behaviour because a weak symbol is not treated
as undefined, so nothing forces it to be pulled in from the .a You
have to force the linker to pull in weak symbols using -u$SYMNAME
which is not a practical approach.
This risk is silent bad linkage that affects runtime behaviour is
not acceptable for a fix that was merely trying to fix the test
suite. So stop using __weak__ again.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Currently all mockable functions are annotated with the 'noinline'
attribute. This is insufficient to guarantee that a function can
be reliably mocked with an LD_PRELOAD. The C language spec allows
the compiler to assume there is only a single implementation of
each function. It can thus do things like propagating constant
return values into the caller at compile time, or creating
multiple specialized copies of the function body each optimized
for a different caller. To prevent these optimizations we must
also set the 'noclone' and 'weak' attributes.
This fixes the test suite when libvirt.so is built with CLang
with optimization enabled.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
CLang's optimizer is more aggressive at inlining functions than
gcc and so will often inline functions that our tests want to
mock-override. This causes the test to fail in bizarre ways.
We don't want to disable inlining completely, but we must at
least prevent inlining of mocked functions. Fortunately there
is a 'noinline' attribute that lets us control this per function.
A syntax check rule is added that parses tests/*mock.c to extract
the list of functions that are mocked (restricted to names starting
with 'vir' prefix). It then checks that src/*.h header file to
ensure it has a 'ATTRIBUTE_NOINLINE' annotation. This should prevent
use from bit-rotting in future.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
We do not need a separate check forbidding whitespace
after the opening parenthesis after a keyword -
we forbid it after all of them.
The only allowed whitespace after an opening parenthesis
is a newline, tune the regex to reflect that.
Instead of matching multiple characters before the parenthesis,
only check for a single whitespace, which is much less cpu-intensive.
This only matches a few dozen of places where they are on an separate
line, filter out those with a separate regex.
We're looking for three consecutive lines, first one is a if/for/while
with a condition and start of body, second one is a body with one and
only semicolon and third is end of the body by itself.
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
Change while () { smth; last; } to if () { smth; } as 'last' in perl is
analogous to 'break' in C. These are probably copy-paste leftovers from
creating new syntax-check rules.
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
In bracket-spacing.pl, the current $line is being modified in $data.
That, however, spoils that $data for another check. Introduce new
$tmpdata variable that can be used for temporary modifications. The
difference between $data and $line is that $data are as much cleaned as
possible from non-code blocks and these changes must be kept.
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
Commit a1cbe4b5 added a check for spaces around assignments and this
patch extends it to checks for spaces around '=='. One exception is
virAssertCmpInt where comma after '==' is acceptable (since it is a
macro and '==' is its argument).
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
Enforce and document the style set up by the previous patches.
* build-aux/bracket-spacing.pl: Add comma checks.
* docs/hacking.html.in: Document the rules.
* HACKING: Regenerate.
Signed-off-by: Eric Blake <eblake@redhat.com>
To register virtual machines and containers with systemd-machined,
and thus have cgroups auto-created, we need to talk over DBus.
This is somewhat tedious code, so introduce a dedicated function
to isolate the DBus call in one place.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Only a few cases are allowed:
1) The expression is empty for "for" loop, E.g.
for (i = 0; ; i++)
2) An empty statement
while (write(statuswrite, &status, 1) == -1 &&
errno == EINTR)
; /* empty */
3) ";" is inside double-quote, I.e, as part of const string. E.g.
vshPrint(ctl, "a ; b ; cd;\n");
The "for" loop in src/rpc/virnettlscontext.c is the special case,
1) applies for it, so change it together in this patch.
This documents the following whitespace rules
if(foo) // Bad
if (foo) // Good
int foo (int wizz) // Bad
int foo(int wizz) // Good
bar = foo (wizz); // Bad
bar = foo(wizz); // Good
typedef int (*foo) (int wizz); // Bad
typedef int (*foo)(int wizz); // Good
int foo( int wizz ); // Bad
int foo(int wizz); // Good
There is a syntax-check rule extension to validate all these rules.
Checking for 'function (...args...)' is quite difficult since it
needs to ignore valid usage with keywords like 'if (...test...)'
and while/for/switch. It must also ignore source comments and
quoted strings.
It is not possible todo this with a simple regex in the normal
syntax-check style. So a short Perl script is created instead
to analyse the source. In practice this works well enough. The
only thing it can't cope with is multi-line quoted strings of
the form
"start of string\
more lines\
more line\
the end"
but this can and should be written as
"start of string"
"more lines"
"more line"
"the end"
with this simple change, the bracket checking script does not
have any false positives across libvirt source, provided it
is only run against .c files. It is not practical to run it
against .h files, since those use whitespace extensively to
get alignment (though this is somewhat inconsistent and could
arguably be fixed).
The only limitation is that it cannot detect a violation where
the first arg starts with a '*', eg
foo(*wizz);
since this generates too many false positives on function
typedefs which can't be supressed efficiently.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
https://www.gnu.org/licenses/gpl-howto.html recommends that
the 'If not, see <url>.' phrase be a separate sentence.
* tests/securityselinuxhelper.c: Remove doubled line.
* tests/securityselinuxtest.c: Likewise.
* globally: s/; If/. If/
The cfg.mk file rule to check for tab characters was not
applied to perl files. Much of our Perl code is full of
tabs as a result. Kill them, kill them all !
When adding new config file parameters, the corresponding
additions to the augeas lens' are constantly forgotten.
Also there are augeas test cases, these don't catch the
error, since they too are never updated.
To address this, the augeas test cases need to be auto-generated
from the example config files.
* build-aux/augeas-gentest.pl: Helper to generate an
augeas test file, substituting in elements from the
example config files
* src/Makefile.am, daemon/Makefile.am: Switch to
auto-generated augeas test cases
* daemon/test_libvirtd.aug, daemon/test_libvirtd.aug.in,
src/locking/test_libvirt_sanlock.aug,
src/locking/test_libvirt_sanlock.aug.in,
src/lxc/test_libvirtd_lxc.aug,
src/lxc/test_libvirtd_lxc.aug.in,
src/qemu/test_libvirtd_qemu.aug,
src/qemu/test_libvirtd_qemu.aug.in: Remove example
config file data, replacing with a ::CONFIG:: placeholder
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Allows bootstrap to work on FreeBSD, where gzip doesn't have a '.'
in its version; and silences false positives in the new
'make syntax-check' rule.
* .gnulib: Update to latest.
* bootstrap: Synchronize to upstream.
* .x-sc_bindtextdomain: New exemptions.
* Makefile.am (syntax_check_exceptions): Ship new file.
* .gitignore: Regenerate per latest bootstrap, anchor entries that
are only in the root directory, and consolidate entries from other
generated .gitignore files.
* build-aux/.gitignore, m4/.gitignore, po/.gitignore: Remove from
version control, since bootstrap generates them.
Picks up fixes for gethostname compilation problems on mingw.
* .gnulib: Update to latest.
* build-aux/.gitignore: Regenerate.
* cfg.mk (local-checks-to-skip): Avoid new test not relevent to
libvirt.
GNUTLS uses gcrypt for its crypto functions. gcrypt requires
that the app/library initializes threading before using it.
We don't want to force apps using libvirt to know about
gcrypt, so we make virInitialize init threading on their
behalf. This location also ensures libvirtd has initialized
it correctly. This initialization is required even if libvirt
itself were only using one thread, since another non-libvirt
library (eg GTK-VNC) could also be using gcrypt from another
thread
* src/libvirt.c: Register thread functions for gcrypt
* configure.in: Add -lgcrypt to linker flags
No longer maintain a version-controlled ChangeLog file, but do
continue to include a ChangeLog file in distribution tarball.
* Makefile.am (gen-ChangeLog): New rule.
(dist-hook): Depend on it.
(EXTRA_DIST): Add ChangeLog-old.
* bootstrap (modules): Add gitlog-to-changelog.
* ChangeLog: Remove file. Renamed to...
* ChangeLog-old: ...this. New file.
* autogen.sh: Touch ChangeLog, to ensure it exists. For automake.
This makes it so we record (via a git submodule)
a snapshot of whatever version of gnulib we're using,
and none of gnulib sources are in the libvirt repository.
The result is that we have as much reproducibility as when
we version-controlled imported copies of the gnulib sources,
but without the hassle of the manual process we used when
syncing with upstream.
Note that when you clone libvirt, you get only the libvirt
repository, but when you first run ./bootstrap, it clones
gnulib (at the SHA1 recorded via the submodule), creating
the .gnulib/ hierarchy. Then, the bootstrap script runs
gnulib-tool to populate gnulib/ with the files that make
up the selected modules.
Put the following in your ~/.gitconfig file.
[alias]
syncsub = submodule foreach git pull origin master
The update procedure is simple:
git syncsub
...build & test...
git commit -m 'gnulib: sync submodule to latest' .gnulib
* .gitmodules: New file.
* .gnulib: Initialize.
* bootstrap: Set up to use the new submodule.
Stop using --no-vc-files.
Don't remove .gitignore files.
Don't use or create .cvsignore.
Diagnose an invalid --gnulib-srcdir=DIR argument.
* build-aux/vc-list-files: Delete file, now pulled from gnulib.
* build-aux/useless-if-before-free: Likewise.
* po/POTFILES.in: Remove gnulib/lib/gai_strerror.c, since
it no longer contains translatable strings.
* gnulib/*: Remove gnulib/ hierarchy.
* Makefile.maint (sync-vcs-ignore-files): New target.
Prompted by a patch from James Morris.
http://thread.gmane.org/gmane.comp.emulators.libvirt/8619/focus=8773
Add all (now-generated) .gitignore files.
* .gitignore: New file.
* build-aux/.gitignore: New file.
* docs/.gitignore: New file.
* docs/devhelp/.gitignore: New file.
* docs/examples/.gitignore: New file.
* docs/examples/python/.gitignore: New file.
* gnulib/lib/.gitignore: New file.
* gnulib/lib/arpa/.gitignore: New file.
* gnulib/lib/netinet/.gitignore: New file.
* gnulib/lib/sys/.gitignore: New file.
* gnulib/tests/.gitignore: New file.
* include/.gitignore: New file.
* include/libvirt/.gitignore: New file.
* po/.gitignore: New file.
* proxy/.gitignore: New file.
* python/.gitignore: New file.
* python/tests/.gitignore: New file.
* qemud/.gitignore: New file.
* src/.gitignore: New file.
* tests/.gitignore: New file.
* tests/confdata/.gitignore: New file.
* tests/sexpr2xmldata/.gitignore: New file.
* tests/virshdata/.gitignore: New file.
* tests/xencapsdata/.gitignore: New file.
* tests/xmconfigdata/.gitignore: New file.
* tests/xml2sexprdata/.gitignore: New file.
* tests/Makefile.am (test_scripts): Add vcpupin.
(EXTRA_DIST): Add test-lib.sh.
* tests/test-lib.sh: Testing framework, from coreutils.
* tests/vcpupin: New file.
* build-aux/mktempd: New file, from gnulib.
* bootstrap: Add posix-shell and mktempd to the list of imported modules.
* gnulib/m4/posix-shell.m4: New file, from gnulib.
* bootstrap (gnulib_tool): Add useless-if-before-free to
the list of modules.
* build-aux/useless-if-before-free: Update from gnulib.
* gnulib/lib/vasnprintf.c: Work around a bug in HPUX 10.20.
* gnulib/m4/vasnprintf.m4: Likewise.
Almost all tests are initially disabled via the list in Makefile.cfg.
* Makefile.am (EXTRA_DIST): Add .x-sc_avoid_if_before_free.
Omit names of files that automake includes automatically.
* .x-sc_avoid_if_before_free: New file.
* build-aux/vc-list-files: Likewise.
* build-aux/find-unnecessary-if-before-free: Likewise.
* GNUmakefile, Makefile.cfg, Makefile.maint: New files.
New files go into these directories:
gnulib/lib
gnulib/m4
gnulib/tests
* bootstrap: A wrapper around gnulib-tool.
* configure.in: Invoke gl_EARLY and gl_INIT, being careful to put gl_EARLY
before any macro that uses AC_COMPILE_IFELSE.
(AC_OUTPUT): Add lib/Makefile and gl-tests/Makefile. Remove m4/Makefile.
* Makefile.am (SUBDIRS): Add gnulib/lib and remove m4. Add gnulib/tests
early enough that those tests run before any libvirt unit tests.
* m4/Makefile.am: Remove file. Not needed.
* src/Makefile.am (INCLUDES): Add -I$(top_srcdir)/gnulib/lib -I../gnulib/lib.
(LDADDS, libvirt_la_LIBADD): Add ../gnulib/lib/libgnu.la.
* src/nodeinfo.c: Include "physmem.h".
* qemud/qemud.c, src/remote_internal.c: Include "getaddrinfo.h".
(MEMINFO_PATH, linuxNodeInfoMemPopulate): Remove definitions.
(virNodeInfoPopulate): Use physmem_total, not linuxNodeInfoMemPopulate.
* tests/Makefile.am (INCLUDES): Add -I$(top_srcdir)/gnulib/lib -I../gnulib/lib.
(LDADDS): Add ../gnulib/lib/libgnu.la.
* qemud/Makefile.am (libvirtd_LDADD): Add ../gnulib/lib/libgnu.la.
* tests/nodeinfotest.c (linuxTestCompareFiles): No longer read total
memory from a file.
Update expected output not to include "Memory: NNNN"
* tests/nodeinfodata/linux-nodeinfo-1.txt:
* tests/nodeinfodata/linux-nodeinfo-2.txt:
* tests/nodeinfodata/linux-nodeinfo-3.txt:
* tests/nodeinfodata/linux-nodeinfo-4.txt:
* tests/nodeinfodata/linux-nodeinfo-5.txt:
* tests/nodeinfodata/linux-nodeinfo-6.txt:
* src/test.c [WITH_TEST]: Remove definition of _GNU_SOURCE that
would conflict with the one now in "config.h".
* autogen.sh: Add -I gnulib/m4.
* src/conf.c, src/sexpr.c: Don't define _GNU_SOURCE.
Instead, include "config.h".
* qemud/qemud.c: Remove definition of _GNU_SOURCE.
* src/openvz_driver.c: Likewise.
* src/qemu_driver.c: Likewise.
* src/remote_internal.c: Likewise.
* configure.in: Use AC_CONFIG_AUX_DIR(build-aux), so that a bunch
of gettextize-generated files go into build-aux/, rather than in
the top-level directory.
* .cvsignore: Adjust.
* build-aux/.cvsignore: New file.
Author: Jim Meyering <meyering@redhat.com>
