Commit Graph

218 Commits

Author SHA1 Message Date
Ján Tomko
0fd2f4310a build-aux: add sc_ prefix to more syntax checks
That way our meson build file will correctly pick them up.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
2020-08-03 13:54:15 +02:00
Ján Tomko
5c01b41763 build-aux: set the PYTHON variable in the Makefile
Some syntax-check rules rely on this variable.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
2020-08-03 13:54:15 +02:00
Pavel Hrdina
0c2aa5b513 meson: add syntax-check
This changes the approach used with autotools where it was separate make
target. With meson it will be part of the `meson test` target but can be
disabled using --no-suite syntax-check or we can run only syntax-check
by using --suite syntax-check.

Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Neal Gompa <ngompa13@gmail.com>
2020-08-03 09:27:06 +02:00
Pavel Hrdina
71f95ea1a4 meson: syntax-check: drop Makefile and m4 related checks
Rewrite to meson will eliminate most of the Makefile and all of m4
files so there is no need to check them.

We still need to ignore mk files otherwise syntax-check.mk would be
considered as well.

Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Neal Gompa <ngompa13@gmail.com>
2020-08-03 09:26:33 +02:00
Pavel Hrdina
c6a0d3ff8b po: change the format of POTFILES.in
There is no need to provide relative paths to the current directory if
we provide search paths using --directory option for xgettext.

In addition it will make libvirt.pot file look cleaner as it will not
contain relative paths to current directory. It improves the situation
for developers which are using different build path as that would
change the relative path in libvirt.pot as well. After this patch
it will not happen anymore.

Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-07-10 09:40:29 +02:00
Jonathon Jongsma
9badcbbb1d nodedev: Add testing for 'mdevctl start'
Test that we run 'mdevctl' with the proper arguments when creating new
mediated devices with virNodeDeviceCreateXML().

Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-06-19 10:39:55 +02:00
Peter Krempa
84a20fe8e3 qemumonitortestutils: Make test monitor failures more prominent
Until now we've tried to report errors from the test monitor code by
passing them back as failures from the qemu we simulate. This doesn't
work well in cases when the monitor logic does not detect failures or
has fallback code. Additionally there isn't much use for continuing the
test execution after first failure as in most cases the test data will
be misaligned and all other calls will fail as well.

To make the errors more obvious this patch moves away from reporting
them via the simulated monitor to reporting them to stderr and
exit()ing afterwards. While this might be less convenient
when developing tests it actually makes failures in the test suite
really obvious and doesn't require any opt-in from the tests themselves.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-04-27 08:13:48 +02:00
Andrea Bolognani
936d598b89 docs: Add advanced-tests.rst
This part contains a lot of useful tips, but presenting all of them
at the same time obfuscated the central message which is, 'make check'
and 'make syntax-check' must pass after each patch in a series. Let's
move them to a separate page.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-04-08 09:32:39 +02:00
Andrea Bolognani
5155f6f4b1 docs: Add coding-style.rst
This part represents the biggest chunk of the existing hacking.rst, and
despite that its utility is very limited because 'make syntax-check'
already guarantees most of the rules are followed over time.

Until the glorious day we finally codify our coding style completely
into a configuration for a tool such as clang-format and thus no longer
need a plain English description of it, move this part to a separate
page.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-04-08 09:32:30 +02:00
Andrea Bolognani
b186b07940 docs: Convert hacking.html to reStructuredText
The conversion has been performed by using pandoc as a first pass,
and then tweaking the result manually until it looked satisfactory.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-04-08 09:32:16 +02:00
Peter Krempa
b659b5f93a syntax-check: Exempt tests/qemucapabilitiesdata from 'sc_prohibit_devname'
QEMU uses the 'devname' string in the QAPI schema so a bump would
trigger this check. Exempt all of the capabilities data from the check.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-04-03 09:36:20 +02:00
Michal Privoznik
bf7454d145 syntax-check.mk: Drop needless _src2 variable
The aim was to break a long line, but it is not long anymore.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2020-03-27 16:25:48 +01:00
Michal Privoznik
c67b6a3996 syntax-check: Disallow plain fork() in src/libvirt.c
The last usage of plain fork() was removed in v0.9.7-rc1~50, but
we forgot to update the syntax-check exemption list accordingly.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2020-03-27 16:25:48 +01:00
Rafael Fonseca
5f9c7f0941 remote: use shared daemon startup code
Signed-off-by: Rafael Fonseca <r4f4rfs@gmail.com>
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-03-27 16:22:49 +01:00
Rafael Fonseca
14b12fd944 logging: use shared daemon startup code
Signed-off-by: Rafael Fonseca <r4f4rfs@gmail.com>
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-03-27 16:22:49 +01:00
Rafael Fonseca
0678140239 locking: use shared daemon startup code
Signed-off-by: Rafael Fonseca <r4f4rfs@gmail.com>
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-03-27 16:22:49 +01:00
Rafael Fonseca
fc5925f1e0 util: introduce shared daemon startup code
Several daemons have similar code around general daemon startup code.
Let's move it into a file and share it among them.

Signed-off-by: Rafael Fonseca <r4f4rfs@gmail.com>
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-03-27 16:22:49 +01:00
Michal Privoznik
c799d150d5 virprocess: Passthru error from virProcessRunInForkHelper
When running a function in a forked child, so far the only thing
we could report is exit status of the child and the error
message. However, it may be beneficial to the caller to know the
actual error that happened in the child.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Pavel Mores <pmores@redhat.com>
2020-03-20 16:42:45 +01:00
Daniel P. Berrangé
fdd941eeb2 build: workaround behaviour regression in gnu make 4.3
We need the "$(space)" variable to contain a single whitespace
character. We do this by assigning and then appending an empty
string to the variable. Variable appends get separated by a
single whitespace historically, but GNU make 4.3 introduced a
behaviour regression.

  https://lists.gnu.org/archive/html/bug-make/2020-01/msg00057.html

[quote]
* WARNING: Backward-incompatibility!
  Previously appending using '+=' to an empty variable would
  result in a value starting with a space.  Now the initial
  space is only added if the variable already contains some
  value.  Similarly, appending an empty string does not
  add a trailing space.
[/quote]

This patch tries a new trick to get a single whitespace by
getting make to expand two non-existant variables separated
by a space.

Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-16 12:07:21 +00:00
Ján Tomko
25c29ac2f5 virclosecallbacks: move to src/hypervisor
Just like virhostdev, this depends on domain_conf and
it's shared by multiple hypervisor drivers.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-02-24 16:47:21 +01:00
Ján Tomko
a504a3c377 virhostdev: move to src/hypervisor
This module depends on domain_conf and is used directly by various
hypervisor drivers.

Move it to src/hypervisor.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-02-24 16:47:21 +01:00
Ján Tomko
a1c2628513 syntax-check: inclusion rule for src/hypervisor
Allow it to be included by anything above mid_dirs.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-02-24 16:47:21 +01:00
Ján Tomko
665c5dfb71 syntax-check: remove some exception mechanisms
Do not look for exception patterns in ${srcdir}./x-$@
nor the VC_LIST_EXCEPT_DEFAULT variable.

This also removes the default exception for ChangeLog.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-02-11 11:48:08 +01:00
Ján Tomko
a275ce2ab7 syntax-check: remove README
This exception is no longer useful since README is just a symlink
to README.md, which is a subject to this check already.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-02-11 11:48:08 +01:00
Ján Tomko
9d8d332dbd syntax-check: exclude: remove virstring
We no longer implement a wrapper over strdup - g_strdup
is preferred.

The use of strncpy was removed in:
commit 7d70a63b94
    util: Improve virStrncpy() implementation

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-02-11 11:48:08 +01:00
Ján Tomko
7a86524ddb syntax-check: exclude: remove deleted files
Both xen/xend_internal and bootstrap.conf have been deleted from git.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-02-11 11:48:08 +01:00
Ján Tomko
73c35a3083 syntax-check: drop update-NEWS-hash
NEWS was replaced by docs/news.html, so the pre-requisite for this
rule is not fulfilled.

Also, PREV_VERSION_REGEXP does not seem to be defined anywhere.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-02-11 11:48:08 +01:00
Ján Tomko
1a97ebae1a syntax-check: drop CVS keyword expansion check
$ date +%Y
  2020

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-02-11 11:48:08 +01:00
Ján Tomko
b491528604 syntax-check: drop vulnerable Makefile checks
As foretold, fixed automake is so common nowadays even Ubuntu 16.04
and Debian 9 have 1.11.6 as the oldest available version.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-02-11 11:48:08 +01:00
Ján Tomko
c642199fdf syntax-check: fix sc-prohibit-cross-inclusion
Using '^' in in_vc_files assumes the build is happening
in srcdir, which is no longer possible as of:
commit f96395e78e

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-02-11 11:48:08 +01:00
Ján Tomko
04cef3b2ed syntax-check: do not enforce ChangeLog syntax
Our last conforming ChangeLog was removed from git in 2009 by:
commit 27b175b9a2
    generate ChangeLog from git logs into distribution tarball

Ten years later, it reappeared in:
commit ce97c33a79
    maint: Stop generating ChangeLog from git
which does not have a single line complying to this syntax
check rule.

At some point between the two commits VC_LIST_EXCEPT gained
a default exception for ChangeLog and it did not seem to be
overriden anywhere.

Just remove the rule.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-02-11 11:48:07 +01:00
Daniel P. Berrangé
2621d48f00 gnulib: delete all gnulib integration
This deletes all trace of gnulib from libvirt. We still
have the keycodemapdb submodule to deal with. The simple
solution taken was to update it when running autogen.sh.

Previously gnulib could auto-trigger refresh when running
'make' too. We could figure out a solution for this, but
with the pending meson rewrite it isn't worth worrying
about, given how infrequently keycodemapdb changes.

Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-02-07 15:03:54 +00:00
Daniel P. Berrangé
f8ab47cb44 util: import an event loop impl based on GMainContext
The libvirt-glib project has provided a GMainContext based
event loop impl for applications. This imports it and sets
it up for use by libvirt as the primary event loop. This
remains a private impl detail of libvirt.

IOW, applications must *NOT* assume that a call to
"virEventRegisterDefaultImpl" results in a GLib based
event loop. They should continue to use the libvirt-glib
API gvir_event_register() if they explicitly want to
guarantee a GLib event loop.

This follows the general principle that the libvirt public
API should not expose the fact that GLib is being used
internally.

Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-02-07 15:03:53 +00:00
Daniel P. Berrangé
5de317b8e7 src: introduce helper API for creating GSource for socket
We need to be able to create event loop watches using the
GSource API for sockets. GIOChannel is able todo this, but
we don't want to use the GIOChannel APIs for reading/writing,
and testing shows just using its GSource APIs is unreliable
on Windows.

This patch thus creates a standalone helper API for creating
a GSource for a socket file descriptor. This impl is derived
from code in QEMU's io/channel-watch.c file that was written
by myself & Paolo Bonzini & thus under Red Hat copyright.

Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-02-07 15:03:53 +00:00
Daniel P. Berrangé
7f3eb533f4 src: conditionalize use of net/if.h
The net/if.h is not portable so we must check for its
existance and avoid using it when missing. Some use
of net/if.h was redundant and could be removed.

Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-01-29 14:51:40 +00:00
Peter Krempa
7e8ed7d782 virsh: Work around virSecretFree quirks
Similarly to other libvirt object freeing APIs the function resets the
libvirt error when called and doesn't take NULL gracefully. Install the
workaround and g_autoptr handlers similarly to the 'virshDomain' type.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-01-28 18:09:57 +01:00
Daniel P. Berrangé
d600667278 qemu: introduce a new "virt-qemu-run" program
The previous "QEMU shim" proof of concept was taking an approach of only
caring about initial spawning of the QEMU process. It was then
registered with the libvirtd daemon who took over management of it. The
intent was that later libvirtd would be refactored so that the shim
retained control over the QEMU monitor and libvirt just forwarded APIs
to each shim as needed. This forwarding of APIs would require quite alot
of significant refactoring of libvirtd to achieve.

This impl thus takes a quite different approach, explicitly deciding to
keep the VMs completely separate from those seen & managed by libvirtd.
Instead it uses the new "qemu:///embed" URI scheme to embed the entire
QEMU driver in the shim, running with a custom root directory.

Once the driver is initialization, the shim starts a VM and then waits
to shutdown automatically when QEMU shuts down, or should kill QEMU if
it is terminated itself. This ought to use the AUTO_DESTROY feature but
that is not yet available in embedded mode, so we rely on installing a
few signal handlers to gracefully kill QEMU. This isn't reliable if
we crash of course, but you can restart with the same root dir.

Note this program does not expose any way to manage the QEMU process,
since there's no RPC interface enabled. It merely starts the VM and
cleans up when the guest shuts down at the end. This program is
installed to /usr/bin/virt-qemu-run enabling direct use by end users.
Most use cases will probably want to integrate the concept directly
into their respective application codebases. This standalone binary
serves as a nice demo though, and also provides a way to measure
performance of the startup process quite simply.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-01-27 11:05:02 +00:00
Pavel Hrdina
c02e9621b9 syntax-check: remove deleted daemon directory from space_indent_check
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-01-17 16:04:26 +01:00
Daniel P. Berrangé
fa434739a0 src: replace verify(expr) with G_STATIC_ASSERT(expr)
G_STATIC_ASSERT() is a drop-in functional equivalent of
the GNULIB verify() macro.

Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-01-17 10:02:01 +00:00
Daniel P. Berrangé
4f128bbbfb util: pull gnulib physmem impl into local code
We don't need all the platforms gnulib deals with, so
this is a cut down version of GNULIB's physmem.c
code. This also allows us to integrate libvirt's
error reporting functions closer to the error cause.

Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-01-17 10:01:57 +00:00
Daniel P. Berrangé
f650e86703 util: add detection of openpty function
All UNIX platforms we care about have openpty() in the libutil
library. Use of pty.h must also be made conditional, excluding
Win32.

Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-01-17 09:59:08 +00:00
Daniel P. Berrangé
913707bb8d build: validate headers against local gnulib not git repo
Some syntax check rules validate usage of headers provided
by gnulib. We want to validate these only against the gnulib
modules we've chosen to use, not all modules, since we're
trying to eliminate them.

Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-01-17 09:59:08 +00:00
Daniel P. Berrangé
f018fbac78 src: remove use of the INT_MULTIPLY_OVERFLOW macro
The GLib g_size_checked_mul() function is not quite the
same signature, and gives compiler warnings due to not
correctly casting from gsize to guint64/32. Implementing
a replacement for INT_MULTIPLY_OVERFLOW is easy enough
to do ourselves.

Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-01-17 09:59:05 +00:00
Daniel P. Berrangé
8812163124 src: remove unused imports of dirname.h
A few places were importing dirname.h without actually using it.

Reviewed-by: Fabiano Fidêncio <fidencio@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-01-03 15:42:13 +00:00
Daniel P. Berrangé
0e09706844 util: add note about event file descriptors on Windows
When using GNULIB with Winsock, libvirt will never see the normal HANDLE
objects, instead GNULIB guarantees that libvirt gets a C runtime file
descriptor. The GNULIB poll impl also expects to get C runtime file
descriptors rather than HANDLE objects. Document this behaviour so that
it is clear to applications providing event loop implementations if they
need Windows portability.

Reviewed-by: Fabiano Fidêncio <fidencio@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-01-03 15:42:12 +00:00
Daniel P. Berrangé
6ca74054b9 tests: rewrite qemu capability grouper in Python
As part of a goal to eliminate Perl from libvirt build tools,
rewrite the group-qemu-caps.pl tool in Python.

This was a straight conversion, manually going line-by-line to
change the syntax from Perl to Python. Thus the overall structure
of the file and approach is the same.

Tested-by: Cole Robinson <crobinso@redhat.com>
Reviewed-by: Cole Robinson <crobinso@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-20 14:23:39 +00:00
Pavel Hrdina
969f83822b syntax-check: update strcase check to refer to GLib
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Cole Robinson <crobinso@redhat.com>
2019-12-10 14:08:53 +01:00
Pavel Hrdina
aa7005d1c0 syntax-check: update c-type checks to refer to Glib
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Cole Robinson <crobinso@redhat.com>
2019-12-10 13:49:24 +01:00
Daniel P. Berrangé
504f831957 tests: rewrite test argv line wrapper in Python
As part of a goal to eliminate Perl from libvirt build tools,
rewrite the test-wrap-argv.pl tool in Python.

This was a straight conversion, manually going line-by-line to
change the syntax from Perl to Python. Thus the overall structure
of the file and approach is the same.

Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-04 13:44:15 +00:00
Daniel P. Berrangé
e0314b6855 src: rewrite remote protocol checker in Python
As part of a goal to eliminate Perl from libvirt build tools,
rewrite the pdwtags processing script in Python.

The original inline shell and perl code was completely
unintelligible. The new python code is a manual conversion
that attempts todo basically the same thing.

Tested-by: Cole Robinson <crobinso@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-04 13:44:15 +00:00
Daniel P. Berrangé
5ffbad2575 docs: move apibuild.py to the scripts/ directory
Reviewed-by: Cole Robinson <crobinso@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-04 11:41:32 +00:00
Daniel P. Berrangé
6f4f52d05f src: rewrite systemtap function generator in Python
As part of a goal to eliminate Perl from libvirt build tools,
rewrite the gensystemtap.pl tool in Python.

This was a straight conversion, manually going line-by-line to
change the syntax from Perl to Python. Thus the overall structure
of the file and approach is the same.

Tested-by: Cole Robinson <crobinso@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-11-20 14:45:25 +00:00
Daniel P. Berrangé
2d6f543b06 build-aux: rewrite header ifdef checker in Python
As part of a goal to eliminate Perl from libvirt build tools,
rewrite the header-ifdef.pl tool in Python.

This was a straight conversion, manually going line-by-line to
change the syntax from Perl to Python. Thus the overall structure
of the file and approach is the same.

Tested-by: Cole Robinson <crobinso@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-11-20 14:45:25 +00:00
Daniel P. Berrangé
b7d00249ea build-aux: rewrite mock inline checker in Python
As part of a goal to eliminate Perl from libvirt build tools,
rewrite the mock-noinline.pl tool in Python.

This was a straight conversion, manually going line-by-line to
change the syntax from Perl to Python. Thus the overall structure
of the file and approach is the same.

Tested-by: Cole Robinson <crobinso@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-11-20 14:45:25 +00:00
Pavel Hrdina
722f55180f syntax-check: forbid usage of snprintf
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-11-20 13:30:34 +01:00
Pavel Hrdina
3c5dcf3427 syntax-check: update of sprintf rule to mention g_snprintf
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-11-20 13:30:11 +01:00
Daniel P. Berrangé
7909359d60 build-aux: rewrite duplicate header checker in Python
As part of an goal to eliminate Perl from libvirt build tools,
rewrite the prohibit-duplicate-header.pl tool in Python.

This was a straight conversion, manually going line-by-line to
change the syntax from Perl to Python. Thus the overall structure
of the file and approach is the same.

Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-11-18 17:24:31 +00:00
Ján Tomko
954f36e078 syntax-check: prefer g_mkstemp_full and g_mkdtemp
Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
2019-11-14 19:02:31 +01:00
Ján Tomko
ec07893a5f util: use g_vsnprintf
Instead of vsnprintf from gnulib, use g_vsnprintf from GLib.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
2019-11-14 14:06:49 +01:00
Michal Privoznik
dd98a6edb9 Drop virAsprintf()
Now that function is no longer used, it can be dropped.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com>
2019-11-12 16:15:59 +01:00
Daniel P. Berrangé
07943d35ea build: comment on why we're not adopting certain flake8 rules
Simplify the list of ignored warnings now that we only have two left,
and document why we're not honouring them.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-11-11 14:24:19 +00:00
Daniel P. Berrangé
3df69e628f python: sanitize indentation after line continuations
Line continuations should be 4 space indented unless a previous opening
brace required different alignment.

docs/apibuild.py:2014:24: E126 continuation line over-indented for hanging indent
                       token[0], token[1]))
                       ^
docs/apibuild.py:74:3: E121 continuation line under-indented for hanging indent
  "ATTRIBUTE_UNUSED": (0, "macro keyword"),
  ^
...more...

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-11-11 14:24:19 +00:00
Daniel P. Berrangé
43d29cb40b python: sanitize spaces either side of operators
There should be a single space either side of operators. Inline
comments should have two spaces before the '#'

src/hyperv/hyperv_wmi_generator.py:130:45: E261 at least two spaces before inline comment
            source += '    { "", "", 0 },\n' # null terminated
                                            ^
src/esx/esx_vi_generator.py:417:25: E221 multiple spaces before operator
    FEATURE__DESERIALIZE  = (1 << 6)
                        ^
tests/cputestdata/cpu-cpuid.py:187:78: E225 missing whitespace around operator
                f.write("  <msr index='0x%x' edx='0x%08x' eax='0x%08x'/>\n" %(
                                                                             ^
docs/apibuild.py:524:47: E226 missing whitespace around arithmetic operator
                            self.line = line[i+2:]
                                              ^
...more...

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-11-11 14:24:19 +00:00
Daniel P. Berrangé
bc59247df9 python: sanitize blank line usage
Coding style expects 1 blank line between each method and 2 blank lines
before each class.

docs/apibuild.py:171:5: E303 too many blank lines (2)
    def set_header(self, header):
    ^
docs/apibuild.py:230:1: E302 expected 2 blank lines, found 1
class index:
^
docs/apibuild.py:175:5: E301 expected 1 blank line, found 0
    def set_module(self, module):
    ^
...more...

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-11-11 14:24:19 +00:00
Daniel P. Berrangé
2ba699cbd5 python: avoid bare 'except:' clause
Exception catching statements should always match on a class name, the
most specific one possible. Rather than analyse the code to look at what
the most specific one is, this just uses the base Exception class.

docs/apibuild.py:255:9: E722 do not use bare 'except'
        except:
        ^
docs/apibuild.py:279:9: E722 do not use bare 'except'
        except:
        ^
...more...

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-11-11 14:24:19 +00:00
Daniel P. Berrangé
cfdd871f26 python: avoid variable named 'l'
Python code style recommends avoiding a variable named 'l' as it is
visually similar to '1'.

docs/apibuild.py:482:13: E741 ambiguous variable name 'l'
            l = len(line)
            ^
docs/apibuild.py:503:21: E741 ambiguous variable name 'l'
                    l = len(line)
                    ^
...more...

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-11-11 14:24:19 +00:00
Daniel P. Berrangé
d5c5d8af45 python: mark regex strings with 'r' prefix
When writing regexes special regex matches like "\d" can get
misinterpreted as normal string escape sequences:

docs/apibuild.py:1359:51: W605 invalid escape sequence '\d'
                        value = value + re.sub("^(\d+)U$", "\\1", token[1])
                                                  ^
docs/apibuild.py:2134:31: W605 invalid escape sequence '\('
                m = re.match("\(?1<<(\d+)\)?", info[0])
                              ^
docs/apibuild.py:2134:38: W605 invalid escape sequence '\d'
                m = re.match("\(?1<<(\d+)\)?", info[0])
                                     ^
docs/apibuild.py:2134:42: W605 invalid escape sequence '\)'
                m = re.match("\(?1<<(\d+)\)?", info[0])
                                         ^

To avoid this probem all regexes should use the r"...." syntax for their
strings, which disables normal string escape sequences.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-11-11 14:24:19 +00:00
Daniel P. Berrangé
524b377e9e python: fix use of undeclared variables in python scripts
docs/apibuild.py:2436:65: F821 undefined name 'first_letter'
                        chunks.append(["chunk%s" % (chunk - 1), first_letter, letter])
                                                                ^
src/hyperv/hyperv_wmi_generator.py:415:57: F821 undefined name 'number'
        report_error("line %d: invalid block header" % (number))
                                                        ^

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-11-11 14:24:19 +00:00
Daniel P. Berrangé
3a5fea6062 build: change flake8 to use blacklist instead of whitelist
The current flake8 check only looks at one item (semicolons at end of
line). This means that our code quality will continue to get worse,
violating an increasing number of checks.

Switching to a whitelist means that we freeze the badness at its
current level & can incrementally fix things up.

We are excluding the following...

Indentation:

  E114 indentation is not a multiple of four (comment)
  E115 expected an indented block (comment)
  E116 unexpected indentation (comment)
  E121 continuation line under-indented for hanging indent
  E125 continuation line with same indent as next logical line
  E126 continuation line over-indented for hanging indent
  E127 continuation line over-indented for visual indent
  E128 continuation line under-indented for visual indent
  E129 visually indented line with same indent as next logical line
  E131 continuation line unaligned for hanging indent

Whitespace:

  E211 whitespace before ‘(‘
  E221 multiple spaces before operator
  E222 multiple spaces after operator
  E225 missing whitespace around operator
  E226 missing whitespace around arithmetic operator
  E231 missing whitespace after ‘,’, ‘;’, or ‘:’
  E261 at least two spaces before inline comment

Blank lines

  E301 expected 1 blank line, found 0
  E302 expected 2 blank lines, found 0
  E303 too many blank lines (3)
  E305 expected 2 blank lines after end of function or class

Line length

  E501 line too long (82 > 79 characters)

Statements

  E722 do not use bare except, specify exception instead
  E741 do not use variables named ‘l’, ‘O’, or ‘I’

Errors:

  F821 undefined name 'name'

Warnings:

  W504 line break after binary operator
  W605 invalid escape sequence ‘x’

Later commits will enable most of these exclusions.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-11-11 14:23:50 +00:00
Pavel Hrdina
4753fd0553 src: remote: generate source files into build directory
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2019-11-08 17:07:57 +01:00
Pavel Hrdina
ae98112a85 src: lxc: generate source files into build directory
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2019-11-08 17:07:57 +01:00
Pavel Hrdina
775d08f8c6 src: logging: generate source files into build directory
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2019-11-08 17:07:57 +01:00
Pavel Hrdina
787ea47680 src: locking: generate source files into build directory
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2019-11-08 17:07:57 +01:00
Pavel Hrdina
d6be9e7f65 src: admin: generate source files into build directory
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2019-11-08 17:07:57 +01:00
Pavel Hrdina
7b9cd113dc src: generate source files into build directory
This affects more than src/Makefile.am as the rule to generate source
files for protocols is generic for all sub-directories.

Affected files are:
    src/admin/admin_protocol.{h,c}
    src/locking/lock_protocol.{h,c}
    src/logging/log_protocol.{h,c}
    src/lxc/lxc_monitor_protocol.{h,c}
    src/remote/{lxc,qemu,remote}_protocol.{h,c}
    src/rpc/{virkeepalive,virnet}protocol.{h,c}

Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2019-11-08 17:07:57 +01:00
Pavel Hrdina
eda04022ca remote: unify rpc server dispatch generated files
Our naming was not consistent.  Use the protocol name as prefix for all
generated files.

Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2019-11-08 17:07:57 +01:00
Pavel Hrdina
8beb7fdd0e po: rewrite the way how we generate files
There was no need to handle files for translation from build directory
but that will change with following patches where we will stop
generating source files into source directory.

In order to have them included for translation we have to prefix each
file with SRCDIR or BUILDDIR.

Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2019-11-08 17:07:57 +01:00
Pavel Hrdina
f4fd068c4e syntax-check.mk: cleanup generated_files list for sc_po_check
Move generated_files variable closer to the sc_po_check rule and
remove non-existent gnulib internal path.

Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2019-11-08 17:07:50 +01:00
Pavel Hrdina
0c4eefe4ad syntax-check.mk: cleanup sc_po_check dependencies
Introduce new rule 'generated-sources' as a helper for PO files check
to make sure that all generated files are prepared and to not duplicate
the list on different places.  This will be used as a dependency for
sc_po_check rule instead of duplicated list of generated files.

Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2019-11-08 17:07:50 +01:00
Pavel Hrdina
c647107922 syntax-check.mk: fix sc_po_check rule
Commit <22d8e27ccd5faf48ee2bf288a1b9059aa7ffd28b> introduced our
syntax-check.mk file based on gnulib rules. However, the rule was
completely ignored as we don't have POTFILES.in file.

Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2019-11-08 17:07:50 +01:00
Daniel P. Berrangé
b2c2a3ae91 build-aux: rewrite po file minimizer in Python
As part of an goal to eliminate Perl from libvirt build tools,
rewrite the minimize-po.pl tool in Python.

This was a straight conversion, manually going line-by-line to
change the syntax from Perl to Python. Thus the overall structure
of the file and approach is the same.

Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-10-18 13:54:03 +01:00
Daniel P. Berrangé
17bbdef5cb build-aux: rewrite augeas test generator in Python
As part of an goal to eliminate Perl from libvirt build tools,
rewrite the augeas-gentest.pl tool in Python.

This was a straight conversion, manually going line-by-line to
change the syntax from Perl to Python. Thus the overall structure
of the file and approach is the same.

The use of $(AUG_GENTEST) as a dependancy in the makefiles needed
to be fixed, because this was assumed to be the filename of the
script, but is in fact a full shell command line.

Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-10-18 13:54:03 +01:00
Ján Tomko
67e72053c1 Use G_N_ELEMENTS instead of ARRAY_CARDINALITY
Prefer the GLib version of the macro.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2019-10-15 16:14:19 +02:00
Ján Tomko
88131931b8 Use G_GNUC_FALLTHROUGH instead of ATTRIBUTE_FALLTHROUGH
Also define the macro for building with GLib older than 2.60

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2019-10-15 16:14:19 +02:00
Ján Tomko
da367c0f9b Use G_GNUC_PRINTF instead of ATTRIBUTE_FMT_PRINTF
Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2019-10-15 16:14:18 +02:00
Ján Tomko
d54153fde3 Use G_GNUC_NO_INLINE instead of ATTRIBUTE_NOINLINE
Define the macro for older GLib versions.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2019-10-15 16:14:18 +02:00
Ján Tomko
ec96b74041 syntax-check: prohibit_attribute_macros: generalize error
Instead of enumerating every single macro variant, just emit
a more generic error.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2019-10-15 16:14:18 +02:00
Ján Tomko
81077d6e8a syntax-check: mock-noinline: fix after G_GNUC attribute invocations
We started using G_GNUC macros instead of ATTRIBUTE for some attributes.
Adjust this syntax-check accordingly.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2019-10-15 16:14:18 +02:00
Ján Tomko
a4f979c06e syntax-check: forbid ATTRIBUTE macros with a GLib replacement
Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2019-10-15 11:25:26 +02:00
Ján Tomko
6afbb7cf5c syntax-check: check for G_GNUC_UNUSED instead of ATTRIBUTE_UNUSED
Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2019-10-15 11:25:26 +02:00
Daniel P. Berrangé
44e7f02915 util: rewrite auto cleanup macros to use glib's equivalent
To facilitate porting over to glib, this rewrites the auto cleanup
macros to use glib's equivalent.

As a result it is now possible to use g_autoptr/VIR_AUTOPTR, and
g_auto/VIR_AUTOCLEAN, g_autofree/VIR_AUTOFREE interchangably, regardless
of which macros were used to declare the cleanup types.

Within the scope of any single method, code must remain consistent
using either GLib or Libvirt macros, never mixing both. New code
must preferentially use the GLib macros, and old code will be
converted incrementally.

Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-10-14 10:54:42 +01:00
Daniel P. Berrangé
22d8e27ccd build: merge all syntax-check logic into one file
The gnulib syntax-check rules are spread across GNUmakefile, cfg.mk and
maint.mk. This made sense when we were getting two of the files from the
gnulib submodule. Now that we own all files though, we can at least
merge maint.mk and cfg.mk together. GNUmakefile can be eliminated when
we switch to meson.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-10-09 13:36:54 +01:00
Daniel P. Berrangé
1b4217b995 build: delete all syntax check rules we're skipping
If we've marked rules as skipped, there's no sense keeping them in the
maint.mk file.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-10-09 13:36:48 +01:00
Daniel P. Berrangé
2931761f27 build: remove all logic unrelated to syntax-check
The standard maint.mk from gnulib provides alot more than just the
'syntax-check' target. This can all be purged to give a more minimal
file.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-10-09 13:36:46 +01:00
Daniel P. Berrangé
de744894bb build: move syntax-check code into build-aux directory
The syntax-check rules are the one bit of make usage that will
stay around for a while after the meson conversion. Move them
into the build-aux directory in preparation for refactoring
to make them independent from automake.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-10-09 13:36:43 +01:00
Daniel P. Berrangé
56bd0665c7 build: import gnulib's syntax-check make rules
We're going to be eliminating autotools and gnulib, but we still wish to
have the 'make syntax-check' functionality.

This imports the minimal set of gnulib files required to keep this
working.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-10-09 13:36:29 +01:00
Daniel P. Berrangé
697371c22e build: use @CONFIG@ instead of ::CONFIG:: in augeas tests
Using @VARNAME@ is a normal style of automake, so lets match that.

Reviewed-by: Andrea Bolognani <abologna@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-08-09 14:06:31 +01:00
Daniel P. Berrangé
06334b97f1 build: make augeas-gentest.pl write to stdout
The augeas-gentest.pl program merges a config file into a augeas
file, saving the output to a new file. It is going to be useful
to further process the output file, and it would be easier if this can
be done with a pipeline, so change augeas-gentest.pl to write to stdout
instead of a file.

Reviewed-by: Andrea Bolognani <abologna@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-08-09 14:05:06 +01:00
Jonathon Jongsma
34c9a5fc06 syntax check: update header guard check
Internal headers should use #pragma once instead of the standard #ifndef
guard. Public headers still require the existing header guard.

Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Ján Tomko <jtomko@redhat.com>
2019-06-20 09:01:07 +02:00
Cole Robinson
ac41ffb8fa build-aux: header-ifdef: Handle #pragma once
If we see it, skip all remaining header guard checks

Acked-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Cole Robinson <crobinso@redhat.com>
2019-04-04 18:42:09 -04:00