Commit Graph

195 Commits

Author SHA1 Message Date
Daniel P. Berrangé
89f8902a68 tests: add targets for building libvirt inside Docker containers
The Travis CI system uses Docker containers for its build environment.
These are pre-built and hosted under quay.io/libvirt so that developers
can use them for reproducing problems locally.

Getting the right Docker command syntax to use them, however, is not
entirely easy. This patch addresses that usability issue by introducing
some make targets. To run a simple build (aka 'make all') using the
Fedora 28 container:

   make ci-build@fedora-28

To also run unit tests

   make ci-check@fedora-28

This is just syntax sugar for calling the previous command with a
custom make target

   make ci-build@fedora-28 CI_MAKE_ARGS="check"

To do a purely interactive build it is possible to request a shell

   make ci-shell@fedora-28

To do a MinGW build, it is currently possible to use the fedora-rawhide
image and request a different configure script

   make ci-build@fedora-rawhide CI_CONFIGURE=mingw32-configure

It is also possible to do cross compiled builds via the Debian containers

   make ci-build@debian-9-cross-s390x

In all cases the GIT source tree is cloned locally into a 'ci-tree/src'
sub-directory which is then exposed to the container at '/src'. It is
setup to use a separate build directory so the build takes place in a
subdir '/src/build'. A source tree build can be requested instead
by passing an empty string CI_VPATH= arg to make.

The make rules are kept in a standalone file that is included into the
main Makefile.am, so that it is possible to run them without having to
invoke autotools first.

It is neccessary to disable the gnulib submodule commit check because
this fails due to the way we have manually cloned submodule repos as
primary git repos with their own .git directory, instead of letting
git treat them as submodules in the top level .git directory.

  make[1]: Entering directory '/src/build'
  fatal: Not a valid object name origin
  fatal: run_command returned non-zero status for .gnulib
  .
  maint.mk: found non-public submodule commit
  make: *** [/src/maint.mk:1448: public-submodule-commit] Error 1

Reviewed-by: Andrea Bolognani <abologna@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-04-11 18:38:56 +01:00
Cole Robinson
fb0d6049cc docs: Remove search.php and all references
libvirt.org/search.php drops into some kind of screen which I guess
is supposed to show a search bar with options, but presently for me
renders as nothing but the following text:

Search the documentation on Libvirt.org

The search service indexes the libvirt APIs and documentation as well as the libvir-list@redhat.com mailing-list archives. To use it simply provide a set of keywords:

The main page search bar now redirects to google, this page is broken,
I say we just remove it and move on.

Reviewed-by: Daniel P. Berrange <berrange@redhat.com>
Signed-off-by: Cole Robinson <crobinso@redhat.com>
2019-04-04 18:45:06 -04:00
Andrea Bolognani
ce97c33a79 maint: Stop generating ChangeLog from git
Our ChangeLog is generated by basically redirecting the output
of 'git log' into it so, as can be expected, it has only gotten
bigger as development has progressed. As of today, its size has
reached pretty much comical levels:

  $ du -sk ChangeLog
  11328 ChangeLog

All of that for information *literally nobody* cares about: end
users and distro maintainers have proper release notes lovingly
compiled for them, while developers peruse the history either by
calling 'git log' directly or through their favorite $EDITOR's
git integration.

Replacing the generated ChangeLog with a short message pointing
interested parties to the git repository does not only reduce
the size of the unpacked sources from 259904 KiB to 248576 KiB
(~4% saving): from a quick test on my laptop, doing so reduces
the size of the *compressed* release archive from 15140 KiB to
12364 KiB (~18% saving) and also takes the time needed to run
'make distcheck' down from 4:44 to 4:21 (~8% saving).

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-04-03 09:45:25 +02:00
Andrea Bolognani
912fe2df9d Drop support for "Red Hat" init scripts
Despite the misleading name, these were supposed to be used
with a System V style init; however, none of the platforms we
target is using that kind of init anymore: almost all Linux
distributions have switched to systemd, those that haven't
(such as Gentoo and Alpine) are mostly using OpenRC with
custom init scripts, and the BSDs have been doing their own
thing all along.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2019-03-15 18:36:19 +01:00
Michal Privoznik
a87a75e579 tools: Keep wireshark plugin registration code in git
In order to be able to dissect libvirt protocol the wireshark
plugin needs to be registered. So far this plugin registration
code was generated on every build using a script that was copied
over from wireshark's tools/ directory.

This is suboptimal, because the way that plugins register changes
across wireshark releases. Therefore, let's keep the generated
file in the git, put the command line used to generate the file
into a comment and remove the script.

This solution allows us to put different registration mechanism
into one file (under #ifdef-s) and thus compile with wider range
of wireshark releases.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-02-12 09:22:59 +01:00
Daniel P. Berrangé
a3857dbeeb po: minimize & canonicalize translations stored in git
Similar to the libvirt.pot, .po files contain line numbers and file
names identifying where in the source a translatable string comes from.
The source locations in the .po files are thrown away and replaced with
content from the libvirt.pot whenever msgmerge is run, so this is not
precious information that needs to be stored in git.

When msgmerge processes a .po file, it will add in any msgids from the
libvirt.pot that were not already present. Thus, if a particular msgid
currently has no translation, it can be considered redundant and again
does not need storing in git.

When msgmerge processes a .po file and can't find an exact existing
translation match, it will try todo fuzzy matching instead, marking such
entries with a "# fuzzy" comment to alert the translator to take a
look and either discard, edit or accept the match. Looking at the
existing fuzzy matches in .po files shows that the quality is awful,
with many having a completely different set of printf format specifiers
between the msgid and fuzzy msgstr entry. Fortunately when msgfmt
generates the .gmo, the fuzzy entries are all ignored anyway. The fuzzy
entries could be useful to translators if they were working on the .po
files directly from git, but Libvirt outsourced translation to the
Fedora Zanata system, so keeping fuzzy matches in git is not much help.

Finally, by default msgids are sorted based on source location. Thus, if
a bit of code with translatable text is moved from one file to another,
it may shift around in the .po file, despite the msgid not itself changing.
If the msgids were sorted alphabetically, the .po files would have
stable ordering when code is refactored.

This patch takes advantage of the above observations to canonicalize
and minimize the content stored for .po files in git. Instead of storing
the real .po files, we now store .mini.po files.

The .mini.po files are the same file format as .po files, but have no
source location comments, are sorted alphabetically, and all fuzzy
msgstrs and msgids with no translation are discarded. This cuts the size
of content in the po directory from 109MB to 19MB.

Users working from a libvirt git checkout who need the full .po files
can run "make update-po", which merges the libvirt.pot and .mini.po
file to create a .po file containing all the content previously stored
in git.

Conversely if a full .po file has been modified, for example, by
downloading new content from Zanata, the .mini.po files can be updated
by running "make update-mini-po". The resulting diffs of the .mini.po
file will clearly show the changed translations without any of the noise
that previously obscured content. Being able to see content changes
clearly actually identified a bug in the zanata python client where it
was adding bogus "fuzzy" annotations to many messages:

  https://bugzilla.redhat.com/show_bug.cgi?id=1564497

Users working from libvirt releases should not see any difference in
behaviour, since the tarballs only contain the full .po files, not the
.mini.po files.

As an added benefit, generating tarballs with "make dist", will no
longer cause creation of dirty files in git, since it won't touch the
.mini.po files, only the .po files which are no longer kept in git.

To avoid creating a single commit 100+MB in size, each language is
minimized separately in a following commit.

Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2018-04-19 11:11:08 +01:00
Daniel P. Berrangé
c0a8ea450d po: provide custom make rules for po file management
Historically we have relied on autopoint/gettextize to install a
standard po/Makefile.in.in. There is very limited scope for customizing
this and it also causes a bunch of extra stuff to be pulled into
configure.ac which potentially clashes with gnulib. Writing make rules
for po file management is no more difficult than any other rules libvirt
has, so stop using autopoint/gettextize.

Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2018-04-19 10:35:58 +01:00
Ján Tomko
ab20f5059c Do not generate polkit rules file
This essentially reverts commit <e1019e9>, which added
an extra step for generating the policy file.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
2018-03-14 12:46:26 +01:00
Daniel P. Berrangé
3f055b5997 daemon: move logrotate files to src/remote/
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2018-02-22 12:26:57 +00:00
Daniel P. Berrangé
21ea3828f7 daemon: move misc libvirtd policy files to src/remote/
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2018-02-22 12:26:50 +00:00
Daniel P. Berrangé
168bcca95d daemon: move configuration files to src/remote
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2018-02-22 12:26:39 +00:00
Daniel P. Berrangé
0f6260ada3 daemon: move init system files into src/remote/
Move the systemd, sysvinit, upstart and sysconfig files into the
src/remote/ directory.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2018-02-22 12:21:21 +00:00
Daniel P. Berrangé
91fd3dc932 daemon: move libvirtd code into src/remote/ directory
Having a daemon/ directory makes little sense from a code structure
point of view, as 90% of the code that is built into libvirtd already
lives in the src/ directory. The virtlockd and virlogd daemons also live
entirely in src/{locking,logging} directories. This moves the source
code for libvirtd into src/remote/, alongside the client code.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2018-02-22 12:21:08 +00:00
Daniel P. Berrange
b1e03dad86 admin: move admins server impl/dispatch into src/admin directory
The admin server functionality is a generic concept that should be wired
up into all libvirt daemons, but is currently integrated with the
libvirtd code. Move it all into the src/admin directory to prepare for
broader reuse.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
2018-01-31 15:12:30 +00:00
Andrea Bolognani
64b474a821 Add YouCompleteMe support
YouCompleteMe[1] is a vim plugin that implements semantic
code completion using libclang.

For non-trivial projects such as libvirt, the plugin needs
some help figuring out where to find the various header
files: generate its configuration file at configure time
so that the plugin works out of the box.

[1] http://valloric.github.io/YouCompleteMe/
2017-05-09 09:51:11 +02:00
Andrea Bolognani
a50dee9f68 Add color_coded support
color_coded[1] is a vim plugin that implements semantic
syntax highlighting using libclang.

For non-trivial projects such as libvirt, the plugin needs
some help figuring out where to find the various header
files: generate its configuration file at configure time
so that the plugin works out of the box.

[1] https://github.com/jeaye/color_coded
2017-05-09 09:51:11 +02:00
Daniel P. Berrange
e6c3b59c19 Add ability to generate man page describing key code names & values
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
2017-04-25 21:20:43 +01:00
Daniel P. Berrange
02fb15fb60 util: switch over to use keycodemapdb GIT submodule
A long time ago we imported the keymaps.csv file from GTK-VNC so we
can do conversions between keycode sets. Meanwhile lots of bug fixes
have gone into this CSV file and libvirt hasn't kept in sync. The
keymaps.csv file and associated generator script has been pulled out
of GTK-VNC into a dedicated GIT repo for use as a submodule. This
allows GTK-VNC, SPICE-GTK, QEMU and libvirt to share the same master
database and tools and pushing updates merely requires a submodule
commit update as with gnulib.

The test suite is updated to cover some extra boundary conditions.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
2017-04-25 21:14:18 +01:00
Jiri Denemark
f0ad8e7ee0 Properly ignore files in build-aux directory
We want to ignore all files except *.pl in build-aux directory, however
the unignore pattern "!/build-aux/*.pl" doesn't have any effect because
a previous "/build-aux/" pattern ignores the directory itself rather
than individual files in it.

https://bugzilla.redhat.com/show_bug.cgi?id=1439994

Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
2017-04-07 10:08:33 +02:00
Andrea Bolognani
be36ea4b52 NEWS: Improve building pipeline
Currently, building the NEWS file involves using a XSLT stylesheet
to extract information from the same HTML file that's used on the
libvirt website.

The process works, but it's quite fiddly in that it requires the
source HTML to be formatted in a very precise way, and a single
missing newline can mess up the resulting plain text considerably.

Moreover, the XSLT stylesheet itself encodes a lot of the details
of converting to plain text in a way that's not necessarily easy
to understand, tweak or fix.

To improve the process, move all existing entries to a new XML
file that contains exactly the information we care about in a
simple structured format, and start generating both the HTML and
plain text versions of the release notes using XSLT stylesheets
that can now afford to be almost trivial.
2017-01-10 19:37:53 +01:00
Erik Skultety
0c169930c0 admin: Add an example demonstrating how to use the logging APIs
Provide a simple C example demonstrating the use of both query APIs as
well as setter APIs.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
2016-12-15 10:36:23 +01:00
Michal Privoznik
4a457adda6 tests: Self test virt-admin
Just like we are running 'virsh self-test' from within our test
suite, we should run 'virt-admin self-test' too.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2016-09-14 13:18:07 +02:00
Erik Skultety
fbb8205de3 examples: admin: Add some examples for the new admin APIs
Some of the examples make use of asprintf and strtol functions (to keep
things simple) which are prohibited to use within our code (enforced by
syntax-check). Therefore besides adding some examples, this patch also updates
cfg.mk to exclude examples directory from asprintf and strtol rules, as well as
updates .gitignore to exclude all the new admin binaries created in the
'examples' dir.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
2016-06-26 00:21:06 +02:00
Ján Tomko
21b56a6055 Do not ignore perl scripts in build-aux
Also remove the duplicate build-aux entry from .gitignore.
2016-06-23 09:38:13 +02:00
Ján Tomko
920ab8bdce Introduce virsh self-test
A new hidden command for virsh that will iterate over
all command groups and commands and print help for every single one.

This involves running vshCmddefOptParse so we can get an error if
one of the command's option structure is invalid.
2016-06-20 18:16:33 +02:00
Andrea Bolognani
2c51fa6ec4 maint: Switch to xz compressed PAX release archives
This allows us to produce releases that are roughly a third in
size, have no limitation on path length, and are still readable
by all supported platforms.
2016-06-15 18:53:34 +02:00
Ján Tomko
cf0974fbca Introduce virschematest
Instead of calling xmllint via a shell script, use our virXMLValidator
API to do it directly via libxml.
2016-06-08 10:03:29 +02:00
Michal Privoznik
6326865e6b virtestmock: Print invalid file accesses into a file
All the accesses to files outside our build or source directories
are now identified and appended into a file for later processing.
The location of the file that contains all the records can be
controlled via VIR_TEST_FILE_ACCESS env variable and defaults to
abs_builddir "/test_file_access.txt".

The script that will process the access file is to be added in
next commit.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2016-05-14 09:46:23 +02:00
Jiri Denemark
8d8f65f0a8 Add qemucapsprobe in .gitignore
Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
2016-05-05 10:09:59 +02:00
Andrea Bolognani
a77720f8b8 maint: Ignore all .init, .service and .socket files
All files with those extensions are generated, so we can have
three generic rules instead of explicitly ignoring every single
instance.
2016-05-03 15:58:09 +02:00
Andrea Bolognani
0f4bd97f9c maint: Don't ignore libvirtd.pod any longer
Starting from commit 5b479f3b5b, libvirtd.pod is a source file
rather than a generated file. Update ignore patterns accordingly.
2016-05-03 15:10:27 +02:00
Cole Robinson
4b98948366 gitignore: ignore virt(log|lock)d.(service|socket) 2016-05-03 08:58:30 -04:00
Andrea Bolognani
10ef8c66ee maint: Ignore all intermediate and generated man pages
Since commit 9b77ce63f1 we create a .in file while building all
man pages, including those in the tools/ directory; update the
ignore patterns to take this change into account.

The new ignore patterns are generic enough that we can get rid of
a few existing ones as well.
2016-04-26 09:20:12 +02:00
Cole Robinson
3b9100a4d2 Revert "daemon: use socket activation with systemd"
This reverts commit 1e9808d3a1.

We shouldn't advertise libvirtd.socket activation, since currently
it means VM/network/... autostart won't work as expected.

We tried to find a middle ground by installing the config file without
an [Install] section, since systemd won't allow .socket to be enabled
without one... or at least it did do that; presently on f24 it allows
activating the socket quite happily. This also caused user confusion[1]

Just remove the socket file. I've filed a new RFE to track coming up
with a solution to the autostart problem[2], we can point users at that
if there's more confusion:

[1]: https://bugzilla.redhat.com/show_bug.cgi?id=1279348
[2]: https://bugzilla.redhat.com/show_bug.cgi?id=1326136
2016-04-20 08:53:38 -04:00
Erik Skultety
6409578790 virt-admin: Introduce first working skeleton
This patch introduces virt-admin client which is based on virsh client,
but had to reimplement several methods to meet virt-admin specific needs
or remove unnecessary virsh specific logic.
2015-11-30 09:44:28 +01:00
Erik Skultety
a20b623748 libvirt: introduce libvirt/libvirt-common.h.in
As it turned out, we need to share some enums and declarations between
libvirt.h and libvirt-admin.h, but since our policy forbids direct includes of
libvirt*.h, there has to be some header exempt from this rule. This patch moves
the relevant part of code from libvirt.h.in to libvirt-common.h.in. Moreover,
since there is no need to have libvirt.h generated anymore, introduce a new
header libvirt.h which was previosly ignored from git and make the common
header ignored and generated instead.
2015-11-30 09:36:19 +01:00
Daniel P. Berrange
323a329b26 Import stripped down virtlockd code as basis of virtlogd
Copy the virtlockd codebase across to form the initial virlogd
code. Simple search & replace of s/lock/log/ and gut the remote
protocol & dispatcher. This gives us a daemon that starts up
and listens for connections, but does nothing with them.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
2015-11-26 14:28:55 +00:00
Michal Privoznik
ea65fdba7f .gitignore: Ignore the correct rename example binary
In e755186c5c we tried to introduce an example demonstrating
new virDomainRename API. Unfortunately, in the .gitignore we had
a different binary listed. It's 'rename' binary which we want git
to ignore, not 'test'.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2015-09-07 10:40:55 +02:00
Martin Kletzander
e755186c5c Add example that renames domain there and back
And in the middle it prints out its name to demonstrate changes in later
patch(es).

Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
2015-09-04 18:55:01 +02:00
Martin Kletzander
14d84db863 Add generated libvirt_admin.syms into .gitignore
Commit a2c5d16a70 switched to generating
libvirt_admin.syms, but forgot to add the generated file into
.gitignore, hence causing tree pollution post-build.

Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
2015-08-20 11:17:27 -07:00
Martin Kletzander
220393bfb0 Revert "Example virt-admin"
This reverts commit 4e7ccf8713.

I mistakenly pushed it along with the Admin API series.
2015-06-16 14:08:23 +02:00
Martin Kletzander
4e7ccf8713 Example virt-admin
You had only one job.  That's what you can say about this example
binary.  In future, parts of virsh that are usable for this binary
should be split into separate shell-utils and virt-admin should gain all
the cool features of virsh without too much code addition.

Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
2015-06-16 13:46:21 +02:00
Martin Kletzander
878bf2a3c9 Add XML files with admin API specification
No online docs are build from it since it doesn't really fit into our
document structure and new page will need to be created for it, but this
is at least a heads-up commit for easier parsing in order to build some
documentation (or python bindings) later on.

Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
2015-06-16 13:46:20 +02:00
Martin Kletzander
fb5a0d3388 Build client headers for admin protocol
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
2015-06-16 13:46:20 +02:00
Martin Kletzander
8c8bed02af Add admin protocol
For now there are only CONNECT_OPEN and CONNECT_CLOSE procedures.

Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
2015-06-16 13:46:20 +02:00
Andrea Bolognani
10525df616 maint: Ignore Vim swap files
This removes some noise when you're working on the repository
and also have a bunch of source files open in Vim in another
terminal.
2015-05-13 08:39:14 -06:00
Pavel Hrdina
bb3cc43cd4 main: add new generated files to .gitignore
This means new libxl-lockd.conf and libxl-sanlock.conf

Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
2015-04-30 18:39:34 +02:00
Jim Fehlig
f7a30375bd Revert "src/xenconfig: Xen-xl parser"
This reverts commit 2c78051a14.

Conflicts:
	src/Makefile.am

Signed-off-by: Jim Fehlig <jfehlig@suse.com>
2015-01-12 10:15:13 -07:00
Kiarie Kahurani
2c78051a14 src/xenconfig: Xen-xl parser
Introduce a Xen xl parser

This parser allows for users to convert the new xl disk format and
spice graphics config to libvirt xml format and vice versa. Regarding
the spice graphics config, the code is pretty much straight forward.
For the disk {formating, parsing}, this parser takes care of the new
xl format which include positional parameters and key/value parameters.
In xl format disk config a <diskspec> consists of parameters separated by
commas. If the parameters do not contain an '=' they are automatically
assigned to certain options following the order below

   target, format, vdev, access

The above are the only mandatory parameters in the <diskspec> but there
are many more disk config options. These options can be specified as
key=value pairs. This takes care of the rest of the options such as

  devtype, backend, backendtype, script, direct-io-safe,

The positional paramters can also be specified in key/value form
for example

    /dev/vg/guest-volume,,hda
    /dev/vg/guest-volume,raw,hda,rw
    format=raw, vdev=hda, access=rw, target=/dev/vg/guest-volume

are interpleted to one config.

In xm format, the above diskspec would be written as

phy:/dev/vg/guest-volume,hda,w

The disk parser is based on the same parser used successfully by
the Xen project for several years now.  Ian Jackson authored the
scanner, which is used by this commit with mimimal changes.  Only
the PREFIX option is changed, to produce function and file names
more consistent with libvirt's convention.

Signed-off-by: Kiarie Kahurani <davidkiarie4@gmail.com>
Signed-off-by: Jim Fehlig <jfehlig@suse.com>
2015-01-03 22:41:07 -07:00
Martin Kletzander
a17bfd6304 gitignore: use wildcard for configure's temporary files
Pushed as trivial and pre-ACK'd here:

https://www.redhat.com/archives/libvir-list/2014-November/msg00180.html

Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
2014-11-06 16:56:21 +01:00