Commit Graph

36013 Commits

Author SHA1 Message Date
Han Han
3f9797e2c9 news: Support for kvm dedicated performance hint in 5.7
Signed-off-by: Han Han <hhan@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
2019-12-05 11:50:42 +01:00
Daniel P. Berrangé
01bf0bafce docs: add a kbase page about RPM packaging options
The libvirt RPM packaging is quite fine grained but it is not obvious to
users which package is best to install. Add a kbase doc that describes
the different RPMs, and illustrates some example deployment use cases.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-04 16:10:28 +00:00
Daniel P. Berrangé
7ec0b4d403 docs: convert kbase/locking-sanlock.html.in to RST
This is a semi-automated conversion. The first conversion is done using
"pandoc -f html -t rst". The result is then editted manually to apply
the desired heading markup, and fix a few things that pandoc gets wrong.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-04 16:10:28 +00:00
Daniel P. Berrangé
cea036f5c6 docs: convert kbase/locking-lockd.html.in to RST
This is a semi-automated conversion. The first conversion is done using
"pandoc -f html -t rst". The result is then editted manually to apply
the desired heading markup, and fix a few things that pandoc gets wrong.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-04 16:10:28 +00:00
Daniel P. Berrangé
944932bd43 docs: convert kbase/locking.html.in to RST
This is a semi-automated conversion. The first conversion is done using
"pandoc -f html -t rst". The result is then editted manually to apply
the desired heading markup, and fix a few things that pandoc gets wrong.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-04 16:10:28 +00:00
Daniel P. Berrangé
1a08fc2010 docs: convert kbase/secureusage.html.in to RST
This is a semi-automated conversion. The first conversion is done using
"pandoc -f html -t rst". The result is then editted manually to apply
the desired heading markup, and fix a few things that pandoc gets wrong.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-04 16:10:28 +00:00
Daniel P. Berrangé
f0bfb1b892 docs: convert kbase/launch_security_sev.html.in to RST
This is a semi-automated conversion. The first conversion is done using
"pandoc -f html -t rst". The result is then editted manually to apply
the desired heading markup, and fix a few things that pandoc gets wrong.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-04 16:10:21 +00:00
Daniel P. Berrangé
d3ad38e2cd docs: convert kbase/domainstatecapture.html.in to RST
This is a semi-automated conversion. The first conversion is done using
"pandoc -f html -t rst". The result is then editted manually to apply
the desired heading markup, and fix a few things that pandoc gets wrong.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-04 15:48:28 +00:00
Daniel P. Berrangé
fc721b0878 docs: add a minimal style guide for writing RST docs
Most importantly we document the required heading markup so that we get
consistency across the docs. Also mention that docs should have a table
of contents if they have headings & are likely longer than one page of
text.

The 3-space indent rule may sound wierd, but that's what python has
recommended and thus what tools like pandoc emit. Rather than try to
reindent things to 4-space, just accept this RST norm.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-04 15:48:28 +00:00
Daniel P. Berrangé
8b928bed08 docs: add styling for <tt> element
Although <tt> is deprecated in HTML5, the rst2html command will still
emit it, in preference to <code> tags, so we must style it too.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-04 15:48:28 +00:00
Daniel P. Berrangé
eb7388b549 docs: relax CSS context match for pretty tables
We currently only render pretty tables if they have the "top_table"
class set. All of our tables set this, except for the ACL & migration
doc tables, which should have set it, and the API reference which does
not want it.

Simplify life by rendering all tables in a pretty style and remove the
need for the "top_table" class entirely. A small rule turns off the
pretty style for the API reference where tables are a hack used to
render enums with horizontal alignment.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-04 15:48:28 +00:00
Daniel P. Berrangé
a329bd0376 docs: generate permalinks correctly for rst2html output
The rst2html output generates the links for headings in a slightly
different way than we do for docs written in HTML, so we must match
another scenario when generating back links.

rst2html will also use <h1> tags for both the document title and
the first level of section titles, so we must expand the matching
to allow for this too.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-04 15:48:28 +00:00
Daniel P. Berrangé
8215da927a docs: adapt filling of <head> section for rst2html output
The HTML from rst2html doesn't have <h1> immediately under the <body>
tag, instead there is at least one <div> in between.

There are also many things added in the <head> section that we don't
want to have copied over, since our templating system already adds
suitable <head> elements.

We only need to copy the <script> to make index.html work.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-04 15:48:28 +00:00
Daniel P. Berrangé
c7f75bf04d docs: introduce rst2html as a mandatory tool for building docs
The rst2html tool is provided by python docutils, and as the name
suggests, it converts RST documents into HTML.

Basic rules are added for integrating RST docs into the website
build process.

This enables us to start writing docs on our website in RST format
instead of HTML, without changing the rest of our website templating
system away from XSLT yet.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-04 15:48:28 +00:00
Daniel P. Berrangé
bfe9f25b49 docs: move docs about remote driver URIs into URI docs
The docs about remote URIs in uri.html are somewhat sparse with the full
docs being in remote.html. Move all the URI content from remote.html
into uri.html so the user only needs to look in one place for URI info.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-04 15:48:28 +00:00
Daniel P. Berrangé
df99aa311a docs: split TLS certificate setup into its own file
The generation and deployment of x509 certificates for TLS is complex
and verbose and thus deserves its own standalone page.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-04 15:48:28 +00:00
Michal Privoznik
e7f16ddc94 tests: Make check-file-access.pl accept files through argv
The script needs two files to open:

  $(builddir)/test_file_access.txt, and
  $(srcdir)/file_access_whitelist.txt.

However, the script is opening the files from the $CWD which
won't work for a VPATH build. Make the script accept paths to the
files through @ARGV and tune the Makefile.am to pass them.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-04 15:43:09 +01:00
Daniel P. Berrangé
504f831957 tests: rewrite test argv line wrapper in Python
As part of a goal to eliminate Perl from libvirt build tools,
rewrite the test-wrap-argv.pl tool in Python.

This was a straight conversion, manually going line-by-line to
change the syntax from Perl to Python. Thus the overall structure
of the file and approach is the same.

Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-04 13:44:15 +00:00
Daniel P. Berrangé
e0314b6855 src: rewrite remote protocol checker in Python
As part of a goal to eliminate Perl from libvirt build tools,
rewrite the pdwtags processing script in Python.

The original inline shell and perl code was completely
unintelligible. The new python code is a manual conversion
that attempts todo basically the same thing.

Tested-by: Cole Robinson <crobinso@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-04 13:44:15 +00:00
Daniel P. Berrangé
f0b3840fb6 build: only support python3 binary
python2 will be end of life by the time of the next
libvirt release. All our supported build targets, including
CentOS7, have a python3 build available.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-04 13:13:01 +00:00
Han Han
bba1ec3d37 rng: Extend the range of driveUnit
Since the max unit of virtio scsi disk is 16383, update the range of
driveUnit to it.

Signed-off-by: Han Han <hhan@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2019-12-04 13:20:57 +01:00
Han Han
0dedea96d0 rng: Separate virtio serial port and ccid slot from driveUnit
Signed-off-by: Han Han <hhan@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2019-12-04 13:20:57 +01:00
Han Han
0e37372291 conf: fix logic error for scsi units
Introduced in c8007fdc5d, it should use 'greater than max' instead of
'equal or greater than max' for the condition of checking invalid scsi
unit.

Signed-off-by: Han Han <hhan@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2019-12-04 13:20:57 +01:00
Daniel P. Berrangé
1fbd80c42a tests: stop static linking to libvirt code in tests
If we static link to libvirt_util.la then we can't override functions in
this file by simply implementing them in the test code. Any tests should
dynamic link to the main libvirt.la and ensure symbols are exported.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-04 12:08:21 +00:00
Daniel P. Berrangé
4560013b6e scripts: fix list indexing when printing mis-ordered symbols
The python array slice syntax expects the first and last indexes,
not the first length and element count.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-04 12:08:21 +00:00
Daniel P. Berrangé
d57d8d6e92 docs: move hyperv_wmi_generator.py to the scripts/ directory
Reviewed-by: Cole Robinson <crobinso@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-04 11:42:28 +00:00
Daniel P. Berrangé
74f270095a docs: move esx_vi_generator.py to the scripts/ directory
Reviewed-by: Cole Robinson <crobinso@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-04 11:42:16 +00:00
Daniel P. Berrangé
4de5d01a4e docs: move reformat-news.py to the scripts/ directory
Reviewed-by: Cole Robinson <crobinso@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-04 11:42:04 +00:00
Daniel P. Berrangé
5ffbad2575 docs: move apibuild.py to the scripts/ directory
Reviewed-by: Cole Robinson <crobinso@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-04 11:41:32 +00:00
Daniel P. Berrangé
951bc5a0b1 tests: stop linking virt tests to secondary drivers
The hard dependancy between the virt drivers and the network
or storage drivers was removed quite a while back now, so
the tests no longer need to link to these drivers.

Reviewed-by: Laine Stump <laine@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-04 09:58:33 +00:00
Cole Robinson
d0d728c7c0 spec: mingw: add deps for genprotocol.pl usage
As of 0985a9597b we no longer distribute generated files. mingw
needs to call genprotocol.pl itself, which means it needs rpcgen
and cpp installed.

Signed-off-by: Cole Robinson <crobinso@redhat.com>
2019-12-03 13:53:44 -05:00
Peter Krempa
cd1e6fd478 qemu: process: Re-process qemu capability lockout in qemuProcessPrepareQEMUCaps
We clear some capabilities here so the lockouts need to be
re-evaluated.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Cole Robinson <crobinso@redhat.com>
2019-12-03 15:26:54 +01:00
Peter Krempa
be1a201533 qemu: capabilities: Lock out incremental backup capability without blockdev
Blockdev is required to do incremental backups properly. Add a helper
function for locking out capabilities and export it to allow re-doing
the processing if a different code path modifies capabilities.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Cole Robinson <crobinso@redhat.com>
2019-12-03 15:26:54 +01:00
Peter Krempa
a64265f440 qemu: caps: Don't check capability before clearing it
Checking whether a qemu capability set right before clearing it without
any other logic doesn't make sense.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Cole Robinson <crobinso@redhat.com>
2019-12-03 15:26:54 +01:00
Peter Krempa
78c2a8b934 qemu: process: Move handling of qemu capability overrides
Do all post-processing of capabilities in qemuProcessPrepareQEMUCaps.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Cole Robinson <crobinso@redhat.com>
2019-12-03 15:26:54 +01:00
Peter Krempa
97c9ece79b qemu: process: Move clearing of QEMU_CAPS_CHARDEV_FD_PASS to qemuProcessPrepareQEMUCaps
Move the post-processing of the QEMU_CAPS_CHARDEV_FD_PASS flag to the
new function.

The clearing of the capability is based on the presence of
VIR_QEMU_PROCESS_START_STANDALONE so we must also pass in the process
start flags.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Cole Robinson <crobinso@redhat.com>
2019-12-03 15:26:54 +01:00
Peter Krempa
3a075524d9 qemu: process: Move clearing of the BLOCKDEV capability to qemuProcessPrepareQEMUCaps
Start aggregating all capability post-processing code in one place.

The comment was modified while moving it as it was mentioning floppies
which are no longer clearing the blockdev capability.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-12-03 15:26:54 +01:00
Peter Krempa
dbbc9a3c40 qemu: Move and rename qemuDomainUpdateQEMUCaps
The function is now used only in qemu_process.c so move it there and
name it 'qemuProcessPrepareQEMUCaps' which is more appropriate to what
it's doing.

The reworded comment now mentions that it will also post-process the
caps for VM startup.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-12-03 15:26:54 +01:00
Peter Krempa
530d7a73f4 qemu: process: Don't try to redetect missing qemuCaps on reconnect
The redetection was originally added in 43c01d3838 as a way to recover
from libvirtd upgrade from the time when we didn't persist the qemu
capabilities in the status XML. Also this the oldest supported qemu by
more than two years.

Even if somebody would have a running VM running at least qemu 1.5 with
such an old libvirt we certainly wouldn't do the right thing by
redetecting the capabilities and then trying to communicate with qemu.

For now it will be the best to just stop considering this scenario any
more and error out for such VM.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-12-03 15:26:54 +01:00
Peter Krempa
2504dbeb5d qemu: process: Make it obvious that virDomainDefPostParse is called with NULL opaque
Commit c90fb5a828 added explicit use of the private copy of the qemu
capabilities to various places. The change to qemuProcessInit was bogus
though as at the point where we re-initiate the post parse callbacks
priv->qemuCaps is still NULL as we clear it after shutdown of the VM and
don't initiate it until a later point.

Using the value from priv->qemuCaps might mislead readers of the code
into thinking that something useful is being passed at that point so go
with an explicit NULL instead.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-12-03 15:26:54 +01:00
Peter Krempa
ab163144eb qemu: Always reset @info in qemuDomainGetJobInfo
qemuDomainGetJobInfo didn't always reset the return data in @info.
Thankfully this wouldn't be a problem as the RPC layer does it but we
should do it anyways.

Since we reset the struct we don't have to set the type to
VIR_DOMAIN_JOB_NONE as the value is 0.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-12-03 15:26:54 +01:00
Peter Krempa
bbd54c10e7 API: Introduce VIR_DOMAIN_JOB_SUCCESS field for virDomainGetJobStats
The statistics fields are used in two places:

1) virDomainGetJobStats where the job type which ultimately holds
whether the job was successful or not is returned via a different
argument.

2) The virConnectDomainEventJobCompleted event where we report just the
statistics via typed parameters.

Since it might be useful to report the event also for jobs which
completed unsuccessfully and we don't have the means to transport the
state via a different variable with the event let's add a new field
which will hold the success state.

Since this is meant primarily for completed jobs a plain boolean is
sufficient to convey whether the job was successful or not.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-12-03 15:26:53 +01:00
Peter Krempa
b1bf5efdc1 virsh: domjobinfo: Add switch to print raw fields
Introduce --rawstats which prints all statistics fields from the new API
similarly to how the virsh event handler prints them.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-12-03 15:26:53 +01:00
Peter Krempa
7f3534a5bb virsh: domjobinfo: Allow printing stats also for failed and other jobs
Introduce the --anystats flag which does not skip the printing of the
stats if the job was unsuccessful.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-12-03 15:26:53 +01:00
Peter Krempa
2a55bcd8da virsh: domjobinfo: Print also job operation for failed jobs
Printing that a job failed is rather unhelpful. Print at least the
operation which failed.

Achieve this by moving the check whether to print stats later but
replace it with a check which will skip printing of the operation if
there's no job.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-12-03 15:26:53 +01:00
Peter Krempa
2dae916de4 qemu: Implement VIR_DOMAIN_JOB_STATS_KEEP_COMPLETED
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-12-03 15:26:53 +01:00
Peter Krempa
7884b40659 virsh: Implement VIR_DOMAIN_JOB_STATS_KEEP_COMPLETED for 'domjobinfo'
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-12-03 15:26:53 +01:00
Peter Krempa
d4a4933a0d api: Allow keeping completed domain job stats when reading them
virDomainGetJobStats destroys the completed statistics on the first
read. Give the user possibility to keep them around if they wish so.

Add a flag VIR_DOMAIN_JOB_STATS_KEEP_COMPLETED which will read the stats
without destroying them.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-12-03 15:26:53 +01:00
Peter Krempa
54a9d6e51e virsh: domain: Extract the code converting domain job stats to virDomainJobInfo
To simplify the stats printer code we convert the new statistics from
the typed parameter list into the old stats structure.

Extract this code since it takes a lot of space.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-12-03 15:26:53 +01:00
Daniel P. Berrangé
0279a51b83 tests: fix name of 32-bit x86 QEMU binary
The 32-bit x86 binary is called qemu-system-i386, not
qemu-system-i686. This mistake across many test XML files was
not noticed because the mistake was also made in testutilsqemu.c
when mocking the capabilities.

Reviewed-by: Erik Skultety <eskultet@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-03 13:57:44 +00:00