Commit Graph

28804 Commits

Author SHA1 Message Date
Daniel P. Berrangé
7495b1a779 rpm: fix incorrect expansion of macros with line continuations for args
Macros in RPMs are expanded before line continuations, so when we write

   %systemd_preun foo \
                  bar

What happens is that it expands to

   if [ $1 -eq 0 ] ; then
        # Package removal, not upgrade
        systemctl --no-reload disable --now foo \ > /dev/null 2>&1 || :
   fi
                 bar

which is obviously complete garbage and not what we expected. It is
simply not safe to ever use line continuations in combination with
macros.

Reviewed-by: Laine Stump <laine@laine.org>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2018-03-21 14:46:08 +00:00
Han Han
ec34c51e9d libvirt: Fix comments for two public APIs.
Fix comments for virConnectListAllNodeDevices and
virConnectListAllSecrets.

Signed-off-by: Han Han <hhan@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
2018-03-21 08:09:33 -04:00
Andrea Bolognani
a8223d8f4a travis: Test both python2 and python3
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2018-03-20 16:38:48 +01:00
Michal Privoznik
630c6e3495 virDomainDeviceDefValidateAliasesIterator: Ignore some hostdevs
https://bugzilla.redhat.com/show_bug.cgi?id=1556828

When defining a domain that has <interface type='hostdev'/> our
parser creates two entries in virDomainDef: one for <interface/>
and one for <hostdev/>. However, some info is shared between the
two which makes user alias validation fail because alias belongs
to the set of shared info.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Jiri Denemark <jdenemar@redhat.com>
2018-03-20 15:30:14 +01:00
Cole Robinson
a5d85adb78 hyperv: Sync generator output between python versions
Yet another dependency on dict() hash ordering

Signed-off-by: Cole Robinson <crobinso@redhat.com>
2018-03-20 09:44:35 -04:00
Andrea Bolognani
eb2392fff4 news: Update for Python 3 compatibility
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2018-03-20 14:03:22 +01:00
Andrea Bolognani
86416a7491 mingw: Use Python 3 for building
While RHEL / CentOS are still using Python 2 for the time being,
Fedora has already switched to Python 3 as the default Python
interpreter a while ago, so on that OS it doesn't make sense to
drag in Python 2 anymore.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2018-03-20 14:03:22 +01:00
Andrea Bolognani
7093e8c7a5 spec: Use Python 3 for building when possible
While RHEL / CentOS are still using Python 2 for the time being,
Fedora has already switched to Python 3 as the default Python
interpreter a while ago, so on that OS it doesn't make sense to
drag in Python 2 anymore; the same applies to future RHEL versions.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2018-03-20 14:03:00 +01:00
Andrea Bolognani
c229e01a1f configure: Allow (and prefer) Python 3
Our build process no longer depends on Python 2, so we can
finally allow Python 3 to satisfy our requirement for a Python
interpreter.

Since several distributions have now switched to installing
Python 3 by default and Python 2 is on its way out, prefer the
former when both are available.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2018-03-20 13:42:46 +01:00
Andrea Bolognani
6c3b44ea6f mingw: Require Fedora
Although it was never formally specified, it was always expected
that the mingw RPM build would happen on Fedora, if anything
because RHEL / CentOS don't ship the necessary mingw dependencies.

Make this fact explicit by erroring out if that's not the case,
the same way we already do in the main spec file.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2018-03-20 13:42:46 +01:00
Radostin Stoyanov
c774562242 apibuild: Simplify getline()
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Radostin Stoyanov <rstoyanov1@gmail.com>
2018-03-20 12:13:36 +00:00
Radostin Stoyanov
437f1eadd4 apibuild: Remove redundant parentheses
http://pylint-messages.wikidot.com/messages:c0325

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Radostin Stoyanov <rstoyanov1@gmail.com>
2018-03-20 12:13:36 +00:00
Radostin Stoyanov
eb752c4675 apibuild: Simplify parsing string tokens
Improve readability and reduce the complexity of the code that is
searching for string tokens (i.e. characters surrounded by a single
or double quote).

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Radostin Stoyanov <rstoyanov1@gmail.com>
2018-03-20 12:13:36 +00:00
Radostin Stoyanov
c8d3bbaad0 apibuild: Simplify merging of preproc tokens
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Radostin Stoyanov <rstoyanov1@gmail.com>
2018-03-20 12:13:36 +00:00
Radostin Stoyanov
c3e611927c apibuild: Use list comprehension insteand of map
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Radostin Stoyanov <rstoyanov1@gmail.com>
2018-03-20 12:13:36 +00:00
Radostin Stoyanov
d2e226df5c apibuild: Simplify type checking of literals
Reduce the number of if-statements used to assign a literals
to corresponding class variables.

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Radostin Stoyanov <rstoyanov1@gmail.com>
2018-03-20 12:13:36 +00:00
Radostin Stoyanov
06462d7927 apibuild: Simplify parseTypeComment()
Improve readability and reduce complexity the method
parseTypeComment().

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Radostin Stoyanov <rstoyanov1@gmail.com>
2018-03-20 12:13:36 +00:00
Radostin Stoyanov
c57b56756a apibuild: Simplify strip_lead_star()
The method strip_lead_star() removes a single leading asterisk
character from a string by ignoring leading whitespace, otherwise it
returns the original string.

This could be achieved with a single if-statement followed by replace.

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Radostin Stoyanov <rstoyanov1@gmail.com>
2018-03-20 12:13:36 +00:00
Radostin Stoyanov
eb327e6c61 apibuild: Fix indentation not multiple of 4
PEP8 recommends that the number of spaces used for indentation of
Python code to be a multiple of four [1] [2].

1: https://lintlyci.github.io/Flake8Rules/rules/E111.html
2: https://lintlyci.github.io/Flake8Rules/rules/E114.html

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Radostin Stoyanov <rstoyanov1@gmail.com>
2018-03-20 12:13:36 +00:00
Radostin Stoyanov
aad30c3e57 apibuild: Drop backslash between brackets
Backslash between brackets in Python is redundant. [1]

1: https://lintlyci.github.io/Flake8Rules/rules/E502.html

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Radostin Stoyanov <rstoyanov1@gmail.com>
2018-03-20 12:13:36 +00:00
Radostin Stoyanov
d65f487a91 python3: cpu-reformat: Use the print() function
Replace the print statement, that is only available in Py2, with a
print function that is available in both Py2 and Py3 and drop the
explicit python version in the shebang.

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Radostin Stoyanov <rstoyanov1@gmail.com>
2018-03-20 12:13:36 +00:00
Radostin Stoyanov
ddfc6db2f5 apibuild: Avoid double sorting of ids
The uniq() function returns a sorted list, there is no need
to sort this list again.

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Radostin Stoyanov <rstoyanov1@gmail.com>
2018-03-20 12:13:36 +00:00
Radostin Stoyanov
e169caa462 apibuild: Simplify uniq function
Use a set (unordered collections of unique elements) [1] to remove
repeated elements in a list.

1: https://docs.python.org/3/tutorial/datastructures.html#sets

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Radostin Stoyanov <rstoyanov1@gmail.com>
2018-03-20 12:13:35 +00:00
Radostin Stoyanov
94f417baa9 WmiClass: Don't share "versions" between instances
Lists in Python are mutable and when used as a default value of a
parameter for class constructor, its value will be shared between
all class instances.

Example:

	class Test:
	    def __init__(self, mylist=[]):
	        self.mylist = mylist

	A = Test()
	B = Test()
	A.mylist.append("mylist from instance A")
	print(B.mylist) # Will print ['mylist from instance A']

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Radostin Stoyanov <rstoyanov1@gmail.com>
2018-03-20 12:13:35 +00:00
Radostin Stoyanov
5e870cbda4 python: Remove space around = in keyword args
PEP8 recommends not having spaces around = in a keyword argument or
a default parameter value.

https://www.python.org/dev/peps/pep-0008/#other-recommendations

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Radostin Stoyanov <rstoyanov1@gmail.com>
2018-03-20 12:13:35 +00:00
Radostin Stoyanov
e390bb1571 apibuild: Simplify conditional statements
Improve readability by reducing the complexity and length of
conditional statements.

Example: The following condition:

	if (o >= 97 and o <= 122) or
	   (o >= 65 and o <= 90) or
	   (o >= 48 and o <= 57) or
	   (" \t(){}:;,+-*/%&!|[]=><".find(line[i]) == -1):

Will be True for every character that is not in string:
	" \t(){}:;,+-*/%&!|[]=><"

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Radostin Stoyanov <rstoyanov1@gmail.com>
2018-03-20 12:13:35 +00:00
Radostin Stoyanov
e9476e1035 cfg.mk: check ctype_macros only on *.[c|h] files
The functions like isalnum(), isalpha(), isdigit(), etc. are also
available in Python, however `make syntax-check` do not intend to
prohibit them.

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Radostin Stoyanov <rstoyanov1@gmail.com>
2018-03-20 12:13:35 +00:00
Radostin Stoyanov
f3a99869ff esx_vi_generator: Simplify alignment function
Generate whitespace using the standard function ljust() that is
available in both Py3 [1] and Py2 [2].

1: https://docs.python.org/3/library/stdtypes.html?highlight=strip#str.ljust
2: https://docs.python.org/2.7/library/string.html#string.ljust

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Radostin Stoyanov <rstoyanov1@gmail.com>
2018-03-20 12:13:35 +00:00
Radostin Stoyanov
b89111afed esx_vi_generator: Simplify get_occurrence_comment
Reduce the number of if-statements and use a single return.
Utilise a dictionary to map between occurrences and values.

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Radostin Stoyanov <rstoyanov1@gmail.com>
2018-03-20 12:13:35 +00:00
Radostin Stoyanov
25409b2dd3 esx_vi_generator: Simplify generate_helper_header
The function generate_helper_header() only returns a formatted string.
This could be achieved without performing string concatenation.

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Radostin Stoyanov <rstoyanov1@gmail.com>
2018-03-20 12:13:35 +00:00
Radostin Stoyanov
68106955da esx_vi_generator: Simplify generate_helper_source
The generate_helper_source() function returns a formatted string.
This could be achieved without the use of a local variable "source"
and string concatenation.

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Radostin Stoyanov <rstoyanov1@gmail.com>
2018-03-20 12:13:35 +00:00
Radostin Stoyanov
293bb82b58 python: Add whitespace around = and % operators
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Radostin Stoyanov <rstoyanov1@gmail.com>
2018-03-20 12:13:35 +00:00
Radostin Stoyanov
28593b5b54 apibuild: Remove whitespace before ', ' and ':'
PEP8 recommends removing whitespace immediately before a comma,
semicolon, or colon [1]. In addition remove multiple spaces after
keyword (PEP8 - E271).

1: https://www.python.org/dev/peps/pep-0008/#whitespace-in-expressions-and-statements

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Radostin Stoyanov <rstoyanov1@gmail.com>
2018-03-20 12:13:35 +00:00
Radostin Stoyanov
6d43a754af apibuild: Split imports on separate lines
PEP8 recommends imports to be on separate lines. [1]

1: https://www.python.org/dev/peps/pep-0008/#imports

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Radostin Stoyanov <rstoyanov1@gmail.com>
2018-03-20 12:13:35 +00:00
Radostin Stoyanov
80559e4879 apibuild: Use isinstance for type checking
The isinstance() function [1] returns true if an object argument is an
instance of a classinfo argument or of a direct, indirect subclass
thereof.

1: https://docs.python.org/3/library/functions.html#isinstance

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Radostin Stoyanov <rstoyanov1@gmail.com>
2018-03-20 12:13:35 +00:00
Jiri Denemark
1554eafbe6 qemu: Don't assign alias to disabled balloon device
<memballoon model='none'/> is the only way to disable balloon driver
since libvirt will add one automatically if the memballoon element is
missing. In other words, there's no balloon device if model is 'none'
and generating an alias for it makes no sense. The alias will be ignored
when parsing the XML and it will disappear once libvirtd is restarted.

Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
2018-03-19 21:22:43 +01:00
Daniel P. Berrangé
8ef5db6581 gnulib: switch to use https:// instead of git:// protocol
Some contributors are behind obnoxious firewalls that block everything
except http(s) traffic, preventing checkout of modules using the git://
protocol. Since git.savannah.gnu.org is using the modern, fast HTTP
transport, there's no real downside to using that by default.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2018-03-19 16:32:34 +00:00
Christian Ehrhardt
77cd862fb5 virt-aa-helper: resolve file symlinks
In a recent change b932ed69: "virt-aa-helper: resolve yet to be created
paths" several cases with symlinks in paths were fixed, but it regressed
cases where the file being last element of the path was the actual link.

In the case of the last element being the symlink realpath can (and shall)
be called on the full path that was passed.

Examples would be zfs/lvm block devices like:
 <disk type='block' device='disk'>
    <driver name='qemu' type='raw'/>
    <source dev='/dev/mapper/testlvm-testvol1'/>
    <target dev='vdd' bus='virtio'/>
 </disk>
With the target being:
 /dev/mapper/testlvm-testvol1 -> ../dm-0

That currently is rendered as
 "/dev/mapper/testlvm-testvol1" rwk,
but instead should be (and is with the fix):
 "/dev/dm-0" rwk,

Fixes: b932ed69: "virt-aa-helper: resolve yet to be created paths"
Fixes: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1756394

Signed-off-by: Christian Ehrhardt <christian.ehrhardt@canonical.com>
2018-03-19 13:06:03 +01:00
Christian Ehrhardt
7e476356c2 tools: fix variable scope in in check_guests_shutdown
libvirt-guests.sh when run with more active guests than requested to
shut down in parallel will run until it times out only shutting down
the first set of guests.

This patch fixes parallel shutdown by fixing a variable scope issue
where check_guests_shutdown unintentionally reset $guests which
prevented further progress.

Fixes: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1688508

Signed-off-by: Christian Ehrhardt <christian.ehrhardt@canonical.com>
2018-03-19 13:06:03 +01:00
Andrea Bolognani
677aaeb128 python: Drop explicit version where possible
Some of our scripts are known to work both with Python 2 and
Python 3, so for them we shouldn't be forcing any specific
version of the interpreter when they're called directly; we
always use $(PYTHON) explicitly in our build rules anyway.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2018-03-19 11:22:02 +01:00
Erik Skultety
e02d102bac qemu: hostdev: Fix the error on VM start with an mdev when IOMMU is off
Commit b4c2ac8d56 made a false assumption that IOMMU support necessary
for an mdev device to be assigned to a VM. Unlike direct PCI assignment,
IOMMU support is not needed for mediated devices, as the physical parent
device provides the isolation, therefore, simply checking for VFIO
presence is enough to successfully start a VM.

Luckily, this issue is not serious, since as of yet, libvirt mandates
mdevs to be pre-created prior to a domain's launch - if it is,
everything does work smoothly even with IOMMU disabled, because the
parent device will ensure the iommu groups we try to access exist.
However, if there are *no* IOMMU groups yet, thus no mdev exists yet, one
would see the following error:

"unsupported configuration: Mediated host device assignment requires VFIO
support"

The error msg above is simply wrong and doesn't even reflect the IOMMU
reality, so after applying this patch one would rather see the following
error in such case instead:

"device not found: mediated device '<UUID>' not found"

Signed-off-by: Erik Skultety <eskultet@redhat.com>
2018-03-19 11:14:40 +01:00
Erik Skultety
3d2e4c3e53 util: mdev: Improve the error msg on non-existent mdev prior to VM start
What one currently gets is:
failed to read '/sys/bus/mdev/devices/<UUID>/mdev_type/device_api': No
such file or directory

This indicates that something is missing within the device's sysfs tree
which likely might be not be the case here because the device simply
doesn't exist yet. So, when creating our internal mdev obj, let's check
whether the device exists first prior to trying to verify the
user-provided model within domain XML.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
2018-03-19 11:14:40 +01:00
Michal Privoznik
29d6bb7045 virNetlinkDumpCommand: Don't leak response buffer
==16451== 32,768 bytes in 2 blocks are definitely lost in loss record 1,007 of 1,013
==16451==    at 0x4C2AF0F: malloc (vg_replace_malloc.c:299)
==16451==    by 0x7CADB40: nl_recv (in /usr/lib64/libnl-3.so.200.23.0)
==16451==    by 0x532DFAC: virNetlinkDumpCommand (virnetlink.c:363)
==16451==    by 0x53236AE: virNetDevIPCheckIPv6Forwarding (virnetdevip.c:641)
==16451==    by 0xE3E4A1A: networkStartNetworkVirtual (bridge_driver.c:2490)
==16451==    by 0xE3E55F5: networkStartNetwork (bridge_driver.c:2832)
==16451==    by 0xE3DFFE5: networkAutostartConfig (bridge_driver.c:531)
==16451==    by 0x53F47E0: virNetworkObjListForEachHelper (virnetworkobj.c:1412)
==16451==    by 0x52FE69F: virHashForEach (virhash.c:606)
==16451==    by 0x53F4857: virNetworkObjListForEach (virnetworkobj.c:1439)
==16451==    by 0xE3E0BF4: networkStateAutoStart (bridge_driver.c:808)
==16451==    by 0x55689CE: virStateInitialize (libvirt.c:758)

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
2018-03-19 10:47:08 +01:00
Andrea Bolognani
9671ecfe5a docs: Import print_function in reformat-news.py
The script already works perfectly fine with Python 2, but that's
more by chance than by design: we have a single occurrence of
print(), and it just so happens that its only argument is an
expression. Importing print_function makes the script more future,
err, past proof.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2018-03-19 10:03:54 +01:00
Chen Hanxiao
3ee5a4ddf7 qemu: fix a mem leak
fix a mem leak

Signed-off-by: Chen Hanxiao <chenhanxiao@gmail.com>
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2018-03-17 19:21:22 +01:00
Chen Hanxiao
70c67d4a15 virarptable: fix some leaks and format issue
fix some leaks and format issue
Also support virArpTableFree to get NULL.

Signed-off-by: Chen Hanxiao <chenhanxiao@gmail.com>
2018-03-17 13:19:07 +01:00
Jim Fehlig
60b3fcd90c libxl: MigratePrepare: use standard begin and end API pattern
libxlDomainMigrationPrepare adds the incoming domain def to the list
of domains via virDomainObjListAdd, but never adds its own ref to the
returned virDomainObj as other callers of virDomainObjListAdd do.
libxlDomainMigrationPrepareTunnel3 suffers the same discrepancy.

Change both to add a ref to the virDomainObj after a successful
virDomainObjListAdd, similar to other callers. This ensures a consistent
pattern throughout the drivers and allows using the virDomainObjEndAPI
function for cleanup.

Signed-off-by: Jim Fehlig <jfehlig@suse.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
2018-03-16 14:37:01 -06:00
John Ferlan
fe51dbda56 libxl: Use virDomainObjListFindBy{UUID|ID}Ref
For libxlDomainLookupByID and libxlDomainLookupByUUID let's
return a locked and referenced @vm object so that callers can
then use the common and more consistent virDomainObjEndAPI in
order to handle cleanup rather than needing to know that the
returned object is locked and calling virObjectUnlock.

The LookupByName already returns the ref counted and locked object,
so this will make things more consistent.

Signed-off-by: John Ferlan <jferlan@redhat.com>
Reviewed-by: Jim Fehlig <jfehlig@suse.com>
2018-03-16 14:26:00 -06:00
John Ferlan
4e6fcdb6fa libxl: Properly cleanup after libxlDomObjFromDomain
Commit id '9ac945078' altered libxlDomObjFromDomain to return
a locked *and* ref counted object for some specific purposes;
however, it neglected to alter all the consumers of the helper
to use virDomainObjEndAPI thus leaving many objects with extra
ref counts.

Signed-off-by: John Ferlan <jferlan@redhat.com>
Reviewed-by: Jim Fehlig <jfehlig@suse.com>
2018-03-16 14:24:39 -06:00
Cole Robinson
e39a7cd256 apibuild: Fix -refs.xml building
Another usage of deprecated 'string' functions. We are just trying to
match ascii letters here, so use a simple regex. And again drop the
aggressive exception handling, it doesn't seem to trigger for anything
in libvirt code.

Signed-off-by: Cole Robinson <crobinso@redhat.com>
2018-03-16 15:07:23 -04:00