21056 Commits

Author SHA1 Message Date
Wim ten Have
03d0959af3 xenconfig: add domxml conversions for xen-xl
This patch converts NUMA configurations between the Xen libxl
configuration file format and libvirt's XML format.

XML HVM domain on a 4 node (2 cores/socket) configuration:

  <cpu>
    <numa>
      <cell id='0' cpus='0-1' memory='2097152' unit='KiB'>
        <distances>
          <sibling id='0' value='10'/>
          <sibling id='1' value='21'/>
          <sibling id='2' value='31'/>
          <sibling id='3' value='21'/>
        </distances>
      </cell>
      <cell id='1' cpus='2-3' memory='2097152' unit='KiB'>
        <distances>
          <sibling id='0' value='21'/>
          <sibling id='1' value='10'/>
          <sibling id='2' value='21'/>
          <sibling id='3' value='31'/>
        </distances>
      </cell>
      <cell id='2' cpus='3-4' memory='2097152' unit='KiB'>
        <distances>
          <sibling id='0' value='31'/>
          <sibling id='1' value='21'/>
          <sibling id='2' value='10'/>
          <sibling id='3' value='21'/>
        </distances>
      </cell>
      <cell id='3' cpus='5-6' memory='2097152' unit='KiB'>
        <distances>
          <sibling id='0' value='21'/>
          <sibling id='1' value='31'/>
          <sibling id='2' value='21'/>
          <sibling id='3' value='10'/>
        </distances>
      </cell>
    </numa>
  </cpu>

Xen xl.cfg domain configuration:

  vnuma = [["pnode=0","size=2048","vcpus=0-1","vdistances=10,21,31,21"],
           ["pnode=1","size=2048","vcpus=2-3","vdistances=21,10,21,31"],
           ["pnode=2","size=2048","vcpus=4-5","vdistances=31,21,10,21"],
           ["pnode=3","size=2048","vcpus=6-7","vdistances=21,31,21,10"]]

If there is no XML <distances> description amongst the <cell> data the
conversion schema from xml to native will generate 10 for local and 20
for all remote instances.

Signed-off-by: Wim ten Have <wim.ten.have@oracle.com>
Reviewed-by: Jim Fehlig <jfehlig@suse.com>
Signed-off-by: Jim Fehlig <jfehlig@suse.com>
2017-11-12 19:39:09 -07:00
Wim ten Have
74119a03f1 numa: describe siblings distances within cells
Add support for describing NUMA distances in a domain's <numa> <cell>
XML description.

Below is an example of a 4 node setup:

  <cpu>
    <numa>
      <cell id='0' cpus='0-3' memory='2097152' unit='KiB'>
        <distances>
          <sibling id='0' value='10'/>
          <sibling id='1' value='21'/>
          <sibling id='2' value='31'/>
          <sibling id='3' value='21'/>
        </distances>
      </cell>
      <cell id='1' cpus='4-7' memory='2097152' unit='KiB'>
        <distances>
          <sibling id='0' value='21'/>
          <sibling id='1' value='10'/>
          <sibling id='2' value='21'/>
          <sibling id='3' value='31'/>
        </distances>
      </cell>
      <cell id='2' cpus='8-11' memory='2097152' unit='KiB'>
        <distances>
          <sibling id='0' value='31'/>
          <sibling id='1' value='21'/>
          <sibling id='2' value='10'/>
          <sibling id='3' value='21'/>
        </distances>
      <cell id='3' cpus='12-15' memory='2097152' unit='KiB'>
        <distances>
          <sibling id='0' value='21'/>
          <sibling id='1' value='31'/>
          <sibling id='2' value='21'/>
          <sibling id='3' value='10'/>
        </distances>
      </cell>
    </numa>
  </cpu>

A <cell> defines a NUMA node. <distances> describes the NUMA distance
from the <cell> to the other NUMA nodes (the <sibling>s).  For example,
in above XML description, the distance between NUMA node0 <cell id='0'
...> and NUMA node2 <sibling id='2' ...> is 31.

Valid distance values are '10 <= value <= 255'.  A distance value of 10
represents the distance to the node itself.  A distance value of 20
represents the default value for remote nodes but other values are
possible depending on the physical topology of the system.

When distances are not fully described, any missing sibling distance
values will default to 10 for local nodes and 20 for remote nodes.

If distance is given for A -> B, then we default B -> A to the same
value instead of 20.

Signed-off-by: Wim ten Have <wim.ten.have@oracle.com>
Reviewed-by: Daniel P. Berrange <berrange@redhat.com>
Signed-off-by: Jim Fehlig <jfehlig@suse.com>
2017-11-12 19:39:00 -07:00
Michal Privoznik
9324f67a57 qemuBuildDriveDevStr: Prefer default aliases for IDE bus
https://bugzilla.redhat.com/show_bug.cgi?id=1434451

When testing user aliases it was discovered that for 440fx
machine type which has default IDE bus builtin, domain cannot
start if IDE controller has the user provided alias. This is
because for 440fx we don't put the IDE controller onto the
command line (since it is builtin) and therefore any device that
is plugged onto the bus must use the default alias.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2017-11-10 15:23:55 +01:00
Viktor Mihajlovski
010f88d5cb numa: avoid failure in nodememstats on non-NUMA systems
libvirt reports a fake NUMA topology in virConnectGetCapabilities
even if built without numactl support. The fake NUMA topology consists
of a single cell representing the host's cpu and memory resources.
Currently this is the case for ARM and s390[x] RPM builds.

A client iterating over NUMA cells obtained via virConnectGetCapabilities
and invoking virNodeGetMemoryStats on them will see an internal failure
"NUMA isn't available on this host" from virNumaGetMaxNode. An example
for such a client is VDSM.

Since the intention seems to be that libvirt always reports at least
a single cell it is necessary to return "fake" node memory statistics
matching the previously reported fake cell in case NUMA isn't supported
on the system.

Signed-off-by: Viktor Mihajlovski <mihajlov@linux.vnet.ibm.com>
2017-11-09 17:11:51 -05:00
Dawid Zamirski
0eda09f490 vbox: Add support for 5.2.x
Simply add the 5.2 SDK header to the existing unified framework. No
other special handling is needed as there's no API break between
existing 5.1 and the just added 5.2.
2017-11-09 17:09:41 -05:00
Dawid Zamirski
8f1c88d9d0 vbox: Add vbox 5.2 CAPI header file.
Extracted from 5.2 SDK and reindented with cppi
2017-11-09 17:09:41 -05:00
Jim Fehlig
299e97c4e8 virconf: properly set the end of content
There was a recent report of the xen-xl converter not handling
config files missing an ending newline

https://www.redhat.com/archives/libvir-list/2017-October/msg01353.html

Commit 3cc2a9e0 fixed a similar problem when parsing content of a
file but missed parsing in-memory content. But AFAICT, the better
fix is to properly set the end of the content when initializing the
virConfParserCtxt in virConfParse().

This commit reverts the part of 3cc2a9e0 that appends a newline to
files missing it, and fixes setting the end of content when
initializing virConfParserCtxt. A test is also added to check
parsing in-memory content missing an ending newline.

Signed-off-by: Jim Fehlig <jfehlig@suse.com>
Reviewed-by: Daniel P. Berrange <berrange@redhat.com>
2017-11-09 08:04:26 -07:00
Peter Krempa
5af63c9aa3 conf: Fix type for @liveStatus in virDomainObjListLoadAllConfigs
Use bool instead of an int.
2017-11-09 10:37:58 +01:00
Michal Privoznik
3d899b9965 qemu-ns: Detect /dev/* mount point duplicates even better
In 4f1570720218302 I've tried to make duplicates detection for
nested /dev mount better. However, I've missed the obvious case
when there are two same mount points. For instance if:

  # mount --bind /dev/blah /dev/blah
  # mount --bind /dev/blah /dev/blah

Yeah, very unlikely (in qemu driver world) but possible.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2017-11-08 16:38:26 +01:00
Peter Krempa
0a3bae7d22 util: storage: Fix parsing of IPv6 portal address for iSCSI
Split on the last colon and avoid parsing port if the split remainder
contains the closing square bracket, so that IPv6 addresses are
interpreted correctly.
2017-11-08 13:10:40 +01:00
Michal Privoznik
fec8f9c49a qemu: Use predictable file names for memory-backend-file
In some cases management application needs to allocate memory for
qemu upfront and then just let qemu use that. Since we don't want
to expose path for memory-backend-file anywhere in the domain
XML, we can generate predictable paths. In this case:

  $memoryBackingDir/libvirt/qemu/$shortName/$alias

where $shortName is result of virDomainDefGetShortName().

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
2017-11-08 10:17:17 +01:00
Michal Privoznik
bb3de478a6 qemu: Destroy whole memory tree
When removing path where huge pages are call virFileDeleteTree
instead of plain rmdir(). The reason is that in the near future
there's going to be more in the path than just files - some
subdirs. Therefore plain rmdir() is not going to be enough.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
2017-11-08 10:17:11 +01:00
Michal Privoznik
eff2b2edb1 qemu: Rename qemuProcessBuildDestroyHugepagesPath
At the same time, move its internals into a separate function so
that they can be reused.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
2017-11-08 10:17:10 +01:00
Michal Privoznik
d52e6043bc qemu: Set alias for memory cell in qemuBuildMemoryCellBackendStr
Very soon qemuBuildMemoryBackendStr() is going to use memory cell
aliases. Therefore set one. At the same time, move it a bit
further - if virAsprintf() fails, there's no point in setting
rest of the members.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
2017-11-08 10:17:08 +01:00
Dawid Zamirski
8b5eefb6bc vbox: Add SAS controller support
In VirtualBox SAS and SCSI are separate controller types whereas libvirt
does not make such distinction. This patch adds support for attaching
the VBOX SAS controllers by mapping the 'lsisas1068' controller model in
libvirt XML to VBOX SAS controller type. If VBOX VM has disks attached
to both SCSI and SAS controller libvirt domain XML will have two
<controller type='scsci'> elements with index and model attributes set
accordingly. In this case, each respective <disk> element must have
<address> element specified to assign it to respective SCSI controller.
2017-11-07 15:38:43 -05:00
Dawid Zamirski
e0054c0e5d vbox: Generate disk address element in dumpxml
This patch adds <address> element to each <disk> device since device
names alone won't adequately reflect the storage device layout in the
VM. With this patch, the ouput produced by dumpxml will faithfully
reproduce the storage layout of the VM if used with define.
2017-11-07 15:38:43 -05:00
Dawid Zamirski
7c0a85e5be vbox: Process empty removable disks in dumpxml
Previously any removable storage device without media attached was
omitted from domain XML dump. They're still (rightfully) omitted in
snapshot XML dump but need to be accounted properly to for the device
names to stay in 'sync' between domain and snapshot XML dumps.
2017-11-07 15:38:35 -05:00
Dawid Zamirski
08b995c8d0 vbox: Cleanup vboxDumpDisks implementation
Primer the code for further changes:

* move variable declarations to the top of the function
* group together free/release statements
* error check and report VBOX API calls used
2017-11-07 14:45:51 -05:00
Dawid Zamirski
a4919338ba vbox: Correctly generate drive name in dumpxml
If a VBOX VM has e.g. a SATA and SCSI disk attached, the XML generated
by dumpxml used to produce "sda" for both of those disks. This is an
invalid domain XML as libvirt does not allow duplicate device names. To
address this, keep the running total of disks that will use "sd" prefix
for device name and pass it to the vboxGenerateMediumName which no
longer tries to "compute" the value based only on current and max
port and slot values. After this the vboxGetMaxPortSlotValues is not
needed and was deleted.
2017-11-07 14:45:51 -05:00
Dawid Zamirski
32a944115b vbox: Swap vboxSnapshotGetReadOnlyDisks arguments
So that the function signature matches vboxSnapshotGetReadWriteDisks
2017-11-07 14:45:51 -05:00
Dawid Zamirski
03a24da4c4 vbox: Do not free disk definitions on cleanup
Both vboxSnapshotGetReadWriteDisks and vboxSnapshotGetReadWriteDisks do
not need to free the def->disks on cleanup because it's being done by
the caller via virDomainSnaphotDefFree
2017-11-07 14:45:51 -05:00
Dawid Zamirski
60c519ee24 vbox: Cleanup/prepare snasphot dumpxml functions
This patch prepares the vboxSnapshotGetReadOnlyDisks and
vboxSnapshotGetReadWriteDisks functions for further changes so that
the code movement does not obstruct the gist of those future changes.
This is done primarily because we'll need to know the type of vbox
storage controller as early as possible and make decisions based on
that info.
2017-11-07 14:45:51 -05:00
Dawid Zamirski
c27f79a895 vbox: Rename vboxDumpIDEHDDs to vboxDumpDisks
Because it deals with other disk types as well not just IDE. Also this
function now returns -1 on error
2017-11-07 14:45:51 -05:00
Dawid Zamirski
8339d273bb vbox: Add vboxDumpStorageControllers 2017-11-07 14:45:51 -05:00
Dawid Zamirski
1cad92c421 vbox: Process <controller> element in domain XML
With this patch, the vbox driver will no longer attach all supported
storage controllers by default even if no disk devices are associated
with them. Instead, it will attach only those that are implicitly added
by virDomainDefAddImplicitController based on <disk> element or if
explicitly specified via the <controller> element.
2017-11-07 14:45:51 -05:00
Dawid Zamirski
8442d01a71 vbox: Cleanup partially-defined VM on failure
Since the VBOX API requires to register an initial VM before proceeding
to attach any remaining devices to it, any failure to attach such
devices should result in automatic cleanup of the initially registered
VM so that the state of VBOX registry remains clean without any leftover
"aborted" VMs in it. Failure to cleanup of such partial VMs results in a
warning log so that actual define error stays on the top of the error
stack.
2017-11-07 14:45:51 -05:00
John Ferlan
8fe48f20f8 storage: Privatize virStoragePoolObj and virStorageVolDefList
Move the structures into virstorageobj so that both are known
within virstorageobj.c.
2017-11-07 14:12:52 -05:00
John Ferlan
ba6575e2f6 storage: Use virStoragePoolObjGetDef accessor for new driver events
Missed from merge from commit id 'b0652192' into commit id 'bfcd8fc92'
were a couple of obj->def-> references.
2017-11-07 14:12:52 -05:00
John Ferlan
121cf746b2 storage: Use virStoragePoolObjGetDef accessor for ZFS backend
In preparation for privatizing the object, use the accessor.
2017-11-07 14:12:52 -05:00
John Ferlan
159f691fae storage: Use virStoragePoolObjGetDef accessor for VSTORAGE backend
In preparation for privatizing the object, use the accessor.
2017-11-07 14:12:52 -05:00
John Ferlan
d823466bc2 storage: Use virStoragePoolObjGetDef accessor for SCSI backend
In preparation for privatizing the object, use the accessor.
2017-11-07 14:12:52 -05:00
John Ferlan
4d95a14faf storage: Use virStoragePoolObjGetDef accessor for RBD backend
In preparation for privatizing the object, use the accessor.
2017-11-07 14:12:52 -05:00
John Ferlan
1d995fbebc storage: Use virStoragePoolObjGetDef accessor for MPATH backend
In preparation for privatizing the object, use the accessor.
2017-11-07 14:12:52 -05:00
John Ferlan
dc48de256e storage: Use virStoragePoolObjGetDef accessor for iSCSI backend
In preparation for privatizing the object, use the accessor.
2017-11-07 14:12:52 -05:00
Peter Krempa
31bbb15884 qemu: block: Add node-names to JSON backing storage strings
Format out the node-name if it was assigned for JSON-based storage
specification.
2017-11-07 16:12:52 +01:00
Peter Krempa
caf71b64fe qemu: block: Add JSON props generator for ssh storage backing 2017-11-07 16:12:52 +01:00
Peter Krempa
8da68d87ff qemu: block: Add JSON props generator for sheepdog storage backing 2017-11-07 16:12:52 +01:00
Peter Krempa
5e9e693262 qemu: block: Add JSON props generator for RBD storage backing 2017-11-07 16:12:29 +01:00
Peter Krempa
84955c6257 qemu: block: Add JSON props generator for NBD storage backing 2017-11-07 16:11:38 +01:00
John Ferlan
ea0cc4f4b0 qemu: block: Add JSON props generator for iSCSI protocol 2017-11-07 16:04:53 +01:00
Peter Krempa
97ee821a17 qemu: block: Add JSON props generator for 'curl' based storage backends
QEMU uses curl for accessing files using http(s) and ftp(s). They share
common options so let's generate them in one helper.
2017-11-07 14:58:48 +01:00
Peter Krempa
90521d0754 storage: Store RBD image name as pool and image name
Similarly to how we store gluster names, split the name into a pool and
image portions when paring the XML and store them separately.
2017-11-07 14:58:04 +01:00
Peter Krempa
bb14d9897b storage: Don't store leading '/' in image name when splitting out volume
Libvirt historically stores storage source path including the volume as
one string in the XML, but that is not really flexible enough when
dealing with the fields in the code. Previously we'd store the slash
separating the two as part of the image name. This was fine for gluster
but it's not necessary and does not scale well when converting other
protocols.

Don't store the slash as part of the path. The resulting change from
absolute to relative path within the gluster driver should be okay,
as the root directory is the default when accessing gluster.
2017-11-07 14:58:04 +01:00
Peter Krempa
1ee7d4d972 qemu: process: Split out useful parts from qemuBuildNetworkDriveURI
Extract the part formatting the basic URI part so that it can be reused
to format JSON backing definitions. Parts specific to the command line
format will remain in qemuBuildNetworkDriveURI. The new function is
called qemuBlockStorageSourceGetURI.
2017-11-07 14:57:58 +01:00
Peter Krempa
bd37213786 qemu: block: Use proper type for servers for VxHS disks
Original implementation used 'SocketAddress' equivalent from qemu for
the disk server field, while qemu documentation specifies
'InetSocketAddress'. The backing store parser uses the correct parsing
function but the formatter used the incorrect one (and also with the
legacy mode enabled which was wrong).
2017-11-07 14:43:12 +01:00
Peter Krempa
1f467d1ab1 qemu: command: Move disk serial validation to qemuCheckDiskConfig 2017-11-07 14:38:11 +01:00
Peter Krempa
cf1fcd7b51 qemu: command: Move blkiotune checks to qemuCheckDiskConfig 2017-11-07 14:33:38 +01:00
Peter Krempa
5121457cad qemu: command: Refactor blkiotune checks to tolerate NULL qemuCaps
To allow aggregating the checks, refactor the code to check capabilities
only if they were provided.
2017-11-07 14:33:23 +01:00
Peter Krempa
1055c1bf24 qemu: command: Merge checks from qemuBuildDriveStrValidate to qemuCheckDiskConfig
Stash all the disk definition and capability checks into one function.
2017-11-07 14:33:18 +01:00
Peter Krempa
5fb7ab2c9e qemu: command: Refactor qemuBuildDriveStrValidate to make qemuCaps optional
To allow merging this with other disk type checks we need to check
qemuCaps only when available, since some of the checks are executed on
disk cold-plug and thus capabilities should not be checked.

Make the checks optional by making them conditional on qemuCaps not
being NULL.
2017-11-07 14:31:21 +01:00