Commit Graph

27723 Commits

Author SHA1 Message Date
Pavel Hrdina
ed4d1653ed qemu: properly handle '=' in the VNC socket path
If a domain name contains a '=' and the unix socket path is
auto-generated or socket path provided by user contains '=' QEMU
is unable to properly parse the command line.  In order to make it
work we need to use the new command line syntax for VNC if it's
available, otherwise we can use the old syntax.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1352529

Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
2017-07-27 10:49:55 +02:00
Pavel Hrdina
cb9e09b502 qemu: capabilities: introduce QEMU_CAPS_VNC_MULTI_SERVERS
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
2017-07-27 10:49:55 +02:00
Peter Krempa
4199a33a19 tests: qemumonitorjson: Test extraction of iSCSI device node names
Test storage was created on a rhel/centos 7 node using targetcli.

Reviewed-by: Eric Blake <eblake@redhat.com>
2017-07-27 10:34:14 +02:00
Peter Krempa
86d8d11c14 tests: qemumonitorjson: Test extraction of LUKS node names
Test file created by:

qemu-img create -f luks /var/lib/libvirt/images/luks 10M \
        -o key-secret=asdf --object secret,id=asdf,format=raw,data=asdf

Used in libvirt as:
 <disk type='file' device='disk'>
   <driver name='qemu' type='raw'/>
   <source file='/var/lib/libvirt/images/luks'/>
   <target dev='vda' bus='virtio'/>
   <encryption format='luks'>
     <secret type='passphrase' uuid='9b2c831a-fdb9-4c09-873c-1959580589e1'/>
   </encryption>
 </disk>

Reviewed-by: Eric Blake <eblake@redhat.com>
2017-07-27 10:34:13 +02:00
Peter Krempa
b717be71dc tests: qemumonitorjson: Add test case for node name detection with blockjob
qemu 2.9 returns an extra layer in the backing data if a block job is
active. Add a test case to see whether our code properly detects and
ignores such layer.

The test data was prepared by creating a backing chain of qcow2 images
(with qemu-img and with libvirt's snapshot feature).

One of the layers was then merged back by doing a block-commit:

virsh blockcommit VM hda --top /var/lib/libvirt/images/b

and then a block-copy job was started and kept in synchronized phase:

virsh blockcopy VM hda /tmp/tgt.img --transient job

Reviewed-by: Eric Blake <eblake@redhat.com>
2017-07-27 10:34:13 +02:00
Peter Krempa
b04a3474e4 tests: qemumonitorjson: Fix 'gluster' node name detection test case
Add the blockstats data and fix the expected output.

Reviewed-by: Eric Blake <eblake@redhat.com>
2017-07-27 10:33:44 +02:00
Peter Krempa
18c5c93dc0 tests: qemumonitorjson: Fix 'relative' node name detection test case
Add the blockstats data and fix the expected output.

Test data was created as:

qemu-img create -f qcow2 img0 10M
qemu-img create -f qcow2 -o "backing_fmt=qcow2,backing_file=img0" img1
qemu-img create -f qcow2 -o "backing_fmt=qcow2,backing_file=img1" img2
...

Reviewed-by: Eric Blake <eblake@redhat.com>
2017-07-27 10:32:22 +02:00
Peter Krempa
9b03ffc7a4 tests: qemumonitorjson: Add data and fix 'same-backing' node detection case
With the new approach we are actually able to correctly detect node
names for the two instances of the same backing file.

Test images were created as:
qemu-img create -f qcow2 /var/lib/libvirt/images/base.qcow2 10M
qemu-img create -f qcow2 \
   -o "backing_fmt=qcow2,backing_file=/var/lib/libvirt/images/base.qcow2 \
   /var/lib/libvirt/images/a.qcow2
qemu-img create -f qcow2 \
   -o "backing_fmt=qcow2,backing_file=/var/lib/libvirt/images/base.qcow2 \
   /var/lib/libvirt/images/b.qcow2

and then used for two separate disks.

Reviewed-by: Eric Blake <eblake@redhat.com>
2017-07-27 10:27:20 +02:00
Peter Krempa
c03e52af57 qemu: block: store and test driver names for detected storage nodes
Store the 'drv' field both for the storage node and for the format node
and format them in the test case.

Reviewed-by: Eric Blake <eblake@redhat.com>
2017-07-27 09:56:41 +02:00
Peter Krempa
acfd2fdcbf tests: qemumonitorjson: Simplify node name detection test
We can now iterate the hash table and print all detected backing chains.
This simplifies calling of the test cases.

Reviewed-by: Eric Blake <eblake@redhat.com>
2017-07-27 09:55:24 +02:00
Peter Krempa
4983368367 tests: Extract mock library for making hash table deterministic
virHashNew calls virRandomBits to initialize seed for the hashing
function. If a test uses iteration through the hash table to produce
results they may/will be non-deterministic. Extract the mock library
which was used for mac address mapping to be universal.

Reviewed-by: Eric Blake <eblake@redhat.com>
2017-07-27 09:54:20 +02:00
Peter Krempa
0175dc6ea0 qemu: block: Refactor node name detection code
Remove the complex and unreliable code which inferred the node name
hierarchy only from data returned by 'query-named-block-nodes'. It turns
out that query-blockstats contain the full hierarchy of nodes as
perceived by qemu so the inference code is not necessary.

In query blockstats, the 'parent' object corresponds to the storage
behind a storage volume and 'backing' corresponds to the lower level of
backing chain. Since all have node names this data can be really easily
used to detect node names.

In addition to the code refactoring the one remaining test case needed
to be fixed along.

Reviewed-by: Eric Blake <eblake@redhat.com>
2017-07-27 09:47:40 +02:00
Peter Krempa
c61d169327 util: storagefile: rename 'nodebacking' to 'nodestorage' in virStorageSource
Make it less confusing by naming the field which refers to the storage
object as 'nodestorage'.

Reviewed-by: Eric Blake <eblake@redhat.com>
2017-07-27 09:44:05 +02:00
Peter Krempa
4a58dfc24e qemu: block: Rename qemuBlockFillNodeData and move it to the top
The same operation will become useful in other places so rename the
function to be more generic and move it to the top so that it can be
reused earlier in the file.

Reviewed-by: Eric Blake <eblake@redhat.com>
2017-07-27 09:42:31 +02:00
Peter Krempa
f0cbe201b8 tests: qemumontitorjson: temporarily disable node name detection tests
To simplify the refactoring patches disable the tests. This will allow
adding test data later.

Reviewed-by: Eric Blake <eblake@redhat.com>
2017-07-27 09:40:52 +02:00
Peter Krempa
e3b0eb5b1c tests: qemumonitorjson: simplify path handling in testBlockNodeNameDetect
Extract the test prefix path into a variable and reuse
virTestLoadFileJSON to load the sample json files rather than doing it
manually.

Reviewed-by: Eric Blake <eblake@redhat.com>
2017-07-27 09:39:54 +02:00
Peter Krempa
e5d03a2c1a tests: utils: Add virTestLoadFileJSON helper
This new helper loads, parses and returns a JSON file from 'abs_srcdir'
By using variable arguments for the function, it's not necessary to
format the path separately in the test cases.

Reviewed-by: Eric Blake <eblake@redhat.com>
2017-07-27 09:38:39 +02:00
Peter Krempa
b650caedf1 tests: qemuhelp: convert to virTestLoadFilePath
As a sample usage of the new helper convert the calls in qemuhelptest to
the new helper.

Reviewed-by: Eric Blake <eblake@redhat.com>
2017-07-27 09:37:21 +02:00
Peter Krempa
d492b19259 tests: utils: Add virTestLoadFilePath helper
This new helper loads and returns a file from 'abs_srcdir'. By using
variable arguments for the function, it's not necessary to format the
path separately in the test cases.

Reviewed-by: Eric Blake <eblake@redhat.com>
2017-07-27 09:34:49 +02:00
Peter Krempa
3c60388591 util: buffer: Add virBufferStrcatVArgs
Split out the worker loop into a separate function and export it.

Reviewed-by: Eric Blake <eblake@redhat.com>
2017-07-27 09:31:14 +02:00
Peter Krempa
55e8314efd tests: utils: Don't calculate file size in virTestLoadFile
The callers don't use it so don't waste a strlen(). Also fix the comment
for the function.

Reviewed-by: Eric Blake <eblake@redhat.com>
2017-07-27 09:29:57 +02:00
Peter Krempa
29f771fd66 tests: qemumonitorjson: Drop redundant data from testBlockNodeNameDetectFormat
The node name and backing file name can be inferred from the hierarchy.
This will also help when converting to detect node names using
query-blockstats data.

Reviewed-by: Eric Blake <eblake@redhat.com>
2017-07-27 09:28:48 +02:00
Peter Krempa
f1ce954b32 tests: qemumonitor: Prepare for more test data in testBlockNodeNameDetect
Rename 'json' and related variables to 'nodeNameJson'. Also rename the
test files along. This is a preparation for modifying how we detect node
names which will also require data from 'query-blockstats'.

Reviewed-by: Eric Blake <eblake@redhat.com>
2017-07-27 09:27:58 +02:00
Peter Krempa
2b5d5d68ca tests: qemumonitorjson: Consolidate basic node name detection test cases
Test cases named '1' and '2' differed only in the length of the backing
chain, so remove test case '2' and rename test '1' to 'basic'.

Reviewed-by: Eric Blake <eblake@redhat.com>
2017-07-27 09:25:55 +02:00
Peter Krempa
51d243bdbf qemu: monitor: Extract call of 'query-blockstats' and add new API for it
Allow getting the raw data from query-blockstats, so that we can use it
to detect the backing chain later on.

Reviewed-by: Eric Blake <eblake@redhat.com>
2017-07-27 09:25:26 +02:00
John Ferlan
43e6686c7f storage: Disallow usage of the HBA for a fc_host backing
Disallow providing the wwnn/wwpn of the HBA in the adapter XML:

  <adapter type='fc_host' [parent='scsi_hostN'] wwnn='HBA_wwnn'
    wwpn='HBA_wwpn'/>

This should be considered a configuration error since a vHBA
would not be created. In order to use the HBA as the backing the
following XML should be used:

  <adapter type='scsi_host' name='scsi_hostN'/>

So add a check prior to the checkParent call to validate that
the provided wwnn/wwpn resolves to a vHBA and not an HBA.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2017-07-26 15:28:06 -04:00
Martin Kletzander
e1bafb0099 qemu_cgroup: Remove unnecessary virQEMUDriverPtr arguments
Since commit 2e6ecba1bc, the pointer to the qemu driver is saved in
domain object's private data and hence does not have to be passed as
yet another parameter if domain object is already one of them.

This is a first (example) patch of this kind of clean up, others will
hopefully follow.

Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
2017-07-26 17:47:25 +02:00
Martin Kletzander
a54c962286 docs: Add build timestamps to generated html/php pages
In order not to make the build even less reproducible, honour
SOURCE_DATE_EPOCH environment variable as specified:

  https://reproducible-builds.org/specs/source-date-epoch/

Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
2017-07-26 17:47:25 +02:00
Martin Kletzander
0ca4e369bb build: Distribute tests/{qemucpumock.c,testutilshostcpus.h}
Missed by 13554a9e7f.

Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
2017-07-26 17:37:41 +02:00
Andrea Bolognani
9e6fb8612b tests: Prepare for stricter NIC model validation
While using "definitely-not-virtio" as a model name is very
cute, it will also cause the relevant test to fail once we
introduce stricter validation.

Use "e1000", which is definitely not virtio but also a valid
model name, instead.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
2017-07-26 17:13:01 +02:00
Nitesh Konkar
e765a9972b docs: Fix typo in memory backing section
Signed-off-by: Nitesh Konkar <nitkon12@linux.vnet.ibm.com>
Signed-off-by: Ján Tomko <jtomko@redhat.com>
2017-07-26 16:48:21 +02:00
John Ferlan
5df1d0d417 test: Use consistent variable names for network test driver APIs
A virNetworkObjPtr will be an 'obj'.

A virNetworkPtr will be a 'net'.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2017-07-26 10:38:01 -04:00
Pavel Hrdina
f444e5b4a2 qemu: privatize _virQEMUCapsCachePriv struct
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Jiri Denemark <jdenemar@redhat.com>
2017-07-26 15:36:47 +02:00
Pavel Hrdina
d03de54e7e qemu: switch QEMU capabilities to use virFileCache
The switch contains considerable amount of changes:

  virQEMUCapsRememberCached() is removed because this is now handled
  by virFileCacheSave().

  virQEMUCapsInitCached() is removed because this is now handled by
  virFileCacheLoad().

  virQEMUCapsNewForBinary() is split into two functions,
  virQEMUCapsNewData() which creates new data if there is nothing
  cached and virQEMUCapsLoadFile() which loads the cached data.
  This is now handled by virFileCacheNewData().

  virQEMUCapsCacheValidate() is removed because this is now handled by
  virFileCacheValidate().

  virQEMUCapsCacheFree() is removed because it's no longer required.

  Add virCapsPtr into virQEMUCapsCachePriv because for each call of
  virFileCacheLookup*() we need to use current virCapsPtr.

Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Jiri Denemark <jdenemar@redhat.com>
2017-07-26 15:36:21 +02:00
Pavel Hrdina
f366cfed62 qemu: pass only host arch instead of the whole virCaps
This is a preparation for following patches where we switch to
virFileCache for QEMU capabilities cache

The host arch will always remain the same but virCaps may change.  Now
the host arch is stored while creating new qemu capabilities cache.
It removes the need to pass virCaps into virQEMUCapsCache*() functions.

Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Jiri Denemark <jdenemar@redhat.com>
2017-07-26 15:35:24 +02:00
Pavel Hrdina
13554a9e7f tests: rewrite host CPU mocking
Move all the host CPU data into a separate file and rewrite qemucpumock
to not use passed @caps.  This is preparation for following patch which
will replace virCaps argument with virArch.

Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Jiri Denemark <jdenemar@redhat.com>
2017-07-26 15:35:02 +02:00
Pavel Hrdina
731cfd5fe8 qemu: introduce struct _virQEMUCapsCachePriv
This will store private data that will be used by following patches
when switching to virFileCache.

Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Jiri Denemark <jdenemar@redhat.com>
2017-07-26 15:33:59 +02:00
Pavel Hrdina
478f0c5b85 tests: add virfilecachetest
Implements 3 test cases that covers how the cache is used.

We have to mock unlink() function because the caching code unlinks
files that are no longer valid and we don't want to do it in our tests.

Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Jiri Denemark <jdenemar@redhat.com>
2017-07-26 15:33:45 +02:00
Pavel Hrdina
ac3eb2ab24 util: introduce virFileCache
The new virFileCache will nicely handle the caching logic for any data
that we would like to cache.  For each type of data we will just need
to implement few handlers that will take care of creating, validating,
loading and saving the cached data.

The cached data must be an instance of virObject.

Currently we cache QEMU capabilities which will start using
virFileCache.

Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Jiri Denemark <jdenemar@redhat.com>
2017-07-26 15:31:25 +02:00
Pavel Hrdina
cc1329b627 qemu: we prefer C89 comment styles over C99
Introduced by commit 'a7bc2c8cfd6f'.

Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
2017-07-25 23:10:00 +02:00
Scott Garfinkle
a7bc2c8cfd Generate unique socket file
It's possible to have more than one unnamed virtio-serial unix channel.
We need to generate a unique name for each channel. Currently, we use
".../unknown.sock" for all of them. Better practice would be to specify
an explicit target path name; however, in the absence of that, we need
uniqueness in the names we generate internally.

Before the changes we'd get /var/lib/libvirt/qemu/channel/target/unknown.sock
for each instance of
    <channel type='unix'>
        <source mode='bind'/>
        <target type='virtio'/>
    </channel>

Now, we get vioser-00-00-01.sock, vioser-00-00-02.sock, etc.

Signed-off-by: Scott Garfinkle <seg@us.ibm.com>
2017-07-25 22:38:35 +02:00
Martin Kletzander
eaf2c9f891 Move machineName generation from virsystemd into domain_conf
It is more related to a domain as we might use it even when there is
no systemd and it does not use any dbus/systemd functions.  In order
not to use code from conf/ in util/ pass machineName in cgroups code
as a parameter.  That also fixes a leak of machineName in the lxc
driver and cleans up and de-duplicates some code.

Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
2017-07-25 17:02:27 +02:00
Martin Kletzander
aa0dfb91d5 lxc: Make lxcProcessStop callable even without PID being available
This way the function can work as a central point of clean-up code and
we don't have to duplicate code.  And it works similarly to the qemu
driver.

Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
2017-07-25 17:02:27 +02:00
Martin Kletzander
2e6ecba1bc qemu: Save qemu driver in qemuDomainObjPrivateData
This way we can finally make it static and not use any externs anywhere.

Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
2017-07-25 17:02:27 +02:00
Martin Kletzander
6e6faf6d62 conf: Pass config.priv to xmlopt->privateData.alloc
This will help us to get to some data more easily.

Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
2017-07-25 17:02:27 +02:00
John Ferlan
867bcc9c78 secret: Handle object list removal and deletion properly
Rather than rely on virSecretObjEndAPI to make the final virObjectUnref
after the call to virSecretObjListRemove, be more explicit by calling
virObjectUnref and setting @obj to NULL for secretUndefine and in
the error path of secretDefineXML. Calling EndAPI will end up calling
Unlock on an already unlocked object which has indeteriminate results
(usually an ignored error).

The virSecretObjEndAPI will both Unref and Unlock the object; however,
the virSecretObjListRemove would have already Unlock'd the object so
calling Unlock again is incorrect. Once the virSecretObjListRemove
is called all that's left is to Unref our interest since that's the
corrollary to the virSecretObjListAdd which returned our ref interest
plus references for each hash table in which the object resides. In math
terms, after an Add there's 2 refs on the object (1 for the object and
1 for the list). After calling Remove there's just 1 ref on the object.
For the Add callers, calling EndAPI removes the ref for the object and
unlocks it, but since it's in a list the other 1 remains.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2017-07-25 09:15:30 -04:00
John Ferlan
d04bc0278d secret: Fix memory leak in virSecretLoad
If the virSecretLoadValue fails, the code jumped to cleanup without
setting @ret = obj, thus calling virSecretObjListRemove which only
accounts for the object reference related to adding the object to
the list during virSecretObjListAdd, but does not account for the
reference to the object itself as the return of @ret would be NULL
so the caller wouldn't call virSecretObjEndAPI on the object recently
added thus reducing the refcnt to zero.

This patch will perform the ObjListRemove in the failure path of
virSecretLoadValue and Unref @obj in order to perform clean up
and return @obj as NULL. The @def will be freed as part of the
virObjectUnref.
2017-07-25 09:15:30 -04:00
John Ferlan
e4c0aff215 secret: Properly handle @def after virSecretObjAdd in driver
Since the virSecretObjListAdd technically consumes @def on success,
the secretDefineXML should set @def = NULL immediately and process
the remaining calls using a new @objDef variable. We can use use
VIR_STEAL_PTR since we know the Add function just stores @def in
obj->def.

Because we steal @def into @objDef, if we jump to restore_backup:
and @backup is set, then we need to ensure the @def would be
free'd properly, so we'll steal it back from @objDef. For the other
condition this fixes a double free of @def if the code had jumped to
@backup == NULL thus calling virSecretObjListRemove without setting
@def = NULL. In this case, the subsequent call to DefFree would
succeed and free @def; however, the call to EndAPI would also
call DefFree because the Unref done would be the last one for
the @obj meaning the obj->def would be used to call DefFree,
but it's already been free'd because @def wasn't managed right
within this error path.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2017-07-25 09:15:30 -04:00
John Ferlan
7ca17da9f2 secret: Remove need for local configFile and base64File in ObjectAdd
Rather than assign to a local variable, let's just assign directly to the
object using the error path for cleanup.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2017-07-25 09:11:48 -04:00
Michal Privoznik
2d3c7122c8 Revert "virthread: Introduce virRWLockInitPreferWriter"
This reverts commit 328bd24443.

As it turns out, this is not portable and very Linux & glibc
specific. Worse, this may lead to not starving writers on Linux
but everywhere else. Revert this and if the starvation occurs
resolve it.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Daniel P. Berrange <berrange@redhat.com>
2017-07-25 10:56:03 +02:00