qemu_domin: Account for NVMe disks when calculating memlock limit on hotplug

During hotplug of a NVMe disk we need to adjust the memlock
limit. The computation of the limit is handled by
qemuDomainGetMemLockLimitBytes() which looks at given domain
definition and accounts for various device types (as different
types require different amounts). But during disk hotplug the
disk is not added to domain definition until the very last
moment. Therefore, qemuDomainGetMemLockLimitBytes() has this
@forceVFIO argument which tells it to assume VFIO even if there
are no signs of VFIO in domain definition. And this kind of
works, until the amount needed for NVMe disks changed (in
v9.3.0-rc1~52). What's missing in the commit is making @forceVFIO
behave the same as if there was an NVMe disk present in the
domain definition.

But, we can do even better - just mimic whatever we're doing for
hostdevs. IOW - introduce qemuDomainAdjustMaxMemLockNVMe() that
behaves the same as qemuDomainAdjustMaxMemLockHostdev().

There are subtle differences though:

1) qemuDomainAdjustMaxMemLockHostdev() can afford placing hostdev
   right at the end of vm->def->hostdevs, because the array was
   already reallocated (at the beginning of
   qemuDomainAttachHostPCIDevice()). But
   qemuDomainAdjustMaxMemLockNVMe() doesn't have that luxury.

2) qemuDomainAdjustMaxMemLockHostdev() places a
   virDomainHostdevDef pointer into domain definition, while
   qemuDomainStorageSourceAccessModifyNVMe() (which calls
   qemuDomainAdjustMaxMemLock()) sees a virStorageSource pointer
   but domain definition contains virDomainDiskDef. But that's
   okay, we can create a dummy disk definition and append it into
   the domain definition.

After this, qemuDomainAdjustMaxMemLock() can be called with
@forceVFIO = false, as the disk is now part of domain definition
(when computing the new limit).

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=2014030#c28
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Martin Kletzander <mkletzan@redhat.com>

src/qemu/qemu_domain.c

@@ -8026,7 +8026,7 @@ qemuDomainStorageSourceAccessModifyNVMe(virQEMUDriver *driver,
         goto revoke;
     }
 
-    if (qemuDomainAdjustMaxMemLock(vm, true) < 0)
+    if (qemuDomainAdjustMaxMemLockNVMe(vm, src) < 0)
         goto revoke;
 
     revoke_maxmemlock = true;
@@ -9779,6 +9779,39 @@ qemuDomainAdjustMaxMemLockHostdev(virDomainObj *vm,
 }
 
 
+/**
+ * qemuDomainAdjustMaxMemLockNVMe:
+ * @vm: domain object
+ * @src: disk source
+ *
+ * Temporarily add the disk source to the domain definition,
+ * adjust the max memlock based in this new definition and
+ * restore the original definition.
+ *
+ * Returns: 0 on success,
+ *         -1 on failure.
+ */
+int
+qemuDomainAdjustMaxMemLockNVMe(virDomainObj *vm,
+                               virStorageSource *src)
+{
+    g_autofree virDomainDiskDef *disk = NULL;
+    int ret = 0;
+
+    disk = g_new0(virDomainDiskDef, 1);
+    disk->src = src;
+
+    VIR_APPEND_ELEMENT_COPY(vm->def->disks, vm->def->ndisks, disk);
+
+    if (qemuDomainAdjustMaxMemLock(vm, false) < 0)
+        ret = -1;
+
+    VIR_DELETE_ELEMENT_INPLACE(vm->def->disks, vm->def->ndisks - 1, vm->def->ndisks);
+
+    return ret;
+}
+
+
 /**
  * qemuDomainHasVcpuPids:
  * @vm: Domain object

src/qemu/qemu_domain.h

@@ -41,6 +41,7 @@
 #include "virdomainmomentobjlist.h"
 #include "virenum.h"
 #include "vireventthread.h"
+#include "storage_source_conf.h"
 
 #define QEMU_DOMAIN_FORMAT_LIVE_FLAGS \
     (VIR_DOMAIN_XML_SECURE)
@@ -859,6 +860,8 @@ int qemuDomainAdjustMaxMemLock(virDomainObj *vm,
                                bool forceVFIO);
 int qemuDomainAdjustMaxMemLockHostdev(virDomainObj *vm,
                                       virDomainHostdevDef *hostdev);
+int qemuDomainAdjustMaxMemLockNVMe(virDomainObj *vm,
+                                   virStorageSource *src);
 int qemuDomainSetMaxMemLock(virDomainObj *vm,
                             unsigned long long limit,
                             unsigned long long *origPtr);