'xmltodict' is a Python module that is not installed by default.
Replace it, so the dependencies of cpu-gather.py do not change
when both scripts are merged.
Signed-off-by: Tim Wiederhake <twiederh@redhat.com>
Reviewed-by: Jiri Denemark <jdenemar@redhat.com>
'xmltodict' is a Python module that is not installed by default.
Replace it, so the dependencies of cpu-gather.py do not change
when both scripts are merged.
Signed-off-by: Tim Wiederhake <twiederh@redhat.com>
Reviewed-by: Jiri Denemark <jdenemar@redhat.com>
Using 'argparse' for argument handling simplifies merging this script
with cpu-gather.py in a later patch.
Signed-off-by: Tim Wiederhake <twiederh@redhat.com>
Reviewed-by: Jiri Denemark <jdenemar@redhat.com>
Make
./cpu-gather.py --gather --parse
an alias of
./cpu-gather.py [--gather] | ./cpu-gather.py --parse
Signed-off-by: Tim Wiederhake <twiederh@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
More reliable, easier to parse, easier to edit.
Signed-off-by: Tim Wiederhake <twiederh@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
This changes the invocation from
./cpu-gather.py | ./cpu-parse.sh
to
./cpu-gather.py [--gather] | ./cpu-gather.py --parse
Signed-off-by: Tim Wiederhake <twiederh@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
This is a preparatory step to replace the output format with
something more readable.
Signed-off-by: Tim Wiederhake <twiederh@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Fixes the leaking file descriptors. Does not silently ignore errors
(e.g. permission denied on /dev/cpu/0/msr if run as non-root) and
always attempt to read from /dev/kvm if /dev/cpu/0/msr failed.
'gather_msr()' returns a dictionary of values, as a later patch will
add more registers to be interrogated.
Signed-off-by: Tim Wiederhake <twiederh@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
This is useful if cpuid was compiled from source in a non-standard
location.
Signed-off-by: Tim Wiederhake <twiederh@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Turn the comment on how to aquire cpuid into a runtime error message.
Use "http" instead of "https" in the URL, as the latter is broken.
Signed-off-by: Tim Wiederhake <twiederh@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Some hardware, e.g. exotic platforms or pre-production hardware, may
report wrong or random data for the cpu model name. As the name of
the created files is derived from that name, this may lead to issues.
Signed-off-by: Tim Wiederhake <twiederh@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
This changes the invocation from
./cpu-gather.sh | ./cpu-parse.sh
to
./cpu-gather.py | ./cpu-parse.sh
Signed-off-by: Tim Wiederhake <twiederh@redhat.com>
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
The feature is never enabled by default on KVM and QEMU dropped it from
the models long ago.
https://bugzilla.redhat.com/show_bug.cgi?id=1798004
Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Reviewed-by: Tim Wiederhake <twiederh@redhat.com>
The feature is filtered by KVM and never automatically enabled. So even
though QEMU definition of EPYC-Rome contains this feature, the guest
won't see it. Also domain capabilities will show it as disabled for KVM
domains. Thus the feature should not really be included in our
definition of EPYC-Rome.
Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
The CPU should be identified as EPYC-Rome, but the QEMU binary used to
gather the original test data did not support this model. Let's update
the supported models to QEMU 5.1.0.
Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
The files contained the "-invalid" marker in their filename, marking
them as test cases that are supposed to fail in the virschematest.
Unfortunately, the "-invalid" marker does not discriminate between
different tests the files might be used in.
A later patch will introduce a new test validating the XML. This
test is not supposed to fail, as the files contain valid XML.
Signed-off-by: Tim Wiederhake <twiederh@redhat.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
Use https: links for websites that support them.
The URIs which are used as namespace identifiers
are left alone.
Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Neal Gompa <ngompa13@gmail.com>
The CPUID data in *-{disabled,enabled}.xml convert feature names from
the corresponding *.json file into raw CPUID and MSR data and thus some
of them may need to be updated when new features are added into the CPU
map.
Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
The stepping range (10-11) is likely incomplete. QEMU uses 10 and the
CPUID data for Cooperlake show 11. We will update the range if needed
once more details about he CPU are available.
Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
The signatures of these two CPU model differ only in stepping as both
report family 6 and model 85. Skylake-Server uses stepping 4 or less and
Cascadelake-Server uses stepping 5..7.
https://bugzilla.redhat.com/show_bug.cgi?id=1761678
Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Skylake-Server with family 6, model 85, stepping 4, which is currently
mis-detected as Cascadelake-Server CPU model.
Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Host-model CPU definitions (and domain capabilities) will use the
original CPU models (without noTSX in their name) and explicitly disable
hle and rtm features. This way domains with host-model CPUs will be
migratable even to older versions of libvirt which do not support the
noTSX model variants.
The new models will be advertised in host capabilities and they may
be used explicitly with custom CPUs.
Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Reviewed-by: Christian Ehrhardt <christian.ehrhardt@canonical.com>
Tested-by: Christian Ehrhardt <christian.ehrhardt@canonical.com>
One of the mitigation methods for TAA[1] is to disable TSX
support on the host system. Linux added a mechanism to disable
TSX globally through the kernel command line, and many Linux
distributions now default to tsx=off. This makes existing CPU
models that have HLE and RTM enabled not usable anymore.
Add new versions of all CPU models that have the HLE and RTM
features enabled, that can be used when TSX is disabled in the
host system.
On systems disabling the features without those types defined
in cpu-maps users end up without modern CPU types in the list
of usable CPUs to use in the likes of virsh domcapabilities
or tools higher in the stack like virt-manager.
This adds:
-Cascadelake-Server-noTSX
-Icelake-Client-noTSX
-Icelake-Server-noTSX
-Skylake-Server-noTSX-IBRS
-Skylake-Client-noTSX-IBRS
Introduced in QEMU by commit v4.2.0-rc2-3-g9ab2237f19 (function)
and commit v4.2.0-rc2-4-g02fa60d101 (names)
References:
[1] TAA, TSX asynchronous Abort:
https://software.intel.com/security-software-guidance/insights/deep-dive-intel-transactional-synchronization-extensions-intel-tsx-asynchronous-aborthttps://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/tsx_async_abort.html
Fixes: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1853200
Signed-off-by: Christian Ehrhardt <christian.ehrhardt@canonical.com>
Message-Id: <20200310104806.2723-2-christian.ehrhardt@canonical.com>
Reviewed-by: Jiri Denemark <jdenemar@redhat.com>
Recently CPU hardware vendors have started to support a new structure
inside the CPU package topology known as a "die". Thus the hierarchy
is now:
sockets > dies > cores > threads
This adds support for "dies" in the XML parser, with the value
defaulting to 1 if not specified for backwards compatibility.
For example a system with 64 logical CPUs might report
<topology sockets="4" dies="2" cores="4" threads="2"/>
Reviewed-by: Jiri Denemark <jdenemar@redhat.com>
Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Qemu commit e900135dcfb67 ("i386: Add CPUID bit for CLZERO and XSAVEERPTR")
adds support for CLZERO CPUID bit.
This commit extends support for this CPUID bit into libvirt.
Signed-off-by: Ani Sinha <ani.sinha@nutanix.com>
Message-Id: <1575371352-99055-1-git-send-email-ani.sinha@nutanix.com>
Reviewed-by: Jiri Denemark <jdenemar@redhat.com>
Add Hygon Dhyana CPU data test case related files.
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Yingle Hou <houyingle@hygon.cn>
python2 will be end of life by the time of the next
libvirt release. All our supported build targets, including
CentOS7, have a python3 build available.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Most likely for historical reasons our CPU def formatting code is
happily adding useless <model fallback='allow'/> for host-model CPUs. We
can just drop it.
Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
The pconfig feature was enabled in QEMU by accident in 3.1.0. All other
newer versions do not support it and it was removed from the
Icelake-Server CPU model in QEMU.
We don't normally change our CPU models even when QEMU does so to avoid
breaking migrations between different versions of libvirt. But we can
safely do so in this specific case. QEMU never supported enabling
pconfig so any domain which was able to start has pconfig disabled.
With a small compatibility hack which explicitly disables pconfig when
CPU model equals Icelake-Server in migratable domain definition, only
one migration scenario stays broken (and there's nothing we can do about
it): from any host to a host with libvirt < 5.10.0 and QEMU > 3.1.0.
https://bugzilla.redhat.com/show_bug.cgi?id=1749672
Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
There should be a single space either side of operators. Inline
comments should have two spaces before the '#'
src/hyperv/hyperv_wmi_generator.py:130:45: E261 at least two spaces before inline comment
source += ' { "", "", 0 },\n' # null terminated
^
src/esx/esx_vi_generator.py:417:25: E221 multiple spaces before operator
FEATURE__DESERIALIZE = (1 << 6)
^
tests/cputestdata/cpu-cpuid.py:187:78: E225 missing whitespace around operator
f.write(" <msr index='0x%x' edx='0x%08x' eax='0x%08x'/>\n" %(
^
docs/apibuild.py:524:47: E226 missing whitespace around arithmetic operator
self.line = line[i+2:]
^
...more...
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Coding style expects 1 blank line between each method and 2 blank lines
before each class.
docs/apibuild.py:171:5: E303 too many blank lines (2)
def set_header(self, header):
^
docs/apibuild.py:230:1: E302 expected 2 blank lines, found 1
class index:
^
docs/apibuild.py:175:5: E301 expected 1 blank line, found 0
def set_module(self, module):
^
...more...
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
CVE-2018-12126, CVE-2018-12127, CVE-2018-12130, CVE-2019-11091
The bit is set when microcode provides the mechanism to invoke a flush
of various exploitable CPU buffers by invoking the VERW instruction.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
This patch adds an inline python code for reading MSR features. Since
reading MSRs is a privileged operation, we have to read them from
/dev/cpu/*/msr if it is readable (i.e., the script runs as root) or
fallback to using KVM ioctl which can be done by any user that can start
virtual machines.
The python code is inlined rather than provided in a separate script
because whenever there's an issue with proper detection of CPU features,
we ask the reporter to run cpu-gather.sh script to give us all data we
need to know about the host CPU. Asking them to run several scripts
would likely result in one of them being ignored or forgotten.
Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
The parseMapFeature for parsing features from CPU map XML can be easily
generalized to support more feature types.
Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Let's make sure the current CPUID specific code is only applied to CPUID
features.
Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
This will let us simplify the code since the dictionary keys will match
attribute names in various XMLs.
Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
leaf["eax"] & eax > 0 check works correctly only if there's at most 1
bit set in eax. Luckily that's been always the case, but fixing this
could save us from future surprises.
Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
The function will have to deal with both CPUID and MSR features.
Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
We don't really need to parse CPU data from QEMU older than 2.9 (i.e.,
before query-cpu-model-expansion) at this point. But even if there's a
need to do so, we can always use an older version of this script to do
the conversion.
Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Introduced in QEMU 3.1.0 by commit
c7a88b52f62b30c04158eeb07f73e3f72221b6a8
Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
This fixes several CPUs which were incorrectly detected as
Skylake-Client.
Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
This fixes several CPUs which were incorrectly detected as a different
CPU model.
Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
The signature computation code is not too complicated and it will likely
never change so testing it is not very important. We do it mostly for a
nice side effect of easily accessible signature numbers for all CPU
data files.
Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
The tests/cputestdata/cpu-parse.sh would produce JSON files with QEMU
replies which wouldn't pass syntax-check. Let's fix this by not emitting
an extra new line after reformatting the JSON file.
Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
The tests/cputestdata/cpu-parse.sh script has been broken since the
cpu_map.xml file was split into several XMLs.
Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
The feature was added to QEMU in 3.1.0 and it is currently blocking
migration, which is expected to change in the future. Luckily 3.1.0 is
new enough to give us migratability hints on each feature via
query-cpu-model-expension, which means we don't need to use the
"migratable" attribute on the CPU map XML.
The kernel calls this feature arch_capabilities and RHEL/CentOS 7.* use
arch-facilities. Apparently some CPU test files were gathered with the
RHEL version of QEMU. Let's update the test files to avoid possible
confusion about the correct naming.
Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
QEMU commits:
e37a5c7fa4 (v2.12.0)
i386: Add Intel Processor Trace feature support
c2f193b538 (v2.7.0)
target-i386: Add support for UMIP and RDPID CPUID bits
aff9e6e46a (v2.12.0)
x86/cpu: Enable new SSE/AVX/AVX512 cpu features
f77543772d (v2.9.0)
x86: add AVX512_VPOPCNTDQ features
5131dc433d (v3.1.0)
i386: Add CPUID bit for PCONFIG
59a80a19ca (v3.1.0)
i386: Add CPUID bit for WBNOINVD
Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Replace the print statement, that is only available in Py2, with a
print function that is available in both Py2 and Py3 and drop the
explicit python version in the shebang.
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Radostin Stoyanov <rstoyanov1@gmail.com>
PEP8 recommends not having spaces around = in a keyword argument or
a default parameter value.
https://www.python.org/dev/peps/pep-0008/#other-recommendations
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Radostin Stoyanov <rstoyanov1@gmail.com>
This is a variant of EPYC with indirect branch prediction protection.
The only difference between EPYC and EPYC-IBPB is the added "ibpb"
feature.
Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
This is a variant of Skylake-Server with indirect branch prediction
protection. The only difference between Skylake-Server and
Skylake-Server-IBRS is the added "spec-ctrl" feature.
Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
This is a variant of Skylake-Client with indirect branch prediction
protection. The only difference between Skylake-Client and
Skylake-Client-IBRS is the added "spec-ctrl" feature.
Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
This is a variant of Broadwell with indirect branch prediction
protection. The only difference between Broadwell and Broadwell-IBRS is
the added "spec-ctrl" feature.
The Broadwell-IBRS model in QEMU is a bit different since Broadwell got
several additional features since we added it in cpu_map.xml:
abm, arat, f16c, rdrand, vme, xsaveopt
Adding them only to the -IBRS variant would confuse our CPU detection
code.
Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
This is a variant of Haswell-noTSX with indirect branch prediction
protection. The only difference between Haswell-noTSX and
Haswell-noTSX-IBRS is the added "spec-ctrl" feature.
The Haswell-noTSX-IBRS model in QEMU is a bit different since
Haswell-noTSX got several additional features since we added it in
cpu_map.xml:
arat, abm, f16c, rdrand, vme, xsaveopt
Adding them only to the -IBRS variant would confuse our CPU detection
code.
Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
