Currently the script will reject any type of contents outside
of a section, but we want to be able to have some useful
comments at the top of each file to help users understand how
they are processed.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
In order to further deduplicate the contents of the various unit
files, we need to be able to merge multiple additional units
into the initial one.
Luckily the merge logic is in no way constrained to working with
just two units, so achieving this is pretty much just a matter
of lifting the existing limitation on the number of arguments
that the script accepts.
As a special case, it's now also possible to call the script
with just the base unit as argument. No merging will be performed
in that case, obviously, but we'll still go through the basic
validation and cleanup steps.
This also fixes a bug in the check for the number of arguments:
sys.argv also contains the name of the script, so we should have
checked that its size was at least 3. The check is now written in
a way that's less prone to misunderstandings.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
We already use templating to generate sockets, which are all
based off libvirtd's. Push the idea further, and extend it to
cover services as well.
This is more challenging, as the various modular daemons each have
their own needs in terms of what system services needs to be
available before they can be started, which other components of
libvirt they depend on, and so on.
In order to make this sort of per-service tweaks possible, we
introduce a Python script that can merge two systemd units
together. The script is aware of the semantics of systemd's unit
definition format, so it can intelligently merge sections
together.
This generic systemd unit merging mechanism will also supersede
the extremely ad-hoc @deps@ variable, which is currently used in
a single scenario.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Fix the syntax-check failures (which can be seen after
python3-flake8-import-order package is installed) with the help
of isort[1]:
289/316 libvirt:syntax-check / flake8 FAIL 5.24s exit status 2
[1]: https://pycqa.github.io/isort/
Signed-off-by: Han Han <hhan@redhat.com>
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Using os.system("cp {0} {1}".format(...)) has two issues, it does not
work on Windows, but more importantly it can cause issues in case one of
the directories has a space in it.
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Wrap the auto-generated pages (API ref and hvsupport.html) in the proper
top level element similarly to what the pages generated from RST have to
remove the extra case when templating our web.
(Best viewed with 'git show -w')
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Common APIs such as virConnectOpen/Close and similar which are used by
the non-hypervisor drivers in libvirt are grouped together with
hypervisor drivers, which makes the table very wide.
Split them out into a separate group and clean up the list of hypervisor
drivers.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Use the proper driver struct member names for the aforementioned APIs so
that the fixup of the versions works properly.
Currently we reported that no of the drivers supported the APIs despite
being only shims above 'open'.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
The only remaining page was 'hvsupport.html' which is generated by
'scripts/hvsupport.py'. The script already has all the data to generate
the table of contents internally so we can remove the whole complicated
template.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Since all messages marked for translation contain permutable format
strings, we can add checks for enforcing them.
The syntax check does not catch all cases as it only checks format
strings between _(" and the first ". In other words messages where \"
appears before the first format string or multi-line messages where the
first format strings is not in the first line will not be checked. On
the other hand, it's run automatically by "meson test".
check-pot.py python script will detect all incorrect format strings, but
it's not as easy to use as it requires libvirt.pot to be regenerated and
this does not happen during a standard build. The following steps are
needed to check messages with check-pot.py:
meson compile libvirt-pot-dep
meson compile libvirt-pot
meson compile libvirt-pot-check
Don't forget to revert changes to libvirt.pot if you run these commands
locally as we don't want each patch series to update libvirt.pot.
Shell scripts (tools/libvirt-guests.sh.in is the only one currently)
need to be exempt from this check as shell's printf function does not
understand the permutable format strings.
Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
As an additional step before processing the API parse the protocol file
and extract all ACL definitions. This way we can distribute them for any
user of the libvirt API XML files. We will be also able to avoid another
call to gendispatch, which generates all this data into a standalone
XML.
The remote procedure to API name is inspired by what rpcgen does.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
If the user of the 'docBuilder' class provides a dict (key is API name,
value is a tuple of arrays (acls, aclfilters), use the dict to generate
ACL definitions into the function definition.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
It's not trivial to figure out the ACL object name from our
documentation. Add it above the table outlining existing permissions.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Both the object name and permission name in ACL use '-' instead of '_'
separator when referring to them in the docs or even when used inside of
polkit. Unfortunately the generators used for generating our docs don't
honour this in certain cases which would result in broken names in the
API docs (once they will be generated).
Rename both object and permission name to use dash and reflect that in
the anchor names in the documentation.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Certain APIs are allowed also without authentication but the ACL page
didn't outline which. Generate a new column with the information.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Check both that a file is referenced from our pages and also that pages
reference existing images.
The mode for dumping external references now also dumps images.
'--ignore-image' can be used repeatedly to suppress errors for specific
images.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Now that we have the source file name as a custom attribute we can use
it to report which file actually needs to be edited to fix the error:
ERROR: 'docs/uri.rst': broken link to: 'drvqemu.html#exaple'
rather than:
broken link targets:
docs/uri.html broken link: drvqemu.html#exaple
which pointed to file which does not exist in the source directory.
This also allows us to delete all the relative path handling needed to
report at least somewhat user-legible errors before.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Force users to pass the path to the root of the webpage the script
should check. The script lives in a different subdirectory so the
default of the current directory doesn't make much sense.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
When building the top level description from a header file the
'parseTopComment' method of the 'CParser' would include all trailing
lines into the <description> field. This was designed to concatenate
multi-line descriptions, but unfortunately in all cases also included
the Copyright statement which followed.
Explicitly end the scanning of the header on a line which starts with
'Copyright (C)' and truncate the spaces from the end of the last item.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
meson already supports $DESTDIR natively, but in this case
we're using a custom script and so we have to do some extra
work ourselves.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
The G_GNUC_NO_INLINE macro will eventually be marked as
deprecated [1] and we are recommended to use G_NO_INLINE instead.
Do the switch now, rather than waiting for compile time warning
to occur.
1: 15cd0f0461
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
Improve:
KeyError: 'virAdmConnectSetDaemonTimeout'
to
Exception: Missing symbol file entry for 'virAdmConnectSetDaemonTimeout'
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
In many cases we move around or rename internal anchors which may break
links leading to the content.
docutils handle the case of links inside a document, but we are lacking
the same form of checking between documents.
Introduce a script which cross-checks all the anchors and links in HTML
output files and prints problems and use it as a test case for the
'docs' directory.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
The parameters of self.warning is inconsistent with its definition, So
fix it.
Signed-off-by: luzhipeng <luzhipeng@cestc.cn>
Reviewed-by: Martin Kletzander <mkletzan@redhat.com>
This makes it mandatory to *not* add 'v' to version numbers.
Signed-off-by: Victor Toso <victortoso@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
scripts/apibuild.py did not consider exporting external variable's
comments into the XML API. This commits fixes that.
Noe that the way that CParser is designed, it is currently possible to
lose a parsed comment when parsing other fields as self.comment in
several places. I've added a comment to highlight this.
Signed-off-by: Victor Toso <victortoso@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
So we can use for comments that are being hold in helper variables.
Signed-off-by: Victor Toso <victortoso@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
This patch adds 'version' parameter to generated XML API for functions
and functypes.
The 'version' metadata has been added with e0e0bf6628 by parsing .syms
files. This commit does not override that but it will warn if there is
not 'Since' metadata with new additions.
There is not clear benefit for keeping both. For now, I've added a
warning in case there is a mismatch between the version provided by
.syms and docstring.
Signed-off-by: Victor Toso <victortoso@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
This patch adds 'version' parameter to the generated XML API for
macros
It'll require, for new additions, to add a comment with the version
that the macro was added. An example bellow of code diff and
the change in the generated XML.
Note that the Since tag is removed from the comment as there is a
proper field for it in the XML.
Signed-off-by: Victor Toso <victortoso@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
This patch adds 'version' parameter to the generated XML API for
typedefs
It'll require, for new additions, to add a comment with the version
that the typedef value was added. An example bellow of code diff and
the change in the generated XML.
Note that the Since tag is removed from the comment as there is a
proper field for it in the XML.
Signed-off-by: Victor Toso <victortoso@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
Removed the TODO as we can rely to the serialize_typedef() the job to
report missing comments.
Signed-off-by: Victor Toso <victortoso@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
This patch adds 'version' parameter to the generated XML API for
enums.
It'll require, for new additions, to add a comment with the version
that the enum value was added.
Note that the Since tag is removed from the comment as there is a
proper field for it in the XML.
Signed-off-by: Victor Toso <victortoso@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
The script can break if the number of files does not fit one invocation and
xargs has to split it. Instead pipe the list of files directly into the script
and in the script read them from stdin instead of the arguments.
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
While being great semantic patching tool, coccinelle fails to
understand some of macros we use (including those provided by
glib). What they have in common is use of __attribute__ under the
hood. We store a list of such macros in a file. But in there,
g_auto() macro is not defined properly. Indeed, g_auto(type)
declares a local variable of given type, for instance from
cocci's POV:
g_auto(virBuffer) buf = VIR_BUFFER_INITIALIZER;
virBuffer buf = VIR_BUFFER_INITIALIZER;
are both the same declaration. Fix declaration of g_auto() stub.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Tim Wiederhake <twiederh@redhat.com>
In order to auto-generate more of the language binding code, it is
desirable to know what libvirt version an API was introduced in.
We can extract this information from the .syms files and expose
it in the API description
eg instead of
<function name='virNodeNumOfDevices' file='libvirt-nodedev'
module='libvirt-nodedev'>
we now have
<function name='virNodeNumOfDevices' file='libvirt-nodedev'
module='libvirt-nodedev' version='0.5.0'>
This will benefit this proposal:
https://gitlab.com/libvirt/libvirt-go-module/-/merge_requests/7
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Tested-by: Victor Toso <victortoso@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
The currrent generated API contains *** pointer types with bogus
whitespace in the middle:
<arg name='keys' type='char ** *' info='pointer to a variable to store authorized keys'/>
because the tokenizer only tries to merge 2 distinct '*' together.
This refactors the code to merge an arbitrary number, resulting
in
<arg name='keys' type='char ***' info='pointer to a variable to store authorized keys'/>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Add a cross reference of the enum value name with the string
representation. This allows a quick cross-reference of the values
without having to open the header and implementation files separately.
To achieve this the checker code at first obtains a list of the
flags and cross-references them when checking the grouping in
syntax-check, thus we are guaranteed to stay in sync.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Martin Kletzander <mkletzan@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Historically, we declared pointer type to our types:
typedef struct _virXXX virXXX;
typedef virXXX *virXXXPtr;
But usefulness of such declaration is questionable, at best.
Unfortunately, we can't drop every such declaration - we have to
carry some over, because they are part of public API (e.g.
virDomainPtr). But for internal types - we can do drop them and
use what every other C project uses 'virXXX *'.
This change was generated by a very ugly shell script that
generated sed script which was then called over each file in the
repository. For the shell script refer to the cover letter:
https://listman.redhat.com/archives/libvir-list/2021-March/msg00537.html
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
All tests which use files with 'ldargs' and 'args' suffix as output now
use the internal and better line splitting.
Remove the test-wrap-argv.py script, the syntax check which used it and
the helper rewrapping the output when regenerating test output.
For any further use, we require code to use virCommand anyways and thus
it has internal wrapping now.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
This script works under two specific conditions. For each opened file,
search for all functions that has ACL calls and store them, and see
if there is a vir*DriverPtr struct declared in it. For each implementation
found, check if there is an ACL verification inside it, and error out if
none was found. The script also supports the concept of stub, where another
function takes the responsibility for the ACL call instead of the
original API.
Unfortunately this is not enough to cover the new scenario we have now,
with domain_driver.c containing helper functions that execute the ACL
calls. The script does not store state between files because, until now,
it wasn't needed to - APIs and stubs and vir*DriverPtr declarations were
always in the same file. Also, the script will not check for ACL in functions
that does not belong to a vir*DriverPtr interface. What we have now in
domain_driver.c breaks both assumptions: the functions are in a different
file, and there is no vir*DriverPtr being implemented in the file that
uses these functions.
This patch changes check-aclrules.py to accomodate this scenario. The helpers
that have ACL checks are stored beforehand in aclFuncHelpers, allowing other
files to use them to recognize a stub situation. In case the current file
being analyzed is domain_driver.c itself, we'll do a manual check using
aclFuncHelpers to verify that these functions indeed have ACL checks.
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
The error message printed by scripts/group-qemu-caps.py and
scripts/test-wrap-argv.py doesn't actually print the filename of the
offending file:
Incorrect line wrapping in $file
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com>
page.xsl was adding '<div id="content">' wrapper for the content picked
up from the <body> element from the original input file. Optionally
class="$DOCNAME" was added for some documents taken from <body>.
Since docs generated from RST by docutils have a '<div class='document'
id='$DOCNAME>' we actually don't need an extra wrapper for them.
Additionally if we standardize on one of them we can use the same styles
for both. I've picked the latter because it makes more sense to use the
document name as 'id'.
This patch:
1) Modifies the XSL trasformation to add the wrapper only if it's not
present.
2) Modifies the XSL transformation to use 'id' for document name and
class='document' for the wrapper element.
3) Changes docs.html/index.html/hvsupport.html to use 'id' instead of
'class' for document name.
4) Modifies the main stylesheet to keep styling the elements properly
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
i686 builds on x86_64 host on Debian 10 result in the RPC structs
getting "__attribute__((packed))" annotations added to them. This is
harmless since we know the XDR protocol aligns and pads struct fields
suitably on the wire. Thus we can safely cull the attribute before doing
the diff comparison.
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
