This will eliminate the need to call xdr_free to clear
pointers from data structures.
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
This replaces use of 'rpcgen' with our new python impl of
the RPC code generator. Since the new impl generates code
that matches our style/coding rules, and does not contain
long standing bugs, we no longer need to post-process the
output.
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
The new program takes the form
rpcgen [--mode source|header|repr] \
[--header include] \
xdr-file output-file
If '--mode' is not given it parses the XDR file but does not
generate anything, which is useful as a syntax check. The
'source' mode gives the '.c' file content, while 'header'
gives the '.h' file content. 'repr' gives a representation
of the abstract syntax tree, mostly useful for debugging
the parser.
If '--header' is given, it is added as a local #include ".."
statement in the output and is valid for either 'header'
or 'source' modes.
Either 'xdr-file' or 'output-file' can be omitted in which
case they default to stdin/stdout respectively.
This rpcgen program will directly include the 'config.h'
header in its output.
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Test the serialization done by libtirpc, so that when we later
switch to our own code, we can prove wire compatibility.
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
This implements a C code generator that emits code that is
(almost) identical to the classic 'rpcgen' program. The
key differences are:
- Skip inlining of calls for struct fields
- Skip K&R style function prototypes in headers
- Use int64_t instead of quad_t for OS portability
- Saner whitespace / indentation
The tests/demo.c and tests/demo.h files were created using
the traditional 'rpcgen' program, and then editted to cut
out the leading boilerplate, and the differences mentioned
above.
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
The visitor API defines an interface for visiting each element
in the XDR protocol spec abstract syntax tree.
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
This adds a parser capable of handling the XDR protocol files.
The parsing grammar requirements are detailed in
https://www.rfc-editor.org/rfc/rfc4506#section-6.3
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
This introduces classes needed to form an abstract syntax
tree representing the XDR protocol language.
The syntax requirements are detailed in
https://www.rfc-editor.org/rfc/rfc4506#section-6.3
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
This adds a lexer capable of handling the XDR protocol files.
The lexical rquirements are detailed in
https://www.rfc-editor.org/rfc/rfc4506#section-6.2
pytest is introduced as a build dependancy for testing python
code.
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Currently the script will reject any type of contents outside
of a section, but we want to be able to have some useful
comments at the top of each file to help users understand how
they are processed.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
In order to further deduplicate the contents of the various unit
files, we need to be able to merge multiple additional units
into the initial one.
Luckily the merge logic is in no way constrained to working with
just two units, so achieving this is pretty much just a matter
of lifting the existing limitation on the number of arguments
that the script accepts.
As a special case, it's now also possible to call the script
with just the base unit as argument. No merging will be performed
in that case, obviously, but we'll still go through the basic
validation and cleanup steps.
This also fixes a bug in the check for the number of arguments:
sys.argv also contains the name of the script, so we should have
checked that its size was at least 3. The check is now written in
a way that's less prone to misunderstandings.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
We already use templating to generate sockets, which are all
based off libvirtd's. Push the idea further, and extend it to
cover services as well.
This is more challenging, as the various modular daemons each have
their own needs in terms of what system services needs to be
available before they can be started, which other components of
libvirt they depend on, and so on.
In order to make this sort of per-service tweaks possible, we
introduce a Python script that can merge two systemd units
together. The script is aware of the semantics of systemd's unit
definition format, so it can intelligently merge sections
together.
This generic systemd unit merging mechanism will also supersede
the extremely ad-hoc @deps@ variable, which is currently used in
a single scenario.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Fix the syntax-check failures (which can be seen after
python3-flake8-import-order package is installed) with the help
of isort[1]:
289/316 libvirt:syntax-check / flake8 FAIL 5.24s exit status 2
[1]: https://pycqa.github.io/isort/
Signed-off-by: Han Han <hhan@redhat.com>
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Using os.system("cp {0} {1}".format(...)) has two issues, it does not
work on Windows, but more importantly it can cause issues in case one of
the directories has a space in it.
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Wrap the auto-generated pages (API ref and hvsupport.html) in the proper
top level element similarly to what the pages generated from RST have to
remove the extra case when templating our web.
(Best viewed with 'git show -w')
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Common APIs such as virConnectOpen/Close and similar which are used by
the non-hypervisor drivers in libvirt are grouped together with
hypervisor drivers, which makes the table very wide.
Split them out into a separate group and clean up the list of hypervisor
drivers.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Use the proper driver struct member names for the aforementioned APIs so
that the fixup of the versions works properly.
Currently we reported that no of the drivers supported the APIs despite
being only shims above 'open'.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
The only remaining page was 'hvsupport.html' which is generated by
'scripts/hvsupport.py'. The script already has all the data to generate
the table of contents internally so we can remove the whole complicated
template.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Since all messages marked for translation contain permutable format
strings, we can add checks for enforcing them.
The syntax check does not catch all cases as it only checks format
strings between _(" and the first ". In other words messages where \"
appears before the first format string or multi-line messages where the
first format strings is not in the first line will not be checked. On
the other hand, it's run automatically by "meson test".
check-pot.py python script will detect all incorrect format strings, but
it's not as easy to use as it requires libvirt.pot to be regenerated and
this does not happen during a standard build. The following steps are
needed to check messages with check-pot.py:
meson compile libvirt-pot-dep
meson compile libvirt-pot
meson compile libvirt-pot-check
Don't forget to revert changes to libvirt.pot if you run these commands
locally as we don't want each patch series to update libvirt.pot.
Shell scripts (tools/libvirt-guests.sh.in is the only one currently)
need to be exempt from this check as shell's printf function does not
understand the permutable format strings.
Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
As an additional step before processing the API parse the protocol file
and extract all ACL definitions. This way we can distribute them for any
user of the libvirt API XML files. We will be also able to avoid another
call to gendispatch, which generates all this data into a standalone
XML.
The remote procedure to API name is inspired by what rpcgen does.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
If the user of the 'docBuilder' class provides a dict (key is API name,
value is a tuple of arrays (acls, aclfilters), use the dict to generate
ACL definitions into the function definition.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
It's not trivial to figure out the ACL object name from our
documentation. Add it above the table outlining existing permissions.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Both the object name and permission name in ACL use '-' instead of '_'
separator when referring to them in the docs or even when used inside of
polkit. Unfortunately the generators used for generating our docs don't
honour this in certain cases which would result in broken names in the
API docs (once they will be generated).
Rename both object and permission name to use dash and reflect that in
the anchor names in the documentation.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Certain APIs are allowed also without authentication but the ACL page
didn't outline which. Generate a new column with the information.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Check both that a file is referenced from our pages and also that pages
reference existing images.
The mode for dumping external references now also dumps images.
'--ignore-image' can be used repeatedly to suppress errors for specific
images.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Now that we have the source file name as a custom attribute we can use
it to report which file actually needs to be edited to fix the error:
ERROR: 'docs/uri.rst': broken link to: 'drvqemu.html#exaple'
rather than:
broken link targets:
docs/uri.html broken link: drvqemu.html#exaple
which pointed to file which does not exist in the source directory.
This also allows us to delete all the relative path handling needed to
report at least somewhat user-legible errors before.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Force users to pass the path to the root of the webpage the script
should check. The script lives in a different subdirectory so the
default of the current directory doesn't make much sense.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
When building the top level description from a header file the
'parseTopComment' method of the 'CParser' would include all trailing
lines into the <description> field. This was designed to concatenate
multi-line descriptions, but unfortunately in all cases also included
the Copyright statement which followed.
Explicitly end the scanning of the header on a line which starts with
'Copyright (C)' and truncate the spaces from the end of the last item.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
meson already supports $DESTDIR natively, but in this case
we're using a custom script and so we have to do some extra
work ourselves.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
The G_GNUC_NO_INLINE macro will eventually be marked as
deprecated [1] and we are recommended to use G_NO_INLINE instead.
Do the switch now, rather than waiting for compile time warning
to occur.
1: 15cd0f0461
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
Improve:
KeyError: 'virAdmConnectSetDaemonTimeout'
to
Exception: Missing symbol file entry for 'virAdmConnectSetDaemonTimeout'
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
In many cases we move around or rename internal anchors which may break
links leading to the content.
docutils handle the case of links inside a document, but we are lacking
the same form of checking between documents.
Introduce a script which cross-checks all the anchors and links in HTML
output files and prints problems and use it as a test case for the
'docs' directory.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
The parameters of self.warning is inconsistent with its definition, So
fix it.
Signed-off-by: luzhipeng <luzhipeng@cestc.cn>
Reviewed-by: Martin Kletzander <mkletzan@redhat.com>
This makes it mandatory to *not* add 'v' to version numbers.
Signed-off-by: Victor Toso <victortoso@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
scripts/apibuild.py did not consider exporting external variable's
comments into the XML API. This commits fixes that.
Noe that the way that CParser is designed, it is currently possible to
lose a parsed comment when parsing other fields as self.comment in
several places. I've added a comment to highlight this.
Signed-off-by: Victor Toso <victortoso@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
So we can use for comments that are being hold in helper variables.
Signed-off-by: Victor Toso <victortoso@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
This patch adds 'version' parameter to generated XML API for functions
and functypes.
The 'version' metadata has been added with e0e0bf6628 by parsing .syms
files. This commit does not override that but it will warn if there is
not 'Since' metadata with new additions.
There is not clear benefit for keeping both. For now, I've added a
warning in case there is a mismatch between the version provided by
.syms and docstring.
Signed-off-by: Victor Toso <victortoso@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
This patch adds 'version' parameter to the generated XML API for
macros
It'll require, for new additions, to add a comment with the version
that the macro was added. An example bellow of code diff and
the change in the generated XML.
Note that the Since tag is removed from the comment as there is a
proper field for it in the XML.
Signed-off-by: Victor Toso <victortoso@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
This patch adds 'version' parameter to the generated XML API for
typedefs
It'll require, for new additions, to add a comment with the version
that the typedef value was added. An example bellow of code diff and
the change in the generated XML.
Note that the Since tag is removed from the comment as there is a
proper field for it in the XML.
Signed-off-by: Victor Toso <victortoso@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
Removed the TODO as we can rely to the serialize_typedef() the job to
report missing comments.
Signed-off-by: Victor Toso <victortoso@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
This patch adds 'version' parameter to the generated XML API for
enums.
It'll require, for new additions, to add a comment with the version
that the enum value was added.
Note that the Since tag is removed from the comment as there is a
proper field for it in the XML.
Signed-off-by: Victor Toso <victortoso@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
The script can break if the number of files does not fit one invocation and
xargs has to split it. Instead pipe the list of files directly into the script
and in the script read them from stdin instead of the arguments.
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
While being great semantic patching tool, coccinelle fails to
understand some of macros we use (including those provided by
glib). What they have in common is use of __attribute__ under the
hood. We store a list of such macros in a file. But in there,
g_auto() macro is not defined properly. Indeed, g_auto(type)
declares a local variable of given type, for instance from
cocci's POV:
g_auto(virBuffer) buf = VIR_BUFFER_INITIALIZER;
virBuffer buf = VIR_BUFFER_INITIALIZER;
are both the same declaration. Fix declaration of g_auto() stub.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Tim Wiederhake <twiederh@redhat.com>
In order to auto-generate more of the language binding code, it is
desirable to know what libvirt version an API was introduced in.
We can extract this information from the .syms files and expose
it in the API description
eg instead of
<function name='virNodeNumOfDevices' file='libvirt-nodedev'
module='libvirt-nodedev'>
we now have
<function name='virNodeNumOfDevices' file='libvirt-nodedev'
module='libvirt-nodedev' version='0.5.0'>
This will benefit this proposal:
https://gitlab.com/libvirt/libvirt-go-module/-/merge_requests/7
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Tested-by: Victor Toso <victortoso@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
The currrent generated API contains *** pointer types with bogus
whitespace in the middle:
<arg name='keys' type='char ** *' info='pointer to a variable to store authorized keys'/>
because the tokenizer only tries to merge 2 distinct '*' together.
This refactors the code to merge an arbitrary number, resulting
in
<arg name='keys' type='char ***' info='pointer to a variable to store authorized keys'/>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Add a cross reference of the enum value name with the string
representation. This allows a quick cross-reference of the values
without having to open the header and implementation files separately.
To achieve this the checker code at first obtains a list of the
flags and cross-references them when checking the grouping in
syntax-check, thus we are guaranteed to stay in sync.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Martin Kletzander <mkletzan@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
