If a value of the first level object contains more objects needing
deflattening which would be wrapped in an actual object the function
would not recurse into them.
By this simple addition we can fully deflatten the objects.
As it turns out sometimes users pass in an arbitrarily nested structure
e.g. for the qemu backing chains JSON pseudo protocol. This new
implementation deflattens now a single object fully even with nested
keys.
Additionally it's not necessary now to stick with the "file." prefix for
the properties.
Currently the function would deflatten the object by dropping the 'file'
prefix from the attributes. This does not really scale well or adhere to
the documentation.
Until we refactor the worker to properly deflatten everything we at
least simulate it by adding the "file" wrapper object back.
Almost none of our virJSONValue*Get* functions accept const virJSONValue
pointers and it wouldn't even make sense since we sometimes modify what
we get. And because there is no reason for preventing callers of
virJSONValueObjectForeachKeyValue from modifying the values they get in
each iteration we can just stop doing it.
Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Provide the Steal API for any code paths that will desire to grab the
object array and then free it afterwards rather than relying to freeing
the whole chain from the reply.
json_reformat uses two spaces for when indenting nested objects, let's
do the same. The result of virJSONValueToString will be exactly the same
as json_reformat would produce.
Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
The internal representation of a JSON array counts the items in
size_t. However, for some reason, when asking for the count it's
reported as int. Firstly, we need the function to return a signed
type as it's returning -1 on an error. But, not every system has
integer the same size as size_t. Therefore, lets return ssize_t.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
We already enable the parser option to detect invalid UTF-8, but
didn't test it. Also, JSON states that behavior of an object
with a duplicated key is undefined; we chose to reject it, but
were not testing it.
With the enhanced tests in place, we can simplify yajl2
initialization by relying on parser defaults being sane.
* src/util/virjson.c (virJSONValueFromString): Simplify.
* tests/jsontest.c (mymain): Test more bad usage.
Signed-off-by: Eric Blake <eblake@redhat.com>
Since older yajl ignores trailing garbage, a client can cause
problems by intentionally ending the wrapper array early. Since
we already track nesting, it's not too much harder to reject
invalid nesting pops.
* src/util/virjson. (_virJSONParser): Add field.
(virJSONValueFromString): Set witness.
(virJSONParserHandleEndArray): Use it to catch abuse.
* tests/jsontest.c (mymain): Test it.
Signed-off-by: Eric Blake <eblake@redhat.com>
Yajl 2 has a nice feature that it can be configured whether to
allow multiple JSON objects parsed from a single stream, defaulting
to off. And yajl 1.0.12 at least provided a way to tell if all
input bytes were parsed, or if trailing bytes remained after a
valid JSON object was parsed. But we target RHEL 6 yajl 1.0.7,
which has neither of these. So fake it by always parsing '[...]'
instead, so that trailing garbage either trips up the array parse,
or is easily detected when unwrapping the result.
* src/util/virjson.c (virJSONValueFromString): With older json,
wrap text to avoid trailing garbage.
* tests/jsontest.c (mymain): Add tests for this.
Signed-off-by: Eric Blake <eblake@redhat.com>
We have been allowing javascript style comments in JSON ever
since commit 9428f2c (v0.7.5), but qemu doesn't send them, and
they are not strict JSON. Reject them for now; if we can later
prove that it is worthwhile, we can reinstate it at that point
(or even make it conditional, by adding a bool parameter to
the libvirt entry point).
* src/util/virjson.c (virJSONValueFromString): Don't enable
comment parsing.
* tests/jsontest.c (mymain): Test it.
Signed-off-by: Eric Blake <eblake@redhat.com>
Commit ceb496e5 fails on RHEL 6, with yajl 1.0.7, because that
version of yajl returns yajl_status_insufficient_data when the
parser is waiting for the rest of a token (this enum value was
dropped in yajl 2, so we have to wrap it). It also exposes a
problem where older yajl silently ignores trailing garbage after
a successful parse, so this patch works around that by changing
the testsuite. Another more invasive patch can add tighter
semantics to json parsing, but this is sufficient for a minimal
clean backport.
While touching this, fix up our error message cleanup. Yajl
documents that error messages produced by yajl_get_error()
MUST be cleaned with yajl_free_error(); this is certainly
true if we were to pass non-NULL allocator callbacks during
yajl_alloc(), but probably harmless in our usage of passing
NULL. But better safe than sorry.
* src/util/virjson.c (virJSONValueFromString): Allow different
error code. Use canonical cleanup of error message.
(VIR_YAJL_STATUS_OK): New helper macro.
* tests/jsontest.c (mymain): Wrap text to avoid difference in
trailing garbage handling
Signed-off-by: Eric Blake <eblake@redhat.com>
While working in qemu_monitor_json, I repeatedly found myself
getting a value then checking if it was an object. Add some
wrappers to make this task easier.
* src/util/virjson.c (virJSONValueObjectGetByType)
(virJSONValueObjectGetObject, virJSONValueObjectGetArray): New
functions.
(virJSONValueObjectGetString, virJSONValueObjectGetNumberInt)
(virJSONValueObjectGetNumberUint)
(virJSONValueObjectGetNumberLong)
(virJSONValueObjectGetNumberUlong)
(virJSONValueObjectGetNumberDouble)
(virJSONValueObjectGetBoolean): Simplify.
(virJSONValueIsNull): Change return type.
* src/util/virjson.h: Reflect changes.
* src/libvirt_private.syms (virjson.h): Export them.
* tests/jsontest.c (testJSONLookup): New test.
Signed-off-by: Eric Blake <eblake@redhat.com>
I was adding a JSON test, and was shocked to find out our parser
treated the input string of "1" as invalid JSON. It turns out
that YAJL specifically documents that it buffers input, and that
if the last input read could be a prefix to a longer token, then
you have to explicitly tell the parser that the buffer has ended
before that token will be processed.
It doesn't help that yajl 2 renamed the function from what it was
in yajl 1.
* src/util/virjson.c (virJSONValueFromString): Complete parse, in
case buffer ends in possible token prefix.
* tests/jsontest.c (mymain): Expose the problem.
Signed-off-by: Eric Blake <eblake@redhat.com>
This helper eases iterating all key=value pairs stored in a JSON
object. Usually we pick only certain known keys from a JSON object, but
this will allow to walk complete objects and have the callback act on
those.
To be able to easily represent nodesets and other data stored in
virBitmaps in libvirt, this patch introduces a set of helpers that allow
to convert the bitmap to and from JSON value objects.
The JSON structure constructor has an option to add JSON arrays to the
constructed object. The description is inaccurate as it can add any json
object even a dict. Change the docs to cover this option and reject
adding NULL objects.
Our qemu monitor code has a converter from key-value pairs to a json
value object. I want to re-use the code later and having it part of the
monitor command generator is inflexible. Split it out into a separate
helper.
Any source file which calls the logging APIs now needs
to have a VIR_LOG_INIT("source.name") declaration at
the start of the file. This provides a static variable
of the virLogSource type.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
If OOM or another error occurs in virJSONValueFromString the
parser state object will be leaked.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
If OOM occurs in virJSONParserHandleStartMap it will free
a variable that is owned by another object. This leads to
a later double-free.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Convert the type of loop iterators named 'i', 'j', k',
'ii', 'jj', 'kk', to be 'size_t' instead of 'int' or
'unsigned int', also santizing 'ii', 'jj', 'kk' to use
the normal 'i', 'j', 'k' naming
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
In an upcoming patch, I need the way to safely transfer a nested
virJSON object out of its parent container for independent use,
even after the parent is freed.
* src/util/virjson.h (virJSONValueObjectRemoveKey): New function.
(_virJSONObject, _virJSONArray): Use correct type.
* src/util/virjson.c (virJSONValueObjectRemoveKey): Implement it.
* src/libvirt_private.syms (virjson.h): Export it.
* tests/jsontest.c (mymain): Test it.
Signed-off-by: Eric Blake <eblake@redhat.com>
The source code base needs to be adapted as well. Some files
include virutil.h just for the string related functions (here,
the include is substituted to match the new file), some include
virutil.h without any need (here, the include is removed), and
some require both.
