summary refs log tree commit diff stats
path: root/scripts/qapi/doc.py (unfollow)
Commit message (Collapse)AuthorFilesLines
2019-09-24tests/qapi-schema: Demonstrate suboptimal lexical errorsMarkus Armbruster6-1/+5
The error message for forgotten quotes around a name shows just the name's first character, which isn't as nice as it could be. Same for attempting to use a number. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20190914153506.2151-7-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
2019-09-24tests/qapi-schema: Demonstrate insufficient 'if' checkingMarkus Armbruster17-0/+96
Cover invalid 'if' in struct members, features, union and alternate branches. Four out of four are broken. Mark FIXME. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20190914153506.2151-6-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> [Comment typo fixed]
2019-09-24tests/qapi-schema: Demonstrate broken discriminator errorsMarkus Armbruster4-10/+6
When the union definition's base is an object, some error messages show it as an OrderedDict. Oops. Mark FIXME. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20190914153506.2151-5-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
2019-09-24tests/qapi-schema: Demonstrate misleading optional tag errorMarkus Armbruster7-2/+17
Test flat-union-optional-discriminator declares its union tag as '*switch': 'Enum', and points to it with 'discriminator': '*switch'. This gets rejected as "discriminator of flat union 'MyUnion' uses invalid name '*switch'". Correct; member 'discriminator' doesn't accept a '*' prefix. However, this merely tests name validity checking, which we already cover elsewhere. More interesting is testing the valid name 'switch'. This reports "discriminator 'switch' is not a member of base struct 'Base'", which is misleading. Copy the existing 'discriminator': '*switch' test to flat-union-discriminator-bad-name, and rewrite its comment. Change flat-union-optional-discriminator to test 'discriminator': 'switch', and mark it FIXME. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20190914153506.2151-4-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
2019-09-24tests/qapi-schema: Delete two redundant testsMarkus Armbruster10-11/+1
Tests duplicate-key and double-data test the same thing. The former predates the latter, and it has a better name. Delete the latter, and tweak the former's comment. Tests include-format-err and include-extra-junk test the same thing. The former predates the latter, but the latter has a better name and a comment. Delete the former. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20190914153506.2151-3-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
2019-09-24tests/qapi-schema: Cover unknown pragmaMarkus Armbruster5-0/+4
Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20190914153506.2151-2-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
2019-09-24qapi: Tweak code to match docs/devel/qapi-code-gen.txtMarkus Armbruster7-18/+18
The previous commit made qapi-code-gen.txt define "(top-level) expression" as either "directive" or "definition". The code still uses "expression" when it really means "definition". Tidy up. The previous commit made qapi-code-gen.txt use "object" rather than "dictionary". The code still uses "dictionary". Tidy up. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20190913201349.24332-17-armbru@redhat.com>
2019-09-24docs/devel/qapi-code-gen: Improve QAPI schema language docMarkus Armbruster1-225/+373
We document the language by giving patterns of valid JSON objects. The patterns contain placeholders we don't define anywhere; their names have to speak for themselves. I guess they do, but I'd prefer a bit more rigor. Provide a grammar instead, and rework the text accordingly. Documentation for QAPI schema conditionals (commit 967c885108, 6cc32b0e14, 87adbbffd4..3e270dcacc) and feature flags (commit 6a8c0b5102) was bolted on. The sections documenting types, commands and events don't mention them. Section "Features" and "Configuring the schema" then provide additional syntax for types, commands and events. I hate that. Fix the sections documenting types, commands and events to provide accurate syntax, and point to "Features" and "Configuring the schema" for details. We talk about "(top-level) expressions other than include and pragma". Adopt more convenient terminology: a (top-level) expression is either a directive (include or pragma) or a definition (anything else). Avoid the terms "dictionary" and "key". Stick to JSON terminology "object" and "member name" instead. While there, make spacing more consistent. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20190913201349.24332-16-armbru@redhat.com>
2019-09-24docs/devel/qapi-code-gen: Rewrite introduction to schemaMarkus Armbruster1-59/+48
The introduction to the QAPI schema is somewhat rambling. Rewrite for clarity. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20190913201349.24332-15-armbru@redhat.com>
2019-09-24docs/devel/qapi-code-gen: Rewrite compatibility considerationsMarkus Armbruster1-35/+64
We have some compatibility advice buried in sections "Enumeration types" and "Struct types". Compatibility is actually about commands and events. It devolves to the types used there. All kinds of types, not just enumerations and structs. Replace the existing advice by a new section "Compatibility considerations". Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20190913201349.24332-14-armbru@redhat.com> [Squash in paragraph on invisible schema changes, as per Eric's review]
2019-09-24docs/devel/qapi-code-gen: Reorder sections for readabilityMarkus Armbruster1-215/+221
Section "QMP/Guest agent schema" starts with a brief introduction, then subsection "Comments", then subsection "Schema overview" (more elaborate introduction), and only then talks about schema entities like types, commands, and so forth. Subsection "Comments" is long and tiring: almost 500 words, mostly about doc comments. Move the doc comment part to its own subsection "Documentation comments" at the very end of "QMP/Guest agent schema". Subsection "Schema overview" explains naming rules at considerable length: 250 words. Move this part to its own subsection "Naming rules and reserved names" right after the subsections on schema entities. Subsection "Enumeration types" is wedged between "Struct types" and "Union types". Move it before "Struct types". Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20190913201349.24332-13-armbru@redhat.com>
2019-09-24qapi: Adjust frontend errors to say enum value, not memberMarkus Armbruster4-6/+11
For consistency with docs/devel/qapi-code-gen.txt. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20190913201349.24332-12-armbru@redhat.com>
2019-09-24qapi: Permit omitting all flat union branchesMarkus Armbruster8-14/+27
Absent flat union branches default to the empty struct (since commit 800877bb16 "qapi: allow empty branches in flat unions"). But an attempt to omit all of them is rejected with "Union 'FOO' has no branches". Harmless oddity, but it's easy to avoid, so do that. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20190913201349.24332-11-armbru@redhat.com> [Commit message typo fixed]
2019-09-24qapi: Permit alternates with just one branchMarkus Armbruster5-10/+12
A union or alternate without branches makes no sense and doesn't work: it can't be instantiated. A union or alternate with just one branch works, but is degenerate. We accept the former, but reject the latter. Weird. docs/devel/qapi-code-gen.txt doesn't mention the difference. It claims an alternate definition is "is similar to a simple union type". Permit degenerate alternates to make them consistent with unions. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20190913201349.24332-10-armbru@redhat.com>
2019-09-24qapi: Permit 'boxed' with empty typeMarkus Armbruster11-25/+19
We reject empty types with 'boxed': true. We don't really need that to work, but making it work is actually simpler than rejecting it, so do that. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20190913201349.24332-9-armbru@redhat.com>
2019-09-24qapi: Drop support for escape sequences other than \\Markus Armbruster19-60/+7
Since the previous commit restricted strings to printable ASCII, \uXXXX's only use is obfuscation. Drop it. This leaves \\, \/, \', and \". Since QAPI schema strings are all names, and names are restricted to ASCII letters, digits, hyphen, and underscore, none of them is useful. The latter three have no test coverage. Drop them. Keep \\ to avoid (more) gratuitous incompatibility with JSON. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20190913201349.24332-8-armbru@redhat.com>
2019-09-24qapi: Restrict strings to printable ASCIIMarkus Armbruster12-21/+20
RFC 8259 on string contents: All Unicode characters may be placed within the quotation marks, except for the characters that MUST be escaped: quotation mark, reverse solidus, and the control characters (U+0000 through U+001F). The QAPI schema parser accepts both less and more than JSON: it accepts only ASCII with \u (less), and accepts control characters other than LF (new line) unescaped. How it treats unescaped non-ASCII input differs between Python 2 and Python 3. Make it accept strictly less: require printable ASCII. Drop support for \b, \f, \n, \r, \t. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20190913201349.24332-7-armbru@redhat.com>
2019-09-24tests/qapi-schema: Demonstrate bad reporting of funny charactersMarkus Armbruster2-2/+3
Invalid name 'not\\possible' is reported as 'not\possible'. Control characters (quoted or not) are even more confusing. Mark FIXME. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20190913201349.24332-6-armbru@redhat.com>
2019-09-24docs/devel/qapi-code-gen: Minor specification fixesMarkus Armbruster1-18/+16
The specification claims "Each expression that isn't an include directive may be preceded by a documentation block", but the code also rejects them for pragma directives. The code is correct. Fix the specification. The specification reserves member names starting with 'has_', but the code also reserves name 'u'. Fix the specification. The specification claims "The string 'max' is not allowed as an enum value". Untrue. Fix the specification. While there, delete the naming advice, because it's redundant with the naming rules in section "Schema overview" The specification claims "No branch of the union can be named 'max', as this would collide with the implicit enum". Untrue. Fix the specification. The specification claims "It is not allowed to name an event 'MAX', since the generator also produces a C enumeration of all event names with a generated _MAX value at the end." Untrue. Fix the specification. The specification claims "All branches of the union must be complex types", but the code permits only struct types. The code is correct. Fix the specification. The specification claims a command's return type "must be the string name of a complex or built-in type, a one-element array containing the name of a complex or built-in type" unless the command is in pragma 'returns-whitelist'. The code does not permit built-in types. Fix the specification. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20190913201349.24332-5-armbru@redhat.com>
2019-09-24qapi: Drop support for boxed alternate argumentsMarkus Armbruster4-19/+12
Commands and events can define their argument type inline (default) or by referring to another type ('boxed': true, since commit c818408e44 "qapi: Implement boxed types for commands/events", v2.7.0). The unboxed inline definition is an (anonymous) struct type. The boxed type may be a struct, union, or alternate type. The latter is problematic: docs/interop/qemu-spec.txt requires the value of the 'data' key to be a json-object, but any non-degenerate alternate type has at least one branch that isn't. Fortunately, we haven't made use of alternates in this context outside tests/. Drop support for them. QAPISchemaAlternateType.is_empty() is now unused. Drop it, too. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20190913201349.24332-4-armbru@redhat.com>
2019-09-24qapi: Drop check_type()'s redundant parameter @allow_optionalMarkus Armbruster1-9/+8
check_type() uses @allow_optional only when @value is a dictionary and @allow_dict is True. All callers that pass allow_dict=True also pass allow_optional=True. Therefore, @allow_optional is always True when check_type() uses it. Drop the redundant parameter. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20190913201349.24332-3-armbru@redhat.com>
2019-09-24scripts/git.orderfile: Match QAPI schema more preciselyMarkus Armbruster1-2/+2
Pattern *.json also matches the tests/qapi-schema/*.json. Separates them from the tests/qapi-schema/*.{err,exit,out} in diffs. I hate that. Change the pattern to match just the "real" QAPI schemata. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-Id: <20190913201349.24332-2-armbru@redhat.com>
2019-09-24make check-unit: use after free in test-opts-visitorAndrey Shinkevich1-4/+22
In the struct OptsVisitor, the 'repeated_opts' member points to a list in the 'unprocessed_opts' hash table after the list has been destroyed. A subsequent call to visit_type_int() references the deleted list. It results in use-after-free issue reproduced by running the test case under the Valgrind: valgrind tests/test-opts-visitor. A new mode ListMode::LM_TRAVERSED is declared to mark the list traversal completed. Suggested-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com> Message-Id: <1565024586-387112-1-git-send-email-andrey.shinkevich@virtuozzo.com>
2019-09-24qapi: Make visit_next_list()'s comment less confusingMarkus Armbruster1-4/+4
visit_next_list() returns non-null on success, null on failure. The comment's phrasing "until NULL return or error occurs" is needlessly confusing. Scratch the "or error occurs" part. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20190802122325.16520-1-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
2019-09-23tests/tcg: target/s390x: Test MVCDavid Hildenbrand2-0/+110
Let's add a test that especially verifies that no data will be touched in case we cross page boundaries and one page access triggers a fault. Before the fault-safe handling fixes, the test failes with: TEST mvc on s390x data modified during a fault make[2]: *** [../Makefile.target:116: run-mvc] Error 1 Acked-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: David Hildenbrand <david@redhat.com>
2019-09-23tests/tcg: target/s390x: Test MVODavid Hildenbrand2-0/+26
Let's add the simple test based on the example from the PoP. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Signed-off-by: David Hildenbrand <david@redhat.com>
2019-09-23s390x/tcg: MVO: Fault-safe handlingDavid Hildenbrand1-12/+15
Each operand can have a maximum length of 16. Make sure to prepare all reads/writes before writing. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: David Hildenbrand <david@redhat.com>
2019-09-23s390x/tcg: MVST: Fault-safe handlingDavid Hildenbrand1-7/+17
Access at most single pages and document why. Using the access helpers might over-indicate watchpoints within the same page, I guess we can live with that. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: David Hildenbrand <david@redhat.com>
2019-09-23s390x/tcg: MVZ: Fault-safe handlingDavid Hildenbrand1-4/+13
We can process a maximum of 256 bytes, crossing two pages. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: David Hildenbrand <david@redhat.com>
2019-09-23s390x/tcg: MVN: Fault-safe handlingDavid Hildenbrand1-4/+13
We can process a maximum of 256 bytes, crossing two pages. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: David Hildenbrand <david@redhat.com>
2019-09-23s390x/tcg: MVCIN: Fault-safe handlingDavid Hildenbrand1-3/+12
We can process a maximum of 256 bytes, crossing two pages. Calculate the accessed range upfront - src is accessed right-to-left. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: David Hildenbrand <david@redhat.com>
2019-09-23s390x/tcg: NC: Fault-safe handlingDavid Hildenbrand1-4/+13
We can process a maximum of 256 bytes, crossing two pages. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: David Hildenbrand <david@redhat.com>
2019-09-23s390x/tcg: XC: Fault-safe handlingDavid Hildenbrand1-6/+12
We can process a maximum of 256 bytes, crossing two pages. While at it, increment the length once. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: David Hildenbrand <david@redhat.com>
2019-09-23s390x/tcg: OC: Fault-safe handlingDavid Hildenbrand1-4/+13
We can process a maximum of 256 bytes, crossing two pages. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: David Hildenbrand <david@redhat.com>
2019-09-23s390x/tcg: MVCLU: Fault-safe handlingDavid Hildenbrand1-3/+5
The last remaining bit is padding with two bytes. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: David Hildenbrand <david@redhat.com>
2019-09-23s390x/tcg: MVC: Fault-safe handling on destructive overlapsDavid Hildenbrand1-2/+3
The last remaining bit for MVC is handling destructive overlaps in a fault-safe way. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: David Hildenbrand <david@redhat.com>
2019-09-23s390x/tcg: MVCS/MVCP: Use access_memmove()David Hildenbrand1-14/+12
As we are moving between address spaces, we can use access_memmove() without checking for destructive overlaps (especially of real storage locations): "Each storage operand is processed left to right. The storage-operand-consistency rules are the same as for MOVE (MVC), except that when the operands overlap in real storage, the use of the common real- storage locations is not necessarily recognized." Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: David Hildenbrand <david@redhat.com>
2019-09-23s390x/tcg: Fault-safe memmoveDavid Hildenbrand1-99/+139
Replace fast_memmove() variants by access_memmove() variants, that first try to probe access to all affected pages (maximum is two pages). Introduce access_get_byte()/access_set_byte(). We might be able to speed up memmove in special cases even further (do single-byte access, use memmove() for remaining bytes in page), however, we'll skip that for now. In MVCOS, simply always call access_memmove_as() and drop the TODO about LAP. LAP is already handled in the MMU. Get rid of adj_len_to_page(), which is now unused. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: David Hildenbrand <david@redhat.com>
2019-09-23s390x/tcg: Fault-safe memsetDavid Hildenbrand1-20/+103
Replace fast_memset() by access_memset(), that first tries to probe access to all affected pages (maximum is two). We'll use the same mechanism for other types of accesses soon. Only in very rare cases (especially TLB_NOTDIRTY), we'll have to fallback to ld/st helpers. Try to speed up that case as suggested by Richard. We'll rework most involved handlers soon to do all accesses via new fault-safe helpers, especially MVC. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: David Hildenbrand <david@redhat.com>
2019-09-23s390x/tcg: Always use MMU_USER_IDX for CONFIG_USER_ONLYDavid Hildenbrand2-0/+8
Although we basically ignore the index all the time for CONFIG_USER_ONLY, let's simply skip all the checks and always return MMU_USER_IDX in cpu_mmu_index() and get_mem_index(). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: David Hildenbrand <david@redhat.com>
2019-09-23s390x/tcg: MVST: Fix storing back the addresses to registersDavid Hildenbrand4-19/+19
24 and 31-bit address space handling is wrong when it comes to storing back the addresses to the register. While at it, read gprs 0 implicitly. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: David Hildenbrand <david@redhat.com>
2019-09-23s390x/tcg: MVST: Check for specification exceptionsDavid Hildenbrand1-0/+3
Bit position 32-55 of general register 0 must be zero. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: David Hildenbrand <david@redhat.com>
2019-09-23s390x/tcg: MVCS/MVCP: Properly wrap the lengthDavid Hildenbrand1-0/+6
... and don't perform any move in case the length is zero. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: David Hildenbrand <david@redhat.com>
2019-09-23s390x/tcg: MVCOS: Lengths are 32 bit in 24/31-bit modeDavid Hildenbrand1-3/+11
Triggered by a review comment from Richard, also MVCOS has a 32-bit length in 24/31-bit addressing mode. Add a new helper. Rename wrap_length() to wrap_length31(). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: David Hildenbrand <david@redhat.com>
2019-09-23s390x/tcg: MVCS/MVCP: Check for special operation exceptionsDavid Hildenbrand1-0/+12
Let's perform the documented checks. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: David Hildenbrand <david@redhat.com>
2019-09-23s390x/tcg: MVCLU/MVCLE: Process max 4k bytes at a timeDavid Hildenbrand1-23/+31
Let's stay within single pages. ... and indicate cc=3 in case there is work remaining. Keep unicode padding simple. While reworking, properly wrap the addresses. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: David Hildenbrand <david@redhat.com>
2019-09-23s390x/tcg: MVPG: Properly wrap the addressesDavid Hildenbrand1-2/+9
We have to mask of any unused bits. While at it, document what exactly is missing. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: David Hildenbrand <david@redhat.com>
2019-09-23s390x/tcg: MVPG: Check for specification exceptionsDavid Hildenbrand1-0/+7
Perform the checks documented in the PoP. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: David Hildenbrand <david@redhat.com>
2019-09-23s390x/tcg: MVC: Use is_destructive_overlap()David Hildenbrand1-1/+1
Let's use the new helper, that also detects destructive overlaps when wrapping. We'll make the remaining code (e.g., fast_memmove()) aware of wrapping later. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: David Hildenbrand <david@redhat.com>
2019-09-23s390x/tcg: MVC: Increment the length onceDavid Hildenbrand1-8/+12
Let's increment the length once. While at it, cleanup the comment. The memset() example is given as a programming note in the PoP, so drop the description. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: David Hildenbrand <david@redhat.com>