summary refs log tree commit diff stats
Commit message (Collapse)AuthorAgeFilesLines
* qcrypto-luks: more rigorous header checkingMaxim Levitsky2019-09-261-0/+52
| | | | | | | | | | | Check that keyslots don't overlap with the data, and check that keyslots don't overlap with each other. (this is done using naive O(n^2) nested loops, but since there are just 8 keyslots, this doesn't really matter. Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
* qcrypto-luks: simplify the math used for keyslot locationsMaxim Levitsky2019-09-261-23/+40
| | | | | | Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
* qcrypto-luks: extract store key functionMaxim Levitsky2019-09-261-123/+181
| | | | | | | | | This function will be used later to store new keys to the luks metadata Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
* qcrypto-luks: extract check and parse headerMaxim Levitsky2019-09-261-98/+125
| | | | | | | | | This is just to make qcrypto_block_luks_open more reasonable in size. Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
* qcrypto-luks: extract store and load headerMaxim Levitsky2019-09-261-62/+93
| | | | | | Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
* qcrypto-luks: purge unused error codes from open callbackMaxim Levitsky2019-09-261-32/+13
| | | | | | | | These values are not used by generic crypto code anyway Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
* qcrypto-luks: use the parsed encryption settings in QCryptoBlockLUKSMaxim Levitsky2019-09-261-90/+79
| | | | | | | | | | | | Prior to that patch, the parsed encryption settings were already stored into the QCryptoBlockLUKS but not used anywhere but in qcrypto_block_luks_get_info Using them simplifies the code Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
* qcrypto-luks: pass keyslot index rather that pointer to the keyslotMaxim Levitsky2019-09-261-3/+3
| | | | | | | | Another minor refactoring Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
* qcrypto-luks: simplify masterkey and masterkey lengthMaxim Levitsky2019-09-261-23/+21
| | | | | | | | | Let the caller allocate masterkey Always use master key len from the header Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
* qcrypto-luks: don't overwrite cipher_mode in headerMaxim Levitsky2019-09-261-3/+6
| | | | | | | | | This way we can store the header we loaded, which will be used in key management code Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
* qcrypto-luks: rename some fields in QCryptoBlockLUKSHeaderMaxim Levitsky2019-09-261-44/+47
| | | | | | | | | | * key_bytes -> master_key_len * payload_offset = payload_offset_sector (to emphasise that this isn't byte offset) * key_offset -> key_offset_sector - same as above for luks slots Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
* Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2019-09-24-v2' ↵Peter Maydell2019-09-268-10/+47
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | into staging nbd patches for 2019-09-24 - Improved error message for plaintext client of encrypted server - Fix various assertions when -object iothread is in use - Silence a Coverity error for use-after-free on error path # gpg: Signature made Wed 25 Sep 2019 14:35:52 BST # gpg: using RSA key 71C2CC22B1C4602927D2F3AAA7A16B4A2527436A # gpg: Good signature from "Eric Blake <eblake@redhat.com>" [full] # gpg: aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>" [full] # gpg: aka "[jpeg image of size 6874]" [full] # Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2 F3AA A7A1 6B4A 2527 436A * remotes/ericb/tags/pull-nbd-2019-09-24-v2: util/qemu-sockets: fix keep_alive handling in inet_connect_saddr tests: Use iothreads during iotest 223 nbd: Grab aio context lock in more places nbd/server: attach client channel to the export's AioContext nbd/client: Add hint when TLS is missing Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
| * util/qemu-sockets: fix keep_alive handling in inet_connect_saddrVladimir Sementsov-Ogievskiy2019-09-251-2/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | In "if (saddr->keep_alive) {" we may already be on error path, with invalid sock < 0. Fix it by returning error earlier. Reported-by: Coverity (CID 1405300) Fixes: aec21d31756cbd Suggested-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <20190910075943.12977-1-vsementsov@virtuozzo.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Signed-off-by: Eric Blake <eblake@redhat.com>
| * tests: Use iothreads during iotest 223Eric Blake2019-09-242-2/+5
| | | | | | | | | | | | | | | | Doing so catches the bugs we just fixed with NBD not properly using correct contexts. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <20190920220729.31801-1-eblake@redhat.com>
| * nbd: Grab aio context lock in more placesEric Blake2019-09-243-6/+31
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When iothreads are in use, the failure to grab the aio context results in an assertion failure when trying to unlock things during blk_unref, when trying to unlock a mutex that was not locked. In short, all calls to nbd_export_put need to done while within the correct aio context. But since nbd_export_put can recursively reach itself via nbd_export_close, and recursively grabbing the context would deadlock, we can't do the context grab directly in those functions, but must do so in their callers. Hoist the use of the correct aio_context from nbd_export_new() to its caller qmp_nbd_server_add(). Then tweak qmp_nbd_server_remove(), nbd_eject_notifier(), and nbd_esport_close_all() to grab the right context, so that all callers during qemu now own the context before nbd_export_put() can call blk_unref(). Remaining uses in qemu-nbd don't matter (since that use case does not support iothreads). Suggested-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <20190917023917.32226-1-eblake@redhat.com> Reviewed-by: Sergio Lopez <slp@redhat.com>
| * nbd/server: attach client channel to the export's AioContextSergio Lopez2019-09-241-0/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | On creation, the export's AioContext is set to the same one as the BlockBackend, while the AioContext in the client QIOChannel is left untouched. As a result, when using data-plane, nbd_client_receive_next_request() schedules coroutines in the IOThread AioContext, while the client's QIOChannel is serviced from the main_loop, potentially triggering the assertion at qio_channel_restart_[read|write]. To fix this, as soon we have the export corresponding to the client, we call qio_channel_attach_aio_context() to attach the QIOChannel context to the export's AioContext. This matches with the logic at blk_aio_attached(). RHBZ: https://bugzilla.redhat.com/show_bug.cgi?id=1748253 Signed-off-by: Sergio Lopez <slp@redhat.com> Message-Id: <20190912110032.26395-1-slp@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Eric Blake <eblake@redhat.com>
| * nbd/client: Add hint when TLS is missingEric Blake2019-09-242-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | I received an off-list report of failure to connect to an NBD server expecting an x509 certificate, when the client was attempting something similar to this command line: $ ./x86_64-softmmu/qemu-system-x86_64 -name 'blah' -machine q35 -nodefaults \ -object tls-creds-x509,id=tls0,endpoint=client,dir=$path_to_certs \ -device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pcie.0,addr=0x6 \ -drive id=drive_image1,if=none,snapshot=off,aio=threads,cache=none,format=raw,file=nbd:localhost:9000,werror=stop,rerror=stop,tls-creds=tls0 \ -device scsi-hd,id=image1,drive=drive_image1,bootindex=0 qemu-system-x86_64: -drive id=drive_image1,if=none,snapshot=off,aio=threads,cache=none,format=raw,file=nbd:localhost:9000,werror=stop,rerror=stop,tls-creds=tls0: TLS negotiation required before option 7 (go) server reported: Option 0x7 not permitted before TLS The problem? As specified, -drive is trying to pass tls-creds to the raw format driver instead of the nbd protocol driver, but before we get to the point where we can detect that raw doesn't know what to do with tls-creds, the nbd driver has already failed because the server complained. The fix to the broken command line? Pass '...,file.tls-creds=tls0' to ensure the tls-creds option is handed to nbd, not raw. But since the error message was rather cryptic, I'm trying to improve the error message. With this patch, the error message adds a line: qemu-system-x86_64: -drive id=drive_image1,if=none,snapshot=off,aio=threads,cache=none,format=raw,file=nbd:localhost:9000,werror=stop,rerror=stop,tls-creds=tls0: TLS negotiation required before option 7 (go) Did you forget a valid tls-creds? server reported: Option 0x7 not permitted before TLS And with luck, someone grepping for that error message will find this commit message and figure out their command line mistake. Sadly, the only mention of file.tls-creds in our docs relates to an --image-opts use of PSK encryption with qemu-img as the client, rather than x509 certificate encryption with qemu-kvm as the client. CC: Tingting Mao <timao@redhat.com> CC: Daniel P. Berrangé <berrange@redhat.com> Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <20190907172055.26870-1-eblake@redhat.com> [eblake: squash in iotest 233 fix] Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
* | Merge remote-tracking branch 'remotes/armbru/tags/pull-qapi-2019-09-24' into ↵Peter Maydell2019-09-25111-781/+967
|\ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | staging QAPI patches for 2019-09-24 # gpg: Signature made Tue 24 Sep 2019 13:10:36 BST # gpg: using RSA key 354BC8B3D7EB2A6B68674E5F3870B400EB918653 # gpg: issuer "armbru@redhat.com" # gpg: Good signature from "Markus Armbruster <armbru@redhat.com>" [full] # gpg: aka "Markus Armbruster <armbru@pond.sub.org>" [full] # Primary key fingerprint: 354B C8B3 D7EB 2A6B 6867 4E5F 3870 B400 EB91 8653 * remotes/armbru/tags/pull-qapi-2019-09-24: (37 commits) qapi: Assert .visit() and .check_clash() run only after .check() qapi: Fix excessive QAPISchemaEntity.check() recursion qapi: Fix to .check() empty structs just once qapi: Delete useless check_exprs() code for simple union kind qapi: Clean up around check_known_keys() qapi: Simplify check_keys() qapi: Normalize 'if' in check_exprs(), like other sugar qapi: Fix missing 'if' checks in struct, union, alternate 'data' qapi: Reject blank 'if' conditions in addition to empty ones qapi: Fix broken discriminator error messages qapi: Remove null from schema language qapi: Improve reporting of lexical errors qapi: Use quotes more consistently in frontend error messages tests/qapi-schema: Demonstrate suboptimal lexical errors tests/qapi-schema: Demonstrate insufficient 'if' checking tests/qapi-schema: Demonstrate broken discriminator errors tests/qapi-schema: Demonstrate misleading optional tag error tests/qapi-schema: Delete two redundant tests tests/qapi-schema: Cover unknown pragma qapi: Tweak code to match docs/devel/qapi-code-gen.txt ... Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
| * | qapi: Assert .visit() and .check_clash() run only after .check()Markus Armbruster2019-09-241-1/+10
| | | | | | | | | | | | | | | | | | | | | | | | Easy since the previous commit provides .checked. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20190914153506.2151-20-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
| * | qapi: Fix excessive QAPISchemaEntity.check() recursionMarkus Armbruster2019-09-241-22/+52
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Entity checking goes back to commit ac88219a6c "qapi: New QAPISchema intermediate representation", v2.5.0. It's designed to work as follows: QAPISchema.check() calls .check() for all the schema's entities. An entity's .check() recurses into another entity's .check() only if the C struct generated for the former contains the C struct generated for the latter (pointers don't count). This is used to detect "object contains itself". There are two instances of this: * An object's C struct contains its base's C struct QAPISchemaObjectType.check() calls self.base.check() * An object's C struct contains its variants' C structs QAPISchemaObjectTypeVariants().check calls v.type.check(). Since commit b807a1e1e3 "qapi: Check for QAPI collisions involving variant members", v2.6.0. Thus, only object types can participate in recursion. QAPISchemaObjectType.check() is made for that: it checks @self when called the first time, recursing into base and variants, it reports an "contains itself" error when this recursion reaches an object being checked, and does nothing it reaches an object that has been checked already. The other .check() may safely assume they get called exactly once. Sadly, this design has since eroded: * QAPISchemaCommand.check() and QAPISchemaEvent.check() call .args_type.check(). Since commit c818408e44 "qapi: Implement boxed types for commands/events", v2.7.0. Harmless, since args_type can only be an object type. * QAPISchemaEntity.check() calls ._ifcond.check() when inheriting the condition from another type. Since commit 4fca21c1b0 qapi: leave the ifcond attribute undefined until check(), v3.0.0. This makes simple union wrapper types recurse into the wrapped type (nothing else uses this condition inheritance). The .check() of types used as simple union branch type get called multiple times. * QAPISchemaObjectType.check() calls its super type's .check() *before* the conditional handling multiple calls. Also since commit 4fca21c1b0. QAPISchemaObjectType.check()'s guard against multiple checking doesn't protect QAPISchemaEntity.check(). * QAPISchemaArrayType.check() calls .element_type.check(). Also since commit 4fca21c1b0. The .check() of types used as array element types get called multiple times. Commit 56a4689582 "qapi: Fix array first used in a different module" (v4.0.0) added more code relying on this .element_type.check(). The absence of explosions suggests the .check() involved happen to be effectively idempotent. Fix the unwanted recursion anyway: * QAPISchemaCommand.check() and QAPISchemaEvent.check() calling .args_type.check() is unnecessary. Delete the calls. * Fix QAPISchemaObjectType.check() to call its super type's .check() after the conditional handling multiple calls. * A QAPISchemaEntity's .ifcond becomes valid at .check(). This is due to arrays and simple unions. Most types get ifcond and info passed to their constructor. Array types don't: they get it from their element type, which becomes known only in .element_type.check(). The implicit wrapper object types for simple union branches don't: they get it from the wrapped type, which might be an array. Ditch the idea to set .ifcond in .check(). Instead, turn it into a property and compute it on demand. Safe because it's only used after the schema has been checked. Most types simply return the ifcond passed to their constructor. Array types forward to their .element_type instead, and the wrapper types forward to the wrapped type. * A QAPISchemaEntity's .module becomes valid at .check(). This is because computing it needs info and schema.fname, and because array types get it from their element type instead. Make it a property just like .ifcond. Additionally, have QAPISchemaEntity.check() assert it gets called at most once, so the design invariant will stick this time. Neglecting that was entirely my fault. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20190914153506.2151-19-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> [Commit message tidied up]
| * | qapi: Fix to .check() empty structs just onceMarkus Armbruster2019-09-241-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | QAPISchemaObjectType.check() does nothing for types that have been checked already. Except the "has been checked" predicate is broken for empty types: self.members is [] then, which isn't true. The bug is harmless, but fix it anyway: use self.member is not None instead. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20190914153506.2151-18-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
| * | qapi: Delete useless check_exprs() code for simple union kindMarkus Armbruster2019-09-241-37/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Commit bceae7697f "qapi script: support enum type as discriminator in union" made check_exprs() add the implicit enum types of simple unions to global @enum_types. I'm not sure it was needed even then. It's certainly not needed now. Delete it. discriminator_find_enum_define() and add_name() parameter @implicit are now dead. Bury them. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20190914153506.2151-17-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
| * | qapi: Clean up around check_known_keys()Markus Armbruster2019-09-241-4/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | All callers pass a dict argument to @keys, except check_keys() passes a dict's .keys(). Drop .keys() there, and rename parameter @keys to @value. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20190914153506.2151-16-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
| * | qapi: Simplify check_keys()Markus Armbruster2019-09-241-11/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | check_keys() parameter expr_elem expects a dictionary with keys 'expr' and 'info'. Passing the two values separately is simpler, so do that. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20190914153506.2151-15-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
| * | qapi: Normalize 'if' in check_exprs(), like other sugarMarkus Armbruster2019-09-241-11/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We normalize shorthand to longhand forms in check_expr(): enumeration values with normalize_enum(), feature values with normalize_features(), struct members, union branches and alternate branches with normalize_members(). If conditions are an exception: we normalize them in QAPISchemaEntity.check() and QAPISchemaMember.__init(), with listify_cond(). The idea goes back to commit 2cbc94376e "qapi: pass 'if' condition into QAPISchemaEntity objects", v3.0.0. Normalize in check_expr() instead, with new helper normalize_if(). Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20190914153506.2151-14-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
| * | qapi: Fix missing 'if' checks in struct, union, alternate 'data'Markus Armbruster2019-09-2413-60/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Commit 87adbbffd4..3e270dcacc "qapi: Add 'if' to (implicit struct|union|alternate) members" (v4.0.0) neglected test coverage, and promptly failed to check the conditions. Review fail. Recent commit "tests/qapi-schema: Demonstrate insufficient 'if' checking" added test coverage, demonstrating the bug. Fix it by add the missing check_if(). Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20190914153506.2151-13-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
| * | qapi: Reject blank 'if' conditions in addition to empty onesMarkus Armbruster2019-09-243-4/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | "'if': 'COND'" generates "#if COND". We reject empty COND because it won't compile. Blank COND won't compile any better, so reject that, too. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20190914153506.2151-12-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
| * | qapi: Fix broken discriminator error messagesMarkus Armbruster2019-09-247-11/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | check_union() checks the discriminator exists in base and makes sense. Two error messages mention the base. These are broken for anonymous bases, as demonstrated by tests flat-union-invalid-discriminator and flat-union-invalid-if-discriminator.err. The third one doesn't bother. First broken when commit ac4338f8eb "qapi: Allow anonymous base for flat union" (v2.6.0) neglected to adjust the "not a member of base" error message. Commit ccadd6bcba "qapi: Add 'if' to implicit struct members" (v4.0.0) then cloned the flawed error message. Dumb them down not to mention the base. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20190914153506.2151-11-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
| * | qapi: Remove null from schema languageMarkus Armbruster2019-09-247-24/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We represent the parse tree as OrderedDict. We fetch optional dict members with .get(). So far, so good. We represent null literals as None. .get() returns None both for "absent" and for "present, value is the null literal". Uh-oh. Test features-if-invalid exposes this bug: "'if': null" is misinterpreted as absent "if". We added null to the schema language to "allow [...] an explicit default value" (commit e53188ada5 "qapi: Allow true, false and null in schema json", v2.4.0). Hasn't happened; null is still unused except as generic invalid value in tests/. To fix, we'd have to replace .get() by something more careful, or represent null differently. Feasible, but we got more and bigger fish to fry right now. Remove the null literal from the schema language. Replace null in tests by another invalid value. Test features-if-invalid now behaves as it should. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20190914153506.2151-10-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
| * | qapi: Improve reporting of lexical errorsMarkus Armbruster2019-09-243-3/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Show text up to next structural character, whitespace, or quote character instead of just the first character. Forgotten quotes now get reported like "Stray 'command'" instead of "Stray 'c'". Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20190914153506.2151-9-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
| * | qapi: Use quotes more consistently in frontend error messagesMarkus Armbruster2019-09-2420-36/+38
| | | | | | | | | | | | | | | | | | | | | | | | | | | Consistently enclose error messages in double quotes. Use single quotes within, except for one case of "'". Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20190914153506.2151-8-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
| * | tests/qapi-schema: Demonstrate suboptimal lexical errorsMarkus Armbruster2019-09-246-1/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The error message for forgotten quotes around a name shows just the name's first character, which isn't as nice as it could be. Same for attempting to use a number. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20190914153506.2151-7-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
| * | tests/qapi-schema: Demonstrate insufficient 'if' checkingMarkus Armbruster2019-09-2417-0/+96
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Cover invalid 'if' in struct members, features, union and alternate branches. Four out of four are broken. Mark FIXME. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20190914153506.2151-6-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> [Comment typo fixed]
| * | tests/qapi-schema: Demonstrate broken discriminator errorsMarkus Armbruster2019-09-244-10/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | When the union definition's base is an object, some error messages show it as an OrderedDict. Oops. Mark FIXME. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20190914153506.2151-5-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
| * | tests/qapi-schema: Demonstrate misleading optional tag errorMarkus Armbruster2019-09-247-2/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Test flat-union-optional-discriminator declares its union tag as '*switch': 'Enum', and points to it with 'discriminator': '*switch'. This gets rejected as "discriminator of flat union 'MyUnion' uses invalid name '*switch'". Correct; member 'discriminator' doesn't accept a '*' prefix. However, this merely tests name validity checking, which we already cover elsewhere. More interesting is testing the valid name 'switch'. This reports "discriminator 'switch' is not a member of base struct 'Base'", which is misleading. Copy the existing 'discriminator': '*switch' test to flat-union-discriminator-bad-name, and rewrite its comment. Change flat-union-optional-discriminator to test 'discriminator': 'switch', and mark it FIXME. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20190914153506.2151-4-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
| * | tests/qapi-schema: Delete two redundant testsMarkus Armbruster2019-09-2410-11/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Tests duplicate-key and double-data test the same thing. The former predates the latter, and it has a better name. Delete the latter, and tweak the former's comment. Tests include-format-err and include-extra-junk test the same thing. The former predates the latter, but the latter has a better name and a comment. Delete the former. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20190914153506.2151-3-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
| * | tests/qapi-schema: Cover unknown pragmaMarkus Armbruster2019-09-245-0/+4
| | | | | | | | | | | | | | | | | | Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20190914153506.2151-2-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
| * | qapi: Tweak code to match docs/devel/qapi-code-gen.txtMarkus Armbruster2019-09-247-18/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The previous commit made qapi-code-gen.txt define "(top-level) expression" as either "directive" or "definition". The code still uses "expression" when it really means "definition". Tidy up. The previous commit made qapi-code-gen.txt use "object" rather than "dictionary". The code still uses "dictionary". Tidy up. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20190913201349.24332-17-armbru@redhat.com>
| * | docs/devel/qapi-code-gen: Improve QAPI schema language docMarkus Armbruster2019-09-241-225/+373
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We document the language by giving patterns of valid JSON objects. The patterns contain placeholders we don't define anywhere; their names have to speak for themselves. I guess they do, but I'd prefer a bit more rigor. Provide a grammar instead, and rework the text accordingly. Documentation for QAPI schema conditionals (commit 967c885108, 6cc32b0e14, 87adbbffd4..3e270dcacc) and feature flags (commit 6a8c0b5102) was bolted on. The sections documenting types, commands and events don't mention them. Section "Features" and "Configuring the schema" then provide additional syntax for types, commands and events. I hate that. Fix the sections documenting types, commands and events to provide accurate syntax, and point to "Features" and "Configuring the schema" for details. We talk about "(top-level) expressions other than include and pragma". Adopt more convenient terminology: a (top-level) expression is either a directive (include or pragma) or a definition (anything else). Avoid the terms "dictionary" and "key". Stick to JSON terminology "object" and "member name" instead. While there, make spacing more consistent. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20190913201349.24332-16-armbru@redhat.com>
| * | docs/devel/qapi-code-gen: Rewrite introduction to schemaMarkus Armbruster2019-09-241-59/+48
| | | | | | | | | | | | | | | | | | | | | | | | | | | The introduction to the QAPI schema is somewhat rambling. Rewrite for clarity. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20190913201349.24332-15-armbru@redhat.com>
| * | docs/devel/qapi-code-gen: Rewrite compatibility considerationsMarkus Armbruster2019-09-241-35/+64
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We have some compatibility advice buried in sections "Enumeration types" and "Struct types". Compatibility is actually about commands and events. It devolves to the types used there. All kinds of types, not just enumerations and structs. Replace the existing advice by a new section "Compatibility considerations". Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20190913201349.24332-14-armbru@redhat.com> [Squash in paragraph on invisible schema changes, as per Eric's review]
| * | docs/devel/qapi-code-gen: Reorder sections for readabilityMarkus Armbruster2019-09-241-215/+221
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Section "QMP/Guest agent schema" starts with a brief introduction, then subsection "Comments", then subsection "Schema overview" (more elaborate introduction), and only then talks about schema entities like types, commands, and so forth. Subsection "Comments" is long and tiring: almost 500 words, mostly about doc comments. Move the doc comment part to its own subsection "Documentation comments" at the very end of "QMP/Guest agent schema". Subsection "Schema overview" explains naming rules at considerable length: 250 words. Move this part to its own subsection "Naming rules and reserved names" right after the subsections on schema entities. Subsection "Enumeration types" is wedged between "Struct types" and "Union types". Move it before "Struct types". Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20190913201349.24332-13-armbru@redhat.com>
| * | qapi: Adjust frontend errors to say enum value, not memberMarkus Armbruster2019-09-244-6/+11
| | | | | | | | | | | | | | | | | | | | | | | | For consistency with docs/devel/qapi-code-gen.txt. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20190913201349.24332-12-armbru@redhat.com>
| * | qapi: Permit omitting all flat union branchesMarkus Armbruster2019-09-248-14/+27
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Absent flat union branches default to the empty struct (since commit 800877bb16 "qapi: allow empty branches in flat unions"). But an attempt to omit all of them is rejected with "Union 'FOO' has no branches". Harmless oddity, but it's easy to avoid, so do that. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20190913201349.24332-11-armbru@redhat.com> [Commit message typo fixed]
| * | qapi: Permit alternates with just one branchMarkus Armbruster2019-09-245-10/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | A union or alternate without branches makes no sense and doesn't work: it can't be instantiated. A union or alternate with just one branch works, but is degenerate. We accept the former, but reject the latter. Weird. docs/devel/qapi-code-gen.txt doesn't mention the difference. It claims an alternate definition is "is similar to a simple union type". Permit degenerate alternates to make them consistent with unions. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20190913201349.24332-10-armbru@redhat.com>
| * | qapi: Permit 'boxed' with empty typeMarkus Armbruster2019-09-2411-25/+19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We reject empty types with 'boxed': true. We don't really need that to work, but making it work is actually simpler than rejecting it, so do that. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20190913201349.24332-9-armbru@redhat.com>
| * | qapi: Drop support for escape sequences other than \\Markus Armbruster2019-09-2419-60/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Since the previous commit restricted strings to printable ASCII, \uXXXX's only use is obfuscation. Drop it. This leaves \\, \/, \', and \". Since QAPI schema strings are all names, and names are restricted to ASCII letters, digits, hyphen, and underscore, none of them is useful. The latter three have no test coverage. Drop them. Keep \\ to avoid (more) gratuitous incompatibility with JSON. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20190913201349.24332-8-armbru@redhat.com>
| * | qapi: Restrict strings to printable ASCIIMarkus Armbruster2019-09-2412-21/+20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | RFC 8259 on string contents: All Unicode characters may be placed within the quotation marks, except for the characters that MUST be escaped: quotation mark, reverse solidus, and the control characters (U+0000 through U+001F). The QAPI schema parser accepts both less and more than JSON: it accepts only ASCII with \u (less), and accepts control characters other than LF (new line) unescaped. How it treats unescaped non-ASCII input differs between Python 2 and Python 3. Make it accept strictly less: require printable ASCII. Drop support for \b, \f, \n, \r, \t. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20190913201349.24332-7-armbru@redhat.com>
| * | tests/qapi-schema: Demonstrate bad reporting of funny charactersMarkus Armbruster2019-09-242-2/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | Invalid name 'not\\possible' is reported as 'not\possible'. Control characters (quoted or not) are even more confusing. Mark FIXME. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20190913201349.24332-6-armbru@redhat.com>
| * | docs/devel/qapi-code-gen: Minor specification fixesMarkus Armbruster2019-09-241-18/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The specification claims "Each expression that isn't an include directive may be preceded by a documentation block", but the code also rejects them for pragma directives. The code is correct. Fix the specification. The specification reserves member names starting with 'has_', but the code also reserves name 'u'. Fix the specification. The specification claims "The string 'max' is not allowed as an enum value". Untrue. Fix the specification. While there, delete the naming advice, because it's redundant with the naming rules in section "Schema overview" The specification claims "No branch of the union can be named 'max', as this would collide with the implicit enum". Untrue. Fix the specification. The specification claims "It is not allowed to name an event 'MAX', since the generator also produces a C enumeration of all event names with a generated _MAX value at the end." Untrue. Fix the specification. The specification claims "All branches of the union must be complex types", but the code permits only struct types. The code is correct. Fix the specification. The specification claims a command's return type "must be the string name of a complex or built-in type, a one-element array containing the name of a complex or built-in type" unless the command is in pragma 'returns-whitelist'. The code does not permit built-in types. Fix the specification. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20190913201349.24332-5-armbru@redhat.com>