summary refs log tree commit diff stats
path: root/docs/tools (follow)
Commit message (Collapse)AuthorAgeFilesLines
...
* docs: Stop calling the top level subsections of our manual 'manuals'Peter Maydell2021-07-181-3/+2
| | | | | | | | | | | | | | We merged our previous multiple-manual setup into a single Sphinx manual, but we left some text in the various index.rst lines that still calls the top level subsections separate 'manuals'. Update them to talk about "this section of the manual" instead, and remove now-obsolete comments about how the index.rst files are the "top level page for the 'foo' manual". Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Acked-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Message-id: 20210705095547.15790-3-peter.maydell@linaro.org
* qemu-img: Reword 'qemu-img map --output=json' docsEric Blake2021-07-121-10/+10
| | | | | | | | | | | Reword the paragraphs to list the JSON key first, rather than in the middle of prose. Suggested-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <20210707184125.2551140-1-eblake@redhat.com> Reviewed-by: Nir Soffer <nsoffer@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
* qemu-img: Make unallocated part of backing chain obvious in mapEric Blake2021-07-121-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The recently-added NBD context qemu:allocation-depth is able to distinguish between locally-present data (even when that data is sparse) [shown as depth 1 over NBD], and data that could not be found anywhere in the backing chain [shown as depth 0]; and the libnbd project was recently patched to give the human-readable name "absent" to an allocation-depth of 0. But qemu-img map --output=json predates that addition, and has the unfortunate behavior that all portions of the backing chain that resolve without finding a hit in any backing layer report the same depth as the final backing layer. This makes it harder to reconstruct a qcow2 backing chain using just 'qemu-img map' output, especially when using "backing":null to artificially limit a backing chain, because it is impossible to distinguish between a QCOW2_CLUSTER_UNALLOCATED (which defers to a [missing] backing file) and a QCOW2_CLUSTER_ZERO_PLAIN cluster (which would override any backing file), since both types of clusters otherwise show as "data":false,"zero":true" (but note that we can distinguish a QCOW2_CLUSTER_ZERO_ALLOCATED, which would also have an "offset": listing). The task of reconstructing a qcow2 chain was made harder in commit 0da9856851 (nbd: server: Report holes for raw images), because prior to that point, it was possible to abuse NBD's block status command to see which portions of a qcow2 file resulted in BDRV_BLOCK_ALLOCATED (showing up as NBD_STATE_ZERO in isolation) vs. missing from the chain (showing up as NBD_STATE_ZERO|NBD_STATE_HOLE); but now qemu reports more accurate sparseness information over NBD. An obvious solution is to make 'qemu-img map --output=json' add an additional "present":false designation to any cluster lacking an allocation anywhere in the chain, without any change to the "depth" parameter to avoid breaking existing clients. The iotests have several examples where this distinction demonstrates the additional accuracy. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <20210701190655.2131223-3-eblake@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> [eblake: fix more iotest fallout] Signed-off-by: Eric Blake <eblake@redhat.com>
* virtiofsd: Add an option to enable/disable posix aclsVivek Goyal2021-07-051-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | fuse has an option FUSE_POSIX_ACL which needs to be opted in by fuse server to enable posix acls. As of now we are not opting in for this, so posix acls are disabled on virtiofs by default. Add virtiofsd option "-o posix_acl/no_posix_acl" to let users enable/disable posix acl support. By default it is disabled as of now due to performance concerns with cache=none. Currently even if file server has not opted in for FUSE_POSIX_ACL, user can still query acl and set acl, and system.posix_acl_access and system.posix_acl_default xattrs show up listxattr response. Miklos said this is confusing. So he said lets block and filter system.posix_acl_access and system.posix_acl_default xattrs in getxattr/setxattr/listxattr if user has explicitly disabled posix acls using -o no_posix_acl. As of now continuing to keeping the existing behavior if user did not specify any option to disable acl support due to concerns about backward compatibility. Signed-off-by: Vivek Goyal <vgoyal@redhat.com> Message-Id: <20210622150852.1507204-8-vgoyal@redhat.com> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
* docs: describe the security considerations with virtiofsd xattr mappingDaniel P. Berrangé2021-07-051-5/+50
| | | | | | | | | | | | | | | | | | | | | | | Different guest xattr prefixes have distinct access control rules applied by the guest. When remapping a guest xattr care must be taken that the remapping does not allow the a guest user to bypass guest kernel access control rules. For example if 'trusted.*' which requires CAP_SYS_ADMIN is remapped to 'user.virtiofs.trusted.*', an unprivileged guest user which can write to 'user.*' can bypass the CAP_SYS_ADMIN control. Thus the target of any remapping must be explicitly blocked from read/writes by the guest, to prevent access control bypass. The examples shown in the virtiofsd man page already do the right thing and ensure safety, but the security implications of getting this wrong were not made explicit. This could lead to host admins and apps unwittingly creating insecure configurations. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-Id: <20210611120427.49736-1-berrange@redhat.com> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
* docs/tools/virtiofsd: Fix bad rst syntaxThomas Huth2021-06-211-3/+5
| | | | | | | | | | | For literal blocks, there has to be an empty line after the two colons, and the block itself should be indented. Signed-off-by: Thomas Huth <thuth@redhat.com> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Message-Id: <20210607180015.924571-1-thuth@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
* docs/tools/virtiofsd.rst: Do not hard-code the QEMU binary nameThomas Huth2021-06-211-7/+7
| | | | | | | | | | | | | | In downstream, we want to use a different name for the QEMU binary, and some people might also use the docs for non-x86 binaries, that's why we already created the |qemu_system| placeholder in the past. Use it now in the virtiofsd doc, too. Signed-off-by: Thomas Huth <thuth@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Message-Id: <20210607174250.920226-1-thuth@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
* Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2021-05-14' ↵Peter Maydell2021-05-171-0/+31
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | into staging Block patches: - drop block/io write notifiers - qemu-iotests enhancements to make debugging easier - rbd parsing fix - HMP qemu-io fix (for iothreads) - mirror job cancel relaxation (do not cancel in-flight requests when a READY mirror job is canceled with force=false) - document qcow2's data_file and data_file_raw features - fix iotest 297 for pylint 2.8 - block/copy-on-read refactoring # gpg: Signature made Fri 14 May 2021 17:43:40 BST # gpg: using RSA key 91BEB60A30DB3E8857D11829F407DB0061D5CF40 # gpg: issuer "mreitz@redhat.com" # gpg: Good signature from "Max Reitz <mreitz@redhat.com>" [full] # Primary key fingerprint: 91BE B60A 30DB 3E88 57D1 1829 F407 DB00 61D5 CF40 * remotes/maxreitz/tags/pull-block-2021-05-14: write-threshold: deal with includes test-write-threshold: drop extra TestStruct structure test-write-threshold: drop extra tests block/write-threshold: drop extra APIs test-write-threshold: rewrite test_threshold_(not_)trigger tests block: drop write notifiers block/write-threshold: don't use write notifiers qemu-iotests: fix pylint 2.8 consider-using-with error block/copy-on-read: use bdrv_drop_filter() and drop s->active Document qemu-img options data_file and data_file_raw qemu-iotests: fix case of SOCK_DIR already in the environment qemu-iotests: let "check" spawn an arbitrary test command qemu-iotests: move command line and environment handling from TestRunner to TestEnv qemu-iotests: allow passing unittest.main arguments to the test scripts qemu-iotests: do not buffer the test output mirror: stop cancelling in-flight requests on non-force cancel in READY monitor: hmp_qemu_io: acquire aio contex, fix crash block/rbd: Add an escape-aware strchr helper iotests/231: Update expected deprecation message Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
| * Document qemu-img options data_file and data_file_rawConnor Kuehl2021-05-141-0/+31
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The contents of this patch were initially developed and posted by Han Han[1], however, it appears the original patch was not applied. Since then, the relevant documentation has been moved and adapted to a new format. I've taken most of the original wording and tweaked it according to some of the feedback from the original patch submission. I've also adapted it to restructured text, which is the format the documentation currently uses. [1] https://lists.nongnu.org/archive/html/qemu-block/2019-10/msg01253.html Fixes: https://bugzilla.redhat.com/1763105 Signed-off-by: Han Han <hhan@redhat.com> Suggested-by: Max Reitz <mreitz@redhat.com> [ Max: provided description of data_file_raw behavior ] Signed-off-by: Connor Kuehl <ckuehl@redhat.com> Message-Id: <20210505195512.391128-1-ckuehl@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
* | sphinx: adopt kernel readthedoc themeMarc-André Lureau2021-05-141-5/+0
|/ | | | | | | | | | | | | | | | | | | The default "alabaster" sphinx theme has a couple shortcomings: - the navbar moves along the page - the search bar is not always at the same place - it lacks some contrast and colours The "rtd" theme from readthedocs.org is a popular third party theme used notably by the kernel, with a custom style sheet. I like it better, perhaps others do too. It also simplifies the "Edit on Gitlab" links. Tweak a bit the custom theme to match qemu.org style, use the QEMU logo, and favicon etc. Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Tested-by: Bin Meng <bmeng.cn@gmail.com> Message-Id: <20210323115328.4146052-1-marcandre.lureau@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com>
* docs: simplify each section titleMarc-André Lureau2021-04-011-2/+2
| | | | | | | | | Now that we merged into one doc, it makes the nav looks nicer. Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Message-Id: <20210323074704.4078381-1-marcandre.lureau@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com>
* qsd: Document FUSE exportsMax Reitz2021-03-291-0/+19
| | | | | | | | | | | Implementing FUSE exports required no changes to the storage daemon, so we forgot to document them there. Considering that both NBD and vhost-user-blk exports are documented in its man page (and NBD exports in its --help text), we should probably do the same for FUSE. Signed-off-by: Max Reitz <mreitz@redhat.com> Message-Id: <20210217115844.62661-1-mreitz@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
* qemu-img: Use user_creatable_process_cmdline() for --objectKevin Wolf2021-03-191-1/+1
| | | | | | | | | | | | | | | | | | | | | | This switches qemu-img from a QemuOpts-based parser for --object to user_creatable_process_cmdline() which uses a keyval parser and enforces the QAPI schema. Apart from being a cleanup, this makes non-scalar properties accessible. As a side effect, fix wrong exit codes in the object parsing error path of 'qemu-img compare'. This was broken in commit 334c43e2c3 because &error_fatal exits with an exit code of 1, while it should have been 2. Document that exit code 0 is also returned when just requested help was printed instead of comparing images. This is preexisting behaviour that isn't changed by this patch, though another instance of it is added with '--object help'. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Acked-by: Paolo Bonzini <pbonzini@redhat.com> Acked-by: Peter Krempa <pkrempa@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
* docs: qsd: Explain --export nbd,name=... defaultKevin Wolf2021-03-081-2/+3
| | | | | | | | | | | | The 'name' option for NBD exports is optional. Add a note that the default for the option is the node name (people could otherwise expect that it's the empty string like for qemu-nbd). Signed-off-by: Kevin Wolf <kwolf@redhat.com> Message-Id: <20210305094856.18964-1-kwolf@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
* docs: replace insecure /tmp examples in qsd docsStefan Hajnoczi2021-03-081-3/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | World-writeable directories have security issues. Avoid showing them in the documentation since someone might accidentally use them in situations where they are insecure. There tend to be 3 security problems: 1. Denial of service. An adversary may be able to create the file beforehand, consume all space/inodes, etc to sabotage us. 2. Impersonation. An adversary may be able to create a listen socket and accept incoming connections that were meant for us. 3. Unauthenticated client access. An adversary may be able to connect to us if we did not set the uid/gid and permissions correctly. These can be prevented or mitigated with private /tmp, carefully setting the umask, etc but that requires special action and does not apply to all situations. Just avoid using /tmp in examples. Reported-by: Richard W.M. Jones <rjones@redhat.com> Reported-by: Daniel P. Berrangé <berrange@redhat.com> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Message-Id: <20210301172728.135331-3-stefanha@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Richard W.M. Jones <rjones@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
* docs: show how to spawn qemu-storage-daemon with fd passingStefan Hajnoczi2021-03-081-2/+40
| | | | | | | | | | | | | | | | | | | | | | | The QMP monitor, NBD server, and vhost-user-blk export all support file descriptor passing. This is a useful technique because it allows the parent process to spawn and wait for qemu-storage-daemon without busy waiting, which may delay startup due to arbitrary sleep() calls. This Python example is inspired by the test case written for libnbd by Richard W.M. Jones <rjones@redhat.com>: https://gitlab.com/nbdkit/libnbd/-/commit/89113f484effb0e6c322314ba75c1cbe07a04543 Thanks to Daniel P. Berrangé <berrange@redhat.com> for suggestions on how to get this working. Now let's document it! Reported-by: Richard W.M. Jones <rjones@redhat.com> Cc: Kevin Wolf <kwolf@redhat.com> Cc: Daniel P. Berrangé <berrange@redhat.com> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Message-Id: <20210301172728.135331-2-stefanha@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Richard W.M. Jones <rjones@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
* qemu-storage-daemon: add --pidfile optionStefan Hajnoczi2021-03-081-0/+14
| | | | | | | | | | | | | | | | | Daemons often have a --pidfile option where the pid is written to a file so that scripts can stop the daemon by sending a signal. The pid file also acts as a lock to prevent multiple instances of the daemon from launching for a given pid file. QEMU, qemu-nbd, qemu-ga, virtiofsd, and qemu-pr-helper all support the --pidfile option. Add it to qemu-storage-daemon too. Reported-by: Richard W.M. Jones <rjones@redhat.com> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Message-Id: <20210302142746.170535-1-stefanha@redhat.com> Reviewed-by: Richard W.M. Jones <rjones@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
* virtiofs: drop remapped security.capability xattr as neededDr. David Alan Gilbert2021-03-041-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | On Linux, the 'security.capability' xattr holds a set of capabilities that can change when an executable is run, giving a limited form of privilege escalation to those programs that the writer of the file deemed worthy. Any write causes the 'security.capability' xattr to be dropped, stopping anyone from gaining privilege by modifying a blessed file. Fuse relies on the daemon to do this dropping, and in turn the daemon relies on the host kernel to drop the xattr for it. However, with the addition of -o xattrmap, the xattr that the guest stores its capabilities in is now not the same as the one that the host kernel automatically clears. Where the mapping changes 'security.capability', explicitly clear the remapped name to preserve the same behaviour. This bug is assigned CVE-2021-20263. Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Reviewed-by: Vivek Goyal <vgoyal@redhat.com>
* docs: update to show preferred boolean syntax for -chardevDaniel P. Berrangé2021-02-251-2/+2
| | | | | | | | | The preferred syntax is to use "foo=on|off", rather than a bare "foo" or "nofoo". Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-Id: <20210216191027.595031-8-berrange@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
* qemu-nbd: Permit --shared=0 for unlimited clientsEric Blake2021-02-121-2/+2
| | | | | | | | | This gives us better feature parity with QMP nbd-server-start, where max-connections defaults to 0 for unlimited. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <20210209152759.209074-3-eblake@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
* docs: Build and install all the docs in a single manualPeter Maydell2021-01-191-37/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When we first converted our documentation to Sphinx, we split it into multiple manuals (system, interop, tools, etc), which are all built separately. The primary driver for this was wanting to be able to avoid shipping the 'devel' manual to end-users. However, this is working against the grain of the way Sphinx wants to be used and causes some annoyances: * Cross-references between documents become much harder or possibly impossible * There is no single index to the whole documentation * Within one manual there's no links or table-of-contents info that lets you easily navigate to the others * The devel manual doesn't get published on the QEMU website (it would be nice to able to refer to it there) Merely hiding our developer documentation from end users seems like it's not enough benefit for these costs. Combine all the documentation into a single manual (the same way that the readthedocs site builds it) and install the whole thing. The previous manual divisions remain as the new top level sections in the manual. * The per-manual conf.py files are no longer needed * The man_pages[] specifications previously in each per-manual conf.py move to the top level conf.py * docs/meson.build logic is simplified as we now only need to run Sphinx once for the HTML and then once for the manpages5B * The old index.html.in that produced the top-level page with links to each manual is no longer needed Unfortunately this means that we now have to build the HTML documentation into docs/manual in the build tree rather than directly into docs/; otherwise it is too awkward to ensure we install only the built manual and not also the dependency info, stamp file, etc. The manual still ends up in the same place in the final installed directory, but anybody who was consulting documentation from within the build tree will have to adjust where they're looking. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Paolo Bonzini <pbonzini@redhat.com> Message-id: 20210115154449.4801-1-peter.maydell@linaro.org
* docs: add qemu-storage-daemon(1) man pageStefan Hajnoczi2020-12-183-0/+151
| | | | | | | | | | | | | Document the qemu-storage-daemon tool. Most of the command-line options are identical to their QEMU counterparts. Perhaps Sphinx hxtool integration could be extended to extract documentation for individual command-line options so they can be shared. For now the qemu-storage-daemon simply refers to the qemu(1) man page where the command-line options are identical. Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Message-Id: <20201209103802.350848-3-stefanha@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
* docs: Split qemu-pr-helper documentation into tools manualPeter Maydell2020-11-233-0/+93
| | | | | | | | Split the documentation of the qemu-pr-helper binary into the tools manual, and give it a manpage like our other standalone executables. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
* docs: Fix some typos (found by codespell)Stefan Weil2020-11-181-1/+1
| | | | | | | | | Fix also a similar typo in a code comment. Signed-off-by: Stefan Weil <sw@weilnetz.de> Message-Id: <20201117193448.393472-1-sw@weilnetz.de> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
* docs: add "page source" link to sphinx documentationDaniel P. Berrangé2020-11-101-0/+5
| | | | | | | | | Add a link to the top of the sidebar in every docs page that takes the user back to the source code in gitlab. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-Id: <20201102130926.161183-5-berrange@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
* nbd: Add 'qemu-nbd -A' to expose allocation depthEric Blake2020-10-301-1/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | Allow the server to expose an additional metacontext to be requested by savvy clients. qemu-nbd adds a new option -A to expose the qemu:allocation-depth metacontext through NBD_CMD_BLOCK_STATUS; this can also be set via QMP when using block-export-add. qemu as client is hacked into viewing the key aspects of this new context by abusing the already-experimental x-dirty-bitmap option to collapse all depths greater than 2, which results in a tri-state value visible in the output of 'qemu-img map --output=json' (yes, that means x-dirty-bitmap is now a bit of a misnomer, but I didn't feel like renaming it as it would introduce a needless break of back-compat, even though we make no compat guarantees with x- members): unallocated (depth 0) => "zero":false, "data":true local (depth 1) => "zero":false, "data":false backing (depth 2+) => "zero":true, "data":true libnbd as client is probably a nicer way to get at the information without having to decipher such hacks in qemu as client. ;) Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <20201027050556.269064-11-eblake@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
* Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into stagingPeter Maydell2020-10-301-2/+8
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Block layer patches: - qcow2: Skip copy-on-write when allocating a zero cluster - qemu-img: add support for rate limit in qemu-img convert/commit - Fix deadlock when deleting a block node during drain_all # gpg: Signature made Tue 27 Oct 2020 15:14:07 GMT # gpg: using RSA key DC3DEB159A9AF95D3D7456FE7F09B272C88F2FD6 # gpg: issuer "kwolf@redhat.com" # gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full] # Primary key fingerprint: DC3D EB15 9A9A F95D 3D74 56FE 7F09 B272 C88F 2FD6 * remotes/kevin/tags/for-upstream: block: End quiescent sections when a BDS is deleted qcow2: Skip copy-on-write when allocating a zero cluster qcow2: Report BDRV_BLOCK_ZERO more accurately in bdrv_co_block_status() qemu-img: add support for rate limit in qemu-img convert qemu-img: add support for rate limit in qemu-img commit Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
| * qemu-img: add support for rate limit in qemu-img convertZhengui2020-10-271-1/+5
| | | | | | | | | | | | | | | | | | add support for rate limit in qemu-img convert. Signed-off-by: Zhengui <lizhengui@huawei.com> Message-Id: <1603205264-17424-3-git-send-email-lizhengui@huawei.com> Reviewed-by: Alberto Garcia <berto@igalia.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
| * qemu-img: add support for rate limit in qemu-img commitZhengui2020-10-271-1/+3
| | | | | | | | | | | | | | | | | | add support for rate limit in qemu-img commit. Signed-off-by: Zhengui <lizhengui@huawei.com> Message-Id: <1603205264-17424-2-git-send-email-lizhengui@huawei.com> Reviewed-by: Alberto Garcia <berto@igalia.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
* | tools/virtiofsd: xattr name mappings: Simple 'map'Dr. David Alan Gilbert2020-10-261-0/+19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The mapping rule system implemented in the last few patches is extremely flexible, but not easy to use. Add a simple 'map' type as a sprinkling of sugar to make it easy. e.g. -o xattrmap=":map::user.virtiofs.:" would be sufficient to prefix all xattr's or -o xattrmap=":map:trusted.:user.virtiofs.:" would just prefix 'trusted.' xattr's and leave everything else alone. Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Message-Id: <20201023165812.36028-6-dgilbert@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
* | tools/virtiofsd: xattr name mapping examplesDr. David Alan Gilbert2020-10-261-0/+50
| | | | | | | | | | | | | | | | | | | | Add a few examples of xattrmaps to the documentation. Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Message-Id: <20201023165812.36028-5-dgilbert@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
* | tools/virtiofsd: xattr name mappings: Add optionDr. David Alan Gilbert2020-10-261-0/+92
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add an option to define mappings of xattr names so that the client and server filesystems see different views. This can be used to have different SELinux mappings as seen by the guest, to run the virtiofsd with less privileges (e.g. in a case where it can't set trusted/system/security xattrs but you want the guest to be able to), or to isolate multiple users of the same name; e.g. trusted attributes used by stacking overlayfs. A mapping engine is used with 3 simple rules; the rules can be combined to allow most useful mapping scenarios. The ruleset is defined by -o xattrmap='rules...'. This patch doesn't use the rule maps yet. Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Message-Id: <20201023165812.36028-2-dgilbert@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
* | virtiofsd: add container-friendly -o sandbox=chroot optionStefan Hajnoczi2020-10-261-7/+25
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | virtiofsd cannot run in a container because CAP_SYS_ADMIN is required to create namespaces. Introduce a weaker sandbox mode that is sufficient in container environments because the container runtime already sets up namespaces. Use chroot to restrict path traversal to the shared directory. virtiofsd loses the following: 1. Mount namespace. The process chroots to the shared directory but leaves the mounts in place. Seccomp rejects mount(2)/umount(2) syscalls. 2. Pid namespace. This should be fine because virtiofsd is the only process running in the container. 3. Network namespace. This should be fine because seccomp already rejects the connect(2) syscall, but an additional layer of security is lost. Container runtime-specific network security policies can be used drop network traffic (except for the vhost-user UNIX domain socket). Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Message-Id: <20201008085534.16070-1-stefanha@redhat.com> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
* tools/virtiofsd: add support for --socket-groupAlex Bennée2020-10-121-0/+4
| | | | | | | | | | | | | | If you like running QEMU as a normal user (very common for TCG runs) but you have to run virtiofsd as a root user you run into connection problems. Adding support for an optional --socket-group allows the users to keep using the command line. Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Message-Id: <20200925125147.26943-2-alex.bennee@linaro.org> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com> dgilbert: Split long line
* virtiofsd: document cache=auto defaultHarry G. Coin2020-09-251-0/+1
| | | | | | | | | | | The virtiofsd --help output documents the cache=auto default value but the man page does not. Fix this. Signed-off-by: Harry G. Coin <hgcoin@gmail.com> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Message-Id: <20200916112250.760245-1-stefanha@redhat.com> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
* docs/: fix some comment spelling errorszhaolichang2020-09-171-1/+1
| | | | | | | | | | | I found that there are many spelling errors in the comments of qemu, so I used the spellcheck tool to check the spelling errors and finally found some spelling errors in the docs folder. Signed-off-by: zhaolichang <zhaolichang@huawei.com> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Message-Id: <20200917075029.313-4-zhaolichang@huawei.com> Signed-off-by: Laurent Vivier <laurent@vivier.eu>
* virtiofsd: Remove "norace" from cmdline help and docsSergio Lopez2020-08-281-3/+0
| | | | | | | | | | | | Commit 93bb3d8d4cda ("virtiofsd: remove symlink fallbacks") removed the implementation of the "norace" option, so remove it from the cmdline help and the documentation too. Signed-off-by: Sergio Lopez <slp@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Reviewed-by: Stefano Garzarella <sgarzare@redhat.com> Message-Id: <20200717121110.50580-1-slp@redhat.com> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
* virtiofsd: Disable remote posix locks by defaultVivek Goyal2020-08-281-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | Right now we enable remote posix locks by default. That means when guest does a posix lock it sends request to server (virtiofsd). But currently we only support non-blocking posix lock and return -EOPNOTSUPP for blocking version. This means that existing applications which are doing blocking posix locks get -EOPNOTSUPP and fail. To avoid this, people have been running virtiosd with option "-o no_posix_lock". For new users it is still a surprise and trial and error takes them to this option. Given posix lock implementation is not complete in virtiofsd, disable it by default. This means that posix locks will work with-in applications in a guest but not across guests. Anyway we don't support sharing filesystem among different guests yet in virtiofs so this should not lead to any kind of surprise or regression and will make life little easier for virtiofs users. Reported-by: Aa Aa <jimbothom@yandex.com> Suggested-by: Miklos Szeredi <mszeredi@redhat.com> Signed-off-by: Vivek Goyal <vgoyal@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Misono Tomohiro <misono.tomohiro@jp.fujitsu.com> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
* qcow2: Deprecate use of qemu-img amend to change backing fileEric Blake2020-07-141-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | The use of 'qemu-img amend' to change qcow2 backing files is not tested very well. In particular, our implementation has a bug where if a new backing file is provided without a format, then the prior format is blindly reused, even if this results in data corruption, but this is not caught by iotests. There are also situations where amending other options needs access to the original backing file (for example, on a downgrade to a v2 image, knowing whether a v3 zero cluster must be allocated or may be left unallocated depends on knowing whether the backing file already reads as zero), but the command line does not have a nice way to tell us both the backing file to use for opening the image as well as the backing file to install after the operation is complete. Even if we do allow changing the backing file, it is redundant with the existing ability to change backing files via 'qemu-img rebase -u'. It is time to deprecate this support (leaving the existing behavior intact, even if it is buggy), and at a point in the future, require the use of only 'qemu-img rebase' for adjusting backing chain relations, saving 'qemu-img amend' for changes unrelated to the backing chain. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <20200706203954.341758-8-eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
* block/amend: add 'force' optionMaxim Levitsky2020-07-061-1/+4
| | | | | | | | | | | | | | 'force' option will be used for some unsafe amend operations. This includes things like erasing last keyslot in luks based formats (which destroys the data, unless the master key is backed up by external means), but that _might_ be desired result. Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <20200608094030.670121-4-mlevitsk@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
* virtiofsd: Allow addition or removal of capabilitiesDr. David Alan Gilbert2020-07-031-0/+5
| | | | | | | | | | | | | | | | | | | | | | | | | Allow capabilities to be added or removed from the allowed set for the daemon; e.g. default: CapPrm: 00000000880000df CapEff: 00000000880000df -o modcaps=+sys_admin CapPrm: 00000000882000df CapEff: 00000000882000df -o modcaps=+sys_admin:-chown CapPrm: 00000000882000de CapEff: 00000000882000de Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Message-Id: <20200629115420.98443-4-dgilbert@redhat.com> Acked-by: Vivek Goyal <vgoyal@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
* qemu-img: Fix doc typo for 'bitmap' subcommandEric Blake2020-06-091-1/+1
| | | | | | | | | Prefer a consistent naming for the --merge argument. Fixes: 3b51ab4bf Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <20200529144527.1943527-1-eblake@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
* qemu-img: Add convert --bitmaps optionEric Blake2020-05-281-1/+5
| | | | | | | | | | | | | | | | | | | | | | | Make it easier to copy all the persistent bitmaps of (the top layer of) a source image along with its guest-visible contents, by adding a boolean flag for use with qemu-img convert. This is basically shorthand, as the same effect could be accomplished with a series of 'qemu-img bitmap --add' and 'qemu-img bitmap --merge -b source' commands, or by their corresponding QMP commands. Note that this command will fail in the same scenarios where 'qemu-img measure' omits a 'bitmaps size:' line, namely, when either the source or the destination lacks persistent bitmap support altogether. See also https://bugzilla.redhat.com/show_bug.cgi?id=1779893 While touching this, clean up a couple coding issues spotted in the same function: an extra blank line, and merging back-to-back 'if (!skip_create)' blocks. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <20200521192137.1120211-5-eblake@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
* qcow2: Expose bitmaps' size during measureEric Blake2020-05-281-0/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | It's useful to know how much space can be occupied by qcow2 persistent bitmaps, even though such metadata is unrelated to the guest-visible data. Report this value as an additional QMP field, present when measuring an existing image and output format that both support bitmaps. Update iotest 178 and 190 to updated output, as well as new coverage in 190 demonstrating non-zero values made possible with the recently-added qemu-img bitmap command (see 3b51ab4b). The new 'bitmaps size:' field is displayed automatically as part of 'qemu-img measure' any time it is present in QMP (that is, any time both the source image being measured and destination format support bitmaps, even if the measurement is 0 because there are no bitmaps present). If the field is absent, it means that no bitmaps can be copied (source, destination, or both lack bitmaps, including when measuring based on size rather than on a source image). This behavior is compatible with an upcoming patch adding 'qemu-img convert --bitmaps': that command will fail in the same situations where this patch omits the field. The addition of a new field demonstrates why we should always zero-initialize qapi C structs; while the qcow2 driver still fully populates all fields, the raw and crypto drivers had to be tweaked to avoid uninitialized data. Consideration was also given towards having a 'qemu-img measure --bitmaps' which errors out when bitmaps are not possible, and otherwise sums the bitmaps into the existing allocation totals rather than displaying as a separate field, as a potential convenience factor. But this was ultimately decided to be more complexity than necessary when the QMP interface was sufficient enough with bitmaps remaining a separate field. See also: https://bugzilla.redhat.com/1779904 Reported-by: Nir Soffer <nsoffer@redhat.com> Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <20200521192137.1120211-3-eblake@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
* qemu-img: Add bitmap sub-commandEric Blake2020-05-191-0/+24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Include actions for --add, --remove, --clear, --enable, --disable, and --merge (note that --clear is a bit of fluff, because the same can be accomplished by removing a bitmap and then adding a new one in its place, but it matches what QMP commands exist). Listing is omitted, because it does not require a bitmap name and because it was already possible with 'qemu-img info'. A single command line can play one or more bitmap commands in sequence on the same bitmap name (although all added bitmaps share the same granularity, and and all merged bitmaps come from the same source file). Merge defaults to other bitmaps in the primary image, but can also be told to merge bitmaps from a distinct image. While this supports --image-opts for the file being modified, I did not think it worth the extra complexity to support that for the source file in a cross-file merges. Likewise, I chose to have --merge only take a single source rather than following the QMP support for multiple merges in one go (although you can still use more than one --merge in the command line); in part because qemu-img is offline and therefore atomicity is not an issue. Upcoming patches will add iotest coverage of these commands while also testing other features. Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <20200513011648.166876-7-eblake@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
* docs: Sort sections on qemu-img subcommand parametersEric Blake2020-05-191-24/+24
| | | | | | | | | | | We already list the subcommand summaries alphabetically, we should do the same for the documentation related to subcommand-specific parameters. Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <20200513011648.166876-2-eblake@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
* qemu-img: Add --start-offset and --max-length to mapEyal Moscovici2020-05-181-1/+1
| | | | | | | | | | | | | | | | | | The mapping operation of large disks especially ones stored over a long chain of QCOW2 files can take a long time to finish. Additionally when mapping fails there was no way recover by restarting the mapping from the failed location. The new options, --start-offset and --max-length allows the user to divide these type of map operations into shorter independent tasks. Reviewed-by: Eric Blake <eblake@redhat.com> Acked-by: Mark Kanda <mark.kanda@oracle.com> Co-developed-by: Yoav Elnekave <yoav.elnekave@oracle.com> Signed-off-by: Yoav Elnekave <yoav.elnekave@oracle.com> Signed-off-by: Eyal Moscovici <eyal.moscovici@oracle.com> Message-Id: <20200513133629.18508-5-eyal.moscovici@oracle.com> Signed-off-by: Eric Blake <eblake@redhat.com>
* docs: Move tools documentation to tools manualPeter Maydell2020-02-257-0/+1427
| | | | | | | | | | | | | | | Move the following tools documentation files to the new tools manual: docs/interop/qemu-img.rst docs/interop/qemu-nbd.rst docs/interop/virtfs-proxy-helper.rst docs/interop/qemu-trace-stap.rst docs/interop/virtiofsd.rst Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Acked-by: Paolo Bonzini <pbonzini@redhat.com> Message-id: 20200217155415.30949-4-peter.maydell@linaro.org
* docs: Create new 'tools' manualPeter Maydell2020-02-252-0/+27
Some of the documentation for QEMU "tools" which are standalone binaries like qemu-img is an awkward fit in our current 5-manual split. We've put it into "interop", but they're not really about interoperability. Create a new top level manual "tools" which will be a better home for this documentation. This commit creates an empty initial manual; we will move the relevant documentation files in a subsequent commit. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Acked-by: Paolo Bonzini <pbonzini@redhat.com> Message-id: 20200217155415.30949-2-peter.maydell@linaro.org