summary refs log tree commit diff stats
path: root/migration/multifd-zlib.c (follow)
Commit message (Collapse)AuthorAgeFilesLines
* migration/multifd: solve zero page causing multiple page faultsYuan Liu2024-04-231-0/+1
| | | | | | | | | | | | | | | | | Implemented recvbitmap tracking of received pages in multifd. If the zero page appears for the first time in the recvbitmap, this page is not checked and set. If the zero page has already appeared in the recvbitmap, there is no need to check the data but directly set the data to 0, because it is unlikely that the zero page will be migrated multiple times. Signed-off-by: Yuan Liu <yuan1.liu@intel.com> Reviewed-by: Fabiano Rosas <farosas@suse.de> Link: https://lore.kernel.org/r/20240401154110.2028453-2-yuan1.liu@intel.com [peterx: touch up the comment, as the bitmap is used outside postcopy now] Signed-off-by: Peter Xu <peterx@redhat.com>
* Merge tag 'pull-request-2024-03-12' of https://gitlab.com/thuth/qemu into ↵Peter Maydell2024-03-121-1/+1
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | staging * Add missing ERRP_GUARD() statements in functions that need it * Prefer fast cpu_env() over slower CPU QOM cast macro # -----BEGIN PGP SIGNATURE----- # # iQJFBAABCAAvFiEEJ7iIR+7gJQEY8+q5LtnXdP5wLbUFAmXwPhYRHHRodXRoQHJl # ZGhhdC5jb20ACgkQLtnXdP5wLbWHvBAAgKx5LHFjz3xREVA+LkDTQ49mz0lK3s32 # SGvNlIHjiaDGVttVYhVC4sinBWUruG4Lyv/2QN72OJBzn6WUsEUQE3KPH1d7Y3/s # wS9X7mj70n4kugWJqeIJP5AXSRasHmWoQ4QJLVQRJd6+Eb9jqwep0x7bYkI1de6D # bL1Q7bIfkFeNQBXaiPWAm2i+hqmT4C1r8HEAGZIjAsMFrjy/hzBEjNV+pnh6ZSq9 # Vp8BsPWRfLU2XHm4WX0o8d89WUMAfUGbVkddEl/XjIHDrUD+Zbd1HAhLyfhsmrnE # jXIwSzm+ML1KX4MoF5ilGtg8Oo0gQDEBy9/xck6G0HCm9lIoLKlgTxK9glr2vdT8 # yxZmrM9Hder7F9hKKxmb127xgU6AmL7rYmVqsoQMNAq22D6Xr4UDpgFRXNk2/wO6 # zZZBkfZ4H4MpZXbd/KJpXvYH5mQA4IpkOy8LJdE+dbcHX7Szy9ksZdPA+Z10hqqf # zqS13qTs3abxymy2Q/tO3hPKSJCk1+vCGUkN60Wm+9VoLWGoU43qMc7gnY/pCS7m # 0rFKtvfwFHhokX1orK0lP/ppVzPv/5oFIeK8YDY9if+N+dU2LCwVZHIuf2/VJPRq # wmgH2vAn3JDoRKPxTGX9ly6AMxuZaeP92qBTOPap0gDhihYzIpaCq9ecEBoTakI7 # tdFhV0iRr08= # =NiP4 # -----END PGP SIGNATURE----- # gpg: Signature made Tue 12 Mar 2024 11:35:50 GMT # gpg: using RSA key 27B88847EEE0250118F3EAB92ED9D774FE702DB5 # gpg: issuer "thuth@redhat.com" # gpg: Good signature from "Thomas Huth <th.huth@gmx.de>" [full] # gpg: aka "Thomas Huth <thuth@redhat.com>" [full] # gpg: aka "Thomas Huth <huth@tuxfamily.org>" [full] # gpg: aka "Thomas Huth <th.huth@posteo.de>" [unknown] # Primary key fingerprint: 27B8 8847 EEE0 2501 18F3 EAB9 2ED9 D774 FE70 2DB5 * tag 'pull-request-2024-03-12' of https://gitlab.com/thuth/qemu: (55 commits) user: Prefer fast cpu_env() over slower CPU QOM cast macro target/xtensa: Prefer fast cpu_env() over slower CPU QOM cast macro target/tricore: Prefer fast cpu_env() over slower CPU QOM cast macro target/sparc: Prefer fast cpu_env() over slower CPU QOM cast macro target/sh4: Prefer fast cpu_env() over slower CPU QOM cast macro target/rx: Prefer fast cpu_env() over slower CPU QOM cast macro target/ppc: Prefer fast cpu_env() over slower CPU QOM cast macro target/openrisc: Prefer fast cpu_env() over slower CPU QOM cast macro target/nios2: Prefer fast cpu_env() over slower CPU QOM cast macro target/mips: Prefer fast cpu_env() over slower CPU QOM cast macro target/microblaze: Prefer fast cpu_env() over slower CPU QOM cast macro target/m68k: Prefer fast cpu_env() over slower CPU QOM cast macro target/loongarch: Prefer fast cpu_env() over slower CPU QOM cast macro target/i386/hvf: Use CPUState typedef target/hexagon: Prefer fast cpu_env() over slower CPU QOM cast macro target/cris: Prefer fast cpu_env() over slower CPU QOM cast macro target/avr: Prefer fast cpu_env() over slower CPU QOM cast macro target/alpha: Prefer fast cpu_env() over slower CPU QOM cast macro target: Replace CPU_GET_CLASS(cpu -> obj) in cpu_reset_hold() handler bulk: Call in place single use cpu_env() ... Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
| * bulk: Access existing variables initialized to &S->F when availablePhilippe Mathieu-Daudé2024-03-121-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When a variable is initialized to &struct->field, use it in place. Rationale: while this makes the code more concise, this also helps static analyzers. Mechanical change using the following Coccinelle spatch script: @@ type S, F; identifier s, m, v; @@ S *s; ... F *v = &s->m; <+... - &s->m + v ...+> Inspired-by: Zhao Liu <zhao1.liu@intel.com> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org> Message-ID: <20240129164514.73104-2-philmd@linaro.org> Acked-by: Fabiano Rosas <farosas@suse.de> Reviewed-by: Zhao Liu <zhao1.liu@intel.com> Reviewed-by: Anthony PERARD <anthony.perard@citrix.com> [thuth: Dropped hunks that need a rebase, and fixed sizeof() in pmu_realize()] Signed-off-by: Thomas Huth <thuth@redhat.com>
* | migration/multifd: Implement zero page transmission on the multifd thread.Hao Xiang2024-03-111-5/+16
|/ | | | | | | | | | | | | | | 1. Add zero_pages field in MultiFDPacket_t. 2. Implements the zero page detection and handling on the multifd threads for non-compression, zlib and zstd compression backends. 3. Added a new value 'multifd' in ZeroPageDetection enumeration. 4. Adds zero page counters and updates multifd send/receive tracing format to track the newly added counters. Signed-off-by: Hao Xiang <hao.xiang@bytedance.com> Acked-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Fabiano Rosas <farosas@suse.de> Link: https://lore.kernel.org/r/20240311180015.3359271-5-hao.xiang@linux.dev Signed-off-by: Peter Xu <peterx@redhat.com>
* migration/multifd: Decouple recv method from pagesFabiano Rosas2024-03-011-3/+3
| | | | | | | | | | | Next patches will abstract the type of data being received by the channels, so do some cleanup now to remove references to pages and dependency on 'normal_num'. Reviewed-by: Peter Xu <peterx@redhat.com> Signed-off-by: Fabiano Rosas <farosas@suse.de> Link: https://lore.kernel.org/r/20240229153017.2221-14-farosas@suse.de Signed-off-by: Peter Xu <peterx@redhat.com>
* migration/multifd: Rename MultiFDSend|RecvParams::data to compress_dataFabiano Rosas2024-03-011-10/+10
| | | | | | | | | | Use a more specific name for the compression data so we can use the generic for the multifd core code. Reviewed-by: Peter Xu <peterx@redhat.com> Signed-off-by: Fabiano Rosas <farosas@suse.de> Link: https://lore.kernel.org/r/20240229153017.2221-13-farosas@suse.de Signed-off-by: Peter Xu <peterx@redhat.com>
* migration/multifd: Move header prepare/fill into send_prepare()Peter Xu2024-02-051-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch redefines the interfacing of ->send_prepare(). It further simplifies multifd_send_thread() especially on zero copy. Now with the new interface, we require the hook to do all the work for preparing the IOVs to send. After it's completed, the IOVs should be ready to be dumped into the specific multifd QIOChannel later. So now the API looks like: p->pages -----------> send_prepare() -------------> IOVs This also prepares for the case where the input can be extended to even not any p->pages. But that's for later. This patch will achieve similar goal of what Fabiano used to propose here: https://lore.kernel.org/r/20240126221943.26628-1-farosas@suse.de However the send() interface may not be necessary. I'm boldly attaching a "Co-developed-by" for Fabiano. Co-developed-by: Fabiano Rosas <farosas@suse.de> Reviewed-by: Fabiano Rosas <farosas@suse.de> Link: https://lore.kernel.org/r/20240202102857.110210-14-peterx@redhat.com Signed-off-by: Peter Xu <peterx@redhat.com>
* migration/multifd: Drop MultiFDSendParams.normal[] arrayPeter Xu2024-02-051-3/+4
| | | | | | | | | | | | | | | | This array is redundant when p->pages exists. Now we extended the life of p->pages to the whole period where pending_job is set, it should be safe to always use p->pages->offset[] rather than p->normal[]. Drop the array. Alongside, the normal_num is also redundant, which is the same to p->pages->num. This doesn't apply to recv side, because there's no extra buffering on recv side, so p->normal[] array is still needed. Reviewed-by: Fabiano Rosas <farosas@suse.de> Link: https://lore.kernel.org/r/20240202102857.110210-6-peterx@redhat.com Signed-off-by: Peter Xu <peterx@redhat.com>
* migration: spelling fixesMichael Tokarev2023-07-251-1/+1
| | | | | Signed-off-by: Michael Tokarev <mjt@tls.msk.ru> Reviewed-by: Fabiano Rosas <farosas@suse.de>
* migration: Move parameters functions to option.cJuan Quintela2023-04-241-0/+1
| | | | | Signed-off-by: Juan Quintela <quintela@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
* multifd: Create page_size fields into both MultiFD{Recv,Send}ParamsJuan Quintela2022-12-151-8/+6
| | | | | | | We were calling qemu_target_page_size() left and right. Signed-off-by: Juan Quintela <quintela@redhat.com> Reviewed-by: Leonardo Bras <leobras@redhat.com>
* multifd: Copy pages before compressing them with zlibIlya Leoshkevich2022-07-201-8/+30
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | zlib_send_prepare() compresses pages of a running VM. zlib does not make any thread-safety guarantees with respect to changing deflate() input concurrently with deflate() [1]. One can observe problems due to this with the IBM zEnterprise Data Compression accelerator capable zlib [2]. When the hardware acceleration is enabled, migration/multifd/tcp/plain/zlib test fails intermittently [3] due to sliding window corruption. The accelerator's architecture explicitly discourages concurrent accesses [4]: Page 26-57, "Other Conditions": As observed by this CPU, other CPUs, and channel programs, references to the parameter block, first, second, and third operands may be multiple-access references, accesses to these storage locations are not necessarily block-concurrent, and the sequence of these accesses or references is undefined. Mark Adler pointed out that vanilla zlib performs double fetches under certain circumstances as well [5], therefore we need to copy data before passing it to deflate(). [1] https://zlib.net/manual.html [2] https://github.com/madler/zlib/pull/410 [3] https://lists.nongnu.org/archive/html/qemu-devel/2022-03/msg03988.html [4] http://publibfp.dhe.ibm.com/epubs/pdf/a227832c.pdf [5] https://lists.gnu.org/archive/html/qemu-devel/2022-07/msg00889.html Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com> Message-Id: <20220705203559.2960949-1-iii@linux.ibm.com> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
* Use g_new() & friends where that makes obvious senseMarkus Armbruster2022-03-211-2/+2
| | | | | | | | | | | | | | | | | | | | | | | g_new(T, n) is neater than g_malloc(sizeof(T) * n). It's also safer, for two reasons. One, it catches multiplication overflowing size_t. Two, it returns T * rather than void *, which lets the compiler catch more type errors. This commit only touches allocations with size arguments of the form sizeof(T). Patch created mechanically with: $ spatch --in-place --sp-file scripts/coccinelle/use-g_new-etc.cocci \ --macro-file scripts/cocci-macro-file.h FILES... Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Reviewed-by: Cédric Le Goater <clg@kaod.org> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Acked-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Message-Id: <20220315144156.1595462-4-armbru@redhat.com> Reviewed-by: Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>
* multifd: recv side only needs the RAMBlock host addressJuan Quintela2022-01-281-1/+1
| | | | | | | So we can remove the MultiFDPages. Signed-off-by: Juan Quintela <quintela@redhat.com> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
* multifd: Use normal pages array on the recv sideJuan Quintela2022-01-281-4/+4
| | | | | | | | | Signed-off-by: Juan Quintela <quintela@redhat.com> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com> --- Rename num_normal_pages to total_normal_pages (peter)
* multifd: Use normal pages array on the send sideJuan Quintela2022-01-281-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We are only sending normal pages through multifd channels. Later on this series, we are going to also send zero pages. We are going to detect if a page is zero or non zero in the multifd channel thread, not on the main thread. So we receive an array of pages page->offset[N] And we will end with: p->normal[N - zero_pages] p->zero[zero_pages]. In this patch, we just copy all the pages in offset to normal. for (i = 0; i < pages->num; i++) { p->narmal[p->normal_num] = pages->offset[i]; p->normal_num++: } Later in the series this becomes: for (i = 0; i < pages->num; i++) { if (buffer_is_zero(page->offset[i])) { p->zerol[p->zero_num] = pages->offset[i]; p->zero_num++: } else { p->narmal[p->normal_num] = pages->offset[i]; p->normal_num++: } } Signed-off-by: Juan Quintela <quintela@redhat.com> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com> --- Improving comment (dave) Renaming num_normal_pages to total_normal_pages (peter)
* multifd: Remove send_write() methodJuan Quintela2022-01-281-17/+0
| | | | | | | Everything use now iov's. Signed-off-by: Juan Quintela <quintela@redhat.com> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
* multifd: Make zlib use iov'sJuan Quintela2022-01-281-4/+4
| | | | | Signed-off-by: Juan Quintela <quintela@redhat.com> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
* multifd: Use proper maximum compression valuesJuan Quintela2022-01-281-2/+2
| | | | | | | | | It happens that there are functions to calculate the worst possible compression size for a packet. Use them. Suggested-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Signed-off-by: Juan Quintela <quintela@redhat.com> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
* migration: All this fields are unsignedJuan Quintela2022-01-281-10/+10
| | | | | | | | | So printing it as %d is wrong. Notice that for the channel id, that is an uint8_t, but I changed it anyways for consistency. Signed-off-by: Juan Quintela <quintela@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Reviewed-by: Peter Xu <peterx@redhat.com>
* multifd: Make zlib compression method not use iovsJuan Quintela2021-12-151-8/+9
| | | | | Signed-off-by: Juan Quintela <quintela@redhat.com> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
* multifd: remove used parameter from send_recv_pages() methodJuan Quintela2021-12-151-5/+4
| | | | | | | It is already there as p->pages->num. Signed-off-by: Juan Quintela <quintela@redhat.com> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
* multifd: remove used parameter from send_prepare() methodJuan Quintela2021-12-151-4/+3
| | | | | | | It is already there as p->pages->num. Signed-off-by: Juan Quintela <quintela@redhat.com> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
* multifd: Add missing documentionJuan Quintela2021-12-151-0/+2
| | | | | Signed-off-by: Juan Quintela <quintela@redhat.com> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
* multifd: Delete useless operationJuan Quintela2021-12-151-9/+4
| | | | | | | | We are dividing by page_size to multiply again in the only use. Once there, improve the comments. Signed-off-by: Juan Quintela <quintela@redhat.com> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
* multifd: Add zlib compression multifd supportJuan Quintela2020-02-281-0/+325
Signed-off-by: Juan Quintela <quintela@redhat.com> Acked-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>