Merge tag 'migration-20231103-pull-request' of https://gitlab.com/juan.quintela/qemu into staging

Migration Pull request (20231103)

Hi

In this PULL:
- dirty limit fixes (hyman)
- coverity issues (juan)

Please apply.

# -----BEGIN PGP SIGNATURE-----
#
# iQIzBAABCAAdFiEEGJn/jt6/WMzuA0uC9IfvGFhy1yMFAmVE4dgACgkQ9IfvGFhy
# 1yPBgxAAvrsCHwU6/m9y+XGokyHTKKKIVysLip/14jAjDL+viLYgxdVyOGQKQGBa
# +yV+XHTaEyKdihKG4Z5nWuC0yM+sdZQpWXQAcHJLPaPs5wDGICICpdAFY2LbAWSK
# jtX9uq7crywIL4mVKiX+HOjRUPCAYUx/2TcqJf2+0+MKDEVC33ikxNbcx8ZELY+Q
# +hGyOws3mkHSQjyaNUVgnnQtGzikYqcNO2efa+zVPdXYd+TUWW2e9I++Qf48r0Hv
# OqeZAB7bSAb39PNRuj0I1gt4d3WTHzHt7BSpX1OuFqQnzLw8vS5iDQH943WAyGkY
# NblZVb8pyzSg1Jy18H/SmrJDXeufRwqFwD+1NHyxGjsF89KOuVUqGrGpRXhMBtmA
# DSzdgn5jqW5lI1po9FqGdlPTFlhstpMH3DSfPQWurvJh42oM38gmSEHLBNpc4tXo
# 8udMYI09H/kHUoNMTZNGjnZO9LfarGsag6eOJP1bMMublhRlKCaL9RIyV9oOHycE
# IeOeQFeBP/BmYFLWbVPeZej7uiqsEc7VPDJK2QXns210UYanaWmggkmpdAr0I0EV
# pEKHSfVv1qlIlFH4d7MhcJzP2/rY62EC5tYQjT0UaBnCRcDInKrNWa3kbDL0akwr
# 0aJgpbT5ipknVChtwnMWJlbqpeW/VUF5g0jVpYQ3jbe/Zf+OtmU=
# =Pv8z
# -----END PGP SIGNATURE-----
# gpg: Signature made Fri 03 Nov 2023 20:04:40 HKT
# gpg:                using RSA key 1899FF8EDEBF58CCEE034B82F487EF185872D723
# gpg: Good signature from "Juan Quintela <quintela@redhat.com>" [full]
# gpg:                 aka "Juan Quintela <quintela@trasno.org>" [full]
# Primary key fingerprint: 1899 FF8E DEBF 58CC EE03  4B82 F487 EF18 5872 D723

* tag 'migration-20231103-pull-request' of https://gitlab.com/juan.quintela/qemu:
  migration: Unlock mutex in error case
  docs/migration: Add the dirty limit section
  tests/migration: Introduce dirty-limit into guestperf
  tests/migration: Introduce dirty-ring-size option into guestperf
  tests: Add migration dirty-limit capability test
  system/dirtylimit: Drop the reduplicative check
  system/dirtylimit: Fix a race situation

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

1 files changed, 71 insertions, 0 deletions

diff --git a/docs/devel/migration.rst b/docs/devel/migration.rst
index 240eb16d90..5adf4f12f7 100644
--- a/docs/devel/migration.rst
+++ b/docs/devel/migration.rst
@@ -594,6 +594,77 @@ path.
      Return path  - opened by main thread, written by main thread AND postcopy
      thread (protected by rp_mutex)
 
+Dirty limit
+=====================
+The dirty limit, short for dirty page rate upper limit, is a new capability
+introduced in the 8.1 QEMU release that uses a new algorithm based on the KVM
+dirty ring to throttle down the guest during live migration.
+
+The algorithm framework is as follows:
+
+::
+
+  ------------------------------------------------------------------------------
+  main   --------------> throttle thread ------------> PREPARE(1) <--------
+  thread  \                                                |              |
+           \                                               |              |
+            \                                              V              |
+             -\                                        CALCULATE(2)       |
+               \                                           |              |
+                \                                          |              |
+                 \                                         V              |
+                  \                                    SET PENALTY(3) -----
+                   -\                                      |
+                     \                                     |
+                      \                                    V
+                       -> virtual CPU thread -------> ACCEPT PENALTY(4)
+  ------------------------------------------------------------------------------
+
+When the qmp command qmp_set_vcpu_dirty_limit is called for the first time,
+the QEMU main thread starts the throttle thread. The throttle thread, once
+launched, executes the loop, which consists of three steps:
+
+  - PREPARE (1)
+
+     The entire work of PREPARE (1) is preparation for the second stage,
+     CALCULATE(2), as the name implies. It involves preparing the dirty
+     page rate value and the corresponding upper limit of the VM:
+     The dirty page rate is calculated via the KVM dirty ring mechanism,
+     which tells QEMU how many dirty pages a virtual CPU has had since the
+     last KVM_EXIT_DIRTY_RING_FULL exception; The dirty page rate upper
+     limit is specified by caller, therefore fetch it directly.
+
+  - CALCULATE (2)
+
+     Calculate a suitable sleep period for each virtual CPU, which will be
+     used to determine the penalty for the target virtual CPU. The
+     computation must be done carefully in order to reduce the dirty page
+     rate progressively down to the upper limit without oscillation. To
+     achieve this, two strategies are provided: the first is to add or
+     subtract sleep time based on the ratio of the current dirty page rate
+     to the limit, which is used when the current dirty page rate is far
+     from the limit; the second is to add or subtract a fixed time when
+     the current dirty page rate is close to the limit.
+
+  - SET PENALTY (3)
+
+     Set the sleep time for each virtual CPU that should be penalized based
+     on the results of the calculation supplied by step CALCULATE (2).
+
+After completing the three above stages, the throttle thread loops back
+to step PREPARE (1) until the dirty limit is reached.
+
+On the other hand, each virtual CPU thread reads the sleep duration and
+sleeps in the path of the KVM_EXIT_DIRTY_RING_FULL exception handler, that
+is ACCEPT PENALTY (4). Virtual CPUs tied with writing processes will
+obviously exit to the path and get penalized, whereas virtual CPUs involved
+with read processes will not.
+
+In summary, thanks to the KVM dirty ring technology, the dirty limit
+algorithm will restrict virtual CPUs as needed to keep their dirty page
+rate inside the limit. This leads to more steady reading performance during
+live migration and can aid in improving large guest responsiveness.
+
 Postcopy
 ========