From patchwork Wed Jul 5 05:49:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: ~hyman X-Patchwork-Id: 1803426 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Qwpgc4pXtz20bK for ; Wed, 5 Jul 2023 15:51:24 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qGvOU-0005UZ-Ji; Wed, 05 Jul 2023 01:49:38 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qGvOF-0005Rr-K2 for qemu-devel@nongnu.org; Wed, 05 Jul 2023 01:49:23 -0400 Received: from mail-b.sr.ht ([173.195.146.151]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qGvOD-00084i-7x for qemu-devel@nongnu.org; Wed, 05 Jul 2023 01:49:23 -0400 Authentication-Results: mail-b.sr.ht; dkim=none Received: from git.sr.ht (unknown [173.195.146.142]) by mail-b.sr.ht (Postfix) with ESMTPSA id E3FBD11EEE1; Wed, 5 Jul 2023 05:49:19 +0000 (UTC) From: ~hyman Date: Wed, 05 Jul 2023 05:49:19 +0000 MIME-Version: 1.0 Subject: [PATCH QEMU v7 0/9] migration: introduce dirtylimit capability Message-ID: <168853615963.17240.15832775267134683267-0@git.sr.ht> X-Mailer: git.sr.ht To: qemu-devel Cc: Peter Xu , Paolo Bonzini , Juan Quintela , "Dr. David Alan Gilbert" , Eric Blake , Markus Armbruster , Thomas Huth , Laurent Vivier , Richard Henderson , Philippe =?utf-8?q?Mathie?= =?utf-8?q?u-Daud=C3=A9?= , Hyman =?utf-8?b?SHVhbmco6buE?= =?utf-8?b?5YuHKQ==?= Received-SPF: pass client-ip=173.195.146.151; envelope-from=outgoing@sr.ht; helo=mail-b.sr.ht X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: ~hyman Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Hi, Juan, this version maybe the last version, i rebase on master and fix the conflicts with the switchover-ack capability. Please be free to the take this version to make PR or the previous version if you already fixed conflicts. v7: 1. Rebase on master and fix conflicts v6: 1. Rebase on master 2. Split the commit "Implement dirty-limit convergence algo" into two as Juan suggested as the following: a. Put the detection logic before auto-converge checking b. Implement dirty-limit convergence algo 3. Put the detection logic before auto-converge checking 4. Sort the migrate_dirty_limit function in commit "Introduce dirty-limit capability" suggested by Juan 5. Substitute the the int64_t to uint64_t in the last 2 commits 6. Fix the comments spell mistake 7. Add helper function in the commit "Implement dirty-limit convergence algo" suggested by Juan v5: 1. Rebase on master and enrich the comment for "dirty-limit" capability, suggesting by Markus. 2. Drop commits that have already been merged. v4: 1. Polish the docs and update the release version suggested by Markus 2. Rename the migrate exported info "dirty-limit-throttle-time-per- round" to "dirty-limit-throttle-time-per-full". v3(resend): - fix the syntax error of the topic. v3: This version make some modifications inspired by Peter and Markus as following: 1. Do the code clean up in [PATCH v2 02/11] suggested by Markus 2. Replace the [PATCH v2 03/11] with a much simpler patch posted by Peter to fix the following bug: https://bugzilla.redhat.com/show_bug.cgi?id=2124756 3. Fix the error path of migrate_params_check in [PATCH v2 04/11] pointed out by Markus. Enrich the commit message to explain why x-vcpu-dirty-limit-period an unstable parameter. 4. Refactor the dirty-limit convergence algo in [PATCH v2 07/11] suggested by Peter: a. apply blk_mig_bulk_active check before enable dirty-limit b. drop the unhelpful check function before enable dirty-limit c. change the migration_cancel logic, just cancel dirty-limit only if dirty-limit capability turned on. d. abstract a code clean commit [PATCH v3 07/10] to adjust the check order before enable auto-converge 5. Change the name of observing indexes during dirty-limit live migration to make them more easy-understanding. Use the maximum throttle time of vpus as "dirty-limit-throttle-time-per-full" 6. Fix some grammatical and spelling errors pointed out by Markus and enrich the document about the dirty-limit live migration observing indexes "dirty-limit-ring-full-time" and "dirty-limit-throttle-time-per-full" 7. Change the default value of x-vcpu-dirty-limit-period to 1000ms, which is optimal value pointed out in cover letter in that testing environment. 8. Drop the 2 guestperf test commits [PATCH v2 10/11], [PATCH v2 11/11] and post them with a standalone series in the future. v2: This version make a little bit modifications comparing with version 1 as following: 1. fix the overflow issue reported by Peter Maydell 2. add parameter check for hmp "set_vcpu_dirty_limit" command 3. fix the racing issue between dirty ring reaper thread and Qemu main thread. 4. add migrate parameter check for x-vcpu-dirty-limit-period and vcpu-dirty-limit. 5. add the logic to forbid hmp/qmp commands set_vcpu_dirty_limit, cancel_vcpu_dirty_limit during dirty-limit live migration when implement dirty-limit convergence algo. 6. add capability check to ensure auto-converge and dirty-limit are mutually exclusive. 7. pre-check if kvm dirty ring size is configured before setting dirty-limit migrate parameter Hyman Huang(黄勇) (9): softmmu/dirtylimit: Add parameter check for hmp "set_vcpu_dirty_limit" qapi/migration: Introduce x-vcpu-dirty-limit-period parameter qapi/migration: Introduce vcpu-dirty-limit parameters migration: Introduce dirty-limit capability migration: Refactor auto-converge capability logic migration: Put the detection logic before auto-converge checking migration: Implement dirty-limit convergence algo migration: Extend query-migrate to provide dirty page limit info tests: Add migration dirty-limit capability test include/sysemu/dirtylimit.h | 2 + migration/migration-hmp-cmds.c | 26 ++++++ migration/migration.c | 13 +++ migration/options.c | 73 ++++++++++++++++ migration/options.h | 1 + migration/ram.c | 61 ++++++++++--- migration/trace-events | 1 + qapi/migration.json | 75 ++++++++++++++-- softmmu/dirtylimit.c | 91 +++++++++++++++++-- tests/qtest/migration-test.c | 155 +++++++++++++++++++++++++++++++++ 10 files changed, 473 insertions(+), 25 deletions(-)