From patchwork Thu Nov 22 07:20:23 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xiao Guangrong X-Patchwork-Id: 1001569 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="Us9aeLgv"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 430rTm0nmRz9s29 for ; Thu, 22 Nov 2018 18:21:30 +1100 (AEDT) Received: from localhost ([::1]:44373 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gPjIh-0002gp-Pv for incoming@patchwork.ozlabs.org; Thu, 22 Nov 2018 02:21:23 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:51044) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gPjI2-0002gT-IQ for qemu-devel@nongnu.org; Thu, 22 Nov 2018 02:20:46 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gPjHy-0001gV-5D for qemu-devel@nongnu.org; Thu, 22 Nov 2018 02:20:42 -0500 Received: from mail-pg1-x541.google.com ([2607:f8b0:4864:20::541]:39996) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gPjHx-0001f9-Sm for qemu-devel@nongnu.org; Thu, 22 Nov 2018 02:20:38 -0500 Received: by mail-pg1-x541.google.com with SMTP id z10so1312366pgp.7 for ; Wed, 21 Nov 2018 23:20:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=/kYC6HFF3VRChbJ95X4GaAaMwwZ2vS0aZhIwXhtjcKI=; b=Us9aeLgv+HcA6bqswIp0+RIAu10tYOaNgwneIebpdncfyB8UiP2TRtQt+afwUfHN68 rP0L+UaRBDpuTH/Jl3m2rt/o9fEnLJbKDXWUTijyCMKAO2cwZM8RJCj5lFK+2HAQx6Nd +vYML/P/lYDEJJjpclOJPoD7hP1UsafAWkWzd5gct00rqISIPJ8+g+qNQwNWWZ25GXFx i/oP9ErBEr/jYf50CT6VX/wQ32D1zy2RbmCqxU2RcuSa0nMhx+D7f8c8aeDpp4RuLV25 cg1+C32h2zOLD7W5uxb2nv7tjPn4VGz/I4+0Bxuj7lC49855wM0YtDZbXO3ZGjQicb3Y A8DQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=/kYC6HFF3VRChbJ95X4GaAaMwwZ2vS0aZhIwXhtjcKI=; b=LJnUYoxqfM/thuEsV1tJG/iKa3kjxRiUMyvR57Vk8l9Bx4/rmjAb47gNv3LzDd36gD 5T7VuShdqtsa6l2MaM1wAvtAfPeXDeYCPthmRW8UxqxJQ916tLfod4nRo+tS3h/GCM+6 AbcUra5lXxHOxmNF/fKK3vkgzbdrNltQE0b2fQwd/9jrpptwAWmTtLS3S4YmZj5ncwjv R9rv1+dceZEltMAyx3R8x8sh4fXarmJXKqyUOJarQaciq500MJsVxKiEuHU5ZmwSOCii 55FwpWVbEtAB/DHKnwrCGVzzQP5hlnj2z1YwGBzB4ZdlWrLFODLeV/UzTYnUJZUKyVwv Kfxg== X-Gm-Message-State: AA+aEWYeKPp/69PXxU2MK3eplxSP2K0T+58JHRscx2tf/L9/syRU6ZuN TrZrT31mW/uvf7DFElp0NMA= X-Google-Smtp-Source: AFSGD/Vk7jZ9jBaDbZJK+FAIU3BIwlb10fxTalaprqkiN/GfuIAKvJv7w3St8WnWa9Vxhs6Xrswxrw== X-Received: by 2002:a62:4e83:: with SMTP id c125mr8397848pfb.101.1542871236636; Wed, 21 Nov 2018 23:20:36 -0800 (PST) Received: from localhost.localdomain ([203.205.141.36]) by smtp.gmail.com with ESMTPSA id 19sm52731312pfs.108.2018.11.21.23.20.32 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 21 Nov 2018 23:20:36 -0800 (PST) From: guangrong.xiao@gmail.com X-Google-Original-From: xiaoguangrong@tencent.com To: pbonzini@redhat.com, mst@redhat.com, mtosatti@redhat.com Date: Thu, 22 Nov 2018 15:20:23 +0800 Message-Id: <20181122072028.22819-1-xiaoguangrong@tencent.com> X-Mailer: git-send-email 2.14.5 X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::541 Subject: [Qemu-devel] [PATCH v3 0/5] migration: improve multithreads X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kvm@vger.kernel.org, quintela@redhat.com, Xiao Guangrong , qemu-devel@nongnu.org, peterx@redhat.com, dgilbert@redhat.com, wei.w.wang@intel.com, cota@braap.org, jiang.biao2@zte.com.cn Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: Xiao Guangrong Changelog in v3: Thanks to Emilio's comments and his example code, the changes in this version are: 1. move @requests from the shared data struct to each single thread 2. move completion ev from the shared data struct to each single thread 3. move bitmaps from the shared data struct to each single thread 4. limit the number of request that each thread need handle to 64, then use uint64_t instead of bitmap pointer. The performance is measured by using the benchmark we introduced in this pachset: ./tests/threaded-workqueue-bench -c 20 -m 16 -t N the data is as followings: The baseline of v2: Thread #, Throughput 1, 0.428024 4, 1.668876 8, 3.501940 12, 5.026403 16, 1.912374 20, 1.174771 24, 1.074085 28, 0.747920 32, 0.651409 36, 0.533240 40, 0.517421 44, 0.482153 48, 0.525176 52, 0.492677 56, 0.798679 60, 0.733868 64, 0.751396 After this patchset: Thread #, Throughput 1, 0.449192 4, 1.849271 8, 3.644339 12, 4.809391 16, 4.709095 20, 4.942153 24, 5.116967 28, 4.921542 32, 5.008816 36, 5.408070 40, 5.166064 44, 4.994953 48, 4.853351 52, 4.797540 56, 4.815153 60, 4.793704 64, 4.913544 To see more detailed compression by each step, please refer to https://ibb.co/hq7u5V Xiao Guangrong (5): bitops: introduce change_bit_atomic util: introduce threaded workqueue migration: use threaded workqueue for compression migration: use threaded workqueue for decompression tests: add threaded-workqueue-bench include/qemu/bitops.h | 13 + include/qemu/threaded-workqueue.h | 106 ++++++++ migration/ram.c | 530 ++++++++++++++------------------------ tests/Makefile.include | 5 +- tests/threaded-workqueue-bench.c | 255 ++++++++++++++++++ util/Makefile.objs | 1 + util/threaded-workqueue.c | 463 +++++++++++++++++++++++++++++++++ 7 files changed, 1029 insertions(+), 344 deletions(-) create mode 100644 include/qemu/threaded-workqueue.h create mode 100644 tests/threaded-workqueue-bench.c create mode 100644 util/threaded-workqueue.c