From patchwork Wed Apr 5 00:08:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thadeu Lima de Souza Cascardo X-Patchwork-Id: 1765250 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=BD46yKkp; dkim-atps=neutral Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4PrlPl5qhxz1yZT for ; Wed, 5 Apr 2023 10:10:03 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1pjqio-0007YL-K5; Wed, 05 Apr 2023 00:09:54 +0000 Received: from smtp-relay-canonical-0.internal ([10.131.114.83] helo=smtp-relay-canonical-0.canonical.com) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1pjqif-0007MD-PE for kernel-team@lists.ubuntu.com; Wed, 05 Apr 2023 00:09:45 +0000 Received: from localhost.localdomain (1.general.cascardo.us.vpn [10.172.70.58]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-canonical-0.canonical.com (Postfix) with ESMTPSA id 82A223F0EB for ; Wed, 5 Apr 2023 00:09:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1680653384; bh=zH9wCafmJMwTjfGv9uZ9YOoLLc0HKDZhKqEj8UNBDhk=; h=From:To:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=BD46yKkpqh37cA4lqDAtQZ5NC83rxEU5NvWLDwSV1xmNxqstz3F/w6twLqPk+zBHe xqWL0d6xS9W17JToL1JUBmRpmlO0+dqKbi0Q/iB83x0gSIxu276YvKe2DsZcx5vKKW iWlnRrsNsXDaLL/FCzEV4DVTJnEnvRP1AhP7citMc0+BP4VIjVFVZx8Gs0GxO1x8sA WBlkmj0j/Gs9OgF7LyYIUzVgNq/D10nQHx+QBp8Vkfx3OwUOGX6B1aOG+kdqhSgmNH d1JvSIzCO8+l/Nh7Xh+pjpm7SsR0HBRtGzoSEEhGQb1QlgMnxELEEKHPcr74wX9ZTD dmkvJpP8K9IrQ== From: Thadeu Lima de Souza Cascardo To: kernel-team@lists.ubuntu.com Subject: [UBUNTU OEM-6.0 3/5] io_uring: cmpxchg for poll arm refs release Date: Tue, 4 Apr 2023 21:08:25 -0300 Message-Id: <20230405000827.2250965-9-cascardo@canonical.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230405000827.2250965-1-cascardo@canonical.com> References: <20230405000827.2250965-1-cascardo@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Pavel Begunkov Replace atomically substracting the ownership reference at the end of arming a poll with a cmpxchg. We try to release ownership by setting 0 assuming that poll_refs didn't change while we were arming. If it did change, we keep the ownership and use it to queue a tw, which is fully capable to process all events and (even tolerates spurious wake ups). It's a bit more elegant as we reduce races b/w setting the cancellation flag and getting refs with this release, and with that we don't have to worry about any kinds of underflows. It's not the fastest path for polling. The performance difference b/w cmpxchg and atomic dec is usually negligible and it's not the fastest path. Cc: stable@vger.kernel.org Fixes: aa43477b04025 ("io_uring: poll rework") Signed-off-by: Pavel Begunkov Link: https://lore.kernel.org/r/0c95251624397ea6def568ff040cad2d7926fd51.1668963050.git.asml.silence@gmail.com Signed-off-by: Jens Axboe (cherry picked from commit 2f3893437a4ebf2e892ca172e9e122841319d675) CVE-2023-0468 Signed-off-by: Thadeu Lima de Souza Cascardo --- io_uring/poll.c | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/io_uring/poll.c b/io_uring/poll.c index 92e3fdd3caa1..9c8064f32aef 100644 --- a/io_uring/poll.c +++ b/io_uring/poll.c @@ -509,7 +509,6 @@ static int __io_arm_poll_handler(struct io_kiocb *req, unsigned issue_flags) { struct io_ring_ctx *ctx = req->ctx; - int v; INIT_HLIST_NODE(&req->hash_node); req->work.cancel_seq = atomic_read(&ctx->cancel_seq); @@ -577,11 +576,10 @@ static int __io_arm_poll_handler(struct io_kiocb *req, if (ipt->owning) { /* - * Release ownership. If someone tried to queue a tw while it was - * locked, kick it off for them. + * Try to release ownership. If we see a change of state, e.g. + * poll was waken up, queue up a tw, it'll deal with it. */ - v = atomic_dec_return(&req->poll_refs); - if (unlikely(v & IO_POLL_REF_MASK)) + if (atomic_cmpxchg(&req->poll_refs, 1, 0) != 1) __io_poll_execute(req, 0); } return 0;