From patchwork Mon Jun 12 09:42:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cengiz Can X-Patchwork-Id: 1793839 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=fR/muZE7; dkim-atps=neutral Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QfmwH73sPz20Vx for ; Mon, 12 Jun 2023 19:43:43 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1q8e5J-00017m-NK; Mon, 12 Jun 2023 09:43:37 +0000 Received: from smtp-relay-internal-1.internal ([10.131.114.114] helo=smtp-relay-internal-1.canonical.com) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1q8e5I-00017c-F9 for kernel-team@lists.ubuntu.com; Mon, 12 Jun 2023 09:43:36 +0000 Received: from mail-lf1-f71.google.com (mail-lf1-f71.google.com [209.85.167.71]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-internal-1.canonical.com (Postfix) with ESMTPS id C16D83F11D for ; Mon, 12 Jun 2023 09:43:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1686563015; bh=58+Zb9nt8am/AlxbbAxKEUXs6GDJWTP81b1/s+Csevo=; h=From:To:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=fR/muZE7qmAsTEqy6SuQtGiy+PA1Iy7e/hgCUOPjK/m+vL9eYyZidRnkf4Cp2qimC txzjN1r08Newsuxmj7I3T29YspRS4hhqX6J9sda2mwq4IhI7/utU7MN0O9PZkdTe9N XD/jHqAgB8g6ErXXbubgkk3rpKh0mfgC/yw70odfQOEzEyDW43D7NcojC+nICB5t3l EE9xrVGqn11L7XkTA9WxcYvM8rw+LlRrfvo5W+2TbcrfFZCRWi+Zoi0D0gEkertnb0 SaX2VpDDtkv8c9AsGaTtjqKqp+IuavYvbfuOd9Szk6ax84iaGsh84YfbjM0RKXMslU tA1BH/QnQLSgA== Received: by mail-lf1-f71.google.com with SMTP id 2adb3069b0e04-4f6461620fcso2973967e87.0 for ; Mon, 12 Jun 2023 02:43:35 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686563015; x=1689155015; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=58+Zb9nt8am/AlxbbAxKEUXs6GDJWTP81b1/s+Csevo=; b=H3qSQgkyCrl0MC67TFRP647BVqPTMAyApd97X05PUZh+EJci/aOTU4yZceQNuOgnmY lQ89M62MnVPLBo8Lao9Ch3mO9CiNBVKoDsKgDfV8oqcAXdhOEbcu5ueBW/fpwjg84sln v7AdDFwrfL0ha2f+1caxRlw7HgkwA+GlTdDIHzxcetLERm2PXUOVPkWwuN9lfV1MYPlv 61js5OXjaTZfQxvRfLqmRTveYzhcWqAgEIbfNSUnaiW/ieEHvmJvWm9DIPphFU6lI+up MMyI5YsPil9uS/V98XBZQFVzDJBLK6x5/2EDIf/RVvlYeVrDNjtOuCxQMcG9wHDaEZZw iFhQ== X-Gm-Message-State: AC+VfDwwHk71ceywLWGpULwdgldeA6QdtbNcM4OBserOUIdrMMGqA2bB x7oFSAfXurbjHo+FQsiESIvl59C+A+hUi/IYp17yUGIeY0sbPUa0VFyZL7YxR0yFMoEyO4nbDFU GWZfA5gyc2iZ3Mr2vV9R1IxvkO+iy+PV/ES3rs3dqa1T5kpb565KaWao= X-Received: by 2002:ac2:5f82:0:b0:4ed:c5a4:28dc with SMTP id r2-20020ac25f82000000b004edc5a428dcmr3289316lfe.38.1686563014845; Mon, 12 Jun 2023 02:43:34 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4bGfet4iRn4l8Bi6j/Q1OZIrOw5B3QCqrVOk6oiTfoMMN/nUAcXnXiDDZmYoN61bddwLp5YQ== X-Received: by 2002:ac2:5f82:0:b0:4ed:c5a4:28dc with SMTP id r2-20020ac25f82000000b004edc5a428dcmr3289308lfe.38.1686563014358; Mon, 12 Jun 2023 02:43:34 -0700 (PDT) Received: from localhost (uk.sesame.canonical.com. [185.125.190.60]) by smtp.gmail.com with ESMTPSA id g26-20020a7bc4da000000b003f60e143d38sm10823401wmk.11.2023.06.12.02.43.33 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Jun 2023 02:43:34 -0700 (PDT) From: Cengiz Can To: kernel-team@lists.ubuntu.com Subject: [SRU Jammy, OEM-5.17, Kinetic, OEM-6.0, Lunar PATCH 1/1] xfs: verify buffer contents when we skip log replay Date: Mon, 12 Jun 2023 12:42:36 +0300 Message-Id: <20230612094235.693450-3-cengiz.can@canonical.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230612094235.693450-1-cengiz.can@canonical.com> References: <20230612094235.693450-1-cengiz.can@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: "Darrick J. Wong" syzbot detected a crash during log recovery: XFS (loop0): Mounting V5 Filesystem bfdc47fc-10d8-4eed-a562-11a831b3f791 XFS (loop0): Torn write (CRC failure) detected at log block 0x180. Truncating head block from 0x200. XFS (loop0): Starting recovery (logdev: internal) ================================================================== BUG: KASAN: slab-out-of-bounds in xfs_btree_lookup_get_block+0x15c/0x6d0 fs/xfs/libxfs/xfs_btree.c:1813 Read of size 8 at addr ffff88807e89f258 by task syz-executor132/5074 CPU: 0 PID: 5074 Comm: syz-executor132 Not tainted 6.2.0-rc1-syzkaller #0 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 10/26/2022 Call Trace: __dump_stack lib/dump_stack.c:88 [inline] dump_stack_lvl+0x1b1/0x290 lib/dump_stack.c:106 print_address_description+0x74/0x340 mm/kasan/report.c:306 print_report+0x107/0x1f0 mm/kasan/report.c:417 kasan_report+0xcd/0x100 mm/kasan/report.c:517 xfs_btree_lookup_get_block+0x15c/0x6d0 fs/xfs/libxfs/xfs_btree.c:1813 xfs_btree_lookup+0x346/0x12c0 fs/xfs/libxfs/xfs_btree.c:1913 xfs_btree_simple_query_range+0xde/0x6a0 fs/xfs/libxfs/xfs_btree.c:4713 xfs_btree_query_range+0x2db/0x380 fs/xfs/libxfs/xfs_btree.c:4953 xfs_refcount_recover_cow_leftovers+0x2d1/0xa60 fs/xfs/libxfs/xfs_refcount.c:1946 xfs_reflink_recover_cow+0xab/0x1b0 fs/xfs/xfs_reflink.c:930 xlog_recover_finish+0x824/0x920 fs/xfs/xfs_log_recover.c:3493 xfs_log_mount_finish+0x1ec/0x3d0 fs/xfs/xfs_log.c:829 xfs_mountfs+0x146a/0x1ef0 fs/xfs/xfs_mount.c:933 xfs_fs_fill_super+0xf95/0x11f0 fs/xfs/xfs_super.c:1666 get_tree_bdev+0x400/0x620 fs/super.c:1282 vfs_get_tree+0x88/0x270 fs/super.c:1489 do_new_mount+0x289/0xad0 fs/namespace.c:3145 do_mount fs/namespace.c:3488 [inline] __do_sys_mount fs/namespace.c:3697 [inline] __se_sys_mount+0x2d3/0x3c0 fs/namespace.c:3674 do_syscall_x64 arch/x86/entry/common.c:50 [inline] do_syscall_64+0x3d/0xb0 arch/x86/entry/common.c:80 entry_SYSCALL_64_after_hwframe+0x63/0xcd RIP: 0033:0x7f89fa3f4aca Code: 83 c4 08 5b 5d c3 66 2e 0f 1f 84 00 00 00 00 00 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 49 89 ca b8 a5 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 c0 ff ff ff f7 d8 64 89 01 48 RSP: 002b:00007fffd5fb5ef8 EFLAGS: 00000206 ORIG_RAX: 00000000000000a5 RAX: ffffffffffffffda RBX: 00646975756f6e2c RCX: 00007f89fa3f4aca RDX: 0000000020000100 RSI: 0000000020009640 RDI: 00007fffd5fb5f10 RBP: 00007fffd5fb5f10 R08: 00007fffd5fb5f50 R09: 000000000000970d R10: 0000000000200800 R11: 0000000000000206 R12: 0000000000000004 R13: 0000555556c6b2c0 R14: 0000000000200800 R15: 00007fffd5fb5f50 The fuzzed image contains an AGF with an obviously garbage agf_refcount_level value of 32, and a dirty log with a buffer log item for that AGF. The ondisk AGF has a higher LSN than the recovered log item. xlog_recover_buf_commit_pass2 reads the buffer, compares the LSNs, and decides to skip replay because the ondisk buffer appears to be newer. Unfortunately, the ondisk buffer is corrupt, but recovery just read the buffer with no buffer ops specified: error = xfs_buf_read(mp->m_ddev_targp, buf_f->blf_blkno, buf_f->blf_len, buf_flags, &bp, NULL); Skipping the buffer leaves its contents in memory unverified. This sets us up for a kernel crash because xfs_refcount_recover_cow_leftovers reads the buffer (which is still around in XBF_DONE state, so no read verification) and creates a refcountbt cursor of height 32. This is impossible so we run off the end of the cursor object and crash. Fix this by invoking the verifier on all skipped buffers and aborting log recovery if the ondisk buffer is corrupt. It might be smarter to force replay the log item atop the buffer and then see if it'll pass the write verifier (like ext4 does) but for now let's go with the conservative option where we stop immediately. Link: https://syzkaller.appspot.com/bug?extid=7e9494b8b399902e994e Signed-off-by: Darrick J. Wong Reviewed-by: Dave Chinner Signed-off-by: Dave Chinner CVE-2023-2124 (cherry picked from commit 22ed903eee23a5b174e240f1cdfa9acf393a5210) Signed-off-by: Cengiz Can --- fs/xfs/xfs_buf_item_recover.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/fs/xfs/xfs_buf_item_recover.c b/fs/xfs/xfs_buf_item_recover.c index ffa94102094d..43167f543afc 100644 --- a/fs/xfs/xfs_buf_item_recover.c +++ b/fs/xfs/xfs_buf_item_recover.c @@ -943,6 +943,16 @@ xlog_recover_buf_commit_pass2( if (lsn && lsn != -1 && XFS_LSN_CMP(lsn, current_lsn) >= 0) { trace_xfs_log_recover_buf_skip(log, buf_f); xlog_recover_validate_buf_type(mp, bp, buf_f, NULLCOMMITLSN); + + /* + * We're skipping replay of this buffer log item due to the log + * item LSN being behind the ondisk buffer. Verify the buffer + * contents since we aren't going to run the write verifier. + */ + if (bp->b_ops) { + bp->b_ops->verify_read(bp); + error = bp->b_error; + } goto out_release; }