From patchwork Tue Jan 2 15:03:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "H.J. Lu" X-Patchwork-Id: 1881645 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20230601 header.b=frQW2UQ/; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=sourceware.org (client-ip=8.43.85.97; helo=server2.sourceware.org; envelope-from=libc-alpha-bounces+incoming=patchwork.ozlabs.org@sourceware.org; receiver=patchwork.ozlabs.org) Received: from server2.sourceware.org (server2.sourceware.org [8.43.85.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4T4GMd60Z6z20LT for ; Wed, 3 Jan 2024 02:03:57 +1100 (AEDT) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id BF4163858C2C for ; Tue, 2 Jan 2024 15:03:55 +0000 (GMT) X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-pf1-x436.google.com (mail-pf1-x436.google.com [IPv6:2607:f8b0:4864:20::436]) by sourceware.org (Postfix) with ESMTPS id 083A53858C41 for ; Tue, 2 Jan 2024 15:03:33 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 083A53858C41 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=gmail.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 083A53858C41 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2607:f8b0:4864:20::436 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1704207816; cv=none; b=XuBpTpUw+1BTvRV2Pz0O8rztrmR2gaO5tC+ioNTPrvkevH7khl6Yyj/WW4t8SkTg3kCIp+j3oQKdbH9w0WrxsczzflFmlFAQ6HchLncGk7BdJdO/IFd+Lx7OP/G/mYieoJoy/j6mUHKscIL3HCUUhQIpVcQDbKO2Z1sKyxBScoo= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1704207816; c=relaxed/simple; bh=73GUKI8g+O+H3z/5HW8S98jH5E5OmdzUTTGJzBIwApI=; h=DKIM-Signature:From:To:Subject:Date:Message-ID:MIME-Version; b=Vjoll3F9FfFUS5YqN6EI4Olhlooq4IVxZUgsAq3N5T0iOqeIVXECVXx909QULSFX7UrnvEpioSZ2E4qenZnV64s5EE496R1gjaP+TzzCiuVlhYV+LTnD/zMhPbizOEN02W9JXBTo7ZsBEqW8DpifNTMb7L5muNXnHLyFjL/vt1w= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-pf1-x436.google.com with SMTP id d2e1a72fcca58-6d9a795cffbso4387969b3a.0 for ; Tue, 02 Jan 2024 07:03:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1704207812; x=1704812612; darn=sourceware.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=pdHh2XNQ4jMTsxbsWNlaBDuUaUOQGTJynEGrEV0FfhA=; b=frQW2UQ/v8BZnFqW8weYWbn1EwqNHUR7EYL/plNwFrNFIdbo8Iwd8uUxZqIYr5L5eG WvaEYnIXJ7NxiUGVhxUGWo2TrWweVBs1J2oLYpeH9ySbIGbO+SGukuSCoVSle6KhrAAb 5CZw8hQQYtZA2Ds4KC2b9QA25pCl5ffLWdiC2CStXUilnnPhkR51I+5i/E6WPLdvYsq5 9rj8nzxBYPY3AYhSrciPfa86y6/zQUWY13I7MgxOJWaSyTifHNhKKWkfuJ4fIf3uxrYL ryK6nOoHlHFa39cUT8TSRQqUAonhXHy6uXGs1capTU3rZjy5BhwtO516dLD+52EFOpXG x8Tw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1704207812; x=1704812612; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=pdHh2XNQ4jMTsxbsWNlaBDuUaUOQGTJynEGrEV0FfhA=; b=hrEgZhNmwcjg1BLyWRl0EGH4iP6xti9bJc7wzYEI350Qnf5AWHip4jJq3nJlkwmCbk tDEUi5lB5nc1GHIA5Wj0IwQV4cjzDxpUL8dSDSNVmVMKpBsfxvUmGWarGJaVFIzbCqOv 8PiYM2sSwl5sq89h3C1DDlq4ycV4GoFVOxjiew/ci6DXU62TOxYo6ZldhBcv3yNqEfC/ 7Q4VEdPUHanZ7kXfs2l781jY3HmkVXrM7qavmP9ckfAqx3ABhEzaIGAa+l0HRIYMdgcb z1ntPYcUDP742RNaJKiqEtTgzakGtndjB14YxmcfBs+mEqoVd7EXQCqYcnNbsPOTlabT dWVQ== X-Gm-Message-State: AOJu0YyA3Gw+SHFq+BZdjDzQuRJMsIV3fqI671a99B1ps1ppizoD1XvF c7+s10XZyi8X0M1g3/q7LMXHQMT4PpA= X-Google-Smtp-Source: AGHT+IHqjHS2Gm6piqRHHrJUo7PhIPme2XWigjc4RRy+tcTcnPvk0sl8Dep2JgHkrYrNPtVV/TcmDA== X-Received: by 2002:a05:6a00:1c85:b0:6d9:bee5:702d with SMTP id y5-20020a056a001c8500b006d9bee5702dmr15255210pfw.18.1704207811652; Tue, 02 Jan 2024 07:03:31 -0800 (PST) Received: from gnu-cfl-3.localdomain ([172.56.169.119]) by smtp.gmail.com with ESMTPSA id ka8-20020a056a00938800b006d9ef76732esm12101939pfb.213.2024.01.02.07.03.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jan 2024 07:03:31 -0800 (PST) Received: from gnu-cfl-3.. (localhost [IPv6:::1]) by gnu-cfl-3.localdomain (Postfix) with ESMTP id C29EB74016D; Tue, 2 Jan 2024 07:03:29 -0800 (PST) From: "H.J. Lu" To: libc-alpha@sourceware.org Cc: rick.p.edgecombe@intel.com Subject: [PATCH v2] x86-64/cet: Check the restore token in longjmp Date: Tue, 2 Jan 2024 07:03:29 -0800 Message-ID: <20240102150329.3152784-1-hjl.tools@gmail.com> X-Mailer: git-send-email 2.43.0 MIME-Version: 1.0 X-Spam-Status: No, score=-3024.6 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FREEMAIL_FROM, GIT_PATCH_0, KAM_SHORT, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces+incoming=patchwork.ozlabs.org@sourceware.org Changes in v2: 1. Merge __longjmp.S and __longjmp_chk.S by adding CHECK_INVALID_LONGJMP. 2. Remove a branch in the restore token searching loop. 3. Cache the target shadow stack pointer. setcontext and swapcontext put a restore token on the old shadow stack which is used to restore the target shadow stack when switching user contexts. When longjmp from a user context, the target shadow stack can be different from the current shadow stack and INCSSP can't be used to restore the shadow stack pointer to the target shadow stack. Update longjmp to search for a restore token. If found, use the token to restore the shadow stack pointer before using INCSSP to pop the shadow stack. Stop the token search and use INCSSP if the shadow stack entry value is the same as the current shadow stack pointer. It is a user error if there is a shadow stack switch without leaving a restore token on the old shadow stack. The only difference between __longjmp.S and __longjmp_chk.S is that __longjmp_chk.S has a check for invalid longjmp usages. Merge __longjmp.S and __longjmp_chk.S by adding the CHECK_INVALID_LONGJMP macro. Reviewed-by: Noah Goldstein --- .../unix/sysv/linux/x86_64/____longjmp_chk.S | 179 ++++-------------- sysdeps/x86/__longjmp_cancel.S | 3 + sysdeps/x86_64/__longjmp.S | 47 ++++- 3 files changed, 84 insertions(+), 145 deletions(-) diff --git a/sysdeps/unix/sysv/linux/x86_64/____longjmp_chk.S b/sysdeps/unix/sysv/linux/x86_64/____longjmp_chk.S index deb6398f43..9aa24620b9 100644 --- a/sysdeps/unix/sysv/linux/x86_64/____longjmp_chk.S +++ b/sysdeps/unix/sysv/linux/x86_64/____longjmp_chk.S @@ -15,18 +15,7 @@ License along with the GNU C Library; if not, see . */ -#include -#include -#include -#include -#include #include -#include - -/* Don't restore shadow stack register if shadow stack isn't enabled. */ -#if !SHSTK_ENABLED -# undef SHADOW_STACK_POINTER_OFFSET -#endif .section .rodata.str1.1,"aMS",@progbits,1 .type longjmp_msg,@object @@ -34,136 +23,48 @@ longjmp_msg: .string "longjmp causes uninitialized stack frame" .size longjmp_msg, .-longjmp_msg - -//#define __longjmp ____longjmp_chk - #ifdef PIC -# define CALL_FAIL sub $8, %RSP_LP; \ - cfi_remember_state; \ - cfi_def_cfa_offset(16); \ - lea longjmp_msg(%rip), %RDI_LP; \ - call HIDDEN_JUMPTARGET(__fortify_fail); \ - nop; \ - cfi_restore_state +# define LOAD_MSG lea longjmp_msg(%rip), %RDI_LP #else -# define CALL_FAIL sub $8, %RSP_LP; \ - cfi_remember_state; \ - cfi_def_cfa_offset(16); \ - mov $longjmp_msg, %RDI_LP; \ - call HIDDEN_JUMPTARGET(__fortify_fail); \ - nop; \ - cfi_restore_state +# define LOAD_MSG mov $longjmp_msg, %RDI_LP #endif -/* Jump to the position specified by ENV, causing the - setjmp call there to return VAL, or 1 if VAL is 0. - void __longjmp (__jmp_buf env, int val). */ - .text -ENTRY(____longjmp_chk) - /* Restore registers. */ - mov (JB_RSP*8)(%rdi), %R8_LP - mov (JB_RBP*8)(%rdi),%R9_LP - mov (JB_PC*8)(%rdi), %RDX_LP -#ifdef PTR_DEMANGLE - PTR_DEMANGLE (%R8_LP) - PTR_DEMANGLE (%R9_LP) - PTR_DEMANGLE (%RDX_LP) -# ifdef __ILP32__ - /* We ignored the high bits of the %rbp value because only the low - bits are mangled. But we cannot presume that %rbp is being used - as a pointer and truncate it, so recover the high bits. */ - movl (JB_RBP*8 + 4)(%rdi), %eax - shlq $32, %rax - orq %rax, %r9 -# endif -#endif - - cmp %R8_LP, %RSP_LP - jbe .Lok - - /* Save function parameters. */ - movq %rdi, %r10 - cfi_register (%rdi, %r10) - movl %esi, %ebx - cfi_register (%rsi, %rbx) - - xorl %edi, %edi - lea -sizeSS(%rsp), %RSI_LP - movl $__NR_sigaltstack, %eax - syscall - /* Without working sigaltstack we cannot perform the test. */ - testl %eax, %eax - jne .Lok2 - testl $1, (-sizeSS + oSS_FLAGS)(%rsp) - jz .Lfail - - mov (-sizeSS + oSS_SP)(%rsp), %RAX_LP - add (-sizeSS + oSS_SIZE)(%rsp), %RAX_LP - sub %R8_LP, %RAX_LP - cmp (-sizeSS + oSS_SIZE)(%rsp), %RAX_LP - jae .Lok2 - -.Lfail: CALL_FAIL - -.Lok2: movq %r10, %rdi - cfi_restore (%rdi) - movl %ebx, %esi - cfi_restore (%rsi) - +#define CHECK_INVALID_LONGJMP \ + cmp %R8_LP, %RSP_LP; \ + jbe .Lok; \ + /* Save function parameters. */ \ + movq %rdi, %r10; \ + cfi_register (%rdi, %r10); \ + movl %esi, %ebx; \ + cfi_register (%rsi, %rbx); \ + xorl %edi, %edi; \ + lea -sizeSS(%rsp), %RSI_LP; \ + movl $__NR_sigaltstack, %eax; \ + syscall; \ + /* Without working sigaltstack we cannot perform the test. */ \ + testl %eax, %eax; \ + jne .Lok2; \ + testl $1, (-sizeSS + oSS_FLAGS)(%rsp); \ + jz .Lfail; \ + mov (-sizeSS + oSS_SP)(%rsp), %RAX_LP; \ + add (-sizeSS + oSS_SIZE)(%rsp), %RAX_LP; \ + sub %R8_LP, %RAX_LP; \ + cmp (-sizeSS + oSS_SIZE)(%rsp), %RAX_LP; \ + jae .Lok2; \ +.Lfail: \ + sub $8, %RSP_LP; \ + cfi_remember_state; \ + cfi_def_cfa_offset(16); \ + LOAD_MSG; \ + call HIDDEN_JUMPTARGET(__fortify_fail); \ + nop; \ + cfi_restore_state; \ +.Lok2: \ + movq %r10, %rdi; \ + cfi_restore (%rdi); \ + movl %ebx, %esi; \ + cfi_restore (%rsi); \ .Lok: -#ifdef SHADOW_STACK_POINTER_OFFSET -# if IS_IN (libc) && defined SHARED && defined FEATURE_1_OFFSET - /* Check if Shadow Stack is enabled. */ - testl $X86_FEATURE_1_SHSTK, %fs:FEATURE_1_OFFSET - jz L(skip_ssp) -# else - xorl %eax, %eax -# endif - /* Check and adjust the Shadow-Stack-Pointer. */ - rdsspq %rax - /* And compare it with the saved ssp value. */ - subq SHADOW_STACK_POINTER_OFFSET(%rdi), %rax - je L(skip_ssp) - /* Count the number of frames to adjust and adjust it - with incssp instruction. The instruction can adjust - the ssp by [0..255] value only thus use a loop if - the number of frames is bigger than 255. */ - negq %rax - shrq $3, %rax - /* NB: We saved Shadow-Stack-Pointer of setjmp. Since we are - restoring Shadow-Stack-Pointer of setjmp's caller, we - need to unwind shadow stack by one more frame. */ - addq $1, %rax - movl $255, %ebx -L(loop): - cmpq %rbx, %rax - cmovb %rax, %rbx - incsspq %rbx - subq %rbx, %rax - ja L(loop) -L(skip_ssp): -#endif - LIBC_PROBE (longjmp, 3, LP_SIZE@%RDI_LP, -4@%esi, LP_SIZE@%RDX_LP) - /* We add unwind information for the target here. */ - cfi_def_cfa(%rdi, 0) - cfi_register(%rsp,%r8) - cfi_register(%rbp,%r9) - cfi_register(%rip,%rdx) - cfi_offset(%rbx,JB_RBX*8) - cfi_offset(%r12,JB_R12*8) - cfi_offset(%r13,JB_R13*8) - cfi_offset(%r14,JB_R14*8) - cfi_offset(%r15,JB_R15*8) - movq (JB_RBX*8)(%rdi), %rbx - movq (JB_R12*8)(%rdi), %r12 - movq (JB_R13*8)(%rdi), %r13 - movq (JB_R14*8)(%rdi), %r14 - movq (JB_R15*8)(%rdi), %r15 - /* Set return value for setjmp. */ - movl %esi, %eax - mov %R8_LP, %RSP_LP - movq %r9,%rbp - LIBC_PROBE (longjmp_target, 3, - LP_SIZE@%RDI_LP, -4@%eax, LP_SIZE@%RDX_LP) - jmpq *%rdx -END (____longjmp_chk) + +#define __longjmp ____longjmp_chk +#include <__longjmp.S> diff --git a/sysdeps/x86/__longjmp_cancel.S b/sysdeps/x86/__longjmp_cancel.S index e71b304257..b03f52b308 100644 --- a/sysdeps/x86/__longjmp_cancel.S +++ b/sysdeps/x86/__longjmp_cancel.S @@ -16,5 +16,8 @@ License along with the GNU C Library; if not, see . */ +/* Don't restore shadow stack register for __longjmp_cancel. */ +#define DO_NOT_RESTORE_SHADOW_STACK + #define __longjmp __longjmp_cancel #include <__longjmp.S> diff --git a/sysdeps/x86_64/__longjmp.S b/sysdeps/x86_64/__longjmp.S index c9f70f8e2a..22fedc4997 100644 --- a/sysdeps/x86_64/__longjmp.S +++ b/sysdeps/x86_64/__longjmp.S @@ -22,14 +22,15 @@ #include #include -/* Don't restore shadow stack register if - 1. Shadow stack isn't enabled. Or - 2. __longjmp is defined for __longjmp_cancel. - */ -#if !SHSTK_ENABLED || defined __longjmp +/* Don't restore shadow stack register if shadow stack isn't enabled. */ +#if !SHSTK_ENABLED || defined DO_NOT_RESTORE_SHADOW_STACK # undef SHADOW_STACK_POINTER_OFFSET #endif +#ifndef CHECK_INVALID_LONGJMP +# define CHECK_INVALID_LONGJMP +#endif + /* Jump to the position specified by ENV, causing the setjmp call there to return VAL, or 1 if VAL is 0. void __longjmp (__jmp_buf env, int val). */ @@ -52,6 +53,9 @@ ENTRY(__longjmp) orq %rax, %r9 # endif #endif + + CHECK_INVALID_LONGJMP + #ifdef SHADOW_STACK_POINTER_OFFSET # if IS_IN (libc) && defined SHARED && defined FEATURE_1_OFFSET /* Check if Shadow Stack is enabled. */ @@ -63,9 +67,40 @@ ENTRY(__longjmp) /* Check and adjust the Shadow-Stack-Pointer. */ /* Get the current ssp. */ rdsspq %rax + /* Save the current ssp. */ + movq %rax, %r10 /* And compare it with the saved ssp value. */ - subq SHADOW_STACK_POINTER_OFFSET(%rdi), %rax + movq SHADOW_STACK_POINTER_OFFSET(%rdi), %rcx + subq %rcx, %rax je L(skip_ssp) + + /* Save the target ssp. */ + movq %rcx, %r11 + +L(find_restore_token_loop): + /* Look for a restore token. */ + movq -8(%rcx), %rbx + andq $-8, %rbx + cmpq %rcx, %rbx + /* Find the restore token. */ + je L(restore_shadow_stack) + + /* Try the next slot. */ + subq $8, %rcx + /* Stop if the current ssp is found. */ + cmpq %rcx, %r10 + jne L(find_restore_token_loop) + jmp L(no_shadow_stack_token) + +L(restore_shadow_stack): + /* Restore the target shadow stack. */ + rstorssp -8(%rcx) + /* Save the restore token on the old shadow stack. */ + saveprevssp + rdsspq %rax + subq %r11, %rax + +L(no_shadow_stack_token): /* Count the number of frames to adjust and adjust it with incssp instruction. The instruction can adjust the ssp by [0..255] value only thus use a loop if