From patchwork Wed Apr 22 17:36:29 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Luke Nelson X-Patchwork-Id: 1275338 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=cs.washington.edu Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=cs.washington.edu header.i=@cs.washington.edu header.a=rsa-sha256 header.s=goo201206 header.b=Wuka4V05; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 496ngF3K88z9sSM for ; Thu, 23 Apr 2020 03:36:57 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726632AbgDVRgx (ORCPT ); Wed, 22 Apr 2020 13:36:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46722 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1726057AbgDVRgw (ORCPT ); Wed, 22 Apr 2020 13:36:52 -0400 Received: from mail-pl1-x642.google.com (mail-pl1-x642.google.com [IPv6:2607:f8b0:4864:20::642]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E1739C03C1AA for ; Wed, 22 Apr 2020 10:36:52 -0700 (PDT) Received: by mail-pl1-x642.google.com with SMTP id z6so1204815plk.10 for ; Wed, 22 Apr 2020 10:36:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cs.washington.edu; s=goo201206; h=from:to:cc:subject:date:message-id; bh=pKJyhXiuj2Ugps4yZFwXPuLBd6CO2XHLk23yr11XPUU=; b=Wuka4V05WpA33NOcmcV0RQM00BdjsKcI5C+v7II9CcXxeP5oTz0fUXRvK2hWnStDuu B+edBNqq5jKuoLBSsqY1Pn9fsX4nweAnbBF6UDJ12raphayaz3bglemue6WYhqD0h0y3 3XsrUI1ME6gxllaG/ZuBxLXkSNvdepQMAF3MA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=pKJyhXiuj2Ugps4yZFwXPuLBd6CO2XHLk23yr11XPUU=; b=VcSGtjASAxEHyY3MMtsrI3U8zgedwGFIG/mmomW94iWV5LouNqIpOg1eWRWlfr9wfD HtcU9gFWIvJjfR+Nx0J6DmHdd5Nl8Je8FgqWde2VpDNJzWW1phjQx4zLvdSLEl/E9AVB o8B7VmnaWfvXUeYD9rVOErsR/d05biFgqRJhJGpBlGI6Kp/Wjju1x8LEF8NCUgdVh6JU OVriEJCzea4Fuli/04KS1sOYI3Aib02+WWGCxcx/g8GsAc+SOHJ3JpU8bnNVe/FPay3V hXRmUss/MYuqmMBTNeiLSTW8v0NtgrSVKfFeKM5S7S5TSAJbTgg0m0n8EpaYaZvUjEE0 U38g== X-Gm-Message-State: AGi0PubIFf57gyjzKjSByR8Cl4Gt3rDm9/KFtknj3Eg7fCSpZr1G5NzC YeMYQulEWhKGrtPVtuR2Qwehbw== X-Google-Smtp-Source: APiQypLrcgjFGa1gJskocnU6csUZlxRb1FTlBXblSlEAwl5l99pthecQNlgX1AdQ79pDzCy2Jf4H2g== X-Received: by 2002:a17:90a:db46:: with SMTP id u6mr13131918pjx.15.1587577012060; Wed, 22 Apr 2020 10:36:52 -0700 (PDT) Received: from localhost.localdomain (c-73-53-94-119.hsd1.wa.comcast.net. [73.53.94.119]) by smtp.gmail.com with ESMTPSA id 1sm59514pff.151.2020.04.22.10.36.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Apr 2020 10:36:51 -0700 (PDT) From: Luke Nelson X-Google-Original-From: Luke Nelson To: bpf@vger.kernel.org Cc: Brian Gerst , Luke Nelson , Xi Wang , Wang YanQing , "David S. Miller" , Alexey Kuznetsov , Hideaki YOSHIFUJI , Thomas Gleixner , Ingo Molnar , Borislav Petkov , x86@kernel.org, "H. Peter Anvin" , Alexei Starovoitov , Daniel Borkmann , Martin KaFai Lau , Song Liu , Yonghong Song , Andrii Nakryiko , John Fastabend , KP Singh , netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH bpf v2 1/2] bpf, x86_32: Fix incorrect encoding in BPF_LDX zero-extension Date: Wed, 22 Apr 2020 10:36:29 -0700 Message-Id: <20200422173630.8351-1-luke.r.nels@gmail.com> X-Mailer: git-send-email 2.17.1 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org The current JIT uses the following sequence to zero-extend into the upper 32 bits of the destination register for BPF_LDX BPF_{B,H,W}, when the destination register is not on the stack: EMIT3(0xC7, add_1reg(0xC0, dst_hi), 0); The problem is that C7 /0 encodes a MOV instruction that requires a 4-byte immediate; the current code emits only 1 byte of the immediate. This means that the first 3 bytes of the next instruction will be treated as the rest of the immediate, breaking the stream of instructions. This patch fixes the problem by instead emitting "xor dst_hi,dst_hi" to clear the upper 32 bits. This fixes the problem and is more efficient than using MOV to load a zero immediate. This bug may not be currently triggerable as BPF_REG_AX is the only register not stored on the stack and the verifier uses it in a limited way, and the verifier implements a zero-extension optimization. But the JIT should avoid emitting incorrect encodings regardless. Fixes: 03f5781be2c7b ("bpf, x86_32: add eBPF JIT compiler for ia32") Signed-off-by: Xi Wang Signed-off-by: Luke Nelson Acked-by: Wang YanQing Reviewed-by: H. Peter Anvin (Intel) --- v1 -> v2: Updated commit message to better reflect the bug. (H. Peter Anvin and Brian Gerst) --- arch/x86/net/bpf_jit_comp32.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/arch/x86/net/bpf_jit_comp32.c b/arch/x86/net/bpf_jit_comp32.c index 4d2a7a764602..cc9ad3892ea6 100644 --- a/arch/x86/net/bpf_jit_comp32.c +++ b/arch/x86/net/bpf_jit_comp32.c @@ -1854,7 +1854,9 @@ static int do_jit(struct bpf_prog *bpf_prog, int *addrs, u8 *image, STACK_VAR(dst_hi)); EMIT(0x0, 4); } else { - EMIT3(0xC7, add_1reg(0xC0, dst_hi), 0); + /* xor dst_hi,dst_hi */ + EMIT2(0x33, + add_2reg(0xC0, dst_hi, dst_hi)); } break; case BPF_DW: From patchwork Wed Apr 22 17:36:30 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Luke Nelson X-Patchwork-Id: 1275339 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=cs.washington.edu Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=cs.washington.edu header.i=@cs.washington.edu header.a=rsa-sha256 header.s=goo201206 header.b=NBfIrE2N; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 496ngH2Bj7z9sSM for ; Thu, 23 Apr 2020 03:36:59 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726746AbgDVRgz (ORCPT ); Wed, 22 Apr 2020 13:36:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46730 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1726006AbgDVRgy (ORCPT ); Wed, 22 Apr 2020 13:36:54 -0400 Received: from mail-pl1-x641.google.com (mail-pl1-x641.google.com [IPv6:2607:f8b0:4864:20::641]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EA62AC03C1A9 for ; Wed, 22 Apr 2020 10:36:53 -0700 (PDT) Received: by mail-pl1-x641.google.com with SMTP id v2so1206123plp.9 for ; Wed, 22 Apr 2020 10:36:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cs.washington.edu; s=goo201206; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=upttJD2FlVOwQ9TcHJ5acrg3a8ThasRlmqF+H/glYss=; b=NBfIrE2NjATpSGVDPNE+eakCKhcK491SMXS/kR2TKZL8ND/4YKdgiho0fYch+YM0x3 XKtxnoC3m7J4rsHW0IJSKDqWhhsUHzLRXNRQ+dXmwE7HM4hPbnbD9SG3pq+bonaFUv1N OZsmFi2XvW6HsoLuZg3mj5gSfzRVJyzs3/4LY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=upttJD2FlVOwQ9TcHJ5acrg3a8ThasRlmqF+H/glYss=; b=FdNd5Eja6miVm/KynqOr9njOqRvUIOnK1/1jZtv1mNoTOWCRQmeTo7EPX+/bwXeyCu uBqUEmm/ym4YmYpSs6f4bQhDiJt4rKGI0uH4mABu1hk1eI5sDWQ0etiQQSnkvfIKE5L3 ZjesrhelsGdqLZR9P//NlQ3FqqKHNOgw2Gl3ftclDHoybjtqzaO3WZ9xxjZPP4mAu9cr ONPadddrcNdgYMe8qdvwgtz0sCIa0FQ1xV4iY1NEHUqGlGMkvTb3FOU2eXzHV5p1Y98m HEk6+xqRt22yQDJ7VKYkbxPenhuSWESz0waQ7jPgR1P02BusvOpJrTjcxlZ8fTHJj1/8 dkjA== X-Gm-Message-State: AGi0PuZxbRxEZsyXFU7NZNgVsdbZIW5p2qvuEG9TWQksm3llhgavHbHr 2v5TuIgTP6qp+pxpanE4QM1Jow== X-Google-Smtp-Source: APiQypLCQlRsvYOCNibbXTv7c6Hr1Yx/DPiTsvnw69F/7aVpdz8tsAYZaMLNIKgW3A4iiDXLYk8n6Q== X-Received: by 2002:a17:90a:b10f:: with SMTP id z15mr11890687pjq.188.1587577013353; Wed, 22 Apr 2020 10:36:53 -0700 (PDT) Received: from localhost.localdomain (c-73-53-94-119.hsd1.wa.comcast.net. [73.53.94.119]) by smtp.gmail.com with ESMTPSA id 1sm59514pff.151.2020.04.22.10.36.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Apr 2020 10:36:52 -0700 (PDT) From: Luke Nelson X-Google-Original-From: Luke Nelson To: bpf@vger.kernel.org Cc: Brian Gerst , Luke Nelson , Xi Wang , "David S. Miller" , Alexey Kuznetsov , Hideaki YOSHIFUJI , Wang YanQing , Thomas Gleixner , Ingo Molnar , Borislav Petkov , x86@kernel.org, "H. Peter Anvin" , Alexei Starovoitov , Daniel Borkmann , Martin KaFai Lau , Song Liu , Yonghong Song , Andrii Nakryiko , John Fastabend , KP Singh , netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH bpf v2 2/2] bpf, x86_32: Fix clobbering of dst for BPF_JSET Date: Wed, 22 Apr 2020 10:36:30 -0700 Message-Id: <20200422173630.8351-2-luke.r.nels@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200422173630.8351-1-luke.r.nels@gmail.com> References: <20200422173630.8351-1-luke.r.nels@gmail.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org The current JIT clobbers the destination register for BPF_JSET BPF_X and BPF_K by using "and" and "or" instructions. This is fine when the destination register is a temporary loaded from a register stored on the stack but not otherwise. This patch fixes the problem (for both BPF_K and BPF_X) by always loading the destination register into temporaries since BPF_JSET should not modify the destination register. This bug may not be currently triggerable as BPF_REG_AX is the only register not stored on the stack and the verifier uses it in a limited way. Fixes: 03f5781be2c7b ("bpf, x86_32: add eBPF JIT compiler for ia32") Signed-off-by: Xi Wang Signed-off-by: Luke Nelson Acked-by: Wang YanQing --- v1 -> v2: No changes. --- arch/x86/net/bpf_jit_comp32.c | 22 ++++++++++++++++++---- 1 file changed, 18 insertions(+), 4 deletions(-) diff --git a/arch/x86/net/bpf_jit_comp32.c b/arch/x86/net/bpf_jit_comp32.c index cc9ad3892ea6..ba7d9ccfc662 100644 --- a/arch/x86/net/bpf_jit_comp32.c +++ b/arch/x86/net/bpf_jit_comp32.c @@ -2015,8 +2015,8 @@ static int do_jit(struct bpf_prog *bpf_prog, int *addrs, u8 *image, case BPF_JMP | BPF_JSET | BPF_X: case BPF_JMP32 | BPF_JSET | BPF_X: { bool is_jmp64 = BPF_CLASS(insn->code) == BPF_JMP; - u8 dreg_lo = dstk ? IA32_EAX : dst_lo; - u8 dreg_hi = dstk ? IA32_EDX : dst_hi; + u8 dreg_lo = IA32_EAX; + u8 dreg_hi = IA32_EDX; u8 sreg_lo = sstk ? IA32_ECX : src_lo; u8 sreg_hi = sstk ? IA32_EBX : src_hi; @@ -2028,6 +2028,13 @@ static int do_jit(struct bpf_prog *bpf_prog, int *addrs, u8 *image, add_2reg(0x40, IA32_EBP, IA32_EDX), STACK_VAR(dst_hi)); + } else { + /* mov dreg_lo,dst_lo */ + EMIT2(0x89, add_2reg(0xC0, dreg_lo, dst_lo)); + if (is_jmp64) + /* mov dreg_hi,dst_hi */ + EMIT2(0x89, + add_2reg(0xC0, dreg_hi, dst_hi)); } if (sstk) { @@ -2052,8 +2059,8 @@ static int do_jit(struct bpf_prog *bpf_prog, int *addrs, u8 *image, case BPF_JMP | BPF_JSET | BPF_K: case BPF_JMP32 | BPF_JSET | BPF_K: { bool is_jmp64 = BPF_CLASS(insn->code) == BPF_JMP; - u8 dreg_lo = dstk ? IA32_EAX : dst_lo; - u8 dreg_hi = dstk ? IA32_EDX : dst_hi; + u8 dreg_lo = IA32_EAX; + u8 dreg_hi = IA32_EDX; u8 sreg_lo = IA32_ECX; u8 sreg_hi = IA32_EBX; u32 hi; @@ -2066,6 +2073,13 @@ static int do_jit(struct bpf_prog *bpf_prog, int *addrs, u8 *image, add_2reg(0x40, IA32_EBP, IA32_EDX), STACK_VAR(dst_hi)); + } else { + /* mov dreg_lo,dst_lo */ + EMIT2(0x89, add_2reg(0xC0, dreg_lo, dst_lo)); + if (is_jmp64) + /* mov dreg_hi,dst_hi */ + EMIT2(0x89, + add_2reg(0xC0, dreg_hi, dst_hi)); } /* mov ecx,imm32 */