From patchwork Sat Nov 17 18:57:12 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 999388 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=sparclinux-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="HCrZnyJK"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 42y4905yLvz9s3x for ; Sun, 18 Nov 2018 05:57:24 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726582AbeKRFO4 (ORCPT ); Sun, 18 Nov 2018 00:14:56 -0500 Received: from mail-pf1-f196.google.com ([209.85.210.196]:40188 "EHLO mail-pf1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726075AbeKRFO4 (ORCPT ); Sun, 18 Nov 2018 00:14:56 -0500 Received: by mail-pf1-f196.google.com with SMTP id x2-v6so12925012pfm.7 for ; Sat, 17 Nov 2018 10:57:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=vKtcwajq6uet7RwpHABNJRRfvhJ5RLh+7DwnrP6tkoU=; b=HCrZnyJKCfFnJaZK1DpCvziEmRstnj4rwYTbT92UFXz73rvqQ9ZSjTGDU8wRnGgNJ+ ExMscyjZDgE+uBnbk94MoYbWGi/Ppcc1XZG5FsvUrcfYFnni03qp7O8Pw8kD/mV3EtA8 4qJ0sf0q5nq3snS/fKcmV1/2jIO7ddsYaFvVw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=vKtcwajq6uet7RwpHABNJRRfvhJ5RLh+7DwnrP6tkoU=; b=B9ZYWgbjrVt8A8/f8DRUgWmYosWwLtoQv4ugGAHA7JbZkuJovW1bbd5m8zhe6ukDMR FKK1nyIU1FlgnY/y9YQcItQdkr+GoFxmG6GhUVqE7Jord0WZUUF+R38zusFane8xDw+e F/7y/GWvFbbKVPyCYDVJJf4ff2JievivuVfMwl78Lc2uB1ht2gF6QkpOr401Hci6is5L UZzMoE9uRyy21JGtWvTp2Rk2lDUv8iQpfFvqE7ASUaOVjbFLe9SL7EeQpBDP6NqHDB5G DHNNYbzkg3tKbhbqIbRFbxwpfbnyYTlTfh1pQAZdlXQ1gz8PGS8q7+rW1FUPyBjNhIVU iqAA== X-Gm-Message-State: AGRZ1gK0idxcB+HPX4sq0P+u9ZId+VoKkMeXJfaCEUwkSBm/UtbTfP4o Pi8rC/bnL/GiYEs8oCu3fHF13Q== X-Google-Smtp-Source: AJdET5ctCscFODEPbn/yoyf1Ns6z0NZ0bHOkoDOM6hIBx8Sdy8wtnp67zeIE7sjr1zu/AsWtPwDO4g== X-Received: by 2002:a62:cac4:: with SMTP id y65-v6mr16264821pfk.27.1542481040868; Sat, 17 Nov 2018 10:57:20 -0800 (PST) Received: from mba13.psav.com ([64.114.255.114]) by smtp.gmail.com with ESMTPSA id u76-v6sm49550745pfa.176.2018.11.17.10.57.19 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 17 Nov 2018 10:57:20 -0800 (PST) From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: Ard Biesheuvel , Daniel Borkmann , Alexei Starovoitov , Rick Edgecombe , Eric Dumazet , Jann Horn , Kees Cook , Jessica Yu , Arnd Bergmann , Catalin Marinas , Will Deacon , Mark Rutland , Ralf Baechle , Paul Burton , James Hogan , Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , "David S. Miller" , linux-arm-kernel@lists.infradead.org, linux-mips@linux-mips.org, linuxppc-dev@lists.ozlabs.org, sparclinux@vger.kernel.org, netdev@vger.kernel.org Subject: [PATCH 1/4] bpf: account for freed JIT allocations in arch code Date: Sat, 17 Nov 2018 10:57:12 -0800 Message-Id: <20181117185715.25198-2-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20181117185715.25198-1-ard.biesheuvel@linaro.org> References: <20181117185715.25198-1-ard.biesheuvel@linaro.org> Sender: sparclinux-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: sparclinux@vger.kernel.org Commit ede95a63b5e84 ("bpf: add bpf_jit_limit knob to restrict unpriv allocations") added a call to bpf_jit_uncharge_modmem() to the routine bpf_jit_binary_free() which is called from the __weak bpf_jit_free(). This function is overridden by arches, some of which do not call bpf_jit_binary_free() to release the memory, and so the released memory is not accounted for, potentially leading to spurious allocation failures. So replace the direct calls to module_memfree() in the arch code with calls to bpf_jit_binary_free(). Signed-off-by: Ard Biesheuvel --- arch/mips/net/bpf_jit.c | 2 +- arch/powerpc/net/bpf_jit_comp.c | 2 +- arch/powerpc/net/bpf_jit_comp64.c | 5 +---- arch/sparc/net/bpf_jit_comp_32.c | 2 +- 4 files changed, 4 insertions(+), 7 deletions(-) diff --git a/arch/mips/net/bpf_jit.c b/arch/mips/net/bpf_jit.c index 4d8cb9bb8365..1b69897274a1 100644 --- a/arch/mips/net/bpf_jit.c +++ b/arch/mips/net/bpf_jit.c @@ -1264,7 +1264,7 @@ void bpf_jit_compile(struct bpf_prog *fp) void bpf_jit_free(struct bpf_prog *fp) { if (fp->jited) - module_memfree(fp->bpf_func); + bpf_jit_binary_free(bpf_jit_binary_hdr(fp)); bpf_prog_unlock_free(fp); } diff --git a/arch/powerpc/net/bpf_jit_comp.c b/arch/powerpc/net/bpf_jit_comp.c index d5bfe24bb3b5..a1ea1ea6b40d 100644 --- a/arch/powerpc/net/bpf_jit_comp.c +++ b/arch/powerpc/net/bpf_jit_comp.c @@ -683,7 +683,7 @@ void bpf_jit_compile(struct bpf_prog *fp) void bpf_jit_free(struct bpf_prog *fp) { if (fp->jited) - module_memfree(fp->bpf_func); + bpf_jit_binary_free(bpf_jit_binary_hdr(fp)); bpf_prog_unlock_free(fp); } diff --git a/arch/powerpc/net/bpf_jit_comp64.c b/arch/powerpc/net/bpf_jit_comp64.c index 50b129785aee..84c8f013a6c6 100644 --- a/arch/powerpc/net/bpf_jit_comp64.c +++ b/arch/powerpc/net/bpf_jit_comp64.c @@ -1024,11 +1024,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp) /* Overriding bpf_jit_free() as we don't set images read-only. */ void bpf_jit_free(struct bpf_prog *fp) { - unsigned long addr = (unsigned long)fp->bpf_func & PAGE_MASK; - struct bpf_binary_header *bpf_hdr = (void *)addr; - if (fp->jited) - bpf_jit_binary_free(bpf_hdr); + bpf_jit_binary_free(bpf_jit_binary_hdr(fp)); bpf_prog_unlock_free(fp); } diff --git a/arch/sparc/net/bpf_jit_comp_32.c b/arch/sparc/net/bpf_jit_comp_32.c index a5ff88643d5c..01bda6bc9e7f 100644 --- a/arch/sparc/net/bpf_jit_comp_32.c +++ b/arch/sparc/net/bpf_jit_comp_32.c @@ -759,7 +759,7 @@ cond_branch: f_offset = addrs[i + filter[i].jf]; void bpf_jit_free(struct bpf_prog *fp) { if (fp->jited) - module_memfree(fp->bpf_func); + bpf_jit_binary_free(bpf_jit_binary_hdr(fp)); bpf_prog_unlock_free(fp); } From patchwork Sat Nov 17 18:57:13 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 999393 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=sparclinux-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="cObMuJm7"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 42y49J3kZLz9s3x for ; Sun, 18 Nov 2018 05:57:40 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726872AbeKRFO5 (ORCPT ); Sun, 18 Nov 2018 00:14:57 -0500 Received: from mail-pg1-f196.google.com ([209.85.215.196]:33631 "EHLO mail-pg1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726376AbeKRFO5 (ORCPT ); Sun, 18 Nov 2018 00:14:57 -0500 Received: by mail-pg1-f196.google.com with SMTP id z11so9398639pgu.0 for ; Sat, 17 Nov 2018 10:57:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=FFyK4Oz1QZNyPCsvoKn4MQd4VSSgFUXXhCBgp8QbhcQ=; b=cObMuJm7WoQyjusLl+MSm4ZIOr4K71j4aNd5tPxwLRhwvXT/sYcvL7RDfA86wcLwA1 MBgV1MZKVJe1BNu/nkLqb6788B61g2+j8XPg6DFJaa+9SwGmzpOHS6y6Xqbefq4fvVkj +/MH7MuuSDpYgCOZS+o5G38042o7Ps9gEKLjw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=FFyK4Oz1QZNyPCsvoKn4MQd4VSSgFUXXhCBgp8QbhcQ=; b=I1KgUzsuOVIE7H7u/ww8adyE11UK6M9p/CvL3yKh/mAGg4v9tDRA7ShWFa1hQcMhug aN0TZdeGtjYQn6AtcN0CM/YiYjoQGZyvq8sH9wzHlLOK9TAPTBPjPc0vZ7eiustAJQ8p 6qMU2i0nVsf+KMotl6m0B9tXuSxpJ2pVRI2v34TA3cApKBib/lU9Fb3Dh5scOuRdX2zp Mej6N8Y9p18rNT5V8Mc2ZjQEpnc3DsoTmUv/AWAfAIfglK5HqvwXbD8OD3mi4ckXF3Ha stgimkQQeYwseVwfEnwHQDO8X/AU2SfQmIEWpOnLVjJv/YppSYuWcehuHBu1piJOyHfh 3jSw== X-Gm-Message-State: AGRZ1gJjNu15CkFQJiKlScrqNpwF0PPX1559hNEIYneZJ3kpSHDks9cR sJ+MKKfFHllTkLkrlsbjeJ4rbQ== X-Google-Smtp-Source: AJdET5eTfTm7MZO1Ys5mfFPU5yjtvrYsgtZia1/NgQb951W2tIaAMrSSEf/Q1za/zZx27f//YDj5kQ== X-Received: by 2002:a62:da5a:: with SMTP id w26mr3755848pfl.106.1542481042106; Sat, 17 Nov 2018 10:57:22 -0800 (PST) Received: from mba13.psav.com ([64.114.255.114]) by smtp.gmail.com with ESMTPSA id u76-v6sm49550745pfa.176.2018.11.17.10.57.20 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 17 Nov 2018 10:57:21 -0800 (PST) From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: Ard Biesheuvel , Daniel Borkmann , Alexei Starovoitov , Rick Edgecombe , Eric Dumazet , Jann Horn , Kees Cook , Jessica Yu , Arnd Bergmann , Catalin Marinas , Will Deacon , Mark Rutland , Ralf Baechle , Paul Burton , James Hogan , Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , "David S. Miller" , linux-arm-kernel@lists.infradead.org, linux-mips@linux-mips.org, linuxppc-dev@lists.ozlabs.org, sparclinux@vger.kernel.org, netdev@vger.kernel.org Subject: [PATCH 2/4] net/bpf: refactor freeing of executable allocations Date: Sat, 17 Nov 2018 10:57:13 -0800 Message-Id: <20181117185715.25198-3-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20181117185715.25198-1-ard.biesheuvel@linaro.org> References: <20181117185715.25198-1-ard.biesheuvel@linaro.org> Sender: sparclinux-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: sparclinux@vger.kernel.org All arch overrides of the __weak bpf_jit_free() amount to the same thing: the allocated memory was never mapped read-only, and so it does not have to be remapped to read-write before being freed. So in preparation of permitting arches to serve allocations for BPF JIT programs from other regions than the module region, refactor the existing bpf_jit_free() implementations to use the shared code where possible, and only specialize the remap and free operations. Signed-off-by: Ard Biesheuvel --- arch/mips/net/bpf_jit.c | 7 ++----- arch/powerpc/net/bpf_jit_comp.c | 7 ++----- arch/powerpc/net/bpf_jit_comp64.c | 9 +++------ arch/sparc/net/bpf_jit_comp_32.c | 7 ++----- kernel/bpf/core.c | 15 +++++---------- 5 files changed, 14 insertions(+), 31 deletions(-) diff --git a/arch/mips/net/bpf_jit.c b/arch/mips/net/bpf_jit.c index 1b69897274a1..5696bd7dccc7 100644 --- a/arch/mips/net/bpf_jit.c +++ b/arch/mips/net/bpf_jit.c @@ -1261,10 +1261,7 @@ void bpf_jit_compile(struct bpf_prog *fp) kfree(ctx.offsets); } -void bpf_jit_free(struct bpf_prog *fp) +void bpf_jit_binary_free(struct bpf_binary_header *hdr) { - if (fp->jited) - bpf_jit_binary_free(bpf_jit_binary_hdr(fp)); - - bpf_prog_unlock_free(fp); + module_memfree(hdr); } diff --git a/arch/powerpc/net/bpf_jit_comp.c b/arch/powerpc/net/bpf_jit_comp.c index a1ea1ea6b40d..5b5ce4a1b44b 100644 --- a/arch/powerpc/net/bpf_jit_comp.c +++ b/arch/powerpc/net/bpf_jit_comp.c @@ -680,10 +680,7 @@ void bpf_jit_compile(struct bpf_prog *fp) return; } -void bpf_jit_free(struct bpf_prog *fp) +void bpf_jit_binary_free(struct bpf_binary_header *hdr) { - if (fp->jited) - bpf_jit_binary_free(bpf_jit_binary_hdr(fp)); - - bpf_prog_unlock_free(fp); + module_memfree(hdr); } diff --git a/arch/powerpc/net/bpf_jit_comp64.c b/arch/powerpc/net/bpf_jit_comp64.c index 84c8f013a6c6..f64f1294bd62 100644 --- a/arch/powerpc/net/bpf_jit_comp64.c +++ b/arch/powerpc/net/bpf_jit_comp64.c @@ -1021,11 +1021,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp) return fp; } -/* Overriding bpf_jit_free() as we don't set images read-only. */ -void bpf_jit_free(struct bpf_prog *fp) +/* Overriding bpf_jit_binary_free() as we don't set images read-only. */ +void bpf_jit_binary_free(struct bpf_binary_header *hdr) { - if (fp->jited) - bpf_jit_binary_free(bpf_jit_binary_hdr(fp)); - - bpf_prog_unlock_free(fp); + module_memfree(hdr); } diff --git a/arch/sparc/net/bpf_jit_comp_32.c b/arch/sparc/net/bpf_jit_comp_32.c index 01bda6bc9e7f..589950d152cc 100644 --- a/arch/sparc/net/bpf_jit_comp_32.c +++ b/arch/sparc/net/bpf_jit_comp_32.c @@ -756,10 +756,7 @@ cond_branch: f_offset = addrs[i + filter[i].jf]; return; } -void bpf_jit_free(struct bpf_prog *fp) +void bpf_jit_binary_free(struct bpf_binary_header *hdr) { - if (fp->jited) - bpf_jit_binary_free(bpf_jit_binary_hdr(fp)); - - bpf_prog_unlock_free(fp); + module_memfree(hdr); } diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c index 1a796e0799ec..29f766dac203 100644 --- a/kernel/bpf/core.c +++ b/kernel/bpf/core.c @@ -646,25 +646,20 @@ bpf_jit_binary_alloc(unsigned int proglen, u8 **image_ptr, return hdr; } -void bpf_jit_binary_free(struct bpf_binary_header *hdr) +void __weak bpf_jit_binary_free(struct bpf_binary_header *hdr) { - u32 pages = hdr->pages; - + bpf_jit_binary_unlock_ro(hdr); module_memfree(hdr); - bpf_jit_uncharge_modmem(pages); } -/* This symbol is only overridden by archs that have different - * requirements than the usual eBPF JITs, f.e. when they only - * implement cBPF JIT, do not set images read-only, etc. - */ -void __weak bpf_jit_free(struct bpf_prog *fp) +void bpf_jit_free(struct bpf_prog *fp) { if (fp->jited) { struct bpf_binary_header *hdr = bpf_jit_binary_hdr(fp); + u32 pages = hdr->pages; - bpf_jit_binary_unlock_ro(hdr); bpf_jit_binary_free(hdr); + bpf_jit_uncharge_modmem(pages); WARN_ON_ONCE(!bpf_prog_kallsyms_verify_off(fp)); } From patchwork Sat Nov 17 18:57:14 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 999390 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=sparclinux-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="XYLhyO5m"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 42y4944MNSz9sCh for ; Sun, 18 Nov 2018 05:57:28 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726967AbeKRFO7 (ORCPT ); Sun, 18 Nov 2018 00:14:59 -0500 Received: from mail-pg1-f196.google.com ([209.85.215.196]:45882 "EHLO mail-pg1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726862AbeKRFO6 (ORCPT ); Sun, 18 Nov 2018 00:14:58 -0500 Received: by mail-pg1-f196.google.com with SMTP id y4so12019480pgc.12 for ; Sat, 17 Nov 2018 10:57:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=26kHMy6e9ZHNMFq26DfYcP5qT10rPVni7XnqYDLJuPI=; b=XYLhyO5mN3FVA4ReKitsYfMKGuHNDNlOhuo5CX3m1WdOPZeKdKBplc4oZZ6V6yFg5A hQ83SmcdvQq6M0spjr5rwWJUh9QdiW39oPX0Z2kNZVWF+j0XU6sZnr/m82angEa1AG65 GQ2t7WMlRjYOWyc5If1zGyUDez/kBmIctb+b4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=26kHMy6e9ZHNMFq26DfYcP5qT10rPVni7XnqYDLJuPI=; b=iTHbZ8HELNV1TYdrTQLlhVZi+FRgqvgmV1KOT1zXRERal9EJJ85E0zFMabvsDb2oCV oeAcv/3ks93a4E3gzefDEjpAHngllM3+yznNnew+4Dsu+WCF+44issksscI/XbuQGqUj wXIwX2OA7QO2yFlryHT2DP/LEjDYxd44mE8jVmF5U6yDZIvl+2H6V9mao7BiXVKV7nii oM+gq4Run8aHGDCeplaPLtFbndOx3a1wwkc3Jp0F9uihz5Wp6pwWOKxHryrihb01ny71 OFd/OhdtmPW/eheM0ZcQoMyT23yJnc+KBu+MzJhu0MnT8P4Z2diWMFqxjZ8Q8C8BY7RL rFBw== X-Gm-Message-State: AGRZ1gLFRoEnzINgFEIHfiQgulZf7fRNP8KKVz5dzJ1Ipw7cPoMZYMqI wSpObSaqd0kii3ofUT81qJAoMA== X-Google-Smtp-Source: AJdET5dbJskaau0oeh2D+j2r5+z2XCOLI6qC1Gpj92MrCzlhVrhcUg2kk12XFX1xpT0h9FiUT1nBcw== X-Received: by 2002:a63:f241:: with SMTP id d1mr14569834pgk.2.1542481043353; Sat, 17 Nov 2018 10:57:23 -0800 (PST) Received: from mba13.psav.com ([64.114.255.114]) by smtp.gmail.com with ESMTPSA id u76-v6sm49550745pfa.176.2018.11.17.10.57.22 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 17 Nov 2018 10:57:22 -0800 (PST) From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: Ard Biesheuvel , Daniel Borkmann , Alexei Starovoitov , Rick Edgecombe , Eric Dumazet , Jann Horn , Kees Cook , Jessica Yu , Arnd Bergmann , Catalin Marinas , Will Deacon , Mark Rutland , Ralf Baechle , Paul Burton , James Hogan , Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , "David S. Miller" , linux-arm-kernel@lists.infradead.org, linux-mips@linux-mips.org, linuxppc-dev@lists.ozlabs.org, sparclinux@vger.kernel.org, netdev@vger.kernel.org Subject: [PATCH 3/4] bpf: add __weak hook for allocating executable memory Date: Sat, 17 Nov 2018 10:57:14 -0800 Message-Id: <20181117185715.25198-4-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20181117185715.25198-1-ard.biesheuvel@linaro.org> References: <20181117185715.25198-1-ard.biesheuvel@linaro.org> Sender: sparclinux-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: sparclinux@vger.kernel.org By default, BPF uses module_alloc() to allocate executable memory, but this is not necessary on all arches and potentially undesirable on some of them. So break out the module_alloc() call into a __weak function to allow it to be overridden in arch code. Signed-off-by: Ard Biesheuvel --- kernel/bpf/core.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c index 29f766dac203..156d6b96ac6c 100644 --- a/kernel/bpf/core.c +++ b/kernel/bpf/core.c @@ -609,6 +609,11 @@ static void bpf_jit_uncharge_modmem(u32 pages) atomic_long_sub(pages, &bpf_jit_current); } +void *__weak bpf_jit_alloc_exec(unsigned long size) +{ + return module_alloc(size); +} + struct bpf_binary_header * bpf_jit_binary_alloc(unsigned int proglen, u8 **image_ptr, unsigned int alignment, @@ -626,7 +631,7 @@ bpf_jit_binary_alloc(unsigned int proglen, u8 **image_ptr, if (bpf_jit_charge_modmem(pages)) return NULL; - hdr = module_alloc(size); + hdr = bpf_jit_alloc_exec(size); if (!hdr) { bpf_jit_uncharge_modmem(pages); return NULL; From patchwork Sat Nov 17 18:57:15 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 999392 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=sparclinux-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="KAW8uS2e"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 42y4993zGyz9sB5 for ; Sun, 18 Nov 2018 05:57:33 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726995AbeKRFPC (ORCPT ); Sun, 18 Nov 2018 00:15:02 -0500 Received: from mail-pg1-f194.google.com ([209.85.215.194]:33633 "EHLO mail-pg1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726930AbeKRFO7 (ORCPT ); Sun, 18 Nov 2018 00:14:59 -0500 Received: by mail-pg1-f194.google.com with SMTP id z11so9398661pgu.0 for ; Sat, 17 Nov 2018 10:57:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=JmtMWaiD9FenKM3ns0kNxP2J4QR+RUeNI/5TJ/tQ45I=; b=KAW8uS2esdTjeBcDfFodpTuBiPuzrfOgpEvhmkdmizzqp7CMEjaFC2ORlF9MeL+yeA stdwlyxwK9cJAK+MAYm3DEfW4lRgMCiJRj3yewaXsrtQx5DmclC4R/K9qjVetVNcal+H pjN2zNK/65b6+lBFp6r24jInRxn3xDvrMAU8w= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=JmtMWaiD9FenKM3ns0kNxP2J4QR+RUeNI/5TJ/tQ45I=; b=E+JKrVCeBSlYd8wLIOK5fFpdQtKk1H5O+4sLLPbq8JvBBPoFnz4pnaWBc84Jd4eVHr BNoP9gQP9VBBG8g5suP7D1STLFWHUS0iRD5F0soh6gmv6wrYlwJVXD/3GcmOeFbFovrX TJ24GaLLYhtyjBFZa/JWpsrRSk0H4BbDEE4BoZLHTjUEJ53WddF36Sn7l6PEm1vrg9nc LnBHAM9hkD8tWs+GleLra3y+teIGWgOm2LVzNDez+928EqTezVAbcZHeOqZXtnGaJMYO dDewUF5ORMt/SlBqeVGRiHieOWB11mUfLNx9jvkby7IO4CrVwaOfBPtqrPwRHSEBfNKk JBLw== X-Gm-Message-State: AGRZ1gJL8XRKZ7kUTg8SjgalS1/cJb5IzcRxOR0el1EnmqMDSbzlQ64k kzPYSMOH4IkTqMSUBsEJWh1JEQ== X-Google-Smtp-Source: AJdET5epwL8opIfWAHhuSBY1hNn/GKz9erLG44ejL6V5gIz84eLRbi+P5B/LB3RozshYttdcdgtQpg== X-Received: by 2002:a62:ed09:: with SMTP id u9-v6mr16036733pfh.188.1542481044594; Sat, 17 Nov 2018 10:57:24 -0800 (PST) Received: from mba13.psav.com ([64.114.255.114]) by smtp.gmail.com with ESMTPSA id u76-v6sm49550745pfa.176.2018.11.17.10.57.23 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 17 Nov 2018 10:57:23 -0800 (PST) From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: Ard Biesheuvel , Daniel Borkmann , Alexei Starovoitov , Rick Edgecombe , Eric Dumazet , Jann Horn , Kees Cook , Jessica Yu , Arnd Bergmann , Catalin Marinas , Will Deacon , Mark Rutland , Ralf Baechle , Paul Burton , James Hogan , Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , "David S. Miller" , linux-arm-kernel@lists.infradead.org, linux-mips@linux-mips.org, linuxppc-dev@lists.ozlabs.org, sparclinux@vger.kernel.org, netdev@vger.kernel.org Subject: [PATCH 4/4] arm64/bpf: don't allocate BPF JIT programs in module memory Date: Sat, 17 Nov 2018 10:57:15 -0800 Message-Id: <20181117185715.25198-5-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20181117185715.25198-1-ard.biesheuvel@linaro.org> References: <20181117185715.25198-1-ard.biesheuvel@linaro.org> Sender: sparclinux-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: sparclinux@vger.kernel.org The arm64 module region is a 128 MB region that is kept close to the core kernel, in order to ensure that relative branches are always in range. So using the same region for programs that do not have this restriction is wasteful, and preferably avoided. Now that the core BPF JIT code permits the alloc/free routines to be overridden, implement them by simple vmalloc_exec()/vfree() calls, which can be served from anywere. This also solves an issue under KASAN, where shadow memory is needlessly allocated for all BPF programs (which don't require KASAN shadow pages since they are not KASAN instrumented) Signed-off-by: Ard Biesheuvel --- arch/arm64/net/bpf_jit_comp.c | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c index a6fdaea07c63..e0c702c2f682 100644 --- a/arch/arm64/net/bpf_jit_comp.c +++ b/arch/arm64/net/bpf_jit_comp.c @@ -940,3 +940,14 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog) tmp : orig_prog); return prog; } + +void *bpf_jit_alloc_exec(unsigned long size) +{ + return vmalloc_exec(size); +} + +void bpf_jit_binary_free(struct bpf_binary_header *hdr) +{ + bpf_jit_binary_unlock_ro(hdr); + vfree(hdr); +}