From patchwork Tue Oct 2 20:35:34 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joe Stringer X-Patchwork-Id: 978018 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=wand.net.nz Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="pXbOqLAR"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 42PrWx1jYyz9s7T for ; Wed, 3 Oct 2018 06:35:57 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727604AbeJCDVE (ORCPT ); Tue, 2 Oct 2018 23:21:04 -0400 Received: from mail-yw1-f67.google.com ([209.85.161.67]:45006 "EHLO mail-yw1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727536AbeJCDVD (ORCPT ); Tue, 2 Oct 2018 23:21:03 -0400 Received: by mail-yw1-f67.google.com with SMTP id s73-v6so1342887ywg.11 for ; Tue, 02 Oct 2018 13:35:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references; bh=IzV0ij/tS/lF2NoYCXuhrcvhkbuOfsplqK9Ux0DTjsQ=; b=pXbOqLARVhEWypiv0xrN8bpTJyze3Rq3Qc+RMOtmtMzS599xrUJ5WECs+nAps+P/Vt SrA6L0tnkH3/HC4MlQEo/uuDXiAOeTMOXcG6GutUKE5YBin5fa9OwyYh05abXrioSMnh pCfSL5gkmmb5PPITMoY0w3X1YZXANC1FndYb+5CZRcVKBF8m5IVMuW71Uy/w99AdTp3a xe+hnHnra+qDw5URb87D/8DKGj6a6Ng6QkGP0PtiP5svgsfiKsdPyWjRZJJYk8lUzn78 KSU4GeZ6G5tCDZ9WdtwTq5k90E1AAGBgbQ+cR+K3iPtB2TdgWikPEnVINqPPT7E0Y11P HqQQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references; bh=IzV0ij/tS/lF2NoYCXuhrcvhkbuOfsplqK9Ux0DTjsQ=; b=IWsw/kfD9ZFJDBY32zP7wb39xRdnlPzpN3sfwvH84lcenqCF0848O0LYQ3c6/n1E4U 0z5KCZHXWtP/hW1wx4BrZVXSYW24jmywAsmdDkESeOBUWC+aybjYhYReNI5egAlzMhQs YLQJhgt4m8e06IhSi9q7qma8ZCaIlw9VJpEjv+ZbpgSaTe0krubs5bA670ecLq2bdnh6 mDM8w0RjXv88syeIruE7gMbf9Vx8g7J6T54cZ+ZrucRtJvV3V8FXmJU6bnHVeyW2jtM/ MH3L1VsBjE63o/qkr60fmYRXWvrcZttfp5zgYTHgIsq09vXcFhPNt2m2zoVCiPer+AoU yFbQ== X-Gm-Message-State: ABuFfohIzsZSxlegfkJ2xtX/fFQBFuYADa3yiDS2ayUwtJTiPF6h7VqY zQ6no28XpONJHmSPHmvjYxY= X-Google-Smtp-Source: ACcGV60gHlmnTNEkrtQ4uC79v6Ju5kB90g+dT1jBD7TdRnPLqO65cLnkwW5t+/TryXau5PKf6egLnA== X-Received: by 2002:a81:8a84:: with SMTP id a126-v6mr9510123ywg.104.1538512553382; Tue, 02 Oct 2018 13:35:53 -0700 (PDT) Received: from localhost.localdomain ([99.0.85.34]) by smtp.gmail.com with ESMTPSA id o131-v6sm4555361ywb.107.2018.10.02.13.35.52 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 02 Oct 2018 13:35:52 -0700 (PDT) From: Joe Stringer To: daniel@iogearbox.net Cc: netdev@vger.kernel.org, ast@kernel.org, john.fastabend@gmail.com, tgraf@suug.ch, kafai@fb.com, nitin.hande@gmail.com, mauricio.vasquez@polito.it Subject: [PATCHv4 bpf-next 06/13] bpf: Macrofy stack state copy Date: Tue, 2 Oct 2018 13:35:34 -0700 Message-Id: <20181002203541.26599-7-joe@wand.net.nz> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20181002203541.26599-1-joe@wand.net.nz> References: <20181002203541.26599-1-joe@wand.net.nz> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org An upcoming commit will need very similar copy/realloc boilerplate, so refactor the existing stack copy/realloc functions into macros to simplify it. Signed-off-by: Joe Stringer Acked-by: Alexei Starovoitov --- kernel/bpf/verifier.c | 106 ++++++++++++++++++++++++------------------ 1 file changed, 60 insertions(+), 46 deletions(-) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index f86386c9affd..11e982381061 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -388,60 +388,74 @@ static void print_verifier_state(struct bpf_verifier_env *env, verbose(env, "\n"); } -static int copy_stack_state(struct bpf_func_state *dst, - const struct bpf_func_state *src) -{ - if (!src->stack) - return 0; - if (WARN_ON_ONCE(dst->allocated_stack < src->allocated_stack)) { - /* internal bug, make state invalid to reject the program */ - memset(dst, 0, sizeof(*dst)); - return -EFAULT; - } - memcpy(dst->stack, src->stack, - sizeof(*src->stack) * (src->allocated_stack / BPF_REG_SIZE)); - return 0; -} +#define COPY_STATE_FN(NAME, COUNT, FIELD, SIZE) \ +static int copy_##NAME##_state(struct bpf_func_state *dst, \ + const struct bpf_func_state *src) \ +{ \ + if (!src->FIELD) \ + return 0; \ + if (WARN_ON_ONCE(dst->COUNT < src->COUNT)) { \ + /* internal bug, make state invalid to reject the program */ \ + memset(dst, 0, sizeof(*dst)); \ + return -EFAULT; \ + } \ + memcpy(dst->FIELD, src->FIELD, \ + sizeof(*src->FIELD) * (src->COUNT / SIZE)); \ + return 0; \ +} +/* copy_stack_state() */ +COPY_STATE_FN(stack, allocated_stack, stack, BPF_REG_SIZE) +#undef COPY_STATE_FN + +#define REALLOC_STATE_FN(NAME, COUNT, FIELD, SIZE) \ +static int realloc_##NAME##_state(struct bpf_func_state *state, int size, \ + bool copy_old) \ +{ \ + u32 old_size = state->COUNT; \ + struct bpf_##NAME##_state *new_##FIELD; \ + int slot = size / SIZE; \ + \ + if (size <= old_size || !size) { \ + if (copy_old) \ + return 0; \ + state->COUNT = slot * SIZE; \ + if (!size && old_size) { \ + kfree(state->FIELD); \ + state->FIELD = NULL; \ + } \ + return 0; \ + } \ + new_##FIELD = kmalloc_array(slot, sizeof(struct bpf_##NAME##_state), \ + GFP_KERNEL); \ + if (!new_##FIELD) \ + return -ENOMEM; \ + if (copy_old) { \ + if (state->FIELD) \ + memcpy(new_##FIELD, state->FIELD, \ + sizeof(*new_##FIELD) * (old_size / SIZE)); \ + memset(new_##FIELD + old_size / SIZE, 0, \ + sizeof(*new_##FIELD) * (size - old_size) / SIZE); \ + } \ + state->COUNT = slot * SIZE; \ + kfree(state->FIELD); \ + state->FIELD = new_##FIELD; \ + return 0; \ +} +/* realloc_stack_state() */ +REALLOC_STATE_FN(stack, allocated_stack, stack, BPF_REG_SIZE) +#undef REALLOC_STATE_FN /* do_check() starts with zero-sized stack in struct bpf_verifier_state to * make it consume minimal amount of memory. check_stack_write() access from * the program calls into realloc_func_state() to grow the stack size. - * Note there is a non-zero parent pointer inside each reg of bpf_verifier_state - * which this function copies over. It points to corresponding reg in previous - * bpf_verifier_state which is never reallocated + * Note there is a non-zero 'parent' pointer inside bpf_verifier_state + * which realloc_stack_state() copies over. It points to previous + * bpf_verifier_state which is never reallocated. */ static int realloc_func_state(struct bpf_func_state *state, int size, bool copy_old) { - u32 old_size = state->allocated_stack; - struct bpf_stack_state *new_stack; - int slot = size / BPF_REG_SIZE; - - if (size <= old_size || !size) { - if (copy_old) - return 0; - state->allocated_stack = slot * BPF_REG_SIZE; - if (!size && old_size) { - kfree(state->stack); - state->stack = NULL; - } - return 0; - } - new_stack = kmalloc_array(slot, sizeof(struct bpf_stack_state), - GFP_KERNEL); - if (!new_stack) - return -ENOMEM; - if (copy_old) { - if (state->stack) - memcpy(new_stack, state->stack, - sizeof(*new_stack) * (old_size / BPF_REG_SIZE)); - memset(new_stack + old_size / BPF_REG_SIZE, 0, - sizeof(*new_stack) * (size - old_size) / BPF_REG_SIZE); - } - state->allocated_stack = slot * BPF_REG_SIZE; - kfree(state->stack); - state->stack = new_stack; - return 0; + return realloc_stack_state(state, size, copy_old); } static void free_func_state(struct bpf_func_state *state)