From patchwork Thu Sep 27 23:26:52 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joe Stringer X-Patchwork-Id: 975962 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=wand.net.nz Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="nLeZNOeZ"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 42LrZ56Znbz9s4Z for ; Fri, 28 Sep 2018 09:27:25 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728530AbeI1Fr5 (ORCPT ); Fri, 28 Sep 2018 01:47:57 -0400 Received: from mail-yw1-f66.google.com ([209.85.161.66]:36170 "EHLO mail-yw1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727635AbeI1Fr5 (ORCPT ); Fri, 28 Sep 2018 01:47:57 -0400 Received: by mail-yw1-f66.google.com with SMTP id e201-v6so1860196ywa.3 for ; Thu, 27 Sep 2018 16:27:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references; bh=vNGfv555h7nKk+YLUXfXqrxur2rRvvSUqT5dkXpNbzE=; b=nLeZNOeZ0CstUvYLhx8c3CfTP8VyUeHJwSEGjgATW3RZdDWXrD2eb8nwjrZ3PJLXQ5 2LObmPB8GTswwYZm+nT0BwBtqGITu6CYcsTpLbk9nkQJK+Y0aTY2U4ZP3tGSq7EQ0GTK mtiYZxTo0XfQxmcaI9/P1EJTgj47fzcOiXVxlq7xvZKL6lu7D4UBBaXuI53EZG954ya7 IRyAt1O+rnN/I4FXnejTAOB2CuBc5K79ZJ51WwBYREoLjtFpFaq/JBL36MNTSv2xWBRN 3bk6OXKVkK9FqF6pDxojtPQfCmv93a5AFkuAIv6IFAFXdpd+KRK1dqfBPgI4NuoQEuuW a5lw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references; bh=vNGfv555h7nKk+YLUXfXqrxur2rRvvSUqT5dkXpNbzE=; b=Jqb6+6zKEuN9tjLVZA+esl68TJ1o5uQ8XmbMUgyiFStq63+tHrodt+MWSkknM1Csc3 iumKfW7wnss2U7GyBiMPBCwL14cprQ+R9FrPmlQzSp8egeLbgO0UFRXqnEVuIZDAdP9L qYCuzJhE/56h4kNYCbu/67Kq6JoPG926N015F47EM0Q7x0z9g/jzR7atfpWJI5xECCTI LyCb7qpZ/awau6hbO/+qXruTQFcG3oRsa++oHeuBg+iyGdhPucYVE+rhnsYAwejgQWj4 EvBgM9/qO6D3r/HOc6lvMKui/s2+dmVGSwXN75EmVCgfn7wmU9llqsoMZKDfnNBEPKrc puVg== X-Gm-Message-State: ABuFfoiVQqhKQviJmQRYUgPpOJA731ji8qRogeqeYrnyGgY7GWdg1BOM XxvMgaMea5WSA8ZSWj2NNNs= X-Google-Smtp-Source: ACcGV61t3i0cnKup6Z9icP/iZCIbPolybME1RJWuXQwgT+BjpbdwytgbT2qipodrHwP/KY2VRyE5yw== X-Received: by 2002:a0d:dac1:: with SMTP id c184-v6mr4990922ywe.332.1538090830726; Thu, 27 Sep 2018 16:27:10 -0700 (PDT) Received: from localhost.localdomain ([99.0.85.34]) by smtp.gmail.com with ESMTPSA id w6-v6sm4810717ywg.3.2018.09.27.16.27.09 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 27 Sep 2018 16:27:10 -0700 (PDT) From: Joe Stringer To: daniel@iogearbox.net Cc: netdev@vger.kernel.org, ast@kernel.org, john.fastabend@gmail.com, tgraf@suug.ch, kafai@fb.com, nitin.hande@gmail.com, mauricio.vasquez@polito.it Subject: [PATCHv3 bpf-next 05/12] bpf: Macrofy stack state copy Date: Thu, 27 Sep 2018 16:26:52 -0700 Message-Id: <20180927232659.14348-6-joe@wand.net.nz> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180927232659.14348-1-joe@wand.net.nz> References: <20180927232659.14348-1-joe@wand.net.nz> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org An upcoming commit will need very similar copy/realloc boilerplate, so refactor the existing stack copy/realloc functions into macros to simplify it. Signed-off-by: Joe Stringer Acked-by: Alexei Starovoitov --- kernel/bpf/verifier.c | 106 ++++++++++++++++++++++++------------------ 1 file changed, 60 insertions(+), 46 deletions(-) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index d4abbf0d5727..cf8704d137fa 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -388,60 +388,74 @@ static void print_verifier_state(struct bpf_verifier_env *env, verbose(env, "\n"); } -static int copy_stack_state(struct bpf_func_state *dst, - const struct bpf_func_state *src) -{ - if (!src->stack) - return 0; - if (WARN_ON_ONCE(dst->allocated_stack < src->allocated_stack)) { - /* internal bug, make state invalid to reject the program */ - memset(dst, 0, sizeof(*dst)); - return -EFAULT; - } - memcpy(dst->stack, src->stack, - sizeof(*src->stack) * (src->allocated_stack / BPF_REG_SIZE)); - return 0; -} +#define COPY_STATE_FN(NAME, COUNT, FIELD, SIZE) \ +static int copy_##NAME##_state(struct bpf_func_state *dst, \ + const struct bpf_func_state *src) \ +{ \ + if (!src->FIELD) \ + return 0; \ + if (WARN_ON_ONCE(dst->COUNT < src->COUNT)) { \ + /* internal bug, make state invalid to reject the program */ \ + memset(dst, 0, sizeof(*dst)); \ + return -EFAULT; \ + } \ + memcpy(dst->FIELD, src->FIELD, \ + sizeof(*src->FIELD) * (src->COUNT / SIZE)); \ + return 0; \ +} +/* copy_stack_state() */ +COPY_STATE_FN(stack, allocated_stack, stack, BPF_REG_SIZE) +#undef COPY_STATE_FN + +#define REALLOC_STATE_FN(NAME, COUNT, FIELD, SIZE) \ +static int realloc_##NAME##_state(struct bpf_func_state *state, int size, \ + bool copy_old) \ +{ \ + u32 old_size = state->COUNT; \ + struct bpf_##NAME##_state *new_##FIELD; \ + int slot = size / SIZE; \ + \ + if (size <= old_size || !size) { \ + if (copy_old) \ + return 0; \ + state->COUNT = slot * SIZE; \ + if (!size && old_size) { \ + kfree(state->FIELD); \ + state->FIELD = NULL; \ + } \ + return 0; \ + } \ + new_##FIELD = kmalloc_array(slot, sizeof(struct bpf_##NAME##_state), \ + GFP_KERNEL); \ + if (!new_##FIELD) \ + return -ENOMEM; \ + if (copy_old) { \ + if (state->FIELD) \ + memcpy(new_##FIELD, state->FIELD, \ + sizeof(*new_##FIELD) * (old_size / SIZE)); \ + memset(new_##FIELD + old_size / SIZE, 0, \ + sizeof(*new_##FIELD) * (size - old_size) / SIZE); \ + } \ + state->COUNT = slot * SIZE; \ + kfree(state->FIELD); \ + state->FIELD = new_##FIELD; \ + return 0; \ +} +/* realloc_stack_state() */ +REALLOC_STATE_FN(stack, allocated_stack, stack, BPF_REG_SIZE) +#undef REALLOC_STATE_FN /* do_check() starts with zero-sized stack in struct bpf_verifier_state to * make it consume minimal amount of memory. check_stack_write() access from * the program calls into realloc_func_state() to grow the stack size. - * Note there is a non-zero parent pointer inside each reg of bpf_verifier_state - * which this function copies over. It points to corresponding reg in previous - * bpf_verifier_state which is never reallocated + * Note there is a non-zero 'parent' pointer inside bpf_verifier_state + * which realloc_stack_state() copies over. It points to previous + * bpf_verifier_state which is never reallocated. */ static int realloc_func_state(struct bpf_func_state *state, int size, bool copy_old) { - u32 old_size = state->allocated_stack; - struct bpf_stack_state *new_stack; - int slot = size / BPF_REG_SIZE; - - if (size <= old_size || !size) { - if (copy_old) - return 0; - state->allocated_stack = slot * BPF_REG_SIZE; - if (!size && old_size) { - kfree(state->stack); - state->stack = NULL; - } - return 0; - } - new_stack = kmalloc_array(slot, sizeof(struct bpf_stack_state), - GFP_KERNEL); - if (!new_stack) - return -ENOMEM; - if (copy_old) { - if (state->stack) - memcpy(new_stack, state->stack, - sizeof(*new_stack) * (old_size / BPF_REG_SIZE)); - memset(new_stack + old_size / BPF_REG_SIZE, 0, - sizeof(*new_stack) * (size - old_size) / BPF_REG_SIZE); - } - state->allocated_stack = slot * BPF_REG_SIZE; - kfree(state->stack); - state->stack = new_stack; - return 0; + return realloc_stack_state(state, size, copy_old); } static void free_func_state(struct bpf_func_state *state)