From patchwork Wed Sep 12 00:36:34 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joe Stringer X-Patchwork-Id: 968779 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=wand.net.nz Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="QmyhljS6"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 4292sk2cKjz9s3x for ; Wed, 12 Sep 2018 10:36:58 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727836AbeILFiq (ORCPT ); Wed, 12 Sep 2018 01:38:46 -0400 Received: from mail-oi0-f65.google.com ([209.85.218.65]:40607 "EHLO mail-oi0-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726454AbeILFip (ORCPT ); Wed, 12 Sep 2018 01:38:45 -0400 Received: by mail-oi0-f65.google.com with SMTP id l202-v6so308415oig.7 for ; Tue, 11 Sep 2018 17:36:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references; bh=MGbUE0L3ue9q1J/Vx78X/5abnlfj38QIZrhEr6OxtvA=; b=QmyhljS6PvCnY0UbZKbZTPSmjbxHQY4EyWmepm8AFP3QllbU7dxKlHduM9L49+e1oz abXbg7VKlJfCv0fN0UdBtiUoEy1M5oPYjGMVBIZPu1Op7yoCo5Ij1KP9udtokz6hCZNN Z40OrKEkZoSvS+FTpDqKJnVuxILY8HCU+JhRR+EnlrrarOt7KQ6vrzWbBnPkx9Zpf2if qG/Uleece5Nd5A4JzqMHhv9aQPmBVR/9pZ7KKPFiaqJ7PCCvqDY6qCzRI72AQvAuh9LY 4Bjv+IlvIQuA9A/m+KjEEjMiqj7AfsLRIdo0oQTdrAIMtJ09yea/1Nm7l6qqpeo+JLSI Erkg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references; bh=MGbUE0L3ue9q1J/Vx78X/5abnlfj38QIZrhEr6OxtvA=; b=Pj2rWRxRWK694osR3T0u1r6oy1Y/XNTLBgJ6RKKaEYNpmAskWbLVFYyzxztWpM/xS/ wPhTlMciOhdUfzb1msrYcd+MV+LofIbcXvDXxa4MQ2CJ1A0YT/+KYj21DY1Z97kG+bb9 QSueSBd3xklze5O4QyMF0eYLYxgllL1QK5xlH30VFbOVwoBSkdquPjNFenaBnZNi6lwP pSKcdYAFrlVj7wbnGOFrYcd17MEz0oKdPjmcezwNqQrqI3YH8F4ZnQ8ImcY2LrF8Onf3 e1FtuyQnEmLtLNM8MkbVy2TjyFOmVgt1+jiNDpINS257UxK/i1K5wfMYeWt+KqXWYICd /mOA== X-Gm-Message-State: APzg51BUgYBC6JbJb+SQ7TV2dpDiKZqnSdItZuo1kj/YB2zz+5eNh4Bx batkhfop73oC1OAF1RwKw18= X-Google-Smtp-Source: ANB0Vdbv6Wve9aFn/TcoqPvKuYol3Z8g/XYuKsQd023YmbSP8y7WqCGWi9v7gNwfKzx1HBaM2DsDVA== X-Received: by 2002:aca:2e86:: with SMTP id u128-v6mr27829165oiu.132.1536712615666; Tue, 11 Sep 2018 17:36:55 -0700 (PDT) Received: from allosaurus.lan ([99.0.85.34]) by smtp.gmail.com with ESMTPSA id s145-v6sm39598850oih.16.2018.09.11.17.36.53 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 11 Sep 2018 17:36:54 -0700 (PDT) From: Joe Stringer To: daniel@iogearbox.net Cc: netdev@vger.kernel.org, ast@kernel.org, john.fastabend@gmail.com, tgraf@suug.ch, kafai@fb.com, nitin.hande@gmail.com, mauricio.vasquez@polito.it Subject: [PATCH bpf-next 05/11] bpf: Macrofy stack state copy Date: Tue, 11 Sep 2018 17:36:34 -0700 Message-Id: <20180912003640.28316-6-joe@wand.net.nz> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180912003640.28316-1-joe@wand.net.nz> References: <20180912003640.28316-1-joe@wand.net.nz> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org An upcoming commit will need very similar copy/realloc boilerplate, so refactor the existing stack copy/realloc functions into macros to simplify it. Signed-off-by: Joe Stringer Acked-by: Alexei Starovoitov --- kernel/bpf/verifier.c | 106 ++++++++++++++++++++++++------------------ 1 file changed, 60 insertions(+), 46 deletions(-) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 111e031cf65d..faa83b3d7011 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -387,60 +387,74 @@ static void print_verifier_state(struct bpf_verifier_env *env, verbose(env, "\n"); } -static int copy_stack_state(struct bpf_func_state *dst, - const struct bpf_func_state *src) -{ - if (!src->stack) - return 0; - if (WARN_ON_ONCE(dst->allocated_stack < src->allocated_stack)) { - /* internal bug, make state invalid to reject the program */ - memset(dst, 0, sizeof(*dst)); - return -EFAULT; - } - memcpy(dst->stack, src->stack, - sizeof(*src->stack) * (src->allocated_stack / BPF_REG_SIZE)); - return 0; -} +#define COPY_STATE_FN(NAME, COUNT, FIELD, SIZE) \ +static int copy_##NAME##_state(struct bpf_func_state *dst, \ + const struct bpf_func_state *src) \ +{ \ + if (!src->FIELD) \ + return 0; \ + if (WARN_ON_ONCE(dst->COUNT < src->COUNT)) { \ + /* internal bug, make state invalid to reject the program */ \ + memset(dst, 0, sizeof(*dst)); \ + return -EFAULT; \ + } \ + memcpy(dst->FIELD, src->FIELD, \ + sizeof(*src->FIELD) * (src->COUNT / SIZE)); \ + return 0; \ +} +/* copy_stack_state() */ +COPY_STATE_FN(stack, allocated_stack, stack, BPF_REG_SIZE) +#undef COPY_STATE_FN + +#define REALLOC_STATE_FN(NAME, COUNT, FIELD, SIZE) \ +static int realloc_##NAME##_state(struct bpf_func_state *state, int size, \ + bool copy_old) \ +{ \ + u32 old_size = state->COUNT; \ + struct bpf_##NAME##_state *new_##FIELD; \ + int slot = size / SIZE; \ + \ + if (size <= old_size || !size) { \ + if (copy_old) \ + return 0; \ + state->COUNT = slot * SIZE; \ + if (!size && old_size) { \ + kfree(state->FIELD); \ + state->FIELD = NULL; \ + } \ + return 0; \ + } \ + new_##FIELD = kmalloc_array(slot, sizeof(struct bpf_##NAME##_state), \ + GFP_KERNEL); \ + if (!new_##FIELD) \ + return -ENOMEM; \ + if (copy_old) { \ + if (state->FIELD) \ + memcpy(new_##FIELD, state->FIELD, \ + sizeof(*new_##FIELD) * (old_size / SIZE)); \ + memset(new_##FIELD + old_size / SIZE, 0, \ + sizeof(*new_##FIELD) * (size - old_size) / SIZE); \ + } \ + state->COUNT = slot * SIZE; \ + kfree(state->FIELD); \ + state->FIELD = new_##FIELD; \ + return 0; \ +} +/* realloc_stack_state() */ +REALLOC_STATE_FN(stack, allocated_stack, stack, BPF_REG_SIZE) +#undef REALLOC_STATE_FN /* do_check() starts with zero-sized stack in struct bpf_verifier_state to * make it consume minimal amount of memory. check_stack_write() access from * the program calls into realloc_func_state() to grow the stack size. - * Note there is a non-zero parent pointer inside each reg of bpf_verifier_state - * which this function copies over. It points to corresponding reg in previous - * bpf_verifier_state which is never reallocated + * Note there is a non-zero 'parent' pointer inside bpf_verifier_state + * which realloc_stack_state() copies over. It points to previous + * bpf_verifier_state which is never reallocated. */ static int realloc_func_state(struct bpf_func_state *state, int size, bool copy_old) { - u32 old_size = state->allocated_stack; - struct bpf_stack_state *new_stack; - int slot = size / BPF_REG_SIZE; - - if (size <= old_size || !size) { - if (copy_old) - return 0; - state->allocated_stack = slot * BPF_REG_SIZE; - if (!size && old_size) { - kfree(state->stack); - state->stack = NULL; - } - return 0; - } - new_stack = kmalloc_array(slot, sizeof(struct bpf_stack_state), - GFP_KERNEL); - if (!new_stack) - return -ENOMEM; - if (copy_old) { - if (state->stack) - memcpy(new_stack, state->stack, - sizeof(*new_stack) * (old_size / BPF_REG_SIZE)); - memset(new_stack + old_size / BPF_REG_SIZE, 0, - sizeof(*new_stack) * (size - old_size) / BPF_REG_SIZE); - } - state->allocated_stack = slot * BPF_REG_SIZE; - kfree(state->stack); - state->stack = new_stack; - return 0; + return realloc_stack_state(state, size, copy_old); } static void free_func_state(struct bpf_func_state *state)