From patchwork Fri May 29 02:26:02 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexei Starovoitov X-Patchwork-Id: 477542 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 0E0C5140DED for ; Fri, 29 May 2015 12:27:03 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754424AbbE2C0R (ORCPT ); Thu, 28 May 2015 22:26:17 -0400 Received: from mail-pd0-f169.google.com ([209.85.192.169]:33179 "EHLO mail-pd0-f169.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753234AbbE2C0P (ORCPT ); Thu, 28 May 2015 22:26:15 -0400 Received: by pdbqa5 with SMTP id qa5so55126674pdb.0 for ; Thu, 28 May 2015 19:26:15 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=o0lCs1cBVp7+N3RBnXJhTrezKCbYRvMUKtU/2rE3JD4=; b=izljUFw9GRED5hw6EWvyosrNa2RjId8S41cmrYlvPwYfmiLq/kmclD6F1rG01rqQ5d YkcM+9Ob0aohvkft5EjAmN2SyFCZTaNEd7fMvaPqGz0TUBQD4dLY9Gam7yIRj+HHroGx mAFakoxSdm5dVyvsXZdKdu9vCS+rNDD2moAt/XZg5Tmk2pRtITqFOhOMbtZ2QiRk+fck y2Ym/T+3HuyMHXNt1BDinRHx2LEKdSXxQEj6FW8L2e4zI2cEWLehfM4ctHJtjKtlxm8q v/sk+4pBeNIOt0oDKbTPvzhC47CCMOoUbnOdX3jAYemWAAA6pYPgtOj2qdEM0pWBOEFa f13w== X-Gm-Message-State: ALoCoQmlbkuyOvq07U2cZs5HHSmTcBjNcsTEBBLBj39sN8udmfArjON5VK7IsbHjnNRvzuHPdxQq X-Received: by 10.68.250.194 with SMTP id ze2mr11026212pbc.24.1432866375113; Thu, 28 May 2015 19:26:15 -0700 (PDT) Received: from localhost.localdomain ([12.229.56.227]) by mx.google.com with ESMTPSA id cz1sm3714257pbc.84.2015.05.28.19.26.13 (version=TLSv1.1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 28 May 2015 19:26:13 -0700 (PDT) From: Alexei Starovoitov To: "David S. Miller" Cc: Daniel Borkmann , Michael Holzheu , Martin Schwidefsky , netdev@vger.kernel.org Subject: [PATCH net-next] bpf: add missing rcu protection when releasing programs from prog_array Date: Thu, 28 May 2015 19:26:02 -0700 Message-Id: <1432866362-8154-1-git-send-email-ast@plumgrid.com> X-Mailer: git-send-email 1.7.9.5 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Normally the program attachment place (like sockets, qdiscs) takes care of rcu protection and calls bpf_prog_put() after a grace period. The programs stored inside prog_array may not be attached anywhere, so prog_array needs to take care of preserving rcu protection. Otherwise bpf_tail_call() will race with bpf_prog_put(). To solve that introduce bpf_prog_put_rcu() helper function and use it in 3 places where unattached program can decrement refcnt: closing program fd, deleting/replacing program in prog_array. Fixes: 04fd61ab36ec ("bpf: allow bpf programs to tail-call other bpf programs") Reported-by: Martin Schwidefsky Signed-off-by: Alexei Starovoitov Acked-by: Daniel Borkmann --- include/linux/bpf.h | 6 +++++- kernel/bpf/arraymap.c | 4 ++-- kernel/bpf/syscall.c | 19 ++++++++++++++++++- 3 files changed, 25 insertions(+), 4 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 8821b9a8689e..5f520f5f087e 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -123,7 +123,10 @@ struct bpf_prog_aux { const struct bpf_verifier_ops *ops; struct bpf_map **used_maps; struct bpf_prog *prog; - struct work_struct work; + union { + struct work_struct work; + struct rcu_head rcu; + }; }; struct bpf_array { @@ -153,6 +156,7 @@ void bpf_register_map_type(struct bpf_map_type_list *tl); struct bpf_prog *bpf_prog_get(u32 ufd); void bpf_prog_put(struct bpf_prog *prog); +void bpf_prog_put_rcu(struct bpf_prog *prog); struct bpf_map *bpf_map_get(struct fd f); void bpf_map_put(struct bpf_map *map); diff --git a/kernel/bpf/arraymap.c b/kernel/bpf/arraymap.c index 614bcd4c1d74..cb31229a6fa4 100644 --- a/kernel/bpf/arraymap.c +++ b/kernel/bpf/arraymap.c @@ -202,7 +202,7 @@ static int prog_array_map_update_elem(struct bpf_map *map, void *key, old_prog = xchg(array->prog + index, prog); if (old_prog) - bpf_prog_put(old_prog); + bpf_prog_put_rcu(old_prog); return 0; } @@ -218,7 +218,7 @@ static int prog_array_map_delete_elem(struct bpf_map *map, void *key) old_prog = xchg(array->prog + index, NULL); if (old_prog) { - bpf_prog_put(old_prog); + bpf_prog_put_rcu(old_prog); return 0; } else { return -ENOENT; diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index 98a69bd83069..a1b14d197a4f 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -432,6 +432,23 @@ static void free_used_maps(struct bpf_prog_aux *aux) kfree(aux->used_maps); } +static void __prog_put_rcu(struct rcu_head *rcu) +{ + struct bpf_prog_aux *aux = container_of(rcu, struct bpf_prog_aux, rcu); + + free_used_maps(aux); + bpf_prog_free(aux->prog); +} + +/* version of bpf_prog_put() that is called after a grace period */ +void bpf_prog_put_rcu(struct bpf_prog *prog) +{ + if (atomic_dec_and_test(&prog->aux->refcnt)) { + prog->aux->prog = prog; + call_rcu(&prog->aux->rcu, __prog_put_rcu); + } +} + void bpf_prog_put(struct bpf_prog *prog) { if (atomic_dec_and_test(&prog->aux->refcnt)) { @@ -445,7 +462,7 @@ static int bpf_prog_release(struct inode *inode, struct file *filp) { struct bpf_prog *prog = filp->private_data; - bpf_prog_put(prog); + bpf_prog_put_rcu(prog); return 0; }