From patchwork Fri Jun 14 08:20:13 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Toshiaki Makita X-Patchwork-Id: 1115818 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="MhRkPIYJ"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 45QD7h6N0xz9s6w for ; Fri, 14 Jun 2019 18:20:32 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726479AbfFNIUc (ORCPT ); Fri, 14 Jun 2019 04:20:32 -0400 Received: from mail-pf1-f193.google.com ([209.85.210.193]:38124 "EHLO mail-pf1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726381AbfFNIUb (ORCPT ); Fri, 14 Jun 2019 04:20:31 -0400 Received: by mail-pf1-f193.google.com with SMTP id a186so976193pfa.5; Fri, 14 Jun 2019 01:20:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=NpCxj9FcXnz9ZeL717BMMIHVMfwWmqjUQFeorp7LQqw=; b=MhRkPIYJjLIHjySiagHfnKAExWNBYElBP2jLEE5fTrlOysSIFjCt9g0gJ1dMZx/nXO s03MImGvldn0vnXcsaf8dEaUqKXSB4m3x+DueiDgHAoFa3ePMLaam7+pmjNLSJ5Awh7x rmUtFzJ7Zl0M0SCTTEz8d8ZlSUGnoJefeKJp1gapSAexBzQbXNpS+4C2jcV72C9rmKLH QJYEgh8EVmJ6SHVQ4KElz5hw03tgSqCGB/4gviLjOREFHz1h6TjRnRv3J/51fimSXZ1f Am1TbrXiY4f282ihYt6cXeyBHD9hNhhWjwGvUQ50iD4m0NR6E87XxLdpe6lJjFO9TzbB QsFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=NpCxj9FcXnz9ZeL717BMMIHVMfwWmqjUQFeorp7LQqw=; b=LD4eKdezqr8Trkh6Yh0Q/mP6Z5ZlTlPCLXDAuoXxDC4kTeDI0EmWwvJq9YLTxqdAEy iXy+bm0AO+QDRfxq7l0S/elL+JJZhvhDjKP7Yic+dx9EcTBAAzV+mfkmAnhJidgNr4DD ToqMLnfJeKTFQrMdiiMNzMiXaIm3ZTMhYq1SwBOC+zc4L87J6ZHm15TQBAc5424iwDXg Y3eV+kTnNQ9/2SJz9ZZXMMNggvE9s249GgVMR9cGVClnY1JSdQf5OTJ10JQDqQt1s45Q hbrKrot3dGWnq0kHlEXfOwL1xe0BYD5EbzwSPe5BajSFdjiByEwrtKAYi/Cd9/4p9hws GDTQ== X-Gm-Message-State: APjAAAVVpzMoZevKTsatXqGVcuwRDHXqcFbnb5EmIC2d18NBF4gdMGLj jiAlqUmog15yxYLSk6+p2S0= X-Google-Smtp-Source: APXvYqwCWiSYEHEhz7eHwJEhiPtR0bPC1xiSV3JkKEIwNH+XGQt6bZ6kVNst7EuyRQOHMctzI6T/8Q== X-Received: by 2002:a17:90a:628a:: with SMTP id d10mr9754676pjj.7.1560500430507; Fri, 14 Jun 2019 01:20:30 -0700 (PDT) Received: from z400-fedora29.kern.oss.ntt.co.jp ([222.151.198.97]) by smtp.gmail.com with ESMTPSA id t18sm3352343pgm.69.2019.06.14.01.20.27 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Fri, 14 Jun 2019 01:20:29 -0700 (PDT) From: Toshiaki Makita To: Alexei Starovoitov , Daniel Borkmann , "David S. Miller" , Jakub Kicinski , Jesper Dangaard Brouer , John Fastabend Cc: Toshiaki Makita , netdev@vger.kernel.org, xdp-newbies@vger.kernel.org, bpf@vger.kernel.org, "Michael S. Tsirkin" , Jason Wang , David Ahern Subject: [PATCH bpf 1/3] devmap: Fix premature entry free on destroying map Date: Fri, 14 Jun 2019 17:20:13 +0900 Message-Id: <20190614082015.23336-2-toshiaki.makita1@gmail.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190614082015.23336-1-toshiaki.makita1@gmail.com> References: <20190614082015.23336-1-toshiaki.makita1@gmail.com> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org dev_map_free() waits for flush_needed bitmap to be empty in order to ensure all flush operations have completed before freeing its entries. However the corresponding clear_bit() was called before using the entries, so the entries could be used after free. All access to the entries needs to be done before clearing the bit. It seems commit a5e2da6e9787 ("bpf: netdev is never null in __dev_map_flush") accidentally changed the clear_bit() and memory access order. Note that the problem happens only in __dev_map_flush(), not in dev_map_flush_old(). dev_map_flush_old() is called only after nulling out the corresponding netdev_map entry, so dev_map_free() never frees the entry thus no such race happens there. Fixes: a5e2da6e9787 ("bpf: netdev is never null in __dev_map_flush") Signed-off-by: Toshiaki Makita --- kernel/bpf/devmap.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/kernel/bpf/devmap.c b/kernel/bpf/devmap.c index 1e525d7..e001fb1 100644 --- a/kernel/bpf/devmap.c +++ b/kernel/bpf/devmap.c @@ -291,10 +291,10 @@ void __dev_map_flush(struct bpf_map *map) if (unlikely(!dev)) continue; - __clear_bit(bit, bitmap); - bq = this_cpu_ptr(dev->bulkq); bq_xmit_all(dev, bq, XDP_XMIT_FLUSH, true); + + __clear_bit(bit, bitmap); } } From patchwork Fri Jun 14 08:20:14 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Toshiaki Makita X-Patchwork-Id: 1115822 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: incoming-bpf@patchwork.ozlabs.org Delivered-To: patchwork-incoming-bpf@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=bpf-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="INj7VTjh"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 45QD7v4zFQz9sDB for ; Fri, 14 Jun 2019 18:20:43 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726700AbfFNIUn (ORCPT ); Fri, 14 Jun 2019 04:20:43 -0400 Received: from mail-pf1-f193.google.com ([209.85.210.193]:37156 "EHLO mail-pf1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726381AbfFNIUe (ORCPT ); Fri, 14 Jun 2019 04:20:34 -0400 Received: by mail-pf1-f193.google.com with SMTP id 19so981845pfa.4; Fri, 14 Jun 2019 01:20:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=5LHylaY5VKPZD8rk+dF82kF3y58VI8JtEPT7IjoFkYs=; b=INj7VTjh3UTvSvxSx+pQ06pJ/yxS6auA5JKxKdwHU+yiSupWy2K7gWenphYxz2nZWa QMnxaThrYyvM53OGFZOd0PVfpexaKA2VuPACNb3lKsxydEEKSMOWaopApsqPkH9HVnSZ FI1hcVQRQh+lbnpQSF9g0LkAtM1J/nWQFiK3i5JLwWs+vsEfGK5O5Ef3WZBvdjBKpQvl cg1CjuuSxKVG7uNdaQu9ElE4Cn0O7ep46+xjHrGV4TIZYTh+xctO+xf6bXY67gpagydl FNfmnvZz/aLFkCjBJ58fMRvWRS43kycgGl6w8YkiGxAc4/fBKqwHSqTvbppDd4DoR5Hi TiyQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=5LHylaY5VKPZD8rk+dF82kF3y58VI8JtEPT7IjoFkYs=; b=OkqvGOfbY7EprX5xCU5cDGDKTbpZCrt1POT4Cp3e2rX4b3qWnSJm2P+b54X8HCyHLz 19OYP674pRxDw150RHxUFGt06Q8af9g/CBfqnCip7jo0oRtEsIop4kupvJf1aGEJAxJr wvJhZRDvzJuvLXFU77RWnBZ+Wl+gQJA69umpuopCOximjYuIkcshNmKHZvDSBK0GT2iM yZS4hLY3q2MBw+zHHROXutqGBlSG34hUDeDWMak2kWrKvheqEw2xf8WdzoNdCw9RvY56 hNorSsYxELPzocvG+2KS78T8RWMxdodLigRGr+NpGl9No8olpCr93WpALTeusmwn6Tr9 tcjQ== X-Gm-Message-State: APjAAAVQw9kBMUYXeE7avLzU2B7VYWtG/hDNYCwhTfs0rukh23TXvOwf /Fd8YxqkFgKIwIXBtSBz8Ic= X-Google-Smtp-Source: APXvYqz5h5VV5pjR5GJn8yTxx0bgbqWpaEQEsO1nMwrUZ50IACJBVvB2jvjVMCUMRyJ3vQezC8VwTA== X-Received: by 2002:a17:90a:fa07:: with SMTP id cm7mr9493891pjb.138.1560500433731; Fri, 14 Jun 2019 01:20:33 -0700 (PDT) Received: from z400-fedora29.kern.oss.ntt.co.jp ([222.151.198.97]) by smtp.gmail.com with ESMTPSA id t18sm3352343pgm.69.2019.06.14.01.20.30 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Fri, 14 Jun 2019 01:20:33 -0700 (PDT) From: Toshiaki Makita To: Alexei Starovoitov , Daniel Borkmann , "David S. Miller" , Jakub Kicinski , Jesper Dangaard Brouer , John Fastabend Cc: Toshiaki Makita , netdev@vger.kernel.org, xdp-newbies@vger.kernel.org, bpf@vger.kernel.org, "Michael S. Tsirkin" , Jason Wang , David Ahern Subject: [PATCH bpf 2/3] devmap: Add missing bulk queue free Date: Fri, 14 Jun 2019 17:20:14 +0900 Message-Id: <20190614082015.23336-3-toshiaki.makita1@gmail.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190614082015.23336-1-toshiaki.makita1@gmail.com> References: <20190614082015.23336-1-toshiaki.makita1@gmail.com> MIME-Version: 1.0 Sender: bpf-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org dev_map_free() forgot to free bulk queue when freeing its entries. Fixes: 5d053f9da431 ("bpf: devmap prepare xdp frames for bulking") Signed-off-by: Toshiaki Makita Acked-by: Jesper Dangaard Brouer --- kernel/bpf/devmap.c | 1 + 1 file changed, 1 insertion(+) diff --git a/kernel/bpf/devmap.c b/kernel/bpf/devmap.c index e001fb1..a126d95 100644 --- a/kernel/bpf/devmap.c +++ b/kernel/bpf/devmap.c @@ -186,6 +186,7 @@ static void dev_map_free(struct bpf_map *map) if (!dev) continue; + free_percpu(dev->bulkq); dev_put(dev->dev); kfree(dev); } From patchwork Fri Jun 14 08:20:15 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Toshiaki Makita X-Patchwork-Id: 1115820 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="PvPXwxHt"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 45QD7s6Kg0z9sBr for ; Fri, 14 Jun 2019 18:20:41 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726693AbfFNIUk (ORCPT ); Fri, 14 Jun 2019 04:20:40 -0400 Received: from mail-pg1-f196.google.com ([209.85.215.196]:43336 "EHLO mail-pg1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726655AbfFNIUh (ORCPT ); Fri, 14 Jun 2019 04:20:37 -0400 Received: by mail-pg1-f196.google.com with SMTP id f25so1107529pgv.10; Fri, 14 Jun 2019 01:20:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=8qHhibZPtQ7Gu7/olGLfM6ZKwz0g2uJPETyalb8BGDY=; b=PvPXwxHtSi2X+SHN7HOb2h9oiAi3X+dtgGr4LGlL6w/ZoiR3fFc/TTcMUtqmmVqTxv M2zHUY9hyPjgtqr16Vn0Mxk/Gx9B3sGiN7NHb8osulGFXGpTm+1DYCVx4u9inWl2/gl0 DzLE4Atzam/qTVBfWos7rj3rqPolGM8z62qMEpTxM/uOeqsQ6NG1I+nZ0NjJZF/yqcwf VghDd/4l1Rq3dVXz6oYuaaOclIP/v1FFJ4kDoUbXxGv+Lm/GWRJCH6lk+08WTsjad5kA WizivxzWG1UDdIxJGMyVAea568VWP8Rf9fveM5YBSHjKziqGIwMAZ2naol6FtGGgXLa5 k/OA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=8qHhibZPtQ7Gu7/olGLfM6ZKwz0g2uJPETyalb8BGDY=; b=SdYcvIo4ByupVkQdMvurz9a7+4EZLc5rFu3K5daN5boIuMf6y4nvcs0Mdcim/qCawI fOvrfjXWa+iEy+K93kVbnZH/8qcexL2K9F+LCftL25GJMAVGcFy6kB0b/prby1k7OiMB +GV2Zup0RG1lg8nkpavUdVUk5IrwXB5/ycjZe/Fhp3T3Tc2QBy8YYJQ4mur2o1Ea0kVw Oa5OdVF8DE4H3KrZjqHv26LAi0a4KlV2jlDw+fuxxYbi+ANq+BGenpyDqeTnR8XJBU3N NS9B/0wLvsQdFX8L91EpVRfKobD0xwapKA7Yvw3KOdjLKw6+BIcSu6WwfLMaxtg3Xb9k 3bog== X-Gm-Message-State: APjAAAVBtvUsrHeqvLMAVXmA9pnhmKsg6F9DRL0pE0Tu68yS3UZfHJ0+ AYChnYunU+knWBNSNFZApSo= X-Google-Smtp-Source: APXvYqxa5lSv6MbyuN3w6u8/9qOHc1kXb15XNTlcw6E4F4L/Q3nHEH5n/44xhn17QFtDJ2YPCg5ZOw== X-Received: by 2002:a17:90a:8d86:: with SMTP id d6mr9387075pjo.127.1560500436830; Fri, 14 Jun 2019 01:20:36 -0700 (PDT) Received: from z400-fedora29.kern.oss.ntt.co.jp ([222.151.198.97]) by smtp.gmail.com with ESMTPSA id t18sm3352343pgm.69.2019.06.14.01.20.33 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Fri, 14 Jun 2019 01:20:36 -0700 (PDT) From: Toshiaki Makita To: Alexei Starovoitov , Daniel Borkmann , "David S. Miller" , Jakub Kicinski , Jesper Dangaard Brouer , John Fastabend Cc: Toshiaki Makita , netdev@vger.kernel.org, xdp-newbies@vger.kernel.org, bpf@vger.kernel.org, "Michael S. Tsirkin" , Jason Wang , David Ahern Subject: [PATCH bpf 3/3] devmap: Add missing RCU read lock on flush Date: Fri, 14 Jun 2019 17:20:15 +0900 Message-Id: <20190614082015.23336-4-toshiaki.makita1@gmail.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190614082015.23336-1-toshiaki.makita1@gmail.com> References: <20190614082015.23336-1-toshiaki.makita1@gmail.com> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org .ndo_xdp_xmit() assumes it is called under RCU. For example virtio_net uses RCU to detect it has setup the resources for tx. The assumption accidentally broke when introducing bulk queue in devmap. Fixes: 5d053f9da431 ("bpf: devmap prepare xdp frames for bulking") Reported-by: David Ahern Signed-off-by: Toshiaki Makita --- kernel/bpf/devmap.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/kernel/bpf/devmap.c b/kernel/bpf/devmap.c index a126d95..1defea4 100644 --- a/kernel/bpf/devmap.c +++ b/kernel/bpf/devmap.c @@ -282,6 +282,7 @@ void __dev_map_flush(struct bpf_map *map) unsigned long *bitmap = this_cpu_ptr(dtab->flush_needed); u32 bit; + rcu_read_lock(); for_each_set_bit(bit, bitmap, map->max_entries) { struct bpf_dtab_netdev *dev = READ_ONCE(dtab->netdev_map[bit]); struct xdp_bulk_queue *bq; @@ -297,6 +298,7 @@ void __dev_map_flush(struct bpf_map *map) __clear_bit(bit, bitmap); } + rcu_read_unlock(); } /* rcu_read_lock (from syscall and BPF contexts) ensures that if a delete and/or @@ -389,6 +391,7 @@ static void dev_map_flush_old(struct bpf_dtab_netdev *dev) int cpu; + rcu_read_lock(); for_each_online_cpu(cpu) { bitmap = per_cpu_ptr(dev->dtab->flush_needed, cpu); __clear_bit(dev->bit, bitmap); @@ -396,6 +399,7 @@ static void dev_map_flush_old(struct bpf_dtab_netdev *dev) bq = per_cpu_ptr(dev->bulkq, cpu); bq_xmit_all(dev, bq, XDP_XMIT_FLUSH, false); } + rcu_read_unlock(); } }