From patchwork Wed Sep 4 18:38:56 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taehee Yoo X-Patchwork-Id: 1157891 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="RPQDVK3r"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 46Nszc6P9Wz9s4Y for ; Thu, 5 Sep 2019 04:39:08 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732710AbfIDSjH (ORCPT ); Wed, 4 Sep 2019 14:39:07 -0400 Received: from mail-pg1-f194.google.com ([209.85.215.194]:44189 "EHLO mail-pg1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732177AbfIDSjH (ORCPT ); Wed, 4 Sep 2019 14:39:07 -0400 Received: by mail-pg1-f194.google.com with SMTP id i18so11686414pgl.11 for ; Wed, 04 Sep 2019 11:39:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=W9x6KjNDgBE0suYD/7dK4xL35oQSe9zrDp6sZJ8t8Vs=; b=RPQDVK3rUiNuV+TZigl2bBgvjisUxFj+Ro0Edc99JTanbrbr8/8Ul1INr89oEJUD4r XMmh7FVOsk2pnf8R0+EsHPFvzePyypVPZf+UveL1EN/4PMu4yw/mOWMR+1lMidGqyzCV Z0g6ratKtfuvVUqjNeKmMvJW6W0ipxjS6YbN71QpuJCHUfTTKuEGiB3kECBsEEHjLC+q QHNEWKGx3MEygNj3iZMokfjh33k9ZyJxFN39KTAROWXBXUBPtT5OC9WIBxc94BeSJSJQ SL5f2h/LGqpNpFCxuXVmzZid9VPGIWg2xJSUaKJ8BX8DdGLKtw56gRvaNKgHEn88oT0F JDvA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=W9x6KjNDgBE0suYD/7dK4xL35oQSe9zrDp6sZJ8t8Vs=; b=LxnHUreHWbn8iY3vfU/jaELX8i59Up+GOpNNhWXajJ8406W2xxK7g7kAPV+dyTkJYX CFdB+sgQjJO8oETq765iSq+wL/xjoLDFKDuTmJhOYKnX0MctdSdS3UCpaNIjIBCp+pMS XOjnpar+ojlpIvxRkwNneSvBqu9USPbI/AgUXhuOiJAZOJcgRsOlw8LS4zbktCixRwjp aZB9ShE3EeXy/HH/AXJJeCmFyyNI0D/EnK7AehmSMjwjYxvRFmx5/36abJGzeKWwO8NJ miCXig1hX1PPaat+glQI/2K8sIcGwXmGtbcfUM9y354ZQEplg4Z7gZ2AtYYF/01zH1Wq 6KEw== X-Gm-Message-State: APjAAAVXp90vKeA5D5JAq6rNegIPZ1BplRt7eDeDWe+7qZPzLqpxGKrn eTCzdYkoHVjfrQnaV1h1GWc= X-Google-Smtp-Source: APXvYqymiGxD9VnVLiaI7sD0Hl6CAkUoICxrRcy79jn2vCobXyWIcOg+RfSDemxd6LdAYAkafLqBmw== X-Received: by 2002:a63:9318:: with SMTP id b24mr35990456pge.31.1567622346044; Wed, 04 Sep 2019 11:39:06 -0700 (PDT) Received: from ap-To-be-filled-by-O-E-M.1.1.1.1 ([14.33.120.60]) by smtp.gmail.com with ESMTPSA id c11sm25392471pfj.114.2019.09.04.11.39.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Sep 2019 11:39:04 -0700 (PDT) From: Taehee Yoo To: davem@davemloft.net, netdev@vger.kernel.org, j.vosburgh@gmail.com, vfalico@gmail.com, andy@greyhouse.net, jiri@resnulli.us, sd@queasysnail.net, roopa@cumulusnetworks.com, saeedm@mellanox.com, manishc@marvell.com, rahulv@marvell.com, kys@microsoft.com, haiyangz@microsoft.com, sthemmin@microsoft.com, sashal@kernel.org, hare@suse.de, varun@chelsio.com, ubraun@linux.ibm.com, kgraul@linux.ibm.com Cc: ap420073@gmail.com Subject: [PATCH net 01/11] net: core: limit nested device depth Date: Thu, 5 Sep 2019 03:38:56 +0900 Message-Id: <20190904183856.14465-1-ap420073@gmail.com> X-Mailer: git-send-email 2.17.1 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Current code doesn't limit the number of nested devices. Nested devices would be handled recursively and this needs huge stack memory. So, unlimited nested devices could make stack overflow. This patch adds upper_level and lower_leve, they are common variables and represent maximum lower/upper depth. When upper/lower device is attached or dettached, {lower/upper}_level are updated. and if maximum depth is bigger than 8, attach routine fails and returns -EMLINK. Test commands: ip link add dummy0 type dummy ip link add link dummy0 name vlan1 type vlan id 1 ip link set vlan1 up for i in {2..100} do let A=$i-1 ip link add name vlan$i link vlan$A type vlan id $i done Splat looks like: [ 140.483124] BUG: looking up invalid subclass: 8 [ 140.483505] turning off the locking correctness validator. [ 140.483505] CPU: 0 PID: 1324 Comm: ip Not tainted 5.3.0-rc7+ #322 [ 140.483505] Hardware name: To be filled by O.E.M. To be filled by O.E.M./Aptio CRB, BIOS 5.6.5 07/08/2015 [ 140.483505] Call Trace: [ 140.483505] dump_stack+0x7c/0xbb [ 140.483505] register_lock_class+0x64d/0x14d0 [ 140.483505] ? is_dynamic_key+0x230/0x230 [ 140.483505] ? module_assert_mutex_or_preempt+0x41/0x70 [ 140.483505] ? __module_address+0x3f/0x3c0 [ 140.483505] lockdep_init_map+0x24e/0x630 [ 140.483505] vlan_dev_init+0x828/0xce0 [8021q] [ 140.483505] register_netdevice+0x24f/0xd70 [ 140.483505] ? netdev_change_features+0xa0/0xa0 [ 140.483505] ? dev_get_nest_level+0xe1/0x170 [ 140.483505] register_vlan_dev+0x29b/0x710 [8021q] [ 140.483505] __rtnl_newlink+0xb75/0x1180 [ ... ] [ 168.446539] WARNING: can't dereference registers at 00000000bef3d701 for ip apic_timer_interrupt+0xf/0x20 [ 168.466843] ================================================================== [ 168.469452] BUG: KASAN: slab-out-of-bounds in __unwind_start+0x71/0x850 [ 168.480707] Write of size 88 at addr ffff8880b8856d38 by task ip/1758 [ 168.480707] [ 168.480707] CPU: 1 PID: 1758 Comm: ip Not tainted 5.3.0-rc7+ #322 [ ... ] [ 168.794493] Rebooting in 5 seconds.. Signed-off-by: Taehee Yoo --- include/linux/netdevice.h | 4 ++ net/core/dev.c | 106 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 110 insertions(+) diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index 88292953aa6f..5bb5756129af 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -1624,6 +1624,8 @@ enum netdev_priv_flags { * @type: Interface hardware type * @hard_header_len: Maximum hardware header length. * @min_header_len: Minimum hardware header length + * @upper_level: Maximum depth level of upper devices. + * @lower_level: Maximum depth level of lower devices. * * @needed_headroom: Extra headroom the hardware may need, but not in all * cases can this be guaranteed @@ -1854,6 +1856,8 @@ struct net_device { unsigned short type; unsigned short hard_header_len; unsigned char min_header_len; + unsigned char upper_level; + unsigned char lower_level; unsigned short needed_headroom; unsigned short needed_tailroom; diff --git a/net/core/dev.c b/net/core/dev.c index 0891f499c1bb..6a4b4ce62204 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -146,6 +146,7 @@ #include "net-sysfs.h" #define MAX_GRO_SKBS 8 +#define MAX_NEST_DEV 8 /* This should be increased if a protocol with a bigger head is added. */ #define GRO_MAX_HEAD (MAX_HEADER + 128) @@ -6602,6 +6603,21 @@ struct net_device *netdev_upper_get_next_dev_rcu(struct net_device *dev, } EXPORT_SYMBOL(netdev_upper_get_next_dev_rcu); +static struct net_device *netdev_next_upper_dev(struct net_device *dev, + struct list_head **iter) +{ + struct netdev_adjacent *upper; + + upper = list_entry((*iter)->next, struct netdev_adjacent, list); + + if (&upper->list == &dev->adj_list.upper) + return NULL; + + *iter = &upper->list; + + return upper->dev; +} + static struct net_device *netdev_next_upper_dev_rcu(struct net_device *dev, struct list_head **iter) { @@ -6619,6 +6635,33 @@ static struct net_device *netdev_next_upper_dev_rcu(struct net_device *dev, return upper->dev; } +int netdev_walk_all_upper_dev(struct net_device *dev, + int (*fn)(struct net_device *dev, + void *data), + void *data) +{ + struct net_device *udev; + struct list_head *iter; + int ret; + + for (iter = &dev->adj_list.upper, + udev = netdev_next_upper_dev(dev, &iter); + udev; + udev = netdev_next_upper_dev(dev, &iter)) { + /* first is the upper device itself */ + ret = fn(udev, data); + if (ret) + return ret; + + /* then look at all of its upper devices */ + ret = netdev_walk_all_upper_dev(udev, fn, data); + if (ret) + return ret; + } + + return 0; +} + int netdev_walk_all_upper_dev_rcu(struct net_device *dev, int (*fn)(struct net_device *dev, void *data), @@ -6785,6 +6828,52 @@ static struct net_device *netdev_next_lower_dev_rcu(struct net_device *dev, return lower->dev; } +static u8 __netdev_upper_depth(struct net_device *dev) +{ + struct net_device *udev; + struct list_head *iter; + u8 max_depth = 0; + + for (iter = &dev->adj_list.upper, + udev = netdev_next_upper_dev(dev, &iter); + udev; + udev = netdev_next_upper_dev(dev, &iter)) { + if (max_depth < udev->upper_level) + max_depth = udev->upper_level; + } + + return max_depth; +} + +static u8 __netdev_lower_depth(struct net_device *dev) +{ + struct net_device *ldev; + struct list_head *iter; + u8 max_depth = 0; + + for (iter = &dev->adj_list.lower, + ldev = netdev_next_lower_dev(dev, &iter); + ldev; + ldev = netdev_next_lower_dev(dev, &iter)) { + if (max_depth < ldev->lower_level) + max_depth = ldev->lower_level; + } + + return max_depth; +} + +static int __netdev_update_upper_level(struct net_device *dev, void *data) +{ + dev->upper_level = __netdev_upper_depth(dev) + 1; + return 0; +} + +static int __netdev_update_lower_level(struct net_device *dev, void *data) +{ + dev->lower_level = __netdev_lower_depth(dev) + 1; + return 0; +} + int netdev_walk_all_lower_dev_rcu(struct net_device *dev, int (*fn)(struct net_device *dev, void *data), @@ -7063,6 +7152,9 @@ static int __netdev_upper_dev_link(struct net_device *dev, if (netdev_has_upper_dev(upper_dev, dev)) return -EBUSY; + if ((dev->lower_level + upper_dev->upper_level) > MAX_NEST_DEV) + return -EMLINK; + if (!master) { if (netdev_has_upper_dev(dev, upper_dev)) return -EEXIST; @@ -7089,6 +7181,12 @@ static int __netdev_upper_dev_link(struct net_device *dev, if (ret) goto rollback; + __netdev_update_upper_level(dev, NULL); + netdev_walk_all_lower_dev(dev, __netdev_update_upper_level, NULL); + + __netdev_update_lower_level(upper_dev, NULL); + netdev_walk_all_upper_dev(upper_dev, __netdev_update_lower_level, NULL); + return 0; rollback: @@ -7171,6 +7269,12 @@ void netdev_upper_dev_unlink(struct net_device *dev, call_netdevice_notifiers_info(NETDEV_CHANGEUPPER, &changeupper_info.info); + + __netdev_update_upper_level(dev, NULL); + netdev_walk_all_lower_dev(dev, __netdev_update_upper_level, NULL); + + __netdev_update_lower_level(upper_dev, NULL); + netdev_walk_all_upper_dev(upper_dev, __netdev_update_lower_level, NULL); } EXPORT_SYMBOL(netdev_upper_dev_unlink); @@ -9157,6 +9261,8 @@ struct net_device *alloc_netdev_mqs(int sizeof_priv, const char *name, dev->gso_max_size = GSO_MAX_SIZE; dev->gso_max_segs = GSO_MAX_SEGS; + dev->upper_level = 1; + dev->lower_level = 1; INIT_LIST_HEAD(&dev->napi_list); INIT_LIST_HEAD(&dev->unreg_list); From patchwork Wed Sep 4 18:39:11 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taehee Yoo X-Patchwork-Id: 1157892 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="pr0sAZCi"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 46Nszt2vvqz9s4Y for ; Thu, 5 Sep 2019 04:39:22 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732872AbfIDSjV (ORCPT ); Wed, 4 Sep 2019 14:39:21 -0400 Received: from mail-pl1-f194.google.com ([209.85.214.194]:33209 "EHLO mail-pl1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732177AbfIDSjV (ORCPT ); Wed, 4 Sep 2019 14:39:21 -0400 Received: by mail-pl1-f194.google.com with SMTP id t11so3873246plo.0 for ; Wed, 04 Sep 2019 11:39:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=laL3gXbsDJ48K3M5hCahWUB01RUhYTepy/2Wo0FLWlQ=; b=pr0sAZCiS5HsO+Aak4raJ6AS0WNLk3Tw0S8ofVhLwUWgH7o+U+Ao1iArnyqA+8G0HA ic8rHKaJDTzAEodu8QmPn+nBXwGSZaG4qfzYSXRqU3XFCJWBVRfbFayYCV6lKhDtfgDI 0foQYndqnoe93jTQ4lCsjhXziw09sN+XXrhBf6bmgI67GFrGtl5LQIyoAUk+GwtafIIO pNOz5pRx3x6Uy0R4NjeP3pKBGVaR53OeS7OxjfYFeX9HrngB5n02AR3ZKuB8CW9IcXV0 me2uDQtlCjBZth3d3g923rLPvxmD382A6LD0DQsVbLJDzvOUTv2X6TGwJEwvY3vLrwKZ fsjw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=laL3gXbsDJ48K3M5hCahWUB01RUhYTepy/2Wo0FLWlQ=; b=H14XPLRyPGAvoremsWQ7pk1hLz85fnUR5K281lHESgGBhRxOW230E510Kl5WDVoarV HUT71zKSRMvRlVJq9W2kvPbd4o5wc5mynwU9Y9chawDlbmEeXzlS4UBxvQffwNgoOcQ0 KSKoCS7l7dTp2CwolzGUEPKb3JIgrK7F8Lw+SwZS03t9hdK+zpO72UxQ35TgoAcK2aet qKb0yasD8QiuUaNDcQBzT0G7Hrvb8Zhn8TBJrM6ZqcXotTWRJ1gE2X8YCL0JJrF4qgLM xu3wfk8KzE8evKAQM/Yjw4+ChEl2e8nDm5xxzv1aBSGZLXttPB6YL6wGFQxxzKh8ES7r lvdg== X-Gm-Message-State: APjAAAVlLBnE7g7sj/Z2wJq/nIjr16l8TM8sCbZMagAtFKH/cmQCzNk2 dICWY5lO5J3Ll1GWd/JJ5I4= X-Google-Smtp-Source: APXvYqxzv5P0YMMj7F7Bt/foAfCs6ef2Tm4JFpNdErV4y7BXVlo64MBNiaWQVWF9JEIwKqfQ5REQ3g== X-Received: by 2002:a17:902:e613:: with SMTP id cm19mr39427142plb.299.1567622360361; Wed, 04 Sep 2019 11:39:20 -0700 (PDT) Received: from ap-To-be-filled-by-O-E-M.1.1.1.1 ([14.33.120.60]) by smtp.gmail.com with ESMTPSA id 74sm9648904pfy.78.2019.09.04.11.39.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Sep 2019 11:39:19 -0700 (PDT) From: Taehee Yoo To: davem@davemloft.net, netdev@vger.kernel.org, j.vosburgh@gmail.com, vfalico@gmail.com, andy@greyhouse.net, jiri@resnulli.us, sd@queasysnail.net, roopa@cumulusnetworks.com, saeedm@mellanox.com, manishc@marvell.com, rahulv@marvell.com, kys@microsoft.com, haiyangz@microsoft.com, sthemmin@microsoft.com, sashal@kernel.org, hare@suse.de, varun@chelsio.com, ubraun@linux.ibm.com, kgraul@linux.ibm.com Cc: ap420073@gmail.com Subject: [PATCH net 02/11] vlan: use dynamic lockdep key instead of subclass Date: Thu, 5 Sep 2019 03:39:11 +0900 Message-Id: <20190904183911.14600-1-ap420073@gmail.com> X-Mailer: git-send-email 2.17.1 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org All VLAN device has same lockdep key and subclass is initialized with nest_level. But actual nest_level value can be changed when a lower device is attached. And at this moment, the subclass should be updated but it seems to be unsafe. So this patch makes VLAN use dynamic lockdep key instead of the subclass. Test commands: ip link add dummy0 type dummy ip link set dummy0 up ip link add bond0 type bond ip link add vlan_dummy1 link dummy0 type vlan id 1 ip link add vlan_bond1 link bond0 type vlan id 2 ip link set vlan_dummy1 master bond0 ip link set bond0 up ip link set vlan_dummy1 up ip link set vlan_bond1 up Both vlan_dummy1 and vlan_bond1 have the same subclass and it makes unnecessary deadlock warning message. Splat looks like: [ 149.244978] ============================================ [ 149.244978] WARNING: possible recursive locking detected [ 149.244978] 5.3.0-rc7+ #322 Not tainted [ 149.244978] -------------------------------------------- [ 149.244978] ip/1340 is trying to acquire lock: [ 149.244978] 000000001399b1a7 (&vlan_netdev_addr_lock_key/1){+...}, at: dev_uc_sync_multiple+0xfa/0x1a0 [ 149.279600] [ 149.279600] but task is already holding lock: [ 149.279600] 00000000b963d9b4 (&vlan_netdev_addr_lock_key/1){+...}, at: dev_set_rx_mode+0x19/0x30 [ 149.279600] [ 149.279600] other info that might help us debug this: [ 149.305981] Possible unsafe locking scenario: [ 149.305981] [ 149.305981] CPU0 [ 149.305981] ---- [ 149.305981] lock(&vlan_netdev_addr_lock_key/1); [ 149.305981] lock(&vlan_netdev_addr_lock_key/1); [ 149.326258] [ 149.326258] *** DEADLOCK *** [ 149.326258] [ 149.326258] May be due to missing lock nesting notation [ 149.326258] [ 149.326258] 4 locks held by ip/1340: [ 149.326258] #0: 00000000927f0698 (rtnl_mutex){+.+.}, at: rtnetlink_rcv_msg+0x466/0x8a0 [ 149.326258] #1: 00000000b963d9b4 (&vlan_netdev_addr_lock_key/1){+...}, at: dev_set_rx_mode+0x19/0x30 [ 149.326258] #2: 0000000027395445 (&dev_addr_list_lock_key/3){+...}, at: dev_mc_sync+0xfa/0x1a0 [ 149.369961] #3: 00000000ce334932 (rcu_read_lock){....}, at: bond_set_rx_mode+0x5/0x3c0 [bonding] [ 149.369961] [ 149.369961] stack backtrace: [ 149.369961] CPU: 1 PID: 1340 Comm: ip Not tainted 5.3.0-rc7+ #322 [ 149.369961] Call Trace: [ 149.369961] dump_stack+0x7c/0xbb [ 149.369961] __lock_acquire+0x26a9/0x3de0 [ 149.369961] ? register_lock_class+0x14d0/0x14d0 [ 149.369961] ? register_lock_class+0x14d0/0x14d0 [ 149.369961] lock_acquire+0x164/0x3b0 [ 149.433970] ? dev_uc_sync_multiple+0xfa/0x1a0 [ 149.433970] _raw_spin_lock_nested+0x2e/0x60 [ 149.433970] ? dev_uc_sync_multiple+0xfa/0x1a0 [ 149.433970] dev_uc_sync_multiple+0xfa/0x1a0 [ 149.433970] bond_set_rx_mode+0x269/0x3c0 [bonding] [ 149.433970] ? bond_init+0x6f0/0x6f0 [bonding] [ 149.433970] dev_mc_sync+0x15a/0x1a0 [ 149.433970] vlan_dev_set_rx_mode+0x37/0x80 [8021q] [ 149.433970] dev_set_rx_mode+0x21/0x30 [ 149.433970] __dev_open+0x202/0x310 [ 149.433970] ? dev_set_rx_mode+0x30/0x30 [ 149.433970] ? mark_held_locks+0xa5/0xe0 [ 149.433970] ? __local_bh_enable_ip+0xe9/0x1b0 [ 149.433970] __dev_change_flags+0x3c3/0x500 [ ... ] Fixes: 0fe1e567d0b4 ("[VLAN]: nested VLAN: fix lockdep's recursive locking warning") Signed-off-by: Taehee Yoo --- include/linux/if_vlan.h | 3 +++ net/8021q/vlan_dev.c | 28 +++++++++++++++------------- 2 files changed, 18 insertions(+), 13 deletions(-) diff --git a/include/linux/if_vlan.h b/include/linux/if_vlan.h index 244278d5c222..1aed9f613e90 100644 --- a/include/linux/if_vlan.h +++ b/include/linux/if_vlan.h @@ -183,6 +183,9 @@ struct vlan_dev_priv { struct netpoll *netpoll; #endif unsigned int nest_level; + + struct lock_class_key xmit_lock_key; + struct lock_class_key addr_lock_key; }; static inline struct vlan_dev_priv *vlan_dev_priv(const struct net_device *dev) diff --git a/net/8021q/vlan_dev.c b/net/8021q/vlan_dev.c index 93eadf179123..12bc80650087 100644 --- a/net/8021q/vlan_dev.c +++ b/net/8021q/vlan_dev.c @@ -494,24 +494,24 @@ static void vlan_dev_set_rx_mode(struct net_device *vlan_dev) * "super class" of normal network devices; split their locks off into a * separate class since they always nest. */ -static struct lock_class_key vlan_netdev_xmit_lock_key; -static struct lock_class_key vlan_netdev_addr_lock_key; - static void vlan_dev_set_lockdep_one(struct net_device *dev, struct netdev_queue *txq, - void *_subclass) + void *_unused) { - lockdep_set_class_and_subclass(&txq->_xmit_lock, - &vlan_netdev_xmit_lock_key, - *(int *)_subclass); + struct vlan_dev_priv *vlan = vlan_dev_priv(dev); + + lockdep_set_class(&txq->_xmit_lock, &vlan->xmit_lock_key); } -static void vlan_dev_set_lockdep_class(struct net_device *dev, int subclass) +static void vlan_dev_set_lockdep_class(struct net_device *dev) { - lockdep_set_class_and_subclass(&dev->addr_list_lock, - &vlan_netdev_addr_lock_key, - subclass); - netdev_for_each_tx_queue(dev, vlan_dev_set_lockdep_one, &subclass); + struct vlan_dev_priv *vlan = vlan_dev_priv(dev); + + lockdep_register_key(&vlan->addr_lock_key); + lockdep_set_class(&dev->addr_list_lock, &vlan->addr_lock_key); + + lockdep_register_key(&vlan->xmit_lock_key); + netdev_for_each_tx_queue(dev, vlan_dev_set_lockdep_one, NULL); } static int vlan_dev_get_lock_subclass(struct net_device *dev) @@ -609,7 +609,7 @@ static int vlan_dev_init(struct net_device *dev) SET_NETDEV_DEVTYPE(dev, &vlan_type); - vlan_dev_set_lockdep_class(dev, vlan_dev_get_lock_subclass(dev)); + vlan_dev_set_lockdep_class(dev); vlan->vlan_pcpu_stats = netdev_alloc_pcpu_stats(struct vlan_pcpu_stats); if (!vlan->vlan_pcpu_stats) @@ -630,6 +630,8 @@ static void vlan_dev_uninit(struct net_device *dev) kfree(pm); } } + lockdep_unregister_key(&vlan->addr_lock_key); + lockdep_unregister_key(&vlan->xmit_lock_key); } static netdev_features_t vlan_dev_fix_features(struct net_device *dev, From patchwork Wed Sep 4 18:39:27 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taehee Yoo X-Patchwork-Id: 1157893 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="p0nKN5Hc"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 46Nt0D1vGFz9sBF for ; Thu, 5 Sep 2019 04:39:40 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1733234AbfIDSji (ORCPT ); Wed, 4 Sep 2019 14:39:38 -0400 Received: from mail-pg1-f193.google.com ([209.85.215.193]:33442 "EHLO mail-pg1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730209AbfIDSji (ORCPT ); Wed, 4 Sep 2019 14:39:38 -0400 Received: by mail-pg1-f193.google.com with SMTP id n190so11712482pgn.0 for ; Wed, 04 Sep 2019 11:39:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=cdoCAIFZgQNw5lEaVpVLvxOxoSwVhwv8+E24vSI/Tj8=; b=p0nKN5HcgX/pkZ4fC/wB5m+GAjISdOCRX6fpCm29tNsvpSdH7PRBN5VYdBZsrcYQXy k94SGIMirvTcrAWwuN0vjkbh/moBqP8CvjAX9gTXk5iqia4IOozQUTSvyx8CKTtHlXuB J1ELXFtFDzKJIXX3MffjuLC+vXmN3G+H/FwPBZEBYfc9W4dG8baLZ5t9tvyUy4VEvZbZ VwurjnQXcZR8Go15RwzqgT8vs2HC2s82RHZ+sggivRQ8W6Y19DJ3oscZYb34o4xESoGt 76lR+pOasn3++4BXd/8o8+qxJbAEoeeB0eMGxKYeZr3PU7pY0eUEE1Akatu0MEA3HYnc EwyQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=cdoCAIFZgQNw5lEaVpVLvxOxoSwVhwv8+E24vSI/Tj8=; b=tUXZFUkYI76QTGG8f/HFz3cv8mD+GZVUogf87rBp31NXXFH5EPirRYOvvG3i/Groke zM9ezzfTfXC24OKK8O9/YsBeGei4i+TWQMazSOke4udAKIyDr/0Q0VswmzP6/hutt0AG 28UaHInpEj9KCuyj4xb9MGuHY4JYIhhi3zBjycCUNydok/9awRbYojucUluSXJ0GRUNX fxy25vtEV8EIenPmzGwhMXbPNU+l5SdaHv2S3Q/m9Br3LsMEhzSKLmJE9Ak/xY+Pd+Ys RmaYd1XXWIch1ldLciiB7Ko8TeONqF/2hGv9jV4wQBjn272t/ZTl5wIgKauBtfRpq8Ye W52w== X-Gm-Message-State: APjAAAXjSHcLqRX6MmA+u4O7hacoyh2sUntKHJ8boBBM9bJywhCZx9+V U05AVKhPQdi3+a+GCqaYy/c= X-Google-Smtp-Source: APXvYqz3mQPuRc54gIJUqeqcgQKLkpnJS5cYxWVsbyIn2PrR1YWhKmWFHoC/s8fBYEjoD+LY0wwYdw== X-Received: by 2002:a65:6096:: with SMTP id t22mr37569053pgu.204.1567622377342; Wed, 04 Sep 2019 11:39:37 -0700 (PDT) Received: from ap-To-be-filled-by-O-E-M.1.1.1.1 ([14.33.120.60]) by smtp.gmail.com with ESMTPSA id 2sm23704855pfa.43.2019.09.04.11.39.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Sep 2019 11:39:36 -0700 (PDT) From: Taehee Yoo To: davem@davemloft.net, netdev@vger.kernel.org, j.vosburgh@gmail.com, vfalico@gmail.com, andy@greyhouse.net, jiri@resnulli.us, sd@queasysnail.net, roopa@cumulusnetworks.com, saeedm@mellanox.com, manishc@marvell.com, rahulv@marvell.com, kys@microsoft.com, haiyangz@microsoft.com, sthemmin@microsoft.com, sashal@kernel.org, hare@suse.de, varun@chelsio.com, ubraun@linux.ibm.com, kgraul@linux.ibm.com Cc: ap420073@gmail.com Subject: [PATCH net 03/11] bonding: split IFF_BONDING into IFF_BONDING and IFF_BONDING_SLAVE Date: Thu, 5 Sep 2019 03:39:27 +0900 Message-Id: <20190904183927.14754-1-ap420073@gmail.com> X-Mailer: git-send-email 2.17.1 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org The IFF_BONDING means bonding master or bonding slave device. ->ndo_add_slave() sets IFF_BONDING flag and ->ndo_del_slave() removes IFF_BONDING flag. This routine makes a problem in the nesting bonding structure. bond1<--bond2 Both bond0 and bond1 are bonding device and these should keep having IFF_BONDING flag until they are removed. But bond1 would lose IFF_BONDING at ->ndo_del_slave because that routine can not check whether the slave device is the bonding type or not. So that this patch splits the IFF_BONDING into theIFF_BONDING and the IFF_BONDING_SLAVE. The IFF_BONDING is bonding master flag and IFF_BONDING_SLAVE is bonding slave flag. Test commands: ip link add bond0 type bond ip link add bond1 type bond ip link set bond1 master bond0 ip link set bond1 nomaster ip link del bond1 type bond ip link add bond1 type bond Splat looks like: [ 149.201107] proc_dir_entry 'bonding/bond1' already registered [ 149.208013] WARNING: CPU: 1 PID: 1308 at fs/proc/generic.c:361 proc_register+0x2a9/0x3e0 [ 149.208866] Modules linked in: bonding veth openvswitch nsh nf_conncount nf_nat nf_conntrack nf_defrag_ipv4 ip_tables6 [ 149.208866] CPU: 1 PID: 1308 Comm: ip Not tainted 5.3.0-rc7+ #322 [ 149.208866] RIP: 0010:proc_register+0x2a9/0x3e0 [ 149.208866] Code: 89 fa 48 c1 ea 03 80 3c 02 00 0f 85 39 01 00 00 48 8b 04 24 48 89 ea 48 c7 c7 a0 a0 13 89 48 8b b0 0 [ 149.208866] RSP: 0018:ffff88810df9f098 EFLAGS: 00010286 [ 149.208866] RAX: dffffc0000000008 RBX: ffff8880b5d3aa50 RCX: ffffffff87cdec92 [ 149.208866] RDX: 0000000000000001 RSI: 0000000000000008 RDI: ffff888116bf6a8c [ 149.208866] RBP: ffff8880b5d3acd3 R08: ffffed1022d7ff71 R09: ffffed1022d7ff71 [ 149.208866] R10: 0000000000000001 R11: ffffed1022d7ff70 R12: ffff8880b5d3abe8 [ 149.208866] R13: ffff8880b5d3acd2 R14: dffffc0000000000 R15: ffffed1016ba759a [ 149.208866] FS: 00007f4bd1f650c0(0000) GS:ffff888116a00000(0000) knlGS:0000000000000000 [ 149.208866] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 149.208866] CR2: 000055e7ca686118 CR3: 0000000106fd4000 CR4: 00000000001006e0 [ 149.208866] Call Trace: [ 149.208866] proc_create_seq_private+0xb3/0xf0 [ 149.208866] bond_create_proc_entry+0x1b3/0x3f0 [bonding] [ 149.208866] bond_netdev_event+0x433/0x970 [bonding] [ 149.208866] ? __module_text_address+0x13/0x140 [ 149.208866] notifier_call_chain+0x90/0x160 [ 149.208866] register_netdevice+0x9b3/0xd70 [ 149.208866] ? alloc_netdev_mqs+0x854/0xc10 [ 149.208866] ? netdev_change_features+0xa0/0xa0 [ 149.208866] ? rtnl_create_link+0x2ed/0xad0 [ 149.208866] bond_newlink+0x2a/0x60 [bonding] [ 149.208866] __rtnl_newlink+0xb75/0x1180 [ ... ] Fixes: 0b680e753724 ("[PATCH] bonding: Add priv_flag to avoid event mishandling") Signed-off-by: Taehee Yoo --- drivers/net/bonding/bond_main.c | 13 +++++-------- .../net/ethernet/qlogic/netxen/netxen_nic_main.c | 2 +- drivers/net/hyperv/netvsc_drv.c | 3 +-- drivers/scsi/fcoe/fcoe.c | 2 +- drivers/target/iscsi/cxgbit/cxgbit_cm.c | 2 +- include/linux/netdevice.h | 9 ++++++--- 6 files changed, 15 insertions(+), 16 deletions(-) diff --git a/drivers/net/bonding/bond_main.c b/drivers/net/bonding/bond_main.c index 931d9d935686..abd008c31c9a 100644 --- a/drivers/net/bonding/bond_main.c +++ b/drivers/net/bonding/bond_main.c @@ -1560,7 +1560,7 @@ int bond_enslave(struct net_device *bond_dev, struct net_device *slave_dev, goto err_restore_mac; } - slave_dev->priv_flags |= IFF_BONDING; + slave_dev->priv_flags |= IFF_BONDING_SLAVE; /* initialize slave stats */ dev_get_stats(new_slave->dev, &new_slave->slave_stats); @@ -1816,7 +1816,7 @@ int bond_enslave(struct net_device *bond_dev, struct net_device *slave_dev, slave_disable_netpoll(new_slave); err_close: - slave_dev->priv_flags &= ~IFF_BONDING; + slave_dev->priv_flags &= ~IFF_BONDING_SLAVE; dev_close(slave_dev); err_restore_mac: @@ -2017,7 +2017,7 @@ static int __bond_release_one(struct net_device *bond_dev, else dev_set_mtu(slave_dev, slave->original_mtu); - slave_dev->priv_flags &= ~IFF_BONDING; + slave_dev->priv_flags &= ~IFF_BONDING_SLAVE; bond_free_slave(slave); @@ -3221,10 +3221,7 @@ static int bond_netdev_event(struct notifier_block *this, netdev_dbg(event_dev, "%s received %s\n", __func__, netdev_cmd_to_name(event)); - if (!(event_dev->priv_flags & IFF_BONDING)) - return NOTIFY_DONE; - - if (event_dev->flags & IFF_MASTER) { + if (netif_is_bond_master(event_dev)) { int ret; ret = bond_master_netdev_event(event, event_dev); @@ -3232,7 +3229,7 @@ static int bond_netdev_event(struct notifier_block *this, return ret; } - if (event_dev->flags & IFF_SLAVE) + if (netif_is_bond_slave(event_dev)) return bond_slave_netdev_event(event, event_dev); return NOTIFY_DONE; diff --git a/drivers/net/ethernet/qlogic/netxen/netxen_nic_main.c b/drivers/net/ethernet/qlogic/netxen/netxen_nic_main.c index 58e2eaf77014..5e0389ba1f13 100644 --- a/drivers/net/ethernet/qlogic/netxen/netxen_nic_main.c +++ b/drivers/net/ethernet/qlogic/netxen/netxen_nic_main.c @@ -3340,7 +3340,7 @@ static void netxen_config_master(struct net_device *dev, unsigned long event) * released and is dev_close()ed in bond_release() * just before IFF_BONDING is stripped. */ - if (!master && dev->priv_flags & IFF_BONDING) + if (!master && netif_is_bond_slave(dev)) netxen_free_ip_list(adapter, true); } diff --git a/drivers/net/hyperv/netvsc_drv.c b/drivers/net/hyperv/netvsc_drv.c index e8fce6d715ef..6831202d9bcb 100644 --- a/drivers/net/hyperv/netvsc_drv.c +++ b/drivers/net/hyperv/netvsc_drv.c @@ -2439,8 +2439,7 @@ static int netvsc_netdev_event(struct notifier_block *this, return NOTIFY_DONE; /* Avoid Bonding master dev with same MAC registering as VF */ - if ((event_dev->priv_flags & IFF_BONDING) && - (event_dev->flags & IFF_MASTER)) + if (netif_is_bond_master(event_dev)) return NOTIFY_DONE; switch (event) { diff --git a/drivers/scsi/fcoe/fcoe.c b/drivers/scsi/fcoe/fcoe.c index 00dd47bcbb1e..750a6540eb9d 100644 --- a/drivers/scsi/fcoe/fcoe.c +++ b/drivers/scsi/fcoe/fcoe.c @@ -307,7 +307,7 @@ static int fcoe_interface_setup(struct fcoe_interface *fcoe, } /* Do not support for bonding device */ - if (netdev->priv_flags & IFF_BONDING && netdev->flags & IFF_MASTER) { + if (netif_is_bond_master(netdev)) { FCOE_NETDEV_DBG(netdev, "Bonded interfaces not supported\n"); return -EOPNOTSUPP; } diff --git a/drivers/target/iscsi/cxgbit/cxgbit_cm.c b/drivers/target/iscsi/cxgbit/cxgbit_cm.c index c70caf4ea490..16c8cae333b2 100644 --- a/drivers/target/iscsi/cxgbit/cxgbit_cm.c +++ b/drivers/target/iscsi/cxgbit/cxgbit_cm.c @@ -247,7 +247,7 @@ struct cxgbit_device *cxgbit_find_device(struct net_device *ndev, u8 *port_id) static struct net_device *cxgbit_get_real_dev(struct net_device *ndev) { - if (ndev->priv_flags & IFF_BONDING) { + if (netif_is_bond_master(ndev) || netif_is_bond_slave(ndev)) { pr_err("Bond devices are not supported. Interface:%s\n", ndev->name); return NULL; diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index 5bb5756129af..a2c47f43e54b 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -1441,7 +1441,7 @@ struct net_device_ops { * * @IFF_802_1Q_VLAN: 802.1Q VLAN device * @IFF_EBRIDGE: Ethernet bridging device - * @IFF_BONDING: bonding master or slave + * @IFF_BONDING: bonding master * @IFF_ISATAP: ISATAP interface (RFC4214) * @IFF_WAN_HDLC: WAN HDLC device * @IFF_XMIT_DST_RELEASE: dev_hard_start_xmit() is allowed to @@ -1474,6 +1474,7 @@ struct net_device_ops { * @IFF_FAILOVER_SLAVE: device is lower dev of a failover master device * @IFF_L3MDEV_RX_HANDLER: only invoke the rx handler of L3 master device * @IFF_LIVE_RENAME_OK: rename is allowed while device is up and running + * @IFF_BONDING_SLAVE: bonding slave */ enum netdev_priv_flags { IFF_802_1Q_VLAN = 1<<0, @@ -1507,6 +1508,7 @@ enum netdev_priv_flags { IFF_FAILOVER_SLAVE = 1<<28, IFF_L3MDEV_RX_HANDLER = 1<<29, IFF_LIVE_RENAME_OK = 1<<30, + IFF_BONDING_SLAVE = 1<<31, }; #define IFF_802_1Q_VLAN IFF_802_1Q_VLAN @@ -1539,6 +1541,7 @@ enum netdev_priv_flags { #define IFF_FAILOVER_SLAVE IFF_FAILOVER_SLAVE #define IFF_L3MDEV_RX_HANDLER IFF_L3MDEV_RX_HANDLER #define IFF_LIVE_RENAME_OK IFF_LIVE_RENAME_OK +#define IFF_BONDING_SLAVE IFF_BONDING_SLAVE /** * struct net_device - The DEVICE structure. @@ -4569,12 +4572,12 @@ static inline bool netif_is_macvlan_port(const struct net_device *dev) static inline bool netif_is_bond_master(const struct net_device *dev) { - return dev->flags & IFF_MASTER && dev->priv_flags & IFF_BONDING; + return dev->priv_flags & IFF_BONDING; } static inline bool netif_is_bond_slave(const struct net_device *dev) { - return dev->flags & IFF_SLAVE && dev->priv_flags & IFF_BONDING; + return dev->priv_flags & IFF_BONDING_SLAVE; } static inline bool netif_supports_nofcs(struct net_device *dev) From patchwork Wed Sep 4 18:39:40 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taehee Yoo X-Patchwork-Id: 1157894 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="XLX/ZmpO"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 46Nt0S1Vl4z9s4Y for ; Thu, 5 Sep 2019 04:39:52 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1733271AbfIDSjv (ORCPT ); Wed, 4 Sep 2019 14:39:51 -0400 Received: from mail-pf1-f196.google.com ([209.85.210.196]:45477 "EHLO mail-pf1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730209AbfIDSjv (ORCPT ); Wed, 4 Sep 2019 14:39:51 -0400 Received: by mail-pf1-f196.google.com with SMTP id y72so6208504pfb.12 for ; Wed, 04 Sep 2019 11:39:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=4z8U2U632+awJQZehELz6R59fq+/SNm4unGVvTkWJHw=; b=XLX/ZmpO8hEZqq6Fm8GbI9Ju64u7zzuM2ZDNOMgL/QlCzHRvGJNcp2dzcN9bHTo1s7 I4uHp4IufO4rocoYRL93on68rHG6Ffh1tHEiWeRHog7P6saM9lNoCgVYpCs4srGvWVU3 iIcteBb/zukFsNhb/uWV5Tv55YWmPxVUKx9OauleZ29Fe6mGv3AhA3BSdteJX5N+bXFh GCSjBQmCyMOsIhj5IQ9yC4IlUIzB+quptqC0rwpr1oW00TmLMLyOu4flLrwbACh9wfZD PImbaSNGGZmN9SHvSaW94YPLKgFHemWWq8MkO0KuiMcKJvZxb9AInv1YAz614QcGRnqg wrMw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=4z8U2U632+awJQZehELz6R59fq+/SNm4unGVvTkWJHw=; b=CiIiqouOtUBnTiDwstEttCBbwfiL8tRAl7XhQ2UIJViAYNyssFBHGP1Rq3MT9FJAlc FSsxQdi7ScCrdGe/N6OTda2gOzBMGEVVXe346lzBk0sNcwlT9RbXShZ6hKjNsRsdXTne BaTEuxWspcU5yQpk+PguTK3cSUJCEtF/hobrLv8xjhLy1srlyOFmfTPqv6mgfaWH0U5h L8ADC91HIqTz3fzxjAyuqESq1l2OvxYb+W2PpLCrM6nFm7ZZqneNoGKT5CPAmvJIzrWr 2sm2muX0GqQ736sVNTY4u6gS0O5G1rY9BQ2MGuAF0GyOWKlljXCFeBwvrO+FM3rxRDLU tI6A== X-Gm-Message-State: APjAAAXLWvhz6xavYHYsjan+E1Ku5Z2uzaeXga9e1wm99AbWAGQSCwgb /VWkFkxs5VcDxLojtKxGnMoq9AYK/co= X-Google-Smtp-Source: APXvYqwuLqJBqSvcI2raUUxBazLlBbeewDDgqoxSHw3HK35wyzMKsbPcTQ6qIUYGfO8gvnQR2Ak30Q== X-Received: by 2002:a62:b415:: with SMTP id h21mr47893217pfn.198.1567622390051; Wed, 04 Sep 2019 11:39:50 -0700 (PDT) Received: from ap-To-be-filled-by-O-E-M.1.1.1.1 ([14.33.120.60]) by smtp.gmail.com with ESMTPSA id 129sm12300796pfd.173.2019.09.04.11.39.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Sep 2019 11:39:49 -0700 (PDT) From: Taehee Yoo To: davem@davemloft.net, netdev@vger.kernel.org, j.vosburgh@gmail.com, vfalico@gmail.com, andy@greyhouse.net, jiri@resnulli.us, sd@queasysnail.net, roopa@cumulusnetworks.com, saeedm@mellanox.com, manishc@marvell.com, rahulv@marvell.com, kys@microsoft.com, haiyangz@microsoft.com, sthemmin@microsoft.com, sashal@kernel.org, hare@suse.de, varun@chelsio.com, ubraun@linux.ibm.com, kgraul@linux.ibm.com Cc: ap420073@gmail.com Subject: [PATCH net 04/11] bonding: use dynamic lockdep key instead of subclass Date: Thu, 5 Sep 2019 03:39:40 +0900 Message-Id: <20190904183940.14853-1-ap420073@gmail.com> X-Mailer: git-send-email 2.17.1 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org All bonding device has same lockdep key and subclass is initialized with nest_level. But actual nest_level value can be changed when a lower device is attached. And at this moment, the subclass should be updated but it seems to be unsafe. So this patch makes bonding use dynamic lockdep key instead of the subclass. Test commands: ip link add bond0 type bond for i in {1..5} do let A=$i-1 ip link add bond$i type bond ip link set bond$i master bond$A done ip link set bond5 master bond0 Splat looks like: [ 327.477830] ============================================ [ 327.477830] WARNING: possible recursive locking detected [ 327.477830] 5.3.0-rc7+ #322 Not tainted [ 327.477830] -------------------------------------------- [ 327.477830] ip/1399 is trying to acquire lock: [ 327.477830] 00000000f604be63 (&(&bond->stats_lock)->rlock#2/2){+.+.}, at: bond_get_stats+0xb8/0x500 [bonding] [ 327.477830] [ 327.477830] but task is already holding lock: [ 327.477830] 00000000e9d31238 (&(&bond->stats_lock)->rlock#2/2){+.+.}, at: bond_get_stats+0xb8/0x500 [bonding] [ 327.477830] [ 327.477830] other info that might help us debug this: [ 327.477830] Possible unsafe locking scenario: [ 327.477830] [ 327.477830] CPU0 [ 327.477830] ---- [ 327.477830] lock(&(&bond->stats_lock)->rlock#2/2); [ 327.477830] lock(&(&bond->stats_lock)->rlock#2/2); [ 327.477830] [ 327.477830] *** DEADLOCK *** [ 327.477830] [ 327.477830] May be due to missing lock nesting notation [ 327.477830] [ 327.477830] 3 locks held by ip/1399: [ 327.477830] #0: 00000000a762c4e3 (rtnl_mutex){+.+.}, at: rtnetlink_rcv_msg+0x466/0x8a0 [ 327.477830] #1: 00000000e9d31238 (&(&bond->stats_lock)->rlock#2/2){+.+.}, at: bond_get_stats+0xb8/0x500 [bonding] [ 327.477830] #2: 000000008f7ebff4 (rcu_read_lock){....}, at: bond_get_stats+0x9f/0x500 [bonding] [ 327.477830] [ 327.477830] stack backtrace: [ 327.477830] CPU: 0 PID: 1399 Comm: ip Not tainted 5.3.0-rc7+ #322 [ 327.477830] Call Trace: [ 327.477830] dump_stack+0x7c/0xbb [ 327.477830] __lock_acquire+0x26a9/0x3de0 [ 327.477830] ? __change_page_attr_set_clr+0x133b/0x1d20 [ 327.477830] ? register_lock_class+0x14d0/0x14d0 [ 327.477830] lock_acquire+0x164/0x3b0 [ 327.477830] ? bond_get_stats+0xb8/0x500 [bonding] [ 327.666914] _raw_spin_lock_nested+0x2e/0x60 [ 327.666914] ? bond_get_stats+0xb8/0x500 [bonding] [ 327.678302] bond_get_stats+0xb8/0x500 [bonding] [ 327.678302] ? bond_arp_rcv+0xf10/0xf10 [bonding] [ 327.678302] ? register_lock_class+0x14d0/0x14d0 [ 327.678302] ? bond_get_stats+0xb8/0x500 [bonding] [ 327.678302] dev_get_stats+0x1ec/0x270 [ 327.678302] bond_get_stats+0x1d1/0x500 [bonding] [ 327.678302] ? lock_acquire+0x164/0x3b0 [ 327.678302] ? bond_arp_rcv+0xf10/0xf10 [bonding] [ 327.678302] ? rtnl_is_locked+0x16/0x30 [ 327.678302] ? devlink_compat_switch_id_get+0x18/0x140 [ 327.678302] ? dev_get_alias+0xe2/0x190 [ 327.731145] ? dev_get_port_parent_id+0x12a/0x340 [ 327.731145] ? rtnl_phys_switch_id_fill+0x88/0xe0 [ 327.731145] dev_get_stats+0x1ec/0x270 [ 327.731145] rtnl_fill_stats+0x44/0xbe0 [ 327.731145] ? nla_put+0xc2/0x140 [ ... ] Fixes: d3fff6c443fe ("net: add netdev_lockdep_set_classes() helper") Signed-off-by: Taehee Yoo --- drivers/net/bonding/bond_main.c | 60 ++++++++++++++++++++++++++++++--- include/net/bonding.h | 3 ++ 2 files changed, 59 insertions(+), 4 deletions(-) diff --git a/drivers/net/bonding/bond_main.c b/drivers/net/bonding/bond_main.c index abd008c31c9a..2b16683bb8b8 100644 --- a/drivers/net/bonding/bond_main.c +++ b/drivers/net/bonding/bond_main.c @@ -1856,6 +1856,32 @@ int bond_enslave(struct net_device *bond_dev, struct net_device *slave_dev, return res; } +static void bond_dev_set_lockdep_one(struct net_device *dev, + struct netdev_queue *txq, + void *_unused) +{ + struct bonding *bond = netdev_priv(dev); + + lockdep_set_class(&txq->_xmit_lock, &bond->xmit_lock_key); +} + +static void bond_update_lock_key(struct net_device *dev) +{ + struct bonding *bond = netdev_priv(dev); + + lockdep_unregister_key(&bond->stats_lock_key); + lockdep_unregister_key(&bond->addr_lock_key); + lockdep_unregister_key(&bond->xmit_lock_key); + + lockdep_register_key(&bond->stats_lock_key); + lockdep_register_key(&bond->addr_lock_key); + lockdep_register_key(&bond->xmit_lock_key); + + lockdep_set_class(&bond->stats_lock, &bond->stats_lock_key); + lockdep_set_class(&dev->addr_list_lock, &bond->addr_lock_key); + netdev_for_each_tx_queue(dev, bond_dev_set_lockdep_one, NULL); +} + /* Try to release the slave device from the bond device * It is legal to access curr_active_slave without a lock because all the function * is RTNL-locked. If "all" is true it means that the function is being called @@ -2020,6 +2046,8 @@ static int __bond_release_one(struct net_device *bond_dev, slave_dev->priv_flags &= ~IFF_BONDING_SLAVE; bond_free_slave(slave); + if (netif_is_bond_master(slave_dev)) + bond_update_lock_key(slave_dev); return 0; } @@ -3454,7 +3482,7 @@ static void bond_get_stats(struct net_device *bond_dev, struct list_head *iter; struct slave *slave; - spin_lock_nested(&bond->stats_lock, bond_get_nest_level(bond_dev)); + spin_lock(&bond->stats_lock); memcpy(stats, &bond->bond_stats, sizeof(*stats)); rcu_read_lock(); @@ -4292,8 +4320,6 @@ void bond_setup(struct net_device *bond_dev) { struct bonding *bond = netdev_priv(bond_dev); - spin_lock_init(&bond->mode_lock); - spin_lock_init(&bond->stats_lock); bond->params = bonding_defaults; /* Initialize pointers */ @@ -4362,6 +4388,9 @@ static void bond_uninit(struct net_device *bond_dev) list_del(&bond->bond_list); + lockdep_unregister_key(&bond->stats_lock_key); + lockdep_unregister_key(&bond->addr_lock_key); + lockdep_unregister_key(&bond->xmit_lock_key); bond_debug_unregister(bond); } @@ -4753,6 +4782,29 @@ static int bond_check_params(struct bond_params *params) return 0; } +static struct lock_class_key qdisc_tx_busylock_key; +static struct lock_class_key qdisc_running_key; + +static void bond_dev_set_lockdep_class(struct net_device *dev) +{ + struct bonding *bond = netdev_priv(dev); + + dev->qdisc_tx_busylock = &qdisc_tx_busylock_key; + dev->qdisc_running_key = &qdisc_running_key; + + spin_lock_init(&bond->mode_lock); + + spin_lock_init(&bond->stats_lock); + lockdep_register_key(&bond->stats_lock_key); + lockdep_set_class(&bond->stats_lock, &bond->stats_lock_key); + + lockdep_register_key(&bond->addr_lock_key); + lockdep_set_class(&dev->addr_list_lock, &bond->addr_lock_key); + + lockdep_register_key(&bond->xmit_lock_key); + netdev_for_each_tx_queue(dev, bond_dev_set_lockdep_one, NULL); +} + /* Called from registration process */ static int bond_init(struct net_device *bond_dev) { @@ -4766,7 +4818,7 @@ static int bond_init(struct net_device *bond_dev) return -ENOMEM; bond->nest_level = SINGLE_DEPTH_NESTING; - netdev_lockdep_set_classes(bond_dev); + bond_dev_set_lockdep_class(bond_dev); list_add_tail(&bond->bond_list, &bn->dev_list); diff --git a/include/net/bonding.h b/include/net/bonding.h index f7fe45689142..c39ac7061e41 100644 --- a/include/net/bonding.h +++ b/include/net/bonding.h @@ -239,6 +239,9 @@ struct bonding { struct dentry *debug_dir; #endif /* CONFIG_DEBUG_FS */ struct rtnl_link_stats64 bond_stats; + struct lock_class_key stats_lock_key; + struct lock_class_key xmit_lock_key; + struct lock_class_key addr_lock_key; }; #define bond_slave_get_rcu(dev) \ From patchwork Wed Sep 4 18:39:53 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taehee Yoo X-Patchwork-Id: 1157895 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="ZshX7DkS"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 46Nt0l2zqMz9s4Y for ; Thu, 5 Sep 2019 04:40:07 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1733308AbfIDSkG (ORCPT ); Wed, 4 Sep 2019 14:40:06 -0400 Received: from mail-pg1-f195.google.com ([209.85.215.195]:43918 "EHLO mail-pg1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730209AbfIDSkG (ORCPT ); Wed, 4 Sep 2019 14:40:06 -0400 Received: by mail-pg1-f195.google.com with SMTP id u72so7507185pgb.10 for ; Wed, 04 Sep 2019 11:40:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=Z97Ow6Uu2tS7g/PTIDrFXebWhPHzfnHD1Ve2g/wBaZI=; b=ZshX7DkSUSSsapoYockUVI4ImPQNeX+0XHdGlzV2Ip4I+TGNtW5465TVHlwcxW7mP9 M2GG2IcFOC0boYkgqJtAPICjr+Czc8ExXeyNGfrjwewlyw0ypbWcI9rTadkmK8pJBEpB hqWR07u98wJENdUkuolk+oQw/DSQwIrDgzhDauuwNLNmcFhcZGBkhovOzGwZpIyj5FiI PfhsnMZ2NjGqe1eKiyzd+vK8sbLfGmRRbrbpvXIjSgPCUbp80/zyxnMub69RY5lR5qkN EAouH76YXMCfYQB7rDrtnYBX12ntVTdMGJXlxFFEMM2db2BryiDHpAZYl3yHE3UI3Snj 026w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=Z97Ow6Uu2tS7g/PTIDrFXebWhPHzfnHD1Ve2g/wBaZI=; b=LV4PgAohp46FE7Wrrs+1UCxSXgga8hQWYJKU9sL1mA79V3LQR5RBUoZENwrEVHCeEP ENt1oKe8V8E7B3LgB0xsgkxMpSIENHjWQ2N8j/s2GVR3Tdzejm5K20YXrdMN5eQs2kov jzCm72pVmDYYS+P+Hr03Bex5cmqVLYmYCNfqtcdojXsYlFclm7X7ABA/PGfO+8+qsdng t4mnYBdcK/CSJ5miA3VEHD22fMs/dL9tvVBnnDaPkyokofuAEWfckZFaZXWuoTqOhFgy MX8QNGcZc6rI/j9y6T38pJ9FRGPw1rFO2p6N/xzogJPUX8B4A/RHpmlQ9PsYaUOz37Sh jH/g== X-Gm-Message-State: APjAAAU0Byr59Yw+SiMA3fU7HMMEQLtwXG42b082yWaLV/oX0O9dJ1v7 KvtTunjGXKvWjWTHLpYqV/8= X-Google-Smtp-Source: APXvYqw8Zie7rIn0Y3WqzWmncZoOFR0zpQWQE7FayoKtS2mCVSP+2Acxof9D9UY5nblZMykGkaMiTg== X-Received: by 2002:aa7:8a86:: with SMTP id a6mr47898098pfc.76.1567622405238; Wed, 04 Sep 2019 11:40:05 -0700 (PDT) Received: from ap-To-be-filled-by-O-E-M.1.1.1.1 ([14.33.120.60]) by smtp.gmail.com with ESMTPSA id k14sm23713978pfi.98.2019.09.04.11.39.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Sep 2019 11:40:04 -0700 (PDT) From: Taehee Yoo To: davem@davemloft.net, netdev@vger.kernel.org, j.vosburgh@gmail.com, vfalico@gmail.com, andy@greyhouse.net, jiri@resnulli.us, sd@queasysnail.net, roopa@cumulusnetworks.com, saeedm@mellanox.com, manishc@marvell.com, rahulv@marvell.com, kys@microsoft.com, haiyangz@microsoft.com, sthemmin@microsoft.com, sashal@kernel.org, hare@suse.de, varun@chelsio.com, ubraun@linux.ibm.com, kgraul@linux.ibm.com Cc: ap420073@gmail.com Subject: [PATCH net 05/11] team: use dynamic lockdep key instead of static key Date: Thu, 5 Sep 2019 03:39:53 +0900 Message-Id: <20190904183953.14960-1-ap420073@gmail.com> X-Mailer: git-send-email 2.17.1 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org In the current code, all team devices have same static lockdep key and team devices could be nested so that it makes unnecessary lockdep warning. Test commands: ip link add team0 type team for i in {1..7} do let A=$i-1 ip link add team$i type team ip link set team$i master team$A done ip link del team0 Splat looks like: [ 137.406730] ============================================ [ 137.412685] WARNING: possible recursive locking detected [ 137.418642] 5.3.0-rc7+ #322 Not tainted [ 137.422941] -------------------------------------------- [ 137.428886] ip/1383 is trying to acquire lock: [ 137.433869] 0000000089571080 (&dev_addr_list_lock_key/1){+...}, at: dev_uc_sync_multiple+0xfa/0x1a0 [ 137.444034] [ 137.444034] but task is already holding lock: [ 137.450572] 00000000d9597252 (&dev_addr_list_lock_key/1){+...}, at: dev_uc_unsync+0x10c/0x1b0 [ 137.460142] [ 137.460142] other info that might help us debug this: [ 137.467458] Possible unsafe locking scenario: [ 137.467458] [ 137.474096] CPU0 [ 137.476828] ---- [ 137.479569] lock(&dev_addr_list_lock_key/1); [ 137.484554] lock(&dev_addr_list_lock_key/1); [ 137.489539] [ 137.489539] *** DEADLOCK *** [ 137.489539] [ 137.496178] May be due to missing lock nesting notation [ 137.496178] [ 137.503789] 5 locks held by ip/1383: [ 137.507797] #0: 00000000d497f415 (rtnl_mutex){+.+.}, at: rtnetlink_rcv_msg+0x466/0x8a0 [ 137.516786] #1: 000000008e4b4656 (&team->lock){+.+.}, at: team_uninit+0x3a/0x1a0 [team] [ 137.525882] #2: 000000005cf248d1 (&dev_addr_list_lock_key){+...}, at: dev_uc_unsync+0x98/0x1b0 [ 137.535649] #3: 00000000d9597252 (&dev_addr_list_lock_key/1){+...}, at: dev_uc_unsync+0x10c/0x1b0 [ 137.545709] #4: 00000000bec134c3 (rcu_read_lock){....}, at: team_set_rx_mode+0x5/0x1d0 [team] [ 137.555384] [ 137.555384] stack backtrace: [ 137.560277] CPU: 0 PID: 1383 Comm: ip Not tainted 5.3.0-rc7+ #322 [ 137.577826] Call Trace: [ 137.580586] dump_stack+0x7c/0xbb [ 137.584307] __lock_acquire+0x26a9/0x3de0 [ 137.588820] ? register_lock_class+0x14d0/0x14d0 [ 137.594008] ? register_lock_class+0x14d0/0x14d0 [ 137.599194] lock_acquire+0x164/0x3b0 [ 137.603310] ? dev_uc_sync_multiple+0xfa/0x1a0 [ 137.608307] _raw_spin_lock_nested+0x2e/0x60 [ 137.613105] ? dev_uc_sync_multiple+0xfa/0x1a0 [ 137.618095] dev_uc_sync_multiple+0xfa/0x1a0 [ 137.622900] team_set_rx_mode+0xa9/0x1d0 [team] [ 137.627993] dev_uc_unsync+0x151/0x1b0 [ 137.632205] team_port_del+0x304/0x790 [team] [ 137.637110] team_uninit+0xb0/0x1a0 [team] [ 137.641717] rollback_registered_many+0x728/0xda0 [ 137.647005] ? generic_xdp_install+0x310/0x310 [ 137.651994] ? __set_pages_p+0xf4/0x150 [ 137.656306] ? check_chain_key+0x236/0x5d0 [ 137.660914] ? __nla_validate_parse+0x98/0x1ad0 [ 137.666006] unregister_netdevice_many.part.120+0x13/0x1b0 [ 137.672167] rtnl_delete_link+0xbc/0x100 [ 137.676575] ? rtnl_af_register+0xc0/0xc0 [ 137.681084] rtnl_dellink+0x2e7/0x870 [ 137.685204] ? find_held_lock+0x39/0x1d0 [ ... ] Fixes: 3d249d4ca7d0 ("net: introduce ethernet teaming device") Signed-off-by: Taehee Yoo --- drivers/net/team/team.c | 61 ++++++++++++++++++++++++++++++++++++++--- include/linux/if_team.h | 5 ++++ 2 files changed, 62 insertions(+), 4 deletions(-) diff --git a/drivers/net/team/team.c b/drivers/net/team/team.c index e8089def5a46..bfcd6ed57493 100644 --- a/drivers/net/team/team.c +++ b/drivers/net/team/team.c @@ -1607,6 +1607,34 @@ static const struct team_option team_options[] = { }, }; +static void team_dev_set_lockdep_one(struct net_device *dev, + struct netdev_queue *txq, + void *_unused) +{ + struct team *team = netdev_priv(dev); + + lockdep_set_class(&txq->_xmit_lock, &team->xmit_lock_key); +} + +static struct lock_class_key qdisc_tx_busylock_key; +static struct lock_class_key qdisc_running_key; + +static void team_dev_set_lockdep_class(struct net_device *dev) +{ + struct team *team = netdev_priv(dev); + + dev->qdisc_tx_busylock = &qdisc_tx_busylock_key; + dev->qdisc_running_key = &qdisc_running_key; + + lockdep_register_key(&team->team_lock_key); + __mutex_init(&team->lock, "team->team_lock_key", &team->team_lock_key); + + lockdep_register_key(&team->addr_lock_key); + lockdep_set_class(&dev->addr_list_lock, &team->addr_lock_key); + + lockdep_register_key(&team->xmit_lock_key); + netdev_for_each_tx_queue(dev, team_dev_set_lockdep_one, NULL); +} static int team_init(struct net_device *dev) { @@ -1615,7 +1643,6 @@ static int team_init(struct net_device *dev) int err; team->dev = dev; - mutex_init(&team->lock); team_set_no_mode(team); team->pcpu_stats = netdev_alloc_pcpu_stats(struct team_pcpu_stats); @@ -1642,7 +1669,7 @@ static int team_init(struct net_device *dev) goto err_options_register; netif_carrier_off(dev); - netdev_lockdep_set_classes(dev); + team_dev_set_lockdep_class(dev); return 0; @@ -1673,6 +1700,11 @@ static void team_uninit(struct net_device *dev) team_queue_override_fini(team); mutex_unlock(&team->lock); netdev_change_features(dev); + + lockdep_unregister_key(&team->team_lock_key); + lockdep_unregister_key(&team->addr_lock_key); + lockdep_unregister_key(&team->xmit_lock_key); + } static void team_destructor(struct net_device *dev) @@ -1967,6 +1999,23 @@ static int team_add_slave(struct net_device *dev, struct net_device *port_dev, return err; } +static void team_update_lock_key(struct net_device *dev) +{ + struct team *team = netdev_priv(dev); + + lockdep_unregister_key(&team->team_lock_key); + lockdep_unregister_key(&team->addr_lock_key); + lockdep_unregister_key(&team->xmit_lock_key); + + lockdep_register_key(&team->team_lock_key); + lockdep_register_key(&team->addr_lock_key); + lockdep_register_key(&team->xmit_lock_key); + + lockdep_set_class(&team->lock, &team->team_lock_key); + lockdep_set_class(&dev->addr_list_lock, &team->addr_lock_key); + netdev_for_each_tx_queue(dev, team_dev_set_lockdep_one, NULL); +} + static int team_del_slave(struct net_device *dev, struct net_device *port_dev) { struct team *team = netdev_priv(dev); @@ -1976,8 +2025,12 @@ static int team_del_slave(struct net_device *dev, struct net_device *port_dev) err = team_port_del(team, port_dev); mutex_unlock(&team->lock); - if (!err) - netdev_change_features(dev); + if (err) + return err; + + if (netif_is_team_master(port_dev)) + team_update_lock_key(port_dev); + netdev_change_features(dev); return err; } diff --git a/include/linux/if_team.h b/include/linux/if_team.h index 06faa066496f..9c97bb19ed34 100644 --- a/include/linux/if_team.h +++ b/include/linux/if_team.h @@ -223,6 +223,11 @@ struct team { atomic_t count_pending; struct delayed_work dw; } mcast_rejoin; + + struct lock_class_key team_lock_key; + struct lock_class_key xmit_lock_key; + struct lock_class_key addr_lock_key; + long mode_priv[TEAM_MODE_PRIV_LONGS]; }; From patchwork Wed Sep 4 18:40:09 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taehee Yoo X-Patchwork-Id: 1157896 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="FZwX5Q8x"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 46Nt111hWKz9s4Y for ; Thu, 5 Sep 2019 04:40:21 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2387587AbfIDSkT (ORCPT ); Wed, 4 Sep 2019 14:40:19 -0400 Received: from mail-pl1-f194.google.com ([209.85.214.194]:45267 "EHLO mail-pl1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2387560AbfIDSkT (ORCPT ); Wed, 4 Sep 2019 14:40:19 -0400 Received: by mail-pl1-f194.google.com with SMTP id x3so4818425plr.12 for ; Wed, 04 Sep 2019 11:40:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=g8qP6SeQAdgRncIT+0H2ZklP/MC9CGr+vwnRZEauE8g=; b=FZwX5Q8xiz5LBi7EXBAGczxXiiStyzmyvpj4HzZvPb2ejTKArH1oPUUfNGdkx7kKH3 AvgMwokKaTEdTRpgpzFsAXVq7yiTfVwKhmKYFCKpotFW66MDXOawdwADKZdoKUDbWpUz CqkhDYtR1CGlhDZoVXMqPhngxJUNZ1+DYQunhmyoRMmEm578EEmrWaBfCFGRMWzz7wwN XsGgfX3LPjvSFqw6cQ6oWXzbtYqZY5B7fFsY+LJztTtBhq7pDY4J/zxUXZUn+dDFIXZU o8m/5LbjGGDmj1Qd4UbGfLbfKMAqTTrsGwWbjF1klJK107FV/4LeSGHev6RVSCSoGw11 BGng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=g8qP6SeQAdgRncIT+0H2ZklP/MC9CGr+vwnRZEauE8g=; b=peP5m8zdkX6udE1G47E6rH/eEyHcwjnYUXLP7DCga2Kds/irJ/EnTO5400/5MfoFe3 LHqzUMnIF9Pp3M+awpz39h4Ncu5tjns9LDhlkr6e1/+J7AcC1xMjbk1cywnwI0LLW7bU fJBBd8fL8wuvUJPLsxy+MJDoDwm9rdfAEQ0QPz6KUeTJZTANhIb7cvsxu78umWsN1b6k 9cOQcu2CbVA7yJ0tgYUy+VVt6U3twudH/1LQB5ff8HTolClKI6hYz+mlAgwscRLquQuR c5FUx9Zh9eG4hR2l8VQ+2YvEv3NMkK1jZkmhuTFbUrqt9TmXUASIlpSjaCdnem6Y7Aje nevQ== X-Gm-Message-State: APjAAAWAGixIAKVi6Bt5SdgGbUfX2vwj2YTmSuhgFuagRZyGy0vNB7Au upUCr1NbeUEh5TlxbLcPjt0= X-Google-Smtp-Source: APXvYqwQbepBj+L7we/Ut9bsYYLBiF7NVOqfRvWiXSQSWPQ6yMtvEMj/hd9NaSpGLlxOz53nebiKgw== X-Received: by 2002:a17:902:9889:: with SMTP id s9mr42885024plp.100.1567622418663; Wed, 04 Sep 2019 11:40:18 -0700 (PDT) Received: from ap-To-be-filled-by-O-E-M.1.1.1.1 ([14.33.120.60]) by smtp.gmail.com with ESMTPSA id z23sm1546798pfn.45.2019.09.04.11.40.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Sep 2019 11:40:17 -0700 (PDT) From: Taehee Yoo To: davem@davemloft.net, netdev@vger.kernel.org, j.vosburgh@gmail.com, vfalico@gmail.com, andy@greyhouse.net, jiri@resnulli.us, sd@queasysnail.net, roopa@cumulusnetworks.com, saeedm@mellanox.com, manishc@marvell.com, rahulv@marvell.com, kys@microsoft.com, haiyangz@microsoft.com, sthemmin@microsoft.com, sashal@kernel.org, hare@suse.de, varun@chelsio.com, ubraun@linux.ibm.com, kgraul@linux.ibm.com Cc: ap420073@gmail.com Subject: [PATCH net 06/11] macsec: use dynamic lockdep key instead of subclass Date: Thu, 5 Sep 2019 03:40:09 +0900 Message-Id: <20190904184009.15070-1-ap420073@gmail.com> X-Mailer: git-send-email 2.17.1 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org All macsec device has same lockdep key and subclass is initialized with nest_level. But actual nest_level value can be changed when a lower device is attached. And at this moment, the subclass should be updated but it seems to be unsafe. So this patch makes macsec use dynamic lockdep key instead of the subclass. Test commands: ip link add bond0 type bond ip link add dummy0 type dummy ip link add macsec0 link bond0 type macsec ip link add macsec1 link dummy0 type macsec ip link set bond0 mtu 1000 ip link set macsec1 master bond0 ip link set bond0 up ip link set macsec0 up ip link set dummy0 up ip link set macsec1 up Splat looks like: [ 146.540123] ============================================ [ 146.540123] WARNING: possible recursive locking detected [ 146.540123] 5.3.0-rc7+ #322 Not tainted [ 146.540123] -------------------------------------------- [ 146.540123] ip/1340 is trying to acquire lock: [ 146.540123] 00000000446fd8bd (&macsec_netdev_addr_lock_key/1){+...}, at: dev_uc_sync_multiple+0xfa/0x1a0 [ 146.540123] [ 146.540123] but task is already holding lock: [ 146.540123] 00000000a9ab6378 (&macsec_netdev_addr_lock_key/1){+...}, at: dev_set_rx_mode+0x19/0x30 [ 146.540123] [ 146.540123] other info that might help us debug this: [ 146.540123] Possible unsafe locking scenario: [ 146.540123] [ 146.540123] CPU0 [ 146.540123] ---- [ 146.540123] lock(&macsec_netdev_addr_lock_key/1); [ 146.540123] lock(&macsec_netdev_addr_lock_key/1); [ 146.623155] [ 146.623155] *** DEADLOCK *** [ 146.623155] [ 146.623155] May be due to missing lock nesting notation [ 146.623155] [ 146.623155] 4 locks held by ip/1340: [ 146.623155] #0: 0000000026436ef0 (rtnl_mutex){+.+.}, at: rtnetlink_rcv_msg+0x466/0x8a0 [ 146.623155] #1: 00000000a9ab6378 (&macsec_netdev_addr_lock_key/1){+...}, at: dev_set_rx_mode+0x19/0x30 [ 146.623155] #2: 00000000a8947dd0 (&dev_addr_list_lock_key/3){+...}, at: dev_mc_sync+0xfa/0x1a0 [ 146.623155] #3: 00000000b62011e9 (rcu_read_lock){....}, at: bond_set_rx_mode+0x5/0x3c0 [bonding] [ 146.674970] [ 146.674970] stack backtrace: [ 146.687145] CPU: 0 PID: 1340 Comm: ip Not tainted 5.3.0-rc7+ #322 [ 146.693024] Call Trace: [ 146.693024] dump_stack+0x7c/0xbb [ 146.693024] __lock_acquire+0x26a9/0x3de0 [ 146.693024] ? register_lock_class+0x14d0/0x14d0 [ 146.693024] ? register_lock_class+0x14d0/0x14d0 [ 146.693024] lock_acquire+0x164/0x3b0 [ 146.693024] ? dev_uc_sync_multiple+0xfa/0x1a0 [ 146.693024] _raw_spin_lock_nested+0x2e/0x60 [ 146.693024] ? dev_uc_sync_multiple+0xfa/0x1a0 [ 146.693024] dev_uc_sync_multiple+0xfa/0x1a0 [ 146.693024] bond_set_rx_mode+0x269/0x3c0 [bonding] [ 146.751163] ? bond_init+0x6f0/0x6f0 [bonding] [ 146.757006] ? do_raw_spin_trylock+0xa9/0x170 [ 146.757006] dev_mc_sync+0x15a/0x1a0 [ 146.757006] macsec_dev_set_rx_mode+0x3a/0x50 [macsec] [ 146.757006] dev_set_rx_mode+0x21/0x30 [ 146.757006] __dev_open+0x202/0x310 [ 146.757006] ? dev_set_rx_mode+0x30/0x30 [ 146.757006] ? mark_held_locks+0xa5/0xe0 [ 146.757006] ? __local_bh_enable_ip+0xe9/0x1b0 [ 146.757006] __dev_change_flags+0x3c3/0x500 [ 146.757006] ? dev_set_allmulti+0x10/0x10 [ 146.757006] ? sched_clock_local+0xd4/0x140 [ 146.757006] ? check_chain_key+0x236/0x5d0 [ 146.757006] dev_change_flags+0x7a/0x160 [ 146.757006] do_setlink+0xa26/0x2f20 [ 146.757006] ? sched_clock_local+0xd4/0x140 [ ... ] Fixes: e20038724552 ("macsec: fix lockdep splats when nesting devices") Signed-off-by: Taehee Yoo --- drivers/net/macsec.c | 37 ++++++++++++++++++++++++++++++++----- 1 file changed, 32 insertions(+), 5 deletions(-) diff --git a/drivers/net/macsec.c b/drivers/net/macsec.c index 8f46aa1ddec0..25a4fc88145d 100644 --- a/drivers/net/macsec.c +++ b/drivers/net/macsec.c @@ -267,6 +267,8 @@ struct macsec_dev { struct pcpu_secy_stats __percpu *stats; struct list_head secys; struct gro_cells gro_cells; + struct lock_class_key xmit_lock_key; + struct lock_class_key addr_lock_key; unsigned int nest_level; }; @@ -2749,7 +2751,32 @@ static netdev_tx_t macsec_start_xmit(struct sk_buff *skb, #define MACSEC_FEATURES \ (NETIF_F_SG | NETIF_F_HIGHDMA | NETIF_F_FRAGLIST) -static struct lock_class_key macsec_netdev_addr_lock_key; + +static void macsec_dev_set_lockdep_one(struct net_device *dev, + struct netdev_queue *txq, + void *_unused) +{ + struct macsec_dev *macsec = macsec_priv(dev); + + lockdep_set_class(&txq->_xmit_lock, &macsec->xmit_lock_key); +} + +static struct lock_class_key qdisc_tx_busylock_key; +static struct lock_class_key qdisc_running_key; + +static void macsec_dev_set_lockdep_class(struct net_device *dev) +{ + struct macsec_dev *macsec = macsec_priv(dev); + + dev->qdisc_tx_busylock = &qdisc_tx_busylock_key; + dev->qdisc_running_key = &qdisc_running_key; + + lockdep_register_key(&macsec->addr_lock_key); + lockdep_set_class(&dev->addr_list_lock, &macsec->addr_lock_key); + + lockdep_register_key(&macsec->xmit_lock_key); + netdev_for_each_tx_queue(dev, macsec_dev_set_lockdep_one, NULL); +} static int macsec_dev_init(struct net_device *dev) { @@ -2780,6 +2807,7 @@ static int macsec_dev_init(struct net_device *dev) if (is_zero_ether_addr(dev->broadcast)) memcpy(dev->broadcast, real_dev->broadcast, dev->addr_len); + macsec_dev_set_lockdep_class(dev); return 0; } @@ -2789,6 +2817,9 @@ static void macsec_dev_uninit(struct net_device *dev) gro_cells_destroy(&macsec->gro_cells); free_percpu(dev->tstats); + + lockdep_unregister_key(&macsec->addr_lock_key); + lockdep_unregister_key(&macsec->xmit_lock_key); } static netdev_features_t macsec_fix_features(struct net_device *dev, @@ -3263,10 +3294,6 @@ static int macsec_newlink(struct net *net, struct net_device *dev, dev_hold(real_dev); macsec->nest_level = dev_get_nest_level(real_dev) + 1; - netdev_lockdep_set_classes(dev); - lockdep_set_class_and_subclass(&dev->addr_list_lock, - &macsec_netdev_addr_lock_key, - macsec_get_nest_level(dev)); err = netdev_upper_dev_link(real_dev, dev, extack); if (err < 0) From patchwork Wed Sep 4 18:40:22 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taehee Yoo X-Patchwork-Id: 1157897 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="OEbTBa3J"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 46Nt1G61DPz9s4Y for ; Thu, 5 Sep 2019 04:40:34 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2387593AbfIDSkd (ORCPT ); Wed, 4 Sep 2019 14:40:33 -0400 Received: from mail-pf1-f194.google.com ([209.85.210.194]:39244 "EHLO mail-pf1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2387560AbfIDSkd (ORCPT ); Wed, 4 Sep 2019 14:40:33 -0400 Received: by mail-pf1-f194.google.com with SMTP id s12so6798684pfe.6 for ; Wed, 04 Sep 2019 11:40:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=Q9ldTQaPqq3RxZUb9jrFFYm4H4tiT7qdPr6VQocTNlQ=; b=OEbTBa3J7nye94cXSi06bDvw32BXcQ93ES7jzB5q1WFQsH2K4UcWrdI0JMgv1kDYu+ PuoLG5YoAPwjIoouIb8jrsPUhV85kLhM6EnD9q21BWbFAk5jNV6xoJU+RVvIHNDxgTCM KGssrKXVaX9dVOIWMuDG3wEnDY7ZLiF3S9Pzzdvy/ViL6lh0ZEEQ7zllQXOX2+sGILxV Jdp4WtrVZdtNfH8WaB9iObPh4+ctRtaqdTLJl0bEuxNkXX48Nz3L+ceTRJsGQQONoZN9 RPr3VvdNsRgVWOuRyeLriYe6o+dKNGO3uCmXQCFW0iLu/Pn7ATZ8ojspCoiiFlR6Whys A/zQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=Q9ldTQaPqq3RxZUb9jrFFYm4H4tiT7qdPr6VQocTNlQ=; b=ZVTegdV7M/Pott3ii7+ChaZZSuUmY2jqfN+IkGMDeBYuEQK1WjjzOlyVlT/2BtWPnI rYygM1gBXabnyVX9iqAHekZgUuTGCZe6NzxKOo7SkPKSO3tgt+yCU88pZj58i4XfZVeO veu+SICDdvAOQE9f4iHtFlE0r+Zoihd03/nHmmv8WQyQrnMK9H3aVn3Nav+nJJ5gYF1B 3IBimj7HMpij2uIk3MvsYr68pRmGF4ua6CTNqIiyTNiORWP55R7GYNOPNs+7hGz//lzu SPpcrHZm4ZQndTRakACRYbY9qCGpf/QuN9mYoGGbz6c2nLd95fbbQoNtBQ5GVqoAR2vd n2qg== X-Gm-Message-State: APjAAAX4MGVq/I7hECL8JTH5OCJqeSwF6YP4FAbvOLLt4g6DDLExRbG8 UscD/HULfZYUJRwjL50bBsg= X-Google-Smtp-Source: APXvYqxq/r6Ha1q1M6upMj7U9eu2IHmIqMyMKP61MkDub3JUrJ6v/BkDHDvq6P8hmoBFKSxG/Fz1Fw== X-Received: by 2002:a62:60c7:: with SMTP id u190mr48373443pfb.54.1567622432606; Wed, 04 Sep 2019 11:40:32 -0700 (PDT) Received: from ap-To-be-filled-by-O-E-M.1.1.1.1 ([14.33.120.60]) by smtp.gmail.com with ESMTPSA id 127sm35126782pfy.56.2019.09.04.11.40.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Sep 2019 11:40:31 -0700 (PDT) From: Taehee Yoo To: davem@davemloft.net, netdev@vger.kernel.org, j.vosburgh@gmail.com, vfalico@gmail.com, andy@greyhouse.net, jiri@resnulli.us, sd@queasysnail.net, roopa@cumulusnetworks.com, saeedm@mellanox.com, manishc@marvell.com, rahulv@marvell.com, kys@microsoft.com, haiyangz@microsoft.com, sthemmin@microsoft.com, sashal@kernel.org, hare@suse.de, varun@chelsio.com, ubraun@linux.ibm.com, kgraul@linux.ibm.com Cc: ap420073@gmail.com Subject: [PATCH net 07/11] macvlan: use dynamic lockdep key instead of subclass Date: Thu, 5 Sep 2019 03:40:22 +0900 Message-Id: <20190904184022.15150-1-ap420073@gmail.com> X-Mailer: git-send-email 2.17.1 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org All macvlan device has same lockdep key and subclass is initialized with nest_level. But actual nest_level value can be changed when a lower device is attached. And at this moment, the subclass should be updated but it seems to be unsafe. So this patch makes macvlan use dynamic lockdep key instead of the subclass. Test commands: ip link add bond0 type bond ip link add dummy0 type dummy ip link add macvlan0 link bond0 type macvlan mode bridge ip link add macvlan1 link dummy0 type macvlan mode bridge ip link set bond0 mtu 1000 ip link set macvlan1 master bond0 ip link set bond0 up ip link set macvlan0 up ip link set dummy0 up ip link set macvlan1 up Splat looks like: [ 165.677603] ============================================ [ 165.679642] WARNING: possible recursive locking detected [ 165.679642] 5.3.0-rc7+ #322 Not tainted [ 165.679642] -------------------------------------------- [ 165.679642] ip/1812 is trying to acquire lock: [ 165.679642] 00000000ae6a8a03 (&macvlan_netdev_addr_lock_key/1){+...}, at: dev_uc_sync_multiple+0xfa/0x1a0 [ 165.679642] [ 165.679642] but task is already holding lock: [ 165.679642] 00000000cec5da0b (&macvlan_netdev_addr_lock_key/1){+...}, at: dev_set_rx_mode+0x19/0x30 [ 165.679642] [ 165.679642] other info that might help us debug this: [ 165.679642] Possible unsafe locking scenario: [ 165.679642] [ 165.679642] CPU0 [ 165.679642] ---- [ 165.679642] lock(&macvlan_netdev_addr_lock_key/1); [ 165.679642] lock(&macvlan_netdev_addr_lock_key/1); [ 165.679642] [ 165.679642] *** DEADLOCK *** [ 165.679642] [ 165.679642] May be due to missing lock nesting notation [ 165.679642] [ 165.679642] 4 locks held by ip/1812: [ 165.679642] #0: 0000000088d10bd8 (rtnl_mutex){+.+.}, at: rtnetlink_rcv_msg+0x466/0x8a0 [ 165.679642] #1: 00000000cec5da0b (&macvlan_netdev_addr_lock_key/1){+...}, at: dev_set_rx_mode+0x19/0x30 [ 165.679642] #2: 000000000ca6fdb5 (&dev_addr_list_lock_key/3){+...}, at: dev_uc_sync+0xfa/0x1a0 [ 165.679642] #3: 00000000dc1495a2 (rcu_read_lock){....}, at: bond_set_rx_mode+0x5/0x3c0 [bonding] [ 165.679642] [ 165.679642] stack backtrace: [ 165.679642] CPU: 1 PID: 1812 Comm: ip Not tainted 5.3.0-rc7+ #322 [ 165.679642] Call Trace: [ 165.679642] dump_stack+0x7c/0xbb [ 165.679642] __lock_acquire+0x26a9/0x3de0 [ 165.679642] ? register_lock_class+0x14d0/0x14d0 [ 165.679642] ? mark_held_locks+0xa5/0xe0 [ 165.679642] ? trace_hardirqs_on_thunk+0x1a/0x20 [ 165.679642] ? register_lock_class+0x14d0/0x14d0 [ 165.679642] lock_acquire+0x164/0x3b0 [ 165.679642] ? dev_uc_sync_multiple+0xfa/0x1a0 [ 165.679642] _raw_spin_lock_nested+0x2e/0x60 [ 165.679642] ? dev_uc_sync_multiple+0xfa/0x1a0 [ 165.679642] dev_uc_sync_multiple+0xfa/0x1a0 [ 165.679642] bond_set_rx_mode+0x269/0x3c0 [bonding] [ 165.679642] ? bond_init+0x6f0/0x6f0 [bonding] [ 165.679642] dev_uc_sync+0x15a/0x1a0 [ 165.679642] macvlan_set_mac_lists+0x55/0x110 [macvlan] [ 165.679642] dev_set_rx_mode+0x21/0x30 [ 165.679642] __dev_open+0x202/0x310 [ 165.679642] ? dev_set_rx_mode+0x30/0x30 [ 165.679642] ? mark_held_locks+0xa5/0xe0 [ 165.679642] ? __local_bh_enable_ip+0xe9/0x1b0 [ 165.679642] __dev_change_flags+0x3c3/0x500 [ 165.679642] ? dev_set_allmulti+0x10/0x10 [ 165.679642] dev_change_flags+0x7a/0x160 [ ...] Fixes: c674ac30c549 ("macvlan: Fix lockdep warnings with stacked macvlan devices") Signed-off-by: Taehee Yoo --- drivers/net/macvlan.c | 35 +++++++++++++++++++++++++++-------- include/linux/if_macvlan.h | 2 ++ 2 files changed, 29 insertions(+), 8 deletions(-) diff --git a/drivers/net/macvlan.c b/drivers/net/macvlan.c index 940192c057b6..dae368a2e8d1 100644 --- a/drivers/net/macvlan.c +++ b/drivers/net/macvlan.c @@ -852,8 +852,6 @@ static int macvlan_do_ioctl(struct net_device *dev, struct ifreq *ifr, int cmd) * "super class" of normal network devices; split their locks off into a * separate class since they always nest. */ -static struct lock_class_key macvlan_netdev_addr_lock_key; - #define ALWAYS_ON_OFFLOADS \ (NETIF_F_SG | NETIF_F_HW_CSUM | NETIF_F_GSO_SOFTWARE | \ NETIF_F_GSO_ROBUST | NETIF_F_GSO_ENCAP_ALL) @@ -874,12 +872,30 @@ static int macvlan_get_nest_level(struct net_device *dev) return ((struct macvlan_dev *)netdev_priv(dev))->nest_level; } -static void macvlan_set_lockdep_class(struct net_device *dev) +static void macvlan_dev_set_lockdep_one(struct net_device *dev, + struct netdev_queue *txq, + void *_unused) +{ + struct macvlan_dev *macvlan = netdev_priv(dev); + + lockdep_set_class(&txq->_xmit_lock, &macvlan->xmit_lock_key); +} + +static struct lock_class_key qdisc_tx_busylock_key; +static struct lock_class_key qdisc_running_key; + +static void macvlan_dev_set_lockdep_class(struct net_device *dev) { - netdev_lockdep_set_classes(dev); - lockdep_set_class_and_subclass(&dev->addr_list_lock, - &macvlan_netdev_addr_lock_key, - macvlan_get_nest_level(dev)); + struct macvlan_dev *macvlan = netdev_priv(dev); + + dev->qdisc_tx_busylock = &qdisc_tx_busylock_key; + dev->qdisc_running_key = &qdisc_running_key; + + lockdep_register_key(&macvlan->addr_lock_key); + lockdep_set_class(&dev->addr_list_lock, &macvlan->addr_lock_key); + + lockdep_register_key(&macvlan->xmit_lock_key); + netdev_for_each_tx_queue(dev, macvlan_dev_set_lockdep_one, NULL); } static int macvlan_init(struct net_device *dev) @@ -900,7 +916,7 @@ static int macvlan_init(struct net_device *dev) dev->gso_max_segs = lowerdev->gso_max_segs; dev->hard_header_len = lowerdev->hard_header_len; - macvlan_set_lockdep_class(dev); + macvlan_dev_set_lockdep_class(dev); vlan->pcpu_stats = netdev_alloc_pcpu_stats(struct vlan_pcpu_stats); if (!vlan->pcpu_stats) @@ -922,6 +938,9 @@ static void macvlan_uninit(struct net_device *dev) port->count -= 1; if (!port->count) macvlan_port_destroy(port->dev); + + lockdep_unregister_key(&vlan->addr_lock_key); + lockdep_unregister_key(&vlan->xmit_lock_key); } static void macvlan_dev_get_stats64(struct net_device *dev, diff --git a/include/linux/if_macvlan.h b/include/linux/if_macvlan.h index 2e55e4cdbd8a..ea5b41823287 100644 --- a/include/linux/if_macvlan.h +++ b/include/linux/if_macvlan.h @@ -31,6 +31,8 @@ struct macvlan_dev { u16 flags; int nest_level; unsigned int macaddr_count; + struct lock_class_key xmit_lock_key; + struct lock_class_key addr_lock_key; #ifdef CONFIG_NET_POLL_CONTROLLER struct netpoll *netpoll; #endif From patchwork Wed Sep 4 18:40:36 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taehee Yoo X-Patchwork-Id: 1157898 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="ffAMriBn"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 46Nt1Z3cXWz9s7T for ; Thu, 5 Sep 2019 04:40:50 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2387731AbfIDSks (ORCPT ); Wed, 4 Sep 2019 14:40:48 -0400 Received: from mail-pf1-f193.google.com ([209.85.210.193]:37737 "EHLO mail-pf1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732465AbfIDSks (ORCPT ); Wed, 4 Sep 2019 14:40:48 -0400 Received: by mail-pf1-f193.google.com with SMTP id y9so13794304pfl.4 for ; Wed, 04 Sep 2019 11:40:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=Vrtqv1cFw27uAgADwsiiIYgq/XSGYJzvztXErTTVhxI=; b=ffAMriBnv/YvkWiIL2QcmeNNjHiBPSn3Tdjbpfsd+xrm9eHnAB+1ZcaBvRVA8Yton9 VVau6qwfo3BYRzfWtlTM/Xkl6+IwLMC0gsa13AaGKs8RSxivRxDSrNYBZR1M1mNb8AM5 nBLMUU1pJrySzI+5+EIWI2OOs2vCsrcbnwb7CfGzLGSaCHsSbud/7aaKu8sQiQEwcz+2 8mKok5j8a0keHuATB8k+XgiKpoZO6x+zgtJrX8/STIhYUEghcfO0AyOLwxnC54dy+Cpi yHsQoMXsPnbAZ5yRwhQBcHeD/7IoamjF8vr+swkq6ySC1ivcbVCXJhhG3G2xGhQf5mcR 8nrg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=Vrtqv1cFw27uAgADwsiiIYgq/XSGYJzvztXErTTVhxI=; b=jA+3z6w3tWUsqTKzd9idohZ/t+z9T7o/HPNDw5slz00L0odn17MpbzE0tJugh1p2eq 3HQbPTS29FAXWvChAxvH7RrZK6zymn/x9wYza6k0nQQojXmLxn0ZJwzTyZkjxDaEw8or Ik4/1U4MsZEUwiWKVNudR3XhEwrEkvNODbyeoF6xu8D9hVYFK1Shj9R0NFkMc4AHITO5 VDOfQug7vfZsjEP14redugpLKN1uzsYpINGrR0y6AARF66OtcoterqcWl9ahwbNVR90Q kawqmfZ0xlyuyaHOg5TFPLkkxBn+Mx61Z7yn+jWsxtFXYKSDz7v91Bs38tvbEv0ssG0v O0+A== X-Gm-Message-State: APjAAAWa7x1EPYZcmVETWBchP0/XmiOfm2xirKRT3Kjn41Fn3z3xeh85 82DnmKmYq2kHEVSRScRR5FYSNwHWW9E= X-Google-Smtp-Source: APXvYqzB9RXhIKrX4n1iQ5ElIkela9+HlLOOZfgr/m9o3U4bwQvc5BgO5CvlEIW83/gneGEuuzYxXQ== X-Received: by 2002:a17:90a:8c01:: with SMTP id a1mr6173268pjo.82.1567622447568; Wed, 04 Sep 2019 11:40:47 -0700 (PDT) Received: from ap-To-be-filled-by-O-E-M.1.1.1.1 ([14.33.120.60]) by smtp.gmail.com with ESMTPSA id r1sm19115921pgv.70.2019.09.04.11.40.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Sep 2019 11:40:46 -0700 (PDT) From: Taehee Yoo To: davem@davemloft.net, netdev@vger.kernel.org, j.vosburgh@gmail.com, vfalico@gmail.com, andy@greyhouse.net, jiri@resnulli.us, sd@queasysnail.net, roopa@cumulusnetworks.com, saeedm@mellanox.com, manishc@marvell.com, rahulv@marvell.com, kys@microsoft.com, haiyangz@microsoft.com, sthemmin@microsoft.com, sashal@kernel.org, hare@suse.de, varun@chelsio.com, ubraun@linux.ibm.com, kgraul@linux.ibm.com Cc: ap420073@gmail.com Subject: [PATCH net 08/11] macsec: fix refcnt leak in module exit routine Date: Thu, 5 Sep 2019 03:40:36 +0900 Message-Id: <20190904184036.15229-1-ap420073@gmail.com> X-Mailer: git-send-email 2.17.1 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org When a macsec interface is created, it increases a refcnt to a lower device(real device). when macsec interface is deleted, the refcnt is decreased in macsec_free_netdev(), which is ->priv_destructor() of macsec interface. The problem scenario is this. When nested macsec interfaces are exiting, the exit routine of the macsec module makes refcnt leaks. Test commands: ip link add dummy0 type dummy ip link add macsec0 link dummy0 type macsec ip link add macsec1 link macsec0 type macsec modprobe -rv macsec [ 208.629433] unregister_netdevice: waiting for macsec0 to become free. Usage count = 1 Steps of exit routine of macsec module are below. 1. Calls ->dellink() in __rtnl_link_unregister(). 2. Checks refcnt and wait refcnt to be 0 if refcnt is not 0 in netdev_run_todo(). 3. Calls ->priv_destruvtor() in netdev_run_todo(). Step2 checks refcnt, but step3 decreases refcnt. So, step2 waits forever. This patch makes the macsec module do not hold a refcnt of the lower device because it already holds a refcnt of the lower device with netdev_upper_dev_link(). Fixes: c09440f7dcb3 ("macsec: introduce IEEE 802.1AE driver") Signed-off-by: Taehee Yoo --- drivers/net/macsec.c | 4 ---- 1 file changed, 4 deletions(-) diff --git a/drivers/net/macsec.c b/drivers/net/macsec.c index 25a4fc88145d..41ec1ed0d545 100644 --- a/drivers/net/macsec.c +++ b/drivers/net/macsec.c @@ -3031,12 +3031,10 @@ static const struct nla_policy macsec_rtnl_policy[IFLA_MACSEC_MAX + 1] = { static void macsec_free_netdev(struct net_device *dev) { struct macsec_dev *macsec = macsec_priv(dev); - struct net_device *real_dev = macsec->real_dev; free_percpu(macsec->stats); free_percpu(macsec->secy.tx_sc.stats); - dev_put(real_dev); } static void macsec_setup(struct net_device *dev) @@ -3291,8 +3289,6 @@ static int macsec_newlink(struct net *net, struct net_device *dev, if (err < 0) return err; - dev_hold(real_dev); - macsec->nest_level = dev_get_nest_level(real_dev) + 1; err = netdev_upper_dev_link(real_dev, dev, extack); From patchwork Wed Sep 4 18:40:52 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taehee Yoo X-Patchwork-Id: 1157900 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="ns4/yhTt"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 46Nt1r1QF3z9s4Y for ; Thu, 5 Sep 2019 04:41:04 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2387759AbfIDSlD (ORCPT ); Wed, 4 Sep 2019 14:41:03 -0400 Received: from mail-pl1-f194.google.com ([209.85.214.194]:44466 "EHLO mail-pl1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732543AbfIDSlD (ORCPT ); Wed, 4 Sep 2019 14:41:03 -0400 Received: by mail-pl1-f194.google.com with SMTP id k1so2607811pls.11 for ; Wed, 04 Sep 2019 11:41:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=xTQKEQ6Ui/AXXfKIWIQiYloo4DpFVjrVF9m6m1aadIw=; b=ns4/yhTtwFuHl0OzCY6V/3cKyZkuDtazC+3sNyDAV3nm0a4dS2ua91e5mva65D6BTE kbf5fiPQ62E1N9ZA+YsCTzbsp/WgItqTtDMuuoC/FxOaxzVRNbMpsuQdc6ZBKYiGEH2U 96+ERAZzU2XWwsVIRK0U/JjQykl5WVQe7XmwizGvdu479/zfy+FfwFDQvDa5T96fW/Gl uksMuN9hRbMJQEbQ17/+RPGMPVpCfKCkC5FNV7eQ2ZWV7NneIi3ZD2x8UGWTmS3DAuyC osXIG21lYBcprV4OGGvkhFGj5t645w18i7Gz102QQhpk3nBEvPhPgWdHHKQ0hnaEZS5h qzTw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=xTQKEQ6Ui/AXXfKIWIQiYloo4DpFVjrVF9m6m1aadIw=; b=l5Iraz30wtqC2F+jKfLRVZAvNwM0KchZTPBUFnzIHeKFAqolhWFALd6x2FdfxZ187Q MF2V+M6QDyzGGAutmdAw9Vw4TaHFQxqIc46IguoAEZBiB25v7mitAbUC6svyZ1CXncc2 72EselWJOyOcsT/hM1XKLpJDYiepYDUUS7uSI/A0xKY8fb7lQxUoVINq1In1llDSG93Z dYEsok0fSrX7JyroMjZQpRCtUrqx2sZw527NFosjVj7o5YjttGZ1LzY6mUivwk9piFcT jOUJLpIyazdoHwtdHKS21Y9p+WPYhm8sVDLBQkCj5V0SIoEMYQ8+fvoz90uJkjrbvJC8 vEtQ== X-Gm-Message-State: APjAAAUPkyGwlSSv2tedm5JfgRemi7wvxH/4nXb/CkWvWsgdJMcDaiFH 6mYHRL6XAIjrl7X2L1cfyHVNGToV9c8= X-Google-Smtp-Source: APXvYqzewZeNFogWpu6R97seVy3a+RZDcCaf6Vctvxnb7MruBiN9hR+ngE7J8u2j9QW7F82bgZh+4A== X-Received: by 2002:a17:902:7791:: with SMTP id o17mr26009686pll.10.1567622462106; Wed, 04 Sep 2019 11:41:02 -0700 (PDT) Received: from ap-To-be-filled-by-O-E-M.1.1.1.1 ([14.33.120.60]) by smtp.gmail.com with ESMTPSA id a17sm24511431pfc.26.2019.09.04.11.40.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Sep 2019 11:41:00 -0700 (PDT) From: Taehee Yoo To: davem@davemloft.net, netdev@vger.kernel.org, j.vosburgh@gmail.com, vfalico@gmail.com, andy@greyhouse.net, jiri@resnulli.us, sd@queasysnail.net, roopa@cumulusnetworks.com, saeedm@mellanox.com, manishc@marvell.com, rahulv@marvell.com, kys@microsoft.com, haiyangz@microsoft.com, sthemmin@microsoft.com, sashal@kernel.org, hare@suse.de, varun@chelsio.com, ubraun@linux.ibm.com, kgraul@linux.ibm.com Cc: ap420073@gmail.com Subject: [PATCH net 09/11] net: core: add ignore flag to netdev_adjacent structure Date: Thu, 5 Sep 2019 03:40:52 +0900 Message-Id: <20190904184052.15324-1-ap420073@gmail.com> X-Mailer: git-send-email 2.17.1 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org In order to link an adjacent node, netdev_upper_dev_link() is used and in order to unlink an adjacent node, netdev_upper_dev_unlink() is used. unlink operation does not fail, but link operation can fail. In order to exchange adjacent nodes, we should unlink an old adjacent node first. then, link a new adjacent node. If link operation is failed, we should link an old adjacent node again. But this link operation can fail too. It eventually breaks the adjacent link relationship. This patch adds an ignore flag into the netdev_adjacent structure. If this flag is set, netdev_upper_dev_link() ignores an old adjacent node for a moment. So we can skip unlink operation before link operation. Signed-off-by: Taehee Yoo --- include/linux/netdevice.h | 4 + net/core/dev.c | 160 +++++++++++++++++++++++++++++++++----- 2 files changed, 144 insertions(+), 20 deletions(-) diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index a2c47f43e54b..d9ca4a79f715 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -4322,6 +4322,10 @@ int netdev_master_upper_dev_link(struct net_device *dev, struct netlink_ext_ack *extack); void netdev_upper_dev_unlink(struct net_device *dev, struct net_device *upper_dev); +void netdev_adjacent_dev_disable(struct net_device *upper_dev, + struct net_device *lower_dev); +void netdev_adjacent_dev_enable(struct net_device *upper_dev, + struct net_device *lower_dev); void netdev_adjacent_rename_links(struct net_device *dev, char *oldname); void *netdev_lower_dev_get_private(struct net_device *dev, struct net_device *lower_dev); diff --git a/net/core/dev.c b/net/core/dev.c index 6a4b4ce62204..ac055b531c96 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -6448,6 +6448,9 @@ struct netdev_adjacent { /* upper master flag, there can only be one master device per list */ bool master; + /* lookup ignore flag */ + bool ignore; + /* counter for the number of times this device was added to us */ u16 ref_nr; @@ -6553,6 +6556,22 @@ struct net_device *netdev_master_upper_dev_get(struct net_device *dev) } EXPORT_SYMBOL(netdev_master_upper_dev_get); +struct net_device *netdev_master_upper_dev_get_ignore(struct net_device *dev) +{ + struct netdev_adjacent *upper; + + ASSERT_RTNL(); + + if (list_empty(&dev->adj_list.upper)) + return NULL; + + upper = list_first_entry(&dev->adj_list.upper, + struct netdev_adjacent, list); + if (likely(upper->master) && !upper->ignore) + return upper->dev; + return NULL; +} + /** * netdev_has_any_lower_dev - Check if device is linked to some device * @dev: device @@ -6603,8 +6622,9 @@ struct net_device *netdev_upper_get_next_dev_rcu(struct net_device *dev, } EXPORT_SYMBOL(netdev_upper_get_next_dev_rcu); -static struct net_device *netdev_next_upper_dev(struct net_device *dev, - struct list_head **iter) +static struct net_device *netdev_next_upper_dev_ignore(struct net_device *dev, + struct list_head **iter, + bool *ignore) { struct netdev_adjacent *upper; @@ -6614,6 +6634,7 @@ static struct net_device *netdev_next_upper_dev(struct net_device *dev, return NULL; *iter = &upper->list; + *ignore = upper->ignore; return upper->dev; } @@ -6635,26 +6656,29 @@ static struct net_device *netdev_next_upper_dev_rcu(struct net_device *dev, return upper->dev; } -int netdev_walk_all_upper_dev(struct net_device *dev, - int (*fn)(struct net_device *dev, - void *data), - void *data) +int netdev_walk_all_upper_dev_ignore(struct net_device *dev, + int (*fn)(struct net_device *dev, + void *data), + void *data) { struct net_device *udev; struct list_head *iter; int ret; + bool ignore; for (iter = &dev->adj_list.upper, - udev = netdev_next_upper_dev(dev, &iter); + udev = netdev_next_upper_dev_ignore(dev, &iter, &ignore); udev; - udev = netdev_next_upper_dev(dev, &iter)) { + udev = netdev_next_upper_dev_ignore(dev, &iter, &ignore)) { + if (ignore) + continue; /* first is the upper device itself */ ret = fn(udev, data); if (ret) return ret; /* then look at all of its upper devices */ - ret = netdev_walk_all_upper_dev(udev, fn, data); + ret = netdev_walk_all_upper_dev_ignore(udev, fn, data); if (ret) return ret; } @@ -6690,6 +6714,15 @@ int netdev_walk_all_upper_dev_rcu(struct net_device *dev, } EXPORT_SYMBOL_GPL(netdev_walk_all_upper_dev_rcu); +bool netdev_has_upper_dev_ignore(struct net_device *dev, + struct net_device *upper_dev) +{ + ASSERT_RTNL(); + + return netdev_walk_all_upper_dev_ignore(dev, __netdev_has_upper_dev, + upper_dev); +} + /** * netdev_lower_get_next_private - Get the next ->private from the * lower neighbour list @@ -6786,6 +6819,23 @@ static struct net_device *netdev_next_lower_dev(struct net_device *dev, return lower->dev; } +static struct net_device *netdev_next_lower_dev_ignore(struct net_device *dev, + struct list_head **iter, + bool *ignore) +{ + struct netdev_adjacent *lower; + + lower = list_entry((*iter)->next, struct netdev_adjacent, list); + + if (&lower->list == &dev->adj_list.lower) + return NULL; + + *iter = &lower->list; + *ignore = lower->ignore; + + return lower->dev; +} + int netdev_walk_all_lower_dev(struct net_device *dev, int (*fn)(struct net_device *dev, void *data), @@ -6814,6 +6864,36 @@ int netdev_walk_all_lower_dev(struct net_device *dev, } EXPORT_SYMBOL_GPL(netdev_walk_all_lower_dev); +int netdev_walk_all_lower_dev_ignore(struct net_device *dev, + int (*fn)(struct net_device *dev, + void *data), + void *data) +{ + struct net_device *ldev; + struct list_head *iter; + int ret; + bool ignore; + + for (iter = &dev->adj_list.lower, + ldev = netdev_next_lower_dev_ignore(dev, &iter, &ignore); + ldev; + ldev = netdev_next_lower_dev_ignore(dev, &iter, &ignore)) { + if (ignore) + continue; + /* first is the lower device itself */ + ret = fn(ldev, data); + if (ret) + return ret; + + /* then look at all of its lower devices */ + ret = netdev_walk_all_lower_dev_ignore(ldev, fn, data); + if (ret) + return ret; + } + + return 0; +} + static struct net_device *netdev_next_lower_dev_rcu(struct net_device *dev, struct list_head **iter) { @@ -6833,11 +6913,14 @@ static u8 __netdev_upper_depth(struct net_device *dev) struct net_device *udev; struct list_head *iter; u8 max_depth = 0; + bool ignore; for (iter = &dev->adj_list.upper, - udev = netdev_next_upper_dev(dev, &iter); + udev = netdev_next_upper_dev_ignore(dev, &iter, &ignore); udev; - udev = netdev_next_upper_dev(dev, &iter)) { + udev = netdev_next_upper_dev_ignore(dev, &iter, &ignore)) { + if (ignore) + continue; if (max_depth < udev->upper_level) max_depth = udev->upper_level; } @@ -6850,11 +6933,14 @@ static u8 __netdev_lower_depth(struct net_device *dev) struct net_device *ldev; struct list_head *iter; u8 max_depth = 0; + bool ignore; for (iter = &dev->adj_list.lower, - ldev = netdev_next_lower_dev(dev, &iter); + ldev = netdev_next_lower_dev_ignore(dev, &iter, &ignore); ldev; - ldev = netdev_next_lower_dev(dev, &iter)) { + ldev = netdev_next_lower_dev_ignore(dev, &iter, &ignore)) { + if (ignore) + continue; if (max_depth < ldev->lower_level) max_depth = ldev->lower_level; } @@ -6999,6 +7085,7 @@ static int __netdev_adjacent_dev_insert(struct net_device *dev, adj->master = master; adj->ref_nr = 1; adj->private = private; + adj->ignore = false; dev_hold(adj_dev); pr_debug("Insert adjacency: dev %s adj_dev %s adj->ref_nr %d; dev_hold on %s\n", @@ -7149,17 +7236,17 @@ static int __netdev_upper_dev_link(struct net_device *dev, return -EBUSY; /* To prevent loops, check if dev is not upper device to upper_dev. */ - if (netdev_has_upper_dev(upper_dev, dev)) + if (netdev_has_upper_dev_ignore(upper_dev, dev)) return -EBUSY; if ((dev->lower_level + upper_dev->upper_level) > MAX_NEST_DEV) return -EMLINK; if (!master) { - if (netdev_has_upper_dev(dev, upper_dev)) + if (netdev_has_upper_dev_ignore(dev, upper_dev)) return -EEXIST; } else { - master_dev = netdev_master_upper_dev_get(dev); + master_dev = netdev_master_upper_dev_get_ignore(dev); if (master_dev) return master_dev == upper_dev ? -EEXIST : -EBUSY; } @@ -7182,10 +7269,12 @@ static int __netdev_upper_dev_link(struct net_device *dev, goto rollback; __netdev_update_upper_level(dev, NULL); - netdev_walk_all_lower_dev(dev, __netdev_update_upper_level, NULL); + netdev_walk_all_lower_dev_ignore(dev, __netdev_update_upper_level, + NULL); __netdev_update_lower_level(upper_dev, NULL); - netdev_walk_all_upper_dev(upper_dev, __netdev_update_lower_level, NULL); + netdev_walk_all_upper_dev_ignore(upper_dev, + __netdev_update_lower_level, NULL); return 0; @@ -7271,13 +7360,44 @@ void netdev_upper_dev_unlink(struct net_device *dev, &changeupper_info.info); __netdev_update_upper_level(dev, NULL); - netdev_walk_all_lower_dev(dev, __netdev_update_upper_level, NULL); + netdev_walk_all_lower_dev_ignore(dev, __netdev_update_upper_level, + NULL); __netdev_update_lower_level(upper_dev, NULL); - netdev_walk_all_upper_dev(upper_dev, __netdev_update_lower_level, NULL); + netdev_walk_all_upper_dev_ignore(upper_dev, + __netdev_update_lower_level, NULL); } EXPORT_SYMBOL(netdev_upper_dev_unlink); +void __netdev_adjacent_dev_set(struct net_device *upper_dev, + struct net_device *lower_dev, + bool val) +{ + struct netdev_adjacent *adj; + + adj = __netdev_find_adj(lower_dev, &upper_dev->adj_list.lower); + if (adj) + adj->ignore = val; + + adj = __netdev_find_adj(upper_dev, &lower_dev->adj_list.upper); + if (adj) + adj->ignore = val; +} + +void netdev_adjacent_dev_disable(struct net_device *upper_dev, + struct net_device *lower_dev) +{ + __netdev_adjacent_dev_set(upper_dev, lower_dev, true); +} +EXPORT_SYMBOL(netdev_adjacent_dev_disable); + +void netdev_adjacent_dev_enable(struct net_device *upper_dev, + struct net_device *lower_dev) +{ + __netdev_adjacent_dev_set(upper_dev, lower_dev, false); +} +EXPORT_SYMBOL(netdev_adjacent_dev_enable); + /** * netdev_bonding_info_change - Dispatch event about slave change * @dev: device From patchwork Wed Sep 4 18:41:05 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taehee Yoo X-Patchwork-Id: 1157901 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="AtznKVfs"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 46Nt251sS1z9s4Y for ; Thu, 5 Sep 2019 04:41:17 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2387761AbfIDSlQ (ORCPT ); Wed, 4 Sep 2019 14:41:16 -0400 Received: from mail-pg1-f196.google.com ([209.85.215.196]:42339 "EHLO mail-pg1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732177AbfIDSlQ (ORCPT ); Wed, 4 Sep 2019 14:41:16 -0400 Received: by mail-pg1-f196.google.com with SMTP id p3so11686941pgb.9 for ; Wed, 04 Sep 2019 11:41:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=lw2VArFgPnaf+jWzHkiHI61Szxhke5CeCJ9DDvWn0zQ=; b=AtznKVfs0EEbiEVtm5kvwui5fyf0cJuQOLziIzgs5GGJG99gT+GEnepRiYQL72onSP 1uLGTiCikJyObdPVKbkyPmFNLehm+y+49xbv2PiygRGSbbWWtmcMhZ9lU7VWdrb6N43Z kI7wxBJGgw7c3u+CHCtDwgWOFJ9x+5fRrEydqmxtn1mSlZhzj7C4w4egRQ97xfffQ90L zDmBYfwMPG6FHYB/nY0fzt/7QvSTsk4mCYT+Tyc38trJ1WOWKuAh/I4FDzk3TFvrbexO pVIMNt3pj+6YGvomQiLB/JbkS//fVYXh5hThU+DHo2x3ChU4APw0zMPZroGKAH8q9I52 HckQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=lw2VArFgPnaf+jWzHkiHI61Szxhke5CeCJ9DDvWn0zQ=; b=D+odfoHuFAG5eVILnkMTCjMRTjzUmWPkENtA+8NUcxem7IXnB8yJ8c3qJd0r2ArHBO 1vdbLg5fM4P46Ex9t38uACIaa5v86jnsi/a+loaYj9EW9902mI9Mo7YRVfuo2CqSyTRo qklAOckLwZMe7Gp5ZJlTlAIIpErfOYrdcX5+/J2rzKo1DWBykZ1LxiVug8JdjTxWsd4d e26/D4pxDccYQ3CARWez7JUXQmbDkhqCLAYu1jMhwvhah1lImZwNrI8xcw1/XCe9e+9D C/r+1wN44jrJBE+kDRYABB/CUh0SogjXPxtr+LGKYWZ2KwdwZJ7/IYyep0L5BWqJBjyg hMUA== X-Gm-Message-State: APjAAAUCMoeI7dPfsmu/eyECTNzMgkbl8N+Z99TFw/DLGUDn6uEoqUYJ ZEazy3OrEmc6vSxcF2VkCLo= X-Google-Smtp-Source: APXvYqz/pA0wumvzAryLCV/9gRe2o6q9G+DDvZ0vYmG8uBSRUkQ1yWUTqT5aWBjxtrO0IKe/sEnhSQ== X-Received: by 2002:a17:90a:9f4b:: with SMTP id q11mr6362182pjv.105.1567622475241; Wed, 04 Sep 2019 11:41:15 -0700 (PDT) Received: from ap-To-be-filled-by-O-E-M.1.1.1.1 ([14.33.120.60]) by smtp.gmail.com with ESMTPSA id n185sm23068417pga.16.2019.09.04.11.41.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Sep 2019 11:41:14 -0700 (PDT) From: Taehee Yoo To: davem@davemloft.net, netdev@vger.kernel.org, j.vosburgh@gmail.com, vfalico@gmail.com, andy@greyhouse.net, jiri@resnulli.us, sd@queasysnail.net, roopa@cumulusnetworks.com, saeedm@mellanox.com, manishc@marvell.com, rahulv@marvell.com, kys@microsoft.com, haiyangz@microsoft.com, sthemmin@microsoft.com, sashal@kernel.org, hare@suse.de, varun@chelsio.com, ubraun@linux.ibm.com, kgraul@linux.ibm.com Cc: ap420073@gmail.com Subject: [PATCH net 10/11] vxlan: add adjacent link to limit depth level Date: Thu, 5 Sep 2019 03:41:05 +0900 Message-Id: <20190904184105.15424-1-ap420073@gmail.com> X-Mailer: git-send-email 2.17.1 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Current vxlan code doesn't limit the number of nested devices. Nested devices would be handled recursively and this routine needs huge stack memory. So, unlimited nested devices could make stack overflow. In order to fix this issue, this patch adds adjacent links. The adjacent link APIs internally check the depth level. Test commands: ip link add dummy0 type dummy ip link add vxlan0 type vxlan id 0 group 239.1.1.1 dev dummy0 \ dstport 4789 for i in {1..100} do let A=$i-1 ip link add vxlan$i type vxlan id $i group 239.1.1.1 \ dev vxlan$A dstport 4789 done ip link del dummy0 The top upper link is vxlan100 and the lowest link is vxlan0. When vxlan0 is deleting, the upper devices will be deleted recursively. It needs huge stack memory so it makes stack overflow. Splat looks like: [ 229.628477] ============================================================================= [ 229.629785] BUG page->ptl (Not tainted): Padding overwritten. 0x0000000026abf214-0x0000000091f6abb2 [ 229.629785] ----------------------------------------------------------------------------- [ 229.629785] [ 229.655439] ================================================================== [ 229.629785] INFO: Slab 0x00000000ff7cfda8 objects=19 used=19 fp=0x00000000fe33776c flags=0x200000000010200 [ 229.655688] BUG: KASAN: stack-out-of-bounds in unmap_single_vma+0x25a/0x2e0 [ 229.655688] Read of size 8 at addr ffff888113076928 by task vlan-network-in/2334 [ 229.655688] [ 229.629785] Padding 0000000026abf214: 00 80 14 0d 81 88 ff ff 68 91 81 14 81 88 ff ff ........h....... [ 229.629785] Padding 0000000001e24790: 38 91 81 14 81 88 ff ff 68 91 81 14 81 88 ff ff 8.......h....... [ 229.629785] Padding 00000000b39397c8: 33 30 62 a7 ff ff ff ff ff eb 60 22 10 f1 ff 1f 30b.......`".... [ 229.629785] Padding 00000000bc98f53a: 80 60 07 13 81 88 ff ff 00 80 14 0d 81 88 ff ff .`.............. [ 229.629785] Padding 000000002aa8123d: 68 91 81 14 81 88 ff ff f7 21 17 a7 ff ff ff ff h........!...... [ 229.629785] Padding 000000001c8c2369: 08 81 14 0d 81 88 ff ff 03 02 00 00 00 00 00 00 ................ [ 229.629785] Padding 000000004e290c5d: 21 90 a2 21 10 ed ff ff 00 00 00 00 00 fc ff df !..!............ [ 229.629785] Padding 000000000e25d731: 18 60 07 13 81 88 ff ff c0 8b 13 05 81 88 ff ff .`.............. [ 229.629785] Padding 000000007adc7ab3: b3 8a b5 41 00 00 00 00 ...A.... [ 229.629785] FIX page->ptl: Restoring 0x0000000026abf214-0x0000000091f6abb2=0x5a [ ... ] Fixes: acaf4e70997f ("net: vxlan: when lower dev unregisters remove vxlan dev as well") Signed-off-by: Taehee Yoo --- drivers/net/vxlan.c | 71 ++++++++++++++++++++++++++++++++++++++------- include/net/vxlan.h | 1 + 2 files changed, 62 insertions(+), 10 deletions(-) diff --git a/drivers/net/vxlan.c b/drivers/net/vxlan.c index 3d9bcc957f7d..0d5c8d22d8a4 100644 --- a/drivers/net/vxlan.c +++ b/drivers/net/vxlan.c @@ -3567,6 +3567,8 @@ static int __vxlan_dev_create(struct net *net, struct net_device *dev, struct vxlan_net *vn = net_generic(net, vxlan_net_id); struct vxlan_dev *vxlan = netdev_priv(dev); struct vxlan_fdb *f = NULL; + struct net_device *remote_dev = NULL; + struct vxlan_rdst *dst = &vxlan->default_dst; bool unregister = false; int err; @@ -3577,14 +3579,14 @@ static int __vxlan_dev_create(struct net *net, struct net_device *dev, dev->ethtool_ops = &vxlan_ethtool_ops; /* create an fdb entry for a valid default destination */ - if (!vxlan_addr_any(&vxlan->default_dst.remote_ip)) { + if (!vxlan_addr_any(&dst->remote_ip)) { err = vxlan_fdb_create(vxlan, all_zeros_mac, - &vxlan->default_dst.remote_ip, + &dst->remote_ip, NUD_REACHABLE | NUD_PERMANENT, vxlan->cfg.dst_port, - vxlan->default_dst.remote_vni, - vxlan->default_dst.remote_vni, - vxlan->default_dst.remote_ifindex, + dst->remote_vni, + dst->remote_vni, + dst->remote_ifindex, NTF_SELF, &f); if (err) return err; @@ -3595,26 +3597,43 @@ static int __vxlan_dev_create(struct net *net, struct net_device *dev, goto errout; unregister = true; + if (dst->remote_ifindex) { + remote_dev = __dev_get_by_index(net, dst->remote_ifindex); + if (!remote_dev) + goto errout; + + err = netdev_upper_dev_link(remote_dev, dev, extack); + if (err) + goto errout; + } + err = rtnl_configure_link(dev, NULL); if (err) - goto errout; + goto unlink; if (f) { - vxlan_fdb_insert(vxlan, all_zeros_mac, - vxlan->default_dst.remote_vni, f); + vxlan_fdb_insert(vxlan, all_zeros_mac, dst->remote_vni, f); /* notify default fdb entry */ err = vxlan_fdb_notify(vxlan, f, first_remote_rtnl(f), RTM_NEWNEIGH, true, extack); if (err) { vxlan_fdb_destroy(vxlan, f, false, false); + if (remote_dev) + netdev_upper_dev_unlink(remote_dev, dev); goto unregister; } } list_add(&vxlan->next, &vn->vxlan_list); + if (remote_dev) { + dst->remote_dev = remote_dev; + dev_hold(remote_dev); + } return 0; - +unlink: + if (remote_dev) + netdev_upper_dev_unlink(remote_dev, dev); errout: /* unregister_netdevice() destroys the default FDB entry with deletion * notification. But the addition notification was not sent yet, so @@ -3936,6 +3955,8 @@ static int vxlan_changelink(struct net_device *dev, struct nlattr *tb[], struct net_device *lowerdev; struct vxlan_config conf; int err; + bool linked = false; + bool disabled = false; err = vxlan_nl2conf(tb, data, dev, &conf, true, extack); if (err) @@ -3946,6 +3967,16 @@ static int vxlan_changelink(struct net_device *dev, struct nlattr *tb[], if (err) return err; + if (lowerdev) { + if (dst->remote_dev && lowerdev != dst->remote_dev) { + netdev_adjacent_dev_disable(dst->remote_dev, dev); + disabled = true; + } + err = netdev_upper_dev_link(lowerdev, dev, extack); + if (err) + goto err; + linked = true; + } /* handle default dst entry */ if (!vxlan_addr_equal(&conf.remote_ip, &dst->remote_ip)) { u32 hash_index = fdb_head_index(vxlan, all_zeros_mac, conf.vni); @@ -3962,7 +3993,7 @@ static int vxlan_changelink(struct net_device *dev, struct nlattr *tb[], NTF_SELF, true, extack); if (err) { spin_unlock_bh(&vxlan->hash_lock[hash_index]); - return err; + goto err; } } if (!vxlan_addr_any(&dst->remote_ip)) @@ -3979,8 +4010,24 @@ static int vxlan_changelink(struct net_device *dev, struct nlattr *tb[], if (conf.age_interval != vxlan->cfg.age_interval) mod_timer(&vxlan->age_timer, jiffies); + if (disabled) { + netdev_adjacent_dev_enable(dst->remote_dev, dev); + netdev_upper_dev_unlink(dst->remote_dev, dev); + dev_put(dst->remote_dev); + } + if (linked) { + dst->remote_dev = lowerdev; + dev_hold(dst->remote_dev); + } + vxlan_config_apply(dev, &conf, lowerdev, vxlan->net, true); return 0; +err: + if (linked) + netdev_upper_dev_unlink(lowerdev, dev); + if (disabled) + netdev_adjacent_dev_enable(dst->remote_dev, dev); + return err; } static void vxlan_dellink(struct net_device *dev, struct list_head *head) @@ -3991,6 +4038,10 @@ static void vxlan_dellink(struct net_device *dev, struct list_head *head) list_del(&vxlan->next); unregister_netdevice_queue(dev, head); + if (vxlan->default_dst.remote_dev) { + netdev_upper_dev_unlink(vxlan->default_dst.remote_dev, dev); + dev_put(vxlan->default_dst.remote_dev); + } } static size_t vxlan_get_size(const struct net_device *dev) diff --git a/include/net/vxlan.h b/include/net/vxlan.h index dc1583a1fb8a..08e237d7aa73 100644 --- a/include/net/vxlan.h +++ b/include/net/vxlan.h @@ -197,6 +197,7 @@ struct vxlan_rdst { u8 offloaded:1; __be32 remote_vni; u32 remote_ifindex; + struct net_device *remote_dev; struct list_head list; struct rcu_head rcu; struct dst_cache dst_cache; From patchwork Wed Sep 4 18:41:18 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taehee Yoo X-Patchwork-Id: 1157902 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="uCghbHqE"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 46Nt2L2tTMz9s4Y for ; Thu, 5 Sep 2019 04:41:30 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2387767AbfIDSl3 (ORCPT ); Wed, 4 Sep 2019 14:41:29 -0400 Received: from mail-pg1-f195.google.com ([209.85.215.195]:35061 "EHLO mail-pg1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1733029AbfIDSl3 (ORCPT ); Wed, 4 Sep 2019 14:41:29 -0400 Received: by mail-pg1-f195.google.com with SMTP id n4so11715984pgv.2 for ; Wed, 04 Sep 2019 11:41:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=aKJ37LNFdD61ZAFwEAqObclP4NKKRn8Qp+/PLgAZr14=; b=uCghbHqEVFfG3lBh/l6QyqHUiUC+tVJt0YR8NFCaQHBGDb/XSbHxzp0Y3+0ycobs3l 4DcSw1GDdKUYZ/rXVUC2PMV/1OrQtCpqsZHxUGXv7BGaSqCAziLGUG6Gyb/COGKySMoi +k+cYbJhvnsspv4xbJmXaAdHMnObUv7rEjbwKa8aXdhpOPCv1XUOCtw5ICTf90u07wMb 9MvXSfOR7SNlLkMO41zzHnXjDTViKYTpSb3PFFzwPZE2CHvDQJUQS7Rm+XoUTFoMWeur YiBADF6GLsM6SLMTFxX/Xh5epKoxxoMpLyEdbFNqUkm0kPEEJQjbGnjPWjCA6dLgB4JB em5A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=aKJ37LNFdD61ZAFwEAqObclP4NKKRn8Qp+/PLgAZr14=; b=tamaqXd1BASN4CUU0+eXixUVHEmM2uHKOVy7YOTY9T3DGlgM38sZSOPAfws8uF7yXE /WN/W54UurQjORcKzjk4AjPspIennZgTOHiNnf99s0efVUbHf8sXarT34aBAWsghH+IM nOhKtXdM9tD+nGrABQjwCGEAvPoImimhIL0uji7b4e3DL3hST3wUQpk9lr7vLkSiWBYl a+d8e9D+xorbTyAOyZmaeP1qlnmUE8PgR+1g+jNkt2uf0vv+1nvA/2lbCHIWkj2W+xuw hCosYs/w+jJPlEAo6UD04a/OZLxJG0nBYYZspI5TvqXMPjLAJaxA9i+Eo0S/4Ry0E11e IGRg== X-Gm-Message-State: APjAAAWIoW22OucCHyblilAmnk29CvqdHOQ/YSjEIJTzg19YgSdO2Rwl zC9XT1X15Kjm9ft/JcSiWCg= X-Google-Smtp-Source: APXvYqx9YPjW5vqW84bm17bgOx8iZk0D6+YjMZA9Qbtk9SpxRntuGvXZbVFl2BNxm3AonkgQJkXAAQ== X-Received: by 2002:a63:5f09:: with SMTP id t9mr36646116pgb.351.1567622487560; Wed, 04 Sep 2019 11:41:27 -0700 (PDT) Received: from ap-To-be-filled-by-O-E-M.1.1.1.1 ([14.33.120.60]) by smtp.gmail.com with ESMTPSA id v7sm24583124pff.87.2019.09.04.11.41.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Sep 2019 11:41:26 -0700 (PDT) From: Taehee Yoo To: davem@davemloft.net, netdev@vger.kernel.org, j.vosburgh@gmail.com, vfalico@gmail.com, andy@greyhouse.net, jiri@resnulli.us, sd@queasysnail.net, roopa@cumulusnetworks.com, saeedm@mellanox.com, manishc@marvell.com, rahulv@marvell.com, kys@microsoft.com, haiyangz@microsoft.com, sthemmin@microsoft.com, sashal@kernel.org, hare@suse.de, varun@chelsio.com, ubraun@linux.ibm.com, kgraul@linux.ibm.com Cc: ap420073@gmail.com Subject: [PATCH net 11/11] net: remove unnecessary variables and callback Date: Thu, 5 Sep 2019 03:41:18 +0900 Message-Id: <20190904184118.15505-1-ap420073@gmail.com> X-Mailer: git-send-email 2.17.1 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org This patch removes variables and callback these are related to the nested device structure. devices that can be nested have their own nest_level variable that represents the depth of nested devices. In the previous patch, new {lower/upper}_level variables are added and they replace old private nest_level variable. So, this patch removes all 'nest_level' variables. In order to avoid lockdep warning, ->ndo_get_lock_subclass() was added to get lockdep subclass value, which is actually lower nested depth value. But now, they use the dynamic lockdep key to avoid lockdep warning instead of the subclass. So, this patch removes ->ndo_get_lock_subclass() callback. Signed-off-by: Taehee Yoo --- drivers/net/bonding/bond_alb.c | 2 +- drivers/net/bonding/bond_main.c | 14 ------------- .../net/ethernet/mellanox/mlx5/core/en_tc.c | 2 +- drivers/net/macsec.c | 9 --------- drivers/net/macvlan.c | 7 ------- include/linux/if_macvlan.h | 1 - include/linux/if_vlan.h | 12 ----------- include/linux/netdevice.h | 12 ----------- include/net/bonding.h | 1 - net/8021q/vlan.c | 1 - net/8021q/vlan_dev.c | 6 ------ net/core/dev.c | 20 ------------------- net/core/dev_addr_lists.c | 12 +++++------ net/smc/smc_core.c | 2 +- net/smc/smc_pnet.c | 2 +- 15 files changed, 10 insertions(+), 93 deletions(-) diff --git a/drivers/net/bonding/bond_alb.c b/drivers/net/bonding/bond_alb.c index 8c79bad2a9a5..4f2e6910c623 100644 --- a/drivers/net/bonding/bond_alb.c +++ b/drivers/net/bonding/bond_alb.c @@ -952,7 +952,7 @@ static int alb_upper_dev_walk(struct net_device *upper, void *_data) struct bond_vlan_tag *tags; if (is_vlan_dev(upper) && - bond->nest_level == vlan_get_encap_level(upper) - 1) { + bond->dev->lower_level == upper->lower_level - 1) { if (upper->addr_assign_type == NET_ADDR_STOLEN) { alb_send_lp_vid(slave, mac_addr, vlan_dev_vlan_proto(upper), diff --git a/drivers/net/bonding/bond_main.c b/drivers/net/bonding/bond_main.c index 2b16683bb8b8..2749423649eb 100644 --- a/drivers/net/bonding/bond_main.c +++ b/drivers/net/bonding/bond_main.c @@ -1733,8 +1733,6 @@ int bond_enslave(struct net_device *bond_dev, struct net_device *slave_dev, goto err_upper_unlink; } - bond->nest_level = dev_get_nest_level(bond_dev) + 1; - /* If the mode uses primary, then the following is handled by * bond_change_active_slave(). */ @@ -1982,9 +1980,6 @@ static int __bond_release_one(struct net_device *bond_dev, if (!bond_has_slaves(bond)) { bond_set_carrier(bond); eth_hw_addr_random(bond_dev); - bond->nest_level = SINGLE_DEPTH_NESTING; - } else { - bond->nest_level = dev_get_nest_level(bond_dev) + 1; } unblock_netpoll_tx(); @@ -3467,13 +3462,6 @@ static void bond_fold_stats(struct rtnl_link_stats64 *_res, } } -static int bond_get_nest_level(struct net_device *bond_dev) -{ - struct bonding *bond = netdev_priv(bond_dev); - - return bond->nest_level; -} - static void bond_get_stats(struct net_device *bond_dev, struct rtnl_link_stats64 *stats) { @@ -4293,7 +4281,6 @@ static const struct net_device_ops bond_netdev_ops = { .ndo_neigh_setup = bond_neigh_setup, .ndo_vlan_rx_add_vid = bond_vlan_rx_add_vid, .ndo_vlan_rx_kill_vid = bond_vlan_rx_kill_vid, - .ndo_get_lock_subclass = bond_get_nest_level, #ifdef CONFIG_NET_POLL_CONTROLLER .ndo_netpoll_setup = bond_netpoll_setup, .ndo_netpoll_cleanup = bond_netpoll_cleanup, @@ -4817,7 +4804,6 @@ static int bond_init(struct net_device *bond_dev) if (!bond->wq) return -ENOMEM; - bond->nest_level = SINGLE_DEPTH_NESTING; bond_dev_set_lockdep_class(bond_dev); list_add_tail(&bond->bond_list, &bn->dev_list); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_tc.c b/drivers/net/ethernet/mellanox/mlx5/core/en_tc.c index 00b2d4a86159..e056f9aad8df 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_tc.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_tc.c @@ -2797,7 +2797,7 @@ static int add_vlan_pop_action(struct mlx5e_priv *priv, struct mlx5_esw_flow_attr *attr, u32 *action) { - int nest_level = vlan_get_encap_level(attr->parse_attr->filter_dev); + int nest_level = attr->parse_attr->filter_dev->lower_level; struct flow_action_entry vlan_act = { .id = FLOW_ACTION_VLAN_POP, }; diff --git a/drivers/net/macsec.c b/drivers/net/macsec.c index 41ec1ed0d545..c0cb595f2bba 100644 --- a/drivers/net/macsec.c +++ b/drivers/net/macsec.c @@ -269,7 +269,6 @@ struct macsec_dev { struct gro_cells gro_cells; struct lock_class_key xmit_lock_key; struct lock_class_key addr_lock_key; - unsigned int nest_level; }; /** @@ -2988,11 +2987,6 @@ static int macsec_get_iflink(const struct net_device *dev) return macsec_priv(dev)->real_dev->ifindex; } -static int macsec_get_nest_level(struct net_device *dev) -{ - return macsec_priv(dev)->nest_level; -} - static const struct net_device_ops macsec_netdev_ops = { .ndo_init = macsec_dev_init, .ndo_uninit = macsec_dev_uninit, @@ -3006,7 +3000,6 @@ static const struct net_device_ops macsec_netdev_ops = { .ndo_start_xmit = macsec_start_xmit, .ndo_get_stats64 = macsec_get_stats64, .ndo_get_iflink = macsec_get_iflink, - .ndo_get_lock_subclass = macsec_get_nest_level, }; static const struct device_type macsec_type = { @@ -3289,8 +3282,6 @@ static int macsec_newlink(struct net *net, struct net_device *dev, if (err < 0) return err; - macsec->nest_level = dev_get_nest_level(real_dev) + 1; - err = netdev_upper_dev_link(real_dev, dev, extack); if (err < 0) goto unregister; diff --git a/drivers/net/macvlan.c b/drivers/net/macvlan.c index dae368a2e8d1..2c14bc606514 100644 --- a/drivers/net/macvlan.c +++ b/drivers/net/macvlan.c @@ -867,11 +867,6 @@ static int macvlan_do_ioctl(struct net_device *dev, struct ifreq *ifr, int cmd) #define MACVLAN_STATE_MASK \ ((1<<__LINK_STATE_NOCARRIER) | (1<<__LINK_STATE_DORMANT)) -static int macvlan_get_nest_level(struct net_device *dev) -{ - return ((struct macvlan_dev *)netdev_priv(dev))->nest_level; -} - static void macvlan_dev_set_lockdep_one(struct net_device *dev, struct netdev_queue *txq, void *_unused) @@ -1180,7 +1175,6 @@ static const struct net_device_ops macvlan_netdev_ops = { .ndo_fdb_add = macvlan_fdb_add, .ndo_fdb_del = macvlan_fdb_del, .ndo_fdb_dump = ndo_dflt_fdb_dump, - .ndo_get_lock_subclass = macvlan_get_nest_level, #ifdef CONFIG_NET_POLL_CONTROLLER .ndo_poll_controller = macvlan_dev_poll_controller, .ndo_netpoll_setup = macvlan_dev_netpoll_setup, @@ -1464,7 +1458,6 @@ int macvlan_common_newlink(struct net *src_net, struct net_device *dev, vlan->dev = dev; vlan->port = port; vlan->set_features = MACVLAN_FEATURES; - vlan->nest_level = dev_get_nest_level(lowerdev) + 1; vlan->mode = MACVLAN_MODE_VEPA; if (data && data[IFLA_MACVLAN_MODE]) diff --git a/include/linux/if_macvlan.h b/include/linux/if_macvlan.h index ea5b41823287..e9202edcf101 100644 --- a/include/linux/if_macvlan.h +++ b/include/linux/if_macvlan.h @@ -29,7 +29,6 @@ struct macvlan_dev { netdev_features_t set_features; enum macvlan_mode mode; u16 flags; - int nest_level; unsigned int macaddr_count; struct lock_class_key xmit_lock_key; struct lock_class_key addr_lock_key; diff --git a/include/linux/if_vlan.h b/include/linux/if_vlan.h index 1aed9f613e90..6f30284a58e5 100644 --- a/include/linux/if_vlan.h +++ b/include/linux/if_vlan.h @@ -182,8 +182,6 @@ struct vlan_dev_priv { #ifdef CONFIG_NET_POLL_CONTROLLER struct netpoll *netpoll; #endif - unsigned int nest_level; - struct lock_class_key xmit_lock_key; struct lock_class_key addr_lock_key; }; @@ -224,11 +222,6 @@ extern void vlan_vids_del_by_dev(struct net_device *dev, extern bool vlan_uses_dev(const struct net_device *dev); -static inline int vlan_get_encap_level(struct net_device *dev) -{ - BUG_ON(!is_vlan_dev(dev)); - return vlan_dev_priv(dev)->nest_level; -} #else static inline struct net_device * __vlan_find_dev_deep_rcu(struct net_device *real_dev, @@ -298,11 +291,6 @@ static inline bool vlan_uses_dev(const struct net_device *dev) { return false; } -static inline int vlan_get_encap_level(struct net_device *dev) -{ - BUG(); - return 0; -} #endif /** diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index d9ca4a79f715..6bf493028c41 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -1408,7 +1408,6 @@ struct net_device_ops { void (*ndo_dfwd_del_station)(struct net_device *pdev, void *priv); - int (*ndo_get_lock_subclass)(struct net_device *dev); int (*ndo_set_tx_maxrate)(struct net_device *dev, int queue_index, u32 maxrate); @@ -4050,16 +4049,6 @@ static inline void netif_addr_lock(struct net_device *dev) spin_lock(&dev->addr_list_lock); } -static inline void netif_addr_lock_nested(struct net_device *dev) -{ - int subclass = SINGLE_DEPTH_NESTING; - - if (dev->netdev_ops->ndo_get_lock_subclass) - subclass = dev->netdev_ops->ndo_get_lock_subclass(dev); - - spin_lock_nested(&dev->addr_list_lock, subclass); -} - static inline void netif_addr_lock_bh(struct net_device *dev) { spin_lock_bh(&dev->addr_list_lock); @@ -4337,7 +4326,6 @@ void netdev_lower_state_changed(struct net_device *lower_dev, extern u8 netdev_rss_key[NETDEV_RSS_KEY_LEN] __read_mostly; void netdev_rss_key_fill(void *buffer, size_t len); -int dev_get_nest_level(struct net_device *dev); int skb_checksum_help(struct sk_buff *skb); int skb_crc32c_csum_help(struct sk_buff *skb); int skb_csum_hwoffload_help(struct sk_buff *skb, diff --git a/include/net/bonding.h b/include/net/bonding.h index c39ac7061e41..74f41dd73866 100644 --- a/include/net/bonding.h +++ b/include/net/bonding.h @@ -203,7 +203,6 @@ struct bonding { struct slave __rcu *primary_slave; struct bond_up_slave __rcu *slave_arr; /* Array of usable slaves */ bool force_primary; - u32 nest_level; s32 slave_cnt; /* never change this value outside the attach/detach wrappers */ int (*recv_probe)(const struct sk_buff *, struct bonding *, struct slave *); diff --git a/net/8021q/vlan.c b/net/8021q/vlan.c index 54728d2eda18..d4bcfd8f95bf 100644 --- a/net/8021q/vlan.c +++ b/net/8021q/vlan.c @@ -172,7 +172,6 @@ int register_vlan_dev(struct net_device *dev, struct netlink_ext_ack *extack) if (err < 0) goto out_uninit_mvrp; - vlan->nest_level = dev_get_nest_level(real_dev) + 1; err = register_netdevice(dev); if (err < 0) goto out_uninit_mvrp; diff --git a/net/8021q/vlan_dev.c b/net/8021q/vlan_dev.c index 12bc80650087..e8707827540c 100644 --- a/net/8021q/vlan_dev.c +++ b/net/8021q/vlan_dev.c @@ -514,11 +514,6 @@ static void vlan_dev_set_lockdep_class(struct net_device *dev) netdev_for_each_tx_queue(dev, vlan_dev_set_lockdep_one, NULL); } -static int vlan_dev_get_lock_subclass(struct net_device *dev) -{ - return vlan_dev_priv(dev)->nest_level; -} - static const struct header_ops vlan_header_ops = { .create = vlan_dev_hard_header, .parse = eth_header_parse, @@ -814,7 +809,6 @@ static const struct net_device_ops vlan_netdev_ops = { .ndo_netpoll_cleanup = vlan_dev_netpoll_cleanup, #endif .ndo_fix_features = vlan_dev_fix_features, - .ndo_get_lock_subclass = vlan_dev_get_lock_subclass, .ndo_get_iflink = vlan_dev_get_iflink, }; diff --git a/net/core/dev.c b/net/core/dev.c index ac055b531c96..73a69a7a3553 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -7510,26 +7510,6 @@ void *netdev_lower_dev_get_private(struct net_device *dev, } EXPORT_SYMBOL(netdev_lower_dev_get_private); - -int dev_get_nest_level(struct net_device *dev) -{ - struct net_device *lower = NULL; - struct list_head *iter; - int max_nest = -1; - int nest; - - ASSERT_RTNL(); - - netdev_for_each_lower_dev(dev, lower, iter) { - nest = dev_get_nest_level(lower); - if (max_nest < nest) - max_nest = nest; - } - - return max_nest + 1; -} -EXPORT_SYMBOL(dev_get_nest_level); - /** * netdev_lower_change - Dispatch event about lower device state change * @lower_dev: device diff --git a/net/core/dev_addr_lists.c b/net/core/dev_addr_lists.c index 6393ba930097..2f949b5a1eb9 100644 --- a/net/core/dev_addr_lists.c +++ b/net/core/dev_addr_lists.c @@ -637,7 +637,7 @@ int dev_uc_sync(struct net_device *to, struct net_device *from) if (to->addr_len != from->addr_len) return -EINVAL; - netif_addr_lock_nested(to); + netif_addr_lock(to); err = __hw_addr_sync(&to->uc, &from->uc, to->addr_len); if (!err) __dev_set_rx_mode(to); @@ -667,7 +667,7 @@ int dev_uc_sync_multiple(struct net_device *to, struct net_device *from) if (to->addr_len != from->addr_len) return -EINVAL; - netif_addr_lock_nested(to); + netif_addr_lock(to); err = __hw_addr_sync_multiple(&to->uc, &from->uc, to->addr_len); if (!err) __dev_set_rx_mode(to); @@ -691,7 +691,7 @@ void dev_uc_unsync(struct net_device *to, struct net_device *from) return; netif_addr_lock_bh(from); - netif_addr_lock_nested(to); + netif_addr_lock(to); __hw_addr_unsync(&to->uc, &from->uc, to->addr_len); __dev_set_rx_mode(to); netif_addr_unlock(to); @@ -858,7 +858,7 @@ int dev_mc_sync(struct net_device *to, struct net_device *from) if (to->addr_len != from->addr_len) return -EINVAL; - netif_addr_lock_nested(to); + netif_addr_lock(to); err = __hw_addr_sync(&to->mc, &from->mc, to->addr_len); if (!err) __dev_set_rx_mode(to); @@ -888,7 +888,7 @@ int dev_mc_sync_multiple(struct net_device *to, struct net_device *from) if (to->addr_len != from->addr_len) return -EINVAL; - netif_addr_lock_nested(to); + netif_addr_lock(to); err = __hw_addr_sync_multiple(&to->mc, &from->mc, to->addr_len); if (!err) __dev_set_rx_mode(to); @@ -912,7 +912,7 @@ void dev_mc_unsync(struct net_device *to, struct net_device *from) return; netif_addr_lock_bh(from); - netif_addr_lock_nested(to); + netif_addr_lock(to); __hw_addr_unsync(&to->mc, &from->mc, to->addr_len); __dev_set_rx_mode(to); netif_addr_unlock(to); diff --git a/net/smc/smc_core.c b/net/smc/smc_core.c index 4ca50ddf8d16..a2e91b8d04b3 100644 --- a/net/smc/smc_core.c +++ b/net/smc/smc_core.c @@ -558,7 +558,7 @@ int smc_vlan_by_tcpsk(struct socket *clcsock, struct smc_init_info *ini) } rtnl_lock(); - nest_lvl = dev_get_nest_level(ndev); + nest_lvl = ndev->lower_level; for (i = 0; i < nest_lvl; i++) { struct list_head *lower = &ndev->adj_list.lower; diff --git a/net/smc/smc_pnet.c b/net/smc/smc_pnet.c index bab2da8cf17a..2920b006f65c 100644 --- a/net/smc/smc_pnet.c +++ b/net/smc/smc_pnet.c @@ -718,7 +718,7 @@ static struct net_device *pnet_find_base_ndev(struct net_device *ndev) int i, nest_lvl; rtnl_lock(); - nest_lvl = dev_get_nest_level(ndev); + nest_lvl = ndev->lower_level; for (i = 0; i < nest_lvl; i++) { struct list_head *lower = &ndev->adj_list.lower;