From patchwork Tue Apr 19 20:12:03 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: KY Srinivasan X-Patchwork-Id: 612259 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 3qqDFD6jdJz9sBg for ; Wed, 20 Apr 2016 04:34:24 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S933683AbcDSSdy (ORCPT ); Tue, 19 Apr 2016 14:33:54 -0400 Received: from p3plsmtps2ded03.prod.phx3.secureserver.net ([208.109.80.60]:33401 "EHLO p3plsmtps2ded03.prod.phx3.secureserver.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S933617AbcDSSdZ (ORCPT ); Tue, 19 Apr 2016 14:33:25 -0400 Received: from linuxonhyperv.com ([72.167.245.219]) by : HOSTING RELAY : with SMTP id saPoaYcmpIUCosaPoadizv; Tue, 19 Apr 2016 11:30:24 -0700 x-originating-ip: 72.167.245.219 Received: by linuxonhyperv.com (Postfix, from userid 507) id A0F02190275; Tue, 19 Apr 2016 13:12:04 -0700 (PDT) From: "K. Y. Srinivasan" To: davem@davemloft.net, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, devel@linuxdriverproject.org, olaf@aepfle.de, apw@canonical.com, jasowang@redhat.com, eli@mellanox.com, jackm@mellanox.com, yevgenyp@mellanox.com, john.ronciak@intel.com, intel-wired-lan@lists.osuosl.org, alexander.duyck@gmail.com Cc: "K. Y. Srinivasan" Subject: [PATCH net-next V4 2/2] intel: ixgbevf: Support Windows hosts (Hyper-V) Date: Tue, 19 Apr 2016 13:12:03 -0700 Message-Id: <1461096723-26471-2-git-send-email-kys@microsoft.com> X-Mailer: git-send-email 1.7.4.1 In-Reply-To: <1461096723-26471-1-git-send-email-kys@microsoft.com> References: <1461096698-26430-1-git-send-email-kys@microsoft.com> <1461096723-26471-1-git-send-email-kys@microsoft.com> X-CMAE-Envelope: MS4wfNJUCAEVwYXuUGCbU9BDUy1lW3RZWb0kqU3XpUmKuWpmVwhRVigj+N315jnB36barBIttuhKBfz5NKhSnw8HXu3oGceNSR6Vzr5a5KNG/C5xiCd4mOAg y7ww3dHFyl0RxeanIKCnr9eO1LoyIeALvyU6u9Z+WGOFxRs9kNQxVlLku20MzcnxueLq6ffnbC2awSjSiq/T+2ytoUgN5+ylinSwwEPG3j8Ho8Zj/CY5rQHe /vjBQcDtb31InkeK6ofh9zwnNb1qL6jJMO6vr90Xdh2LqMoPZmlPs+wNaXMCL25pLrdxnos+xloQ504L8ftsFIKn9THoSqRnTk0j/JfzBIoXwvEr5iZJ0nO3 B0daofkJeq/RuVrR24hTVli4rL/3YMAWIGHYTVFwA2n+vcwZv++AD49S10S611GM7+Bw6peBFQs4Z9FrdjhfSRfaVkbHcXw8cV6f3Okjp7z5+kKf1RCau47Y F2dsvgpytBnM2DE+JGj9UOtCvj0+MTa94UK/1hmSyJCtfPTV78fw8QhEsosX6pSRlx4ShCt8fSLN8g/qkCH78AEtD1bsnwNq2UQyeDYAhKMJOw7vvejbU0g5 dUdRYYpYhZs6Efe5ekcXGzqvretfVby1HtbkUJKfGsz29w== Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org On Hyper-V, the VF/PF communication is a via software mediated path as opposed to the hardware mailbox. Make the necessary adjustments to support Hyper-V. Signed-off-by: K. Y. Srinivasan --- V2: Addressed most of the comments from Alexander Duyck and Rustad, Mark D . V3: Addressed additional comments from Alexander Duyck V4: Addressed kbuild errors reported by: kbuild test robot drivers/net/ethernet/intel/ixgbevf/ixgbevf.h | 12 ++ drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c | 31 +++- drivers/net/ethernet/intel/ixgbevf/mbx.c | 12 ++ drivers/net/ethernet/intel/ixgbevf/vf.c | 216 +++++++++++++++++++++ drivers/net/ethernet/intel/ixgbevf/vf.h | 2 + 5 files changed, 266 insertions(+), 7 deletions(-) diff --git a/drivers/net/ethernet/intel/ixgbevf/ixgbevf.h b/drivers/net/ethernet/intel/ixgbevf/ixgbevf.h index 5ac60ee..3296d27 100644 --- a/drivers/net/ethernet/intel/ixgbevf/ixgbevf.h +++ b/drivers/net/ethernet/intel/ixgbevf/ixgbevf.h @@ -460,9 +460,13 @@ enum ixbgevf_state_t { enum ixgbevf_boards { board_82599_vf, + board_82599_vf_hv, board_X540_vf, + board_X540_vf_hv, board_X550_vf, + board_X550_vf_hv, board_X550EM_x_vf, + board_X550EM_x_vf_hv, }; enum ixgbevf_xcast_modes { @@ -477,6 +481,13 @@ extern const struct ixgbevf_info ixgbevf_X550_vf_info; extern const struct ixgbevf_info ixgbevf_X550EM_x_vf_info; extern const struct ixgbe_mbx_operations ixgbevf_mbx_ops; + +extern const struct ixgbevf_info ixgbevf_82599_vf_hv_info; +extern const struct ixgbevf_info ixgbevf_X540_vf_hv_info; +extern const struct ixgbevf_info ixgbevf_X550_vf_hv_info; +extern const struct ixgbevf_info ixgbevf_X550EM_x_vf_hv_info; +extern const struct ixgbe_mbx_operations ixgbevf_hv_mbx_ops; + /* needed by ethtool.c */ extern const char ixgbevf_driver_name[]; extern const char ixgbevf_driver_version[]; @@ -494,6 +505,7 @@ void ixgbevf_free_rx_resources(struct ixgbevf_ring *); void ixgbevf_free_tx_resources(struct ixgbevf_ring *); void ixgbevf_update_stats(struct ixgbevf_adapter *adapter); int ethtool_ioctl(struct ifreq *ifr); +bool ixgbevf_on_hyperv(struct ixgbe_hw *hw); extern void ixgbevf_write_eitr(struct ixgbevf_q_vector *q_vector); diff --git a/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c b/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c index 007cbe0..c4bb480 100644 --- a/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c +++ b/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c @@ -62,10 +62,14 @@ static char ixgbevf_copyright[] = "Copyright (c) 2009 - 2015 Intel Corporation."; static const struct ixgbevf_info *ixgbevf_info_tbl[] = { - [board_82599_vf] = &ixgbevf_82599_vf_info, - [board_X540_vf] = &ixgbevf_X540_vf_info, - [board_X550_vf] = &ixgbevf_X550_vf_info, - [board_X550EM_x_vf] = &ixgbevf_X550EM_x_vf_info, + [board_82599_vf] = &ixgbevf_82599_vf_info, + [board_82599_vf_hv] = &ixgbevf_82599_vf_hv_info, + [board_X540_vf] = &ixgbevf_X540_vf_info, + [board_X540_vf_hv] = &ixgbevf_X540_vf_hv_info, + [board_X550_vf] = &ixgbevf_X550_vf_info, + [board_X550_vf_hv] = &ixgbevf_X550_vf_hv_info, + [board_X550EM_x_vf] = &ixgbevf_X550EM_x_vf_info, + [board_X550EM_x_vf_hv] = &ixgbevf_X550EM_x_vf_hv_info, }; /* ixgbevf_pci_tbl - PCI Device ID Table @@ -78,9 +82,13 @@ static const struct ixgbevf_info *ixgbevf_info_tbl[] = { */ static const struct pci_device_id ixgbevf_pci_tbl[] = { {PCI_VDEVICE(INTEL, IXGBE_DEV_ID_82599_VF), board_82599_vf }, + {PCI_VDEVICE(INTEL, IXGBE_DEV_ID_82599_VF_HV), board_82599_vf_hv }, {PCI_VDEVICE(INTEL, IXGBE_DEV_ID_X540_VF), board_X540_vf }, + {PCI_VDEVICE(INTEL, IXGBE_DEV_ID_X540_VF_HV), board_X540_vf_hv }, {PCI_VDEVICE(INTEL, IXGBE_DEV_ID_X550_VF), board_X550_vf }, + {PCI_VDEVICE(INTEL, IXGBE_DEV_ID_X550_VF_HV), board_X550_vf_hv }, {PCI_VDEVICE(INTEL, IXGBE_DEV_ID_X550EM_X_VF), board_X550EM_x_vf }, + {PCI_VDEVICE(INTEL, IXGBE_DEV_ID_X550EM_X_VF_HV), board_X550EM_x_vf_hv}, /* required last entry */ {0, } }; @@ -1795,7 +1803,10 @@ static void ixgbevf_configure_rx(struct ixgbevf_adapter *adapter) ixgbevf_setup_vfmrqc(adapter); /* notify the PF of our intent to use this size of frame */ - ixgbevf_rlpml_set_vf(hw, netdev->mtu + ETH_HLEN + ETH_FCS_LEN); + if (!ixgbevf_on_hyperv(hw)) + ixgbevf_rlpml_set_vf(hw, netdev->mtu + ETH_HLEN + ETH_FCS_LEN); + else + ixgbevf_hv_rlpml_set_vf(hw, netdev->mtu + ETH_HLEN + ETH_FCS_LEN); /* Setup the HW Rx Head and Tail Descriptor Pointers and * the Base and Length of the Rx Descriptor Ring @@ -2056,7 +2067,10 @@ static void ixgbevf_negotiate_api(struct ixgbevf_adapter *adapter) spin_lock_bh(&adapter->mbx_lock); while (api[idx] != ixgbe_mbox_api_unknown) { - err = ixgbevf_negotiate_api_version(hw, api[idx]); + if (!ixgbevf_on_hyperv(hw)) + err = ixgbevf_negotiate_api_version(hw, api[idx]); + else + err = ixgbevf_hv_negotiate_api_version(hw, api[idx]); if (!err) break; idx++; @@ -3727,7 +3741,10 @@ static int ixgbevf_change_mtu(struct net_device *netdev, int new_mtu) netdev->mtu = new_mtu; /* notify the PF of our intent to use this size of frame */ - ixgbevf_rlpml_set_vf(hw, max_frame); + if (!ixgbevf_on_hyperv(hw)) + ixgbevf_rlpml_set_vf(hw, max_frame); + else + ixgbevf_hv_rlpml_set_vf(hw, max_frame); return 0; } diff --git a/drivers/net/ethernet/intel/ixgbevf/mbx.c b/drivers/net/ethernet/intel/ixgbevf/mbx.c index dc68fea..298a0da 100644 --- a/drivers/net/ethernet/intel/ixgbevf/mbx.c +++ b/drivers/net/ethernet/intel/ixgbevf/mbx.c @@ -346,3 +346,15 @@ const struct ixgbe_mbx_operations ixgbevf_mbx_ops = { .check_for_rst = ixgbevf_check_for_rst_vf, }; +/** + * Mailbox operations when running on Hyper-V. + * On Hyper-V, PF/VF communiction is not through the + * hardware mailbox; this communication is through + * a software mediated path. + * Most mail box operations are noop while running on + * Hyper-V. + */ +const struct ixgbe_mbx_operations ixgbevf_hv_mbx_ops = { + .init_params = ixgbevf_init_mbx_params_vf, + .check_for_rst = ixgbevf_check_for_rst_vf, +}; diff --git a/drivers/net/ethernet/intel/ixgbevf/vf.c b/drivers/net/ethernet/intel/ixgbevf/vf.c index 4d613a4..8f277b9 100644 --- a/drivers/net/ethernet/intel/ixgbevf/vf.c +++ b/drivers/net/ethernet/intel/ixgbevf/vf.c @@ -27,6 +27,13 @@ #include "vf.h" #include "ixgbevf.h" +/* + * On Hyper-V, to reset, we need to read from this offset + * from the PCI config space. This is the mechanism used on + * Hyper-V to support PF/VF communication. + */ +#define IXGBE_HV_RESET_OFFSET 0x201 + /** * ixgbevf_start_hw_vf - Prepare hardware for Tx/Rx * @hw: pointer to hardware structure @@ -126,6 +133,27 @@ static s32 ixgbevf_reset_hw_vf(struct ixgbe_hw *hw) } /** + * Hyper-V variant; the VF/PF communication is through the PCI + * config space. + */ +static s32 ixgbevf_hv_reset_hw_vf(struct ixgbe_hw *hw) +{ +#if IS_ENABLED(CONFIG_PCI_MMCONFIG) + struct ixgbevf_adapter *adapter = hw->back; + int i; + + for (i = 0; i < 6; i++) + pci_read_config_byte(adapter->pdev, + (i + IXGBE_HV_RESET_OFFSET), + &hw->mac.perm_addr[i]); +#else + pr_err("PCI_MMCONFIG needs to be enabled for Hyper-V\n"); +#endif + + return 0; +} + +/** * ixgbevf_stop_hw_vf - Generic stop Tx/Rx units * @hw: pointer to hardware structure * @@ -258,6 +286,11 @@ static s32 ixgbevf_set_uc_addr_vf(struct ixgbe_hw *hw, u32 index, u8 *addr) return ret_val; } +static s32 ixgbevf_hv_set_uc_addr_vf(struct ixgbe_hw *hw, u32 index, u8 *addr) +{ + return -EOPNOTSUPP; +} + /** * ixgbevf_get_reta_locked - get the RSS redirection table (RETA) contents. * @adapter: pointer to the port handle @@ -416,6 +449,26 @@ static s32 ixgbevf_set_rar_vf(struct ixgbe_hw *hw, u32 index, u8 *addr, return ret_val; } +/** + * ixgbevf_hv_set_rar_vf - set device MAC address Hyper-V variant + * @hw: pointer to hardware structure + * @index: Receive address register to write + * @addr: Address to put into receive address register + * @vmdq: Unused in this implementation + * + * We don't really allow setting the device MAC address. However, + * if the address being set is the permanent MAC address we will + * permit that. + **/ +static s32 ixgbevf_hv_set_rar_vf(struct ixgbe_hw *hw, u32 index, u8 *addr, + u32 vmdq) +{ + if (ether_addr_equal(addr, hw->mac.perm_addr)) + return 0; + + return -EOPNOTSUPP; +} + static void ixgbevf_write_msg_read_ack(struct ixgbe_hw *hw, u32 *msg, u16 size) { @@ -473,6 +526,15 @@ static s32 ixgbevf_update_mc_addr_list_vf(struct ixgbe_hw *hw, } /** + * Hyper-V variant - just a stub. + */ +static s32 ixgbevf_hv_update_mc_addr_list_vf(struct ixgbe_hw *hw, + struct net_device *netdev) +{ + return -EOPNOTSUPP; +} + +/** * ixgbevf_update_xcast_mode - Update Multicast mode * @hw: pointer to the HW structure * @netdev: pointer to net device structure @@ -513,6 +575,15 @@ static s32 ixgbevf_update_xcast_mode(struct ixgbe_hw *hw, } /** + * Hyper-V variant - just a stub. + */ +static s32 ixgbevf_hv_update_xcast_mode(struct ixgbe_hw *hw, + struct net_device *netdev, int xcast_mode) +{ + return -EOPNOTSUPP; +} + +/** * ixgbevf_set_vfta_vf - Set/Unset VLAN filter table address * @hw: pointer to the HW structure * @vlan: 12 bit VLAN ID @@ -551,6 +622,15 @@ mbx_err: } /** + * Hyper-V variant - just a stub. + */ +static s32 ixgbevf_hv_set_vfta_vf(struct ixgbe_hw *hw, u32 vlan, u32 vind, + bool vlan_on) +{ + return -EOPNOTSUPP; +} + +/** * ixgbevf_setup_mac_link_vf - Setup MAC link settings * @hw: pointer to hardware structure * @speed: Unused in this implementation @@ -656,6 +736,67 @@ out: } /** + * Hyper-V variant; there is no mailbox communication. + */ +static s32 ixgbevf_hv_check_mac_link_vf(struct ixgbe_hw *hw, + ixgbe_link_speed *speed, + bool *link_up, + bool autoneg_wait_to_complete) +{ + struct ixgbe_mbx_info *mbx = &hw->mbx; + struct ixgbe_mac_info *mac = &hw->mac; + u32 links_reg; + + /* If we were hit with a reset drop the link */ + if (!mbx->ops.check_for_rst(hw) || !mbx->timeout) + mac->get_link_status = true; + + if (!mac->get_link_status) + goto out; + + /* if link status is down no point in checking to see if pf is up */ + links_reg = IXGBE_READ_REG(hw, IXGBE_VFLINKS); + if (!(links_reg & IXGBE_LINKS_UP)) + goto out; + + /* for SFP+ modules and DA cables on 82599 it can take up to 500usecs + * before the link status is correct + */ + if (mac->type == ixgbe_mac_82599_vf) { + int i; + + for (i = 0; i < 5; i++) { + udelay(100); + links_reg = IXGBE_READ_REG(hw, IXGBE_VFLINKS); + + if (!(links_reg & IXGBE_LINKS_UP)) + goto out; + } + } + + switch (links_reg & IXGBE_LINKS_SPEED_82599) { + case IXGBE_LINKS_SPEED_10G_82599: + *speed = IXGBE_LINK_SPEED_10GB_FULL; + break; + case IXGBE_LINKS_SPEED_1G_82599: + *speed = IXGBE_LINK_SPEED_1GB_FULL; + break; + case IXGBE_LINKS_SPEED_100_82599: + *speed = IXGBE_LINK_SPEED_100_FULL; + break; + } + + /* if we passed all the tests above then the link is up and we no + * longer need to check for link + */ + mac->get_link_status = false; + +out: + *link_up = !mac->get_link_status; + return 0; +} + +/** * ixgbevf_rlpml_set_vf - Set the maximum receive packet length * @hw: pointer to the HW structure * @max_size: value to assign to max frame size @@ -670,6 +811,25 @@ void ixgbevf_rlpml_set_vf(struct ixgbe_hw *hw, u16 max_size) } /** + * ixgbevf_hv_rlpml_set_vf - Set the maximum receive packet length + * @hw: pointer to the HW structure + * @max_size: value to assign to max frame size + * Hyper-V variant. + **/ +void ixgbevf_hv_rlpml_set_vf(struct ixgbe_hw *hw, u16 max_size) +{ + u32 reg; + + /* If we are on Hyper-V, we implement + * this functionality differently. + */ + reg = IXGBE_READ_REG(hw, IXGBE_VFRXDCTL(0)); + /* CRC == 4 */ + reg |= ((max_size + 4) | IXGBE_RXDCTL_RLPML_EN); + IXGBE_WRITE_REG(hw, IXGBE_VFRXDCTL(0), reg); +} + +/** * ixgbevf_negotiate_api_version - Negotiate supported API version * @hw: pointer to the HW structure * @api: integer containing requested API version @@ -703,6 +863,22 @@ int ixgbevf_negotiate_api_version(struct ixgbe_hw *hw, int api) return err; } +/** + * ixgbevf_hv_negotiate_api_version - Negotiate supported API version + * @hw: pointer to the HW structure + * @api: integer containing requested API version + * Hyper-V version - only ixgbe_mbox_api_10 supported. + **/ +int ixgbevf_hv_negotiate_api_version(struct ixgbe_hw *hw, int api) +{ + /* Hyper-V only supports api version ixgbe_mbox_api_10 + */ + if (api != ixgbe_mbox_api_10) + return IXGBE_ERR_INVALID_ARGUMENT; + + return 0; +} + int ixgbevf_get_queues(struct ixgbe_hw *hw, unsigned int *num_tcs, unsigned int *default_tc) { @@ -776,22 +952,62 @@ static const struct ixgbe_mac_operations ixgbevf_mac_ops = { .set_vfta = ixgbevf_set_vfta_vf, }; +static const struct ixgbe_mac_operations ixgbevf_hv_mac_ops = { + .init_hw = ixgbevf_init_hw_vf, + .reset_hw = ixgbevf_hv_reset_hw_vf, + .start_hw = ixgbevf_start_hw_vf, + .get_mac_addr = ixgbevf_get_mac_addr_vf, + .stop_adapter = ixgbevf_stop_hw_vf, + .setup_link = ixgbevf_setup_mac_link_vf, + .check_link = ixgbevf_hv_check_mac_link_vf, + .set_rar = ixgbevf_hv_set_rar_vf, + .update_mc_addr_list = ixgbevf_hv_update_mc_addr_list_vf, + .update_xcast_mode = ixgbevf_hv_update_xcast_mode, + .set_uc_addr = ixgbevf_hv_set_uc_addr_vf, + .set_vfta = ixgbevf_hv_set_vfta_vf, +}; + const struct ixgbevf_info ixgbevf_82599_vf_info = { .mac = ixgbe_mac_82599_vf, .mac_ops = &ixgbevf_mac_ops, }; +const struct ixgbevf_info ixgbevf_82599_vf_hv_info = { + .mac = ixgbe_mac_82599_vf, + .mac_ops = &ixgbevf_hv_mac_ops, +}; + const struct ixgbevf_info ixgbevf_X540_vf_info = { .mac = ixgbe_mac_X540_vf, .mac_ops = &ixgbevf_mac_ops, }; +const struct ixgbevf_info ixgbevf_X540_vf_hv_info = { + .mac = ixgbe_mac_X540_vf, + .mac_ops = &ixgbevf_hv_mac_ops, +}; + const struct ixgbevf_info ixgbevf_X550_vf_info = { .mac = ixgbe_mac_X550_vf, .mac_ops = &ixgbevf_mac_ops, }; +const struct ixgbevf_info ixgbevf_X550_vf_hv_info = { + .mac = ixgbe_mac_X550_vf, + .mac_ops = &ixgbevf_hv_mac_ops, +}; + const struct ixgbevf_info ixgbevf_X550EM_x_vf_info = { .mac = ixgbe_mac_X550EM_x_vf, .mac_ops = &ixgbevf_mac_ops, }; + +const struct ixgbevf_info ixgbevf_X550EM_x_vf_hv_info = { + .mac = ixgbe_mac_X550EM_x_vf, + .mac_ops = &ixgbevf_hv_mac_ops, +}; + +bool ixgbevf_on_hyperv(struct ixgbe_hw *hw) +{ + return hw->mbx.ops.check_for_msg == NULL; +} diff --git a/drivers/net/ethernet/intel/ixgbevf/vf.h b/drivers/net/ethernet/intel/ixgbevf/vf.h index ef9f773..658883e 100644 --- a/drivers/net/ethernet/intel/ixgbevf/vf.h +++ b/drivers/net/ethernet/intel/ixgbevf/vf.h @@ -208,7 +208,9 @@ static inline u32 ixgbe_read_reg_array(struct ixgbe_hw *hw, u32 reg, #define IXGBE_READ_REG_ARRAY(h, r, o) ixgbe_read_reg_array(h, r, o) void ixgbevf_rlpml_set_vf(struct ixgbe_hw *hw, u16 max_size); +void ixgbevf_hv_rlpml_set_vf(struct ixgbe_hw *hw, u16 max_size); int ixgbevf_negotiate_api_version(struct ixgbe_hw *hw, int api); +int ixgbevf_hv_negotiate_api_version(struct ixgbe_hw *hw, int api); int ixgbevf_get_queues(struct ixgbe_hw *hw, unsigned int *num_tcs, unsigned int *default_tc); int ixgbevf_get_reta_locked(struct ixgbe_hw *hw, u32 *reta, int num_rx_queues);