From patchwork Tue Oct 29 02:11:06 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jordan Niethe X-Patchwork-Id: 1185803 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 472FVJ1Q0Hz9sP3 for ; Tue, 29 Oct 2019 13:12:56 +1100 (AEDT) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="CCNnEEEP"; dkim-atps=neutral Received: from bilbo.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 472FVH6t2YzF07t for ; Tue, 29 Oct 2019 13:12:55 +1100 (AEDT) X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gmail.com (client-ip=2607:f8b0:4864:20::443; helo=mail-pf1-x443.google.com; envelope-from=jniethe5@gmail.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="CCNnEEEP"; dkim-atps=neutral Received: from mail-pf1-x443.google.com (mail-pf1-x443.google.com [IPv6:2607:f8b0:4864:20::443]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 472FT62sFBzDsFk for ; Tue, 29 Oct 2019 13:11:54 +1100 (AEDT) Received: by mail-pf1-x443.google.com with SMTP id v4so8365234pff.6 for ; Mon, 28 Oct 2019 19:11:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=d7pE1f3AO4sjRfLcsm3FyvwfhQ4s1AGUa07bV+q/U/0=; b=CCNnEEEPHRC55QU2pHfLtOg9YgQWH7BlcElHtHKc0KdnH4IEZO3A83w4gXpmL8Q0t9 hCia9wHTGk2BFpuT0k7776UA05mRCgI4vANHxvZIKODZ8e7FpmXfCrMh8wjPI3D1idsS yf9U6cdbLFTPjgGiJfuK/lmNNqafL4zySo27VoIbRZFFc7DSVHigwXYIlVVXeBK12eIi rjCq8KKpOzYFUVZaxWRMYK//BdSRruPRS9fJrMKRjiljjgpVtt9hch4zZYwkoLMv6q8P c8hKAa68uqN6gJVlwAatXYW6ASZKkvs+hogEX13wyZqOEeYh04t/CqddniCxVqXMfvkM ZCAw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=d7pE1f3AO4sjRfLcsm3FyvwfhQ4s1AGUa07bV+q/U/0=; b=JMH9qjfHm/GBh503SKTYYCgBvo8tXJs8Gi4xKDfOdBlvwM05RIzfJdGAM82YkI4Ecz Zs9l+1ZEb2dh5e/0T0pqExh3LcxX2VhxHAx/+dwBe8X59cxyTMqlN4beou/la4La7RfZ JCeQub8BGXnZS728vaOUAcaN/1HLmd/j7mSOqTGbDHpjPdaIo14BeneGhhlL1S5M5prg 56ka8fLHvOlDr18tU7k9RxHXXvs4zly4K7mJLis2CRETJtIbwOjm+1yWHVzjfdi0atFQ j8rdvsOLEKT4DA6gy91adkRTsGs1RwfWIAOdHTiHOv2FBiqu9WxXLZHWzzpwrQaR6N56 9zug== X-Gm-Message-State: APjAAAUZOZrs3awSxu8ak2eTXssoAEc1hoDRfdKtxu0pFiBWzwSDxQU3 64hWtywtWIQIANSfaz2RM//1fhkK X-Google-Smtp-Source: APXvYqwcnoLqV4WJo0zNxc/EJeUCRVlSA7JjTT5zK5H1uzpxbY/OYWldhmgzBNo4KJ2SEIivlbraMg== X-Received: by 2002:a62:ce4a:: with SMTP id y71mr157985pfg.54.1572315112421; Mon, 28 Oct 2019 19:11:52 -0700 (PDT) Received: from sol.ozlabs.ibm.com ([122.99.82.10]) by smtp.gmail.com with ESMTPSA id w14sm13710999pge.56.2019.10.28.19.11.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 28 Oct 2019 19:11:52 -0700 (PDT) From: Jordan Niethe To: skiboot@lists.ozlabs.org Date: Tue, 29 Oct 2019 13:11:06 +1100 Message-Id: <20191029021106.29396-4-jniethe5@gmail.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20191029021106.29396-1-jniethe5@gmail.com> References: <20191029021106.29396-1-jniethe5@gmail.com> MIME-Version: 1.0 Subject: [Skiboot] [RFC PATCH v2 3/3] core/pcie-slot: Restore slot power controller state X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" After a fundamental reset we currently assume that each slot's power controllers maintain their correct state. This is an invalid assumption. We need to make sure that this state gets restored. If the power controller does not get turned on but we think it is on things will go badly. We can handle this with extra states in our restore_buses() state machine function. If a PCI devices is behind a switch, check for a power controller and if present restore the power state before waiting for the link to activate. Signed-off-by: Jordan Niethe --- v2: - Make sure we actually change to PHB4_SLOT_BUSES_PD_WAIT_SWITCH_OFF state - Use get_power_state() for reading the slot power state core/pcie-slot.c | 55 ++++++++++++++++++++++++++++++++++++++++++++-- include/pci-slot.h | 8 ++++--- 2 files changed, 58 insertions(+), 5 deletions(-) diff --git a/core/pcie-slot.c b/core/pcie-slot.c index 048a8b279c71..06e80ee3d689 100644 --- a/core/pcie-slot.c +++ b/core/pcie-slot.c @@ -267,6 +267,9 @@ int64_t pcie_slot_sm_restore_buses(struct pci_slot *slot) struct pci_device *pd, *parent; uint32_t link_cap = 0; uint16_t link_sts = 0; + uint32_t slot_cap = 0; + uint16_t slot_ctl = 0; + uint8_t power_state; int32_t ecap = 0; struct phb *phb; uint32_t vdid; @@ -296,9 +299,8 @@ int64_t pcie_slot_sm_restore_buses(struct pci_slot *slot) if (!pd->is_vf && !(pd->bdfn & 7) && pd->parent != NULL && pd->parent->dev_type == PCIE_TYPE_SWITCH_DNPORT) { PCIDBG(phb, pd->bdfn, "BUSES: Behind a switch\n"); - slot->retries = PCI_BUS_SWITCH_LINK_RETRIES; pci_slot_set_state(slot, - PCI_SLOT_BUSES_PD_WAIT_SWITCH_LINK); + PCI_SLOT_BUSES_PD_SWITCH_POWER); return pci_slot_set_sm_timeout(slot, msecs_to_tb(1)); } else { PCIDBG(phb, pd->bdfn, "BUSES: Not behind a switch\n"); @@ -306,6 +308,55 @@ int64_t pcie_slot_sm_restore_buses(struct pci_slot *slot) pci_slot_set_state(slot, PCI_SLOT_BUSES_PD_WAIT_CRS); return pci_slot_set_sm_timeout(slot, msecs_to_tb(1)); } + case PCI_SLOT_BUSES_PD_SWITCH_POWER: + if (pci_has_cap(parent, PCI_CFG_CAP_ID_EXP, false)) { + ecap = pci_cap(parent, PCI_CFG_CAP_ID_EXP, false); + pci_cfg_read32(phb, parent->bdfn, + ecap + PCICAP_EXP_SLOTCAP, &slot_cap); + } + + if (!(slot_cap & PCICAP_EXP_SLOTCAP_PWCTRL)) { + PCIDBG(phb, parent->bdfn, + "BUSES: Parent: No power control\n"); + slot->retries = PCI_BUS_SWITCH_LINK_RETRIES; + pci_slot_set_state(slot, PCI_SLOT_BUSES_PD_WAIT_SWITCH_LINK); + return pci_slot_set_sm_timeout(slot, msecs_to_tb(1)); + } + + PCIDBG(phb, parent->bdfn, "BUSES: Parent: Have power control\n"); + /* Always turn it off */ + PCIDBG(phb, parent->bdfn, "BUSES: Parent: Turning slot off\n"); + pci_cfg_read16(phb, parent->bdfn, + ecap + PCICAP_EXP_SLOTCTL, &slot_ctl); + slot_ctl |= PCICAP_EXP_SLOTCTL_PWRCTLR; + slot_ctl |= SETFIELD(PCICAP_EXP_SLOTCTL_PWRI, 0, PCIE_INDIC_OFF); + pci_cfg_write16(phb, parent->bdfn, + ecap + PCICAP_EXP_SLOTCTL, slot_ctl); + pci_slot_set_state(slot, PCI_SLOT_BUSES_PD_WAIT_SWITCH_OFF); + return pci_slot_set_sm_timeout(slot, secs_to_tb(1)); + case PCI_SLOT_BUSES_PD_WAIT_SWITCH_OFF: + parent->slot->ops.get_power_state(parent->slot, &power_state); + if (power_state == PCI_SLOT_POWER_OFF) { + PCIDBG(phb, parent->bdfn, + "BUSES: Parent: Slot power state is off. Nothing to do\n"); + slot->phb->current_pd = pci_device_iter_next(slot->phb, + NULL); + pci_slot_set_state(slot, PCI_SLOT_BUSES_PD_START); + return pci_slot_set_sm_timeout(slot, msecs_to_tb(1)); + } + + PCIDBG(phb, parent->bdfn, "BUSES: Parent: Turning slot on\n"); + ecap = pci_cap(parent, PCI_CFG_CAP_ID_EXP, false); + pci_cfg_read16(phb, parent->bdfn, ecap + PCICAP_EXP_SLOTCTL, + &slot_ctl); + slot_ctl &= ~PCICAP_EXP_SLOTCTL_PWRCTLR; + slot_ctl |= SETFIELD(PCICAP_EXP_SLOTCTL_PWRI, 0, PCIE_INDIC_ON); + pci_cfg_write16(phb, parent->bdfn, + ecap + PCICAP_EXP_SLOTCTL, slot_ctl); + + slot->retries = PCI_BUS_SWITCH_LINK_RETRIES; + pci_slot_set_state(slot, PCI_SLOT_BUSES_PD_WAIT_SWITCH_LINK); + return pci_slot_set_sm_timeout(slot, msecs_to_tb(1)); case PCI_SLOT_BUSES_PD_WAIT_SWITCH_LINK: PCIDBG(phb, pd->bdfn, "BUSES: Wait for Switch Link\n"); diff --git a/include/pci-slot.h b/include/pci-slot.h index 353efc7e96e1..a95a387bf6aa 100644 --- a/include/pci-slot.h +++ b/include/pci-slot.h @@ -134,9 +134,11 @@ struct pci_slot_ops { #define PCI_SLOT_BUSES PCI_SLOT_STATE_BUSES #define PCI_SLOT_BUSES_START (PCI_SLOT_BUSES + 1) #define PCI_SLOT_BUSES_PD_START (PCI_SLOT_BUSES + 2) -#define PCI_SLOT_BUSES_PD_WAIT_SWITCH_LINK (PCI_SLOT_BUSES + 3) -#define PCI_SLOT_BUSES_PD_WAIT_CRS (PCI_SLOT_BUSES + 4) -#define PCI_SLOT_BUSES_PD_END (PCI_SLOT_BUSES + 5) +#define PCI_SLOT_BUSES_PD_SWITCH_POWER (PCI_SLOT_BUSES + 3) +#define PCI_SLOT_BUSES_PD_WAIT_SWITCH_OFF (PCI_SLOT_BUSES + 4) +#define PCI_SLOT_BUSES_PD_WAIT_SWITCH_LINK (PCI_SLOT_BUSES + 5) +#define PCI_SLOT_BUSES_PD_WAIT_CRS (PCI_SLOT_BUSES + 6) +#define PCI_SLOT_BUSES_PD_END (PCI_SLOT_BUSES + 7) #define PCI_BUS_CRS_RETRIES 40 #define PCI_BUS_SWITCH_LINK_RETRIES 100