| Message ID | 20180703102433.24558-1-vaibhav@linux.ibm.com |
|---|---|
| State | Superseded |
| Headers | show |
| Series | phb4/capp: Calculate STQ/DMA read engines based on link-width for PEC2 | expand |
On 03/07/18 20:24, Vaibhav Jain wrote: > Presently in CAPI mode the number of STQ/DMA-read engines allocated on > PEC2 for CAPP is fixed to 6 and 0-30 respectively irrespective of the > PCI link width. These values are only suitable for x8 cards and > quickly run out if a x16 card is plugged to a PEC2 attached slot. This > usually manifests as CAPP reporting TLBI timeout due to these messages > getting stalled due to insufficient STQs. > > To fix this we update enable_capi_mode() to check if PEC2 chiplet is > in x16 mode and if yes then we allocate 4/0-47 STQ/DMA-read engines > for the CAPP traffic. > > Fixes: 37ea3cfdc852("capi: Enable capi mode for PHB4") > Signed-off-by: Vaibhav Jain <vaibhav@linux.ibm.com> This should probably head to stable, right? One minor nitpick below Reviewed-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com> > --- > hw/phb4.c | 35 ++++++++++++++++++++++++++--------- > include/phb4-regs.h | 6 ++++++ > 2 files changed, 32 insertions(+), 9 deletions(-) > > diff --git a/hw/phb4.c b/hw/phb4.c > index ae584d67..62443d7e 100644 > --- a/hw/phb4.c > +++ b/hw/phb4.c > @@ -3918,8 +3918,6 @@ static int64_t enable_capi_mode(struct phb4 *p, uint64_t pe_number, > return OPAL_HARDWARE; > } > > - /* CAPP Control Register. Enable CAPP Mode */ > - reg = 0x8000000000000000ULL; /* PEC works in CAPP Mode */ > if (p->index == CAPP0_PHB_INDEX) { > /* PBCQ is operating as a x16 stack > * - The maximum number of engines give to CAPP will be > @@ -3929,17 +3927,36 @@ static int64_t enable_capi_mode(struct phb4 *p, uint64_t pe_number, > stq_eng = 0x000E000000000000ULL; /* 14 CAPP msg engines */ > dma_eng = 0x0000FFFFFFFFFFFFULL; /* 48 CAPP Read machines */ > } > + > if (p->index == CAPP1_PHB_INDEX) { > - /* PBCQ is operating as a x8 stack > - * - The maximum number of engines given to CAPP should > - * be 6 and will be assigned in the order of 7 to 2. > - * - 0-30 (Read machines) are available for capp use. > - */ > - stq_eng = 0x0006000000000000ULL; /* 6 CAPP msg engines */ > - dma_eng = 0x0000FFFFF00E0000ULL; /* 30 Read machines for CAPP Minus 20-27 for DMA */ > + /* Check if PEC is in x8 or x16 mode */ > + xscom_read(p->chip_id, XPEC_PCI2_CPLT_CONF1, ®); > + > + if ((reg & XPEC_PCI2_IOVALID_MASK) == XPEC_PCI2_IOVALID_X16) { > + /* PBCQ is operating as a x16 stack > + * - The maximum number of engines give to CAPP will be > + * 14 and will be assigned in the order of STQ 15 to 2. > + * - 0-47 (Read machines) are available for capp use. > + */ > + stq_eng = 0x000E000000000000ULL; > + dma_eng = 0x0000FFFFFFFFFFFFULL; > + } else { > + > + /* PBCQ is operating as a x8 stack > + * - The maximum number of engines given to CAPP should > + * be 6 and will be assigned in the order of 7 to 2. > + * - 0-30 (Read machines) are available for capp use. is it worth keeping the "minus 20-27 for DMA" comment in here? > + */ > + stq_eng = 0x0006000000000000ULL; > + dma_eng = 0x0000FFFFF00E0000ULL; > + } > } > + > if (capp_eng & CAPP_MIN_STQ_ENGINES) > stq_eng = 0x0002000000000000ULL; /* 2 capp msg engines */ > + > + /* CAPP Control Register. Enable CAPP Mode */ > + reg = 0x8000000000000000ULL; /* PEC works in CAPP Mode */ > reg |= stq_eng; > if (capp_eng & CAPP_MAX_DMA_READ_ENGINES) > dma_eng = 0x0000FF0000000000ULL; /* 16 CAPP Read machines */ > diff --git a/include/phb4-regs.h b/include/phb4-regs.h > index 3f87ddcd..e7a190ee 100644 > --- a/include/phb4-regs.h > +++ b/include/phb4-regs.h > @@ -400,6 +400,12 @@ > #define XETU_HV_IND_ADDR_AUTOINC PPC_BIT(2) > #define XETU_HV_IND_DATA 0x1 > > + > +/* PCI Chiplet Config Register */ > +#define XPEC_PCI2_CPLT_CONF1 0x000000000F000009ULL > +#define XPEC_PCI2_IOVALID_MASK PPC_BITMASK(4, 6) > +#define XPEC_PCI2_IOVALID_X16 PPC_BIT(4) > + > /* > * IODA3 on-chip tables > */ >
diff --git a/hw/phb4.c b/hw/phb4.c index ae584d67..62443d7e 100644 --- a/hw/phb4.c +++ b/hw/phb4.c @@ -3918,8 +3918,6 @@ static int64_t enable_capi_mode(struct phb4 *p, uint64_t pe_number, return OPAL_HARDWARE; } - /* CAPP Control Register. Enable CAPP Mode */ - reg = 0x8000000000000000ULL; /* PEC works in CAPP Mode */ if (p->index == CAPP0_PHB_INDEX) { /* PBCQ is operating as a x16 stack * - The maximum number of engines give to CAPP will be @@ -3929,17 +3927,36 @@ static int64_t enable_capi_mode(struct phb4 *p, uint64_t pe_number, stq_eng = 0x000E000000000000ULL; /* 14 CAPP msg engines */ dma_eng = 0x0000FFFFFFFFFFFFULL; /* 48 CAPP Read machines */ } + if (p->index == CAPP1_PHB_INDEX) { - /* PBCQ is operating as a x8 stack - * - The maximum number of engines given to CAPP should - * be 6 and will be assigned in the order of 7 to 2. - * - 0-30 (Read machines) are available for capp use. - */ - stq_eng = 0x0006000000000000ULL; /* 6 CAPP msg engines */ - dma_eng = 0x0000FFFFF00E0000ULL; /* 30 Read machines for CAPP Minus 20-27 for DMA */ + /* Check if PEC is in x8 or x16 mode */ + xscom_read(p->chip_id, XPEC_PCI2_CPLT_CONF1, ®); + + if ((reg & XPEC_PCI2_IOVALID_MASK) == XPEC_PCI2_IOVALID_X16) { + /* PBCQ is operating as a x16 stack + * - The maximum number of engines give to CAPP will be + * 14 and will be assigned in the order of STQ 15 to 2. + * - 0-47 (Read machines) are available for capp use. + */ + stq_eng = 0x000E000000000000ULL; + dma_eng = 0x0000FFFFFFFFFFFFULL; + } else { + + /* PBCQ is operating as a x8 stack + * - The maximum number of engines given to CAPP should + * be 6 and will be assigned in the order of 7 to 2. + * - 0-30 (Read machines) are available for capp use. + */ + stq_eng = 0x0006000000000000ULL; + dma_eng = 0x0000FFFFF00E0000ULL; + } } + if (capp_eng & CAPP_MIN_STQ_ENGINES) stq_eng = 0x0002000000000000ULL; /* 2 capp msg engines */ + + /* CAPP Control Register. Enable CAPP Mode */ + reg = 0x8000000000000000ULL; /* PEC works in CAPP Mode */ reg |= stq_eng; if (capp_eng & CAPP_MAX_DMA_READ_ENGINES) dma_eng = 0x0000FF0000000000ULL; /* 16 CAPP Read machines */ diff --git a/include/phb4-regs.h b/include/phb4-regs.h index 3f87ddcd..e7a190ee 100644 --- a/include/phb4-regs.h +++ b/include/phb4-regs.h @@ -400,6 +400,12 @@ #define XETU_HV_IND_ADDR_AUTOINC PPC_BIT(2) #define XETU_HV_IND_DATA 0x1 + +/* PCI Chiplet Config Register */ +#define XPEC_PCI2_CPLT_CONF1 0x000000000F000009ULL +#define XPEC_PCI2_IOVALID_MASK PPC_BITMASK(4, 6) +#define XPEC_PCI2_IOVALID_X16 PPC_BIT(4) + /* * IODA3 on-chip tables */
Presently in CAPI mode the number of STQ/DMA-read engines allocated on PEC2 for CAPP is fixed to 6 and 0-30 respectively irrespective of the PCI link width. These values are only suitable for x8 cards and quickly run out if a x16 card is plugged to a PEC2 attached slot. This usually manifests as CAPP reporting TLBI timeout due to these messages getting stalled due to insufficient STQs. To fix this we update enable_capi_mode() to check if PEC2 chiplet is in x16 mode and if yes then we allocate 4/0-47 STQ/DMA-read engines for the CAPP traffic. Fixes: 37ea3cfdc852("capi: Enable capi mode for PHB4") Signed-off-by: Vaibhav Jain <vaibhav@linux.ibm.com> --- hw/phb4.c | 35 ++++++++++++++++++++++++++--------- include/phb4-regs.h | 6 ++++++ 2 files changed, 32 insertions(+), 9 deletions(-)