| Message ID | 20240621020608.28964-1-zhoushengqing@ttyinfo.com |
|---|---|
| State | New |
| Headers | show |
| Series | PCI: Enable io space 1k granularity for intel cpu root port | expand |
On Fri, Jun 21, 2024 at 02:06:08AM +0000, Zhou Shengqing wrote: > This patch add 1k granularity for intel root port bridge.Intel latest > server CPU support 1K granularity,And there is an BIOS setup item named > "EN1K",but linux doesn't support it. if an IIO has 5 IOU (SPR has 5 IOUs) > all are bifurcated 2x8.In a 2P server system,There are 20 P2P bridges > present.if keep 4K granularity allocation,it need 20*4=80k io space, > exceeding 64k.I test it in a 16*nvidia 4090s system under intel eaglestrem > platform.There are six 4090s that cannot be allocated I/O resources. > So I applied this patch.And I found a similar implementation in quirks.c, > but it only targets the Intel P64H2 platform. > > Signed-off-by: Zhou Shengqing <zhoushengqing@ttyinfo.com> > --- > drivers/pci/probe.c | 22 ++++++++++++++++++++++ > 1 file changed, 22 insertions(+) > > diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c > index 5fbabb4e3425..3f0c901c6653 100644 > --- a/drivers/pci/probe.c > +++ b/drivers/pci/probe.c > @@ -461,6 +461,8 @@ static void pci_read_bridge_windows(struct pci_dev *bridge) > u32 buses; > u16 io; > u32 pmem, tmp; > + u16 ven_id, dev_id; > + u16 en1k = 0; > struct resource res; > > pci_read_config_dword(bridge, PCI_PRIMARY_BUS, &buses); > @@ -478,6 +480,26 @@ static void pci_read_bridge_windows(struct pci_dev *bridge) > } > if (io) { > bridge->io_window = 1; > + if (pci_is_root_bus(bridge->bus)) { > + list_for_each_entry(dev, &bridge->bus->devices, bus_list) { > + pci_read_config_word(dev, PCI_VENDOR_ID, &ven_id); > + pci_read_config_word(dev, PCI_DEVICE_ID, &dev_id); > + if (ven_id == PCI_VENDOR_ID_INTEL && dev_id == 0x09a2) { > + /*IIO MISC Control offset 0x1c0*/ > + pci_read_config_word(dev, 0x1c0, &en1k); > + } > + } > + /* > + *Intel ICX SPR EMR GNR > + *IIO MISC Control (IIOMISCCTRL_1_5_0_CFG) — Offset 1C0h > + *bit 2:Enable 1K (EN1K) > + *This bit when set, enables 1K granularity for I/O space decode > + *in each of the virtual P2P bridges > + *corresponding to root ports, and DMI ports. > + */ > + if (en1k & 0x4) > + bridge->io_window_1k = 1; > + } Can you implement this as a quirk similar to quirk_p64h2_1k_io()? I don't want to clutter the generic code with device-specific things like this. > pci_read_bridge_io(bridge, &res, true); > } > > -- > 2.39.2 >
>> This patch add 1k granularity for intel root port bridge.Intel latest >> server CPU support 1K granularity,And there is an BIOS setup item named >> "EN1K",but linux doesn't support it. if an IIO has 5 IOU (SPR has 5 IOUs) >> all are bifurcated 2x8.In a 2P server system,There are 20 P2P bridges >> present.if keep 4K granularity allocation,it need 20*4=80k io space, >> exceeding 64k.I test it in a 16*nvidia 4090s system under intel eaglestrem >> platform.There are six 4090s that cannot be allocated I/O resources. >> So I applied this patch.And I found a similar implementation in quirks.c, >> but it only targets the Intel P64H2 platform. >> >> Signed-off-by: Zhou Shengqing <zhoushengqing@ttyinfo.com> >> --- >> drivers/pci/probe.c | 22 ++++++++++++++++++++++ >> 1 file changed, 22 insertions(+) >> >> diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c >> index 5fbabb4e3425..3f0c901c6653 100644 >> --- a/drivers/pci/probe.c >> +++ b/drivers/pci/probe.c >> @@ -461,6 +461,8 @@ static void pci_read_bridge_windows(struct pci_dev *bridge) >> u32 buses; >> u16 io; >> u32 pmem, tmp; >> + u16 ven_id, dev_id; >> + u16 en1k = 0; >> struct resource res; >> >> pci_read_config_dword(bridge, PCI_PRIMARY_BUS, &buses); >> @@ -478,6 +480,26 @@ static void pci_read_bridge_windows(struct pci_dev *bridge) >> } >> if (io) { >> bridge->io_window = 1; >> + if (pci_is_root_bus(bridge->bus)) { >> + list_for_each_entry(dev, &bridge->bus->devices, bus_list) { >> + pci_read_config_word(dev, PCI_VENDOR_ID, &ven_id); >> + pci_read_config_word(dev, PCI_DEVICE_ID, &dev_id); >> + if (ven_id == PCI_VENDOR_ID_INTEL && dev_id == 0x09a2) { >> + /*IIO MISC Control offset 0x1c0*/ >> + pci_read_config_word(dev, 0x1c0, &en1k); >> + } >> + } >> + /* >> + *Intel ICX SPR EMR GNR >> + *IIO MISC Control (IIOMISCCTRL_1_5_0_CFG) — Offset 1C0h >> + *bit 2:Enable 1K (EN1K) >> + *This bit when set, enables 1K granularity for I/O space decode >> + *in each of the virtual P2P bridges >> + *corresponding to root ports, and DMI ports. >> + */ >> + if (en1k & 0x4) >> + bridge->io_window_1k = 1; >> + } > >Can you implement this as a quirk similar to quirk_p64h2_1k_io()? > >I don't want to clutter the generic code with device-specific things >like this. I have attempted to implement this patch in quirks.c.But there doesn't seem to be a suitable DECLARE_PCI_FIXUP* to do this.because the patch is not targeting the device itself, It targets other P2P devices with the same bus number. Any other suggestions? Thanks. > >> pci_read_bridge_io(bridge, &res, true); >> } >> >> -- >> 2.39.2 >> > > -- > 2.39.2 >
On Sat, Jun 22, 2024 at 11:06:18PM +0800, zhoushengqing@ttyinfo.com wrote: > >> This patch add 1k granularity for intel root port bridge.Intel latest > > > > >> server CPU support 1K granularity,And there is an BIOS setup item named > > > I don't know what your email agent is doing to add all these extra blank lines, but it makes it painful to read/reply: https://lore.kernel.org/all/2024062223061743562815@ttyinfo.com/ > >Can you implement this as a quirk similar to quirk_p64h2_1k_io()? > > > >I don't want to clutter the generic code with device-specific > >things like this. > > I have attempted to implement this patch in quirks.c.But there > doesn't seem to be a suitable DECLARE_PCI_FIXUP* to do this.because > the patch is not targeting the device itself, It targets other P2P > devices with the same bus number. If I understand the patch correctly, if a [8086:09a2] device on a root bus has EN1K set, *every* bridge (every Root Port in this case because I assume this is a PCIe configuration) on the same bus supports 1K granularity? That seems like a really broken kind of encapsulation. I'd be surprised if there were not a bit in each of those Root Ports that indicates this.
>On Sat, Jun 22, 2024 at 11:06:18PM +0800, zhoushengqing@ttyinfo.com wrote: >> >> This patch add 1k granularity for intel root port bridge.Intel latest >> >> >> >> >> server CPU support 1K granularity,And there is an BIOS setup item named >> >> >> > >I don't know what your email agent is doing to add all these extra >blank lines, but it makes it painful to read/reply: >https://lore.kernel.org/all/2024062223061743562815@ttyinfo.com/ > I'm sorry for the mistake I made in my email client settings. >> >Can you implement this as a quirk similar to quirk_p64h2_1k_io()? >> > >> >I don't want to clutter the generic code with device-specific >> >things like this. >> >> I have attempted to implement this patch in quirks.c.But there >> doesn't seem to be a suitable DECLARE_PCI_FIXUP* to do this.because >> the patch is not targeting the device itself, It targets other P2P >> devices with the same bus number. > >If I understand the patch correctly, if a [8086:09a2] device on a root >bus has EN1K set, *every* bridge (every Root Port in this case because >I assume this is a PCIe configuration) on the same bus supports 1K >granularity? > >That seems like a really broken kind of encapsulation. I'd be >surprised if there were not a bit in each of those Root Ports that >indicates this. Your understanding is completely correct.intel ICX SPR RMR (even GNR) CPU EDS Vol2(register) spec says "This bit when set, enables 1K granularity for I/O space decode in each of the virtual P2P bridges corresponding to root ports,and DMI ports." it targets all P2P bridges within the same root bus, not the root port itself.The root ports configuration space doesn't have a "EN1K" bit.
> > That seems like a really broken kind of encapsulation. I'd be > > surprised if there were not a bit in each of those Root Ports that > > indicates this. > Your understanding is completely correct.intel ICX SPR RMR (even GNR) > CPU EDS Vol2(register) spec says "This bit when set, enables 1K granularity > for I/O space decode in each of the virtual P2P bridges corresponding to > root ports,and DMI ports." it targets all P2P bridges within the same root bus, > not the root port itself.The root ports configuration space doesn't have a > "EN1K" bit. For this reason, it doesn't seem feasible to implement this patch in fixup.c or quirks.c.Would it be reasonable to implement this patch in the pcibios_fixup_bus function in /x86/pci/common.c? I also think adding this patch in pci/probe.c is not a good idea.
Hi Zhou,
kernel test robot noticed the following build errors:
[auto build test ERROR on pci/next]
[also build test ERROR on pci/for-linus linus/master v6.10-rc5 next-20240625]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Zhou-Shengqing/PCI-Enable-io-space-1k-granularity-for-intel-cpu-root-port/20240625-161818
base: https://git.kernel.org/pub/scm/linux/kernel/git/pci/pci.git next
patch link: https://lore.kernel.org/r/20240621020608.28964-1-zhoushengqing%40ttyinfo.com
patch subject: [PATCH] PCI: Enable io space 1k granularity for intel cpu root port
config: x86_64-defconfig (https://download.01.org/0day-ci/archive/20240626/202406261552.dr7kOKOM-lkp@intel.com/config)
compiler: gcc-13 (Ubuntu 13.2.0-4ubuntu3) 13.2.0
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20240626/202406261552.dr7kOKOM-lkp@intel.com/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202406261552.dr7kOKOM-lkp@intel.com/
All errors (new ones prefixed by >>):
In file included from include/linux/smp.h:12,
from include/linux/lockdep.h:14,
from include/linux/spinlock.h:63,
from include/linux/sched.h:2142,
from include/linux/delay.h:23,
from drivers/pci/probe.c:7:
drivers/pci/probe.c: In function 'pci_read_bridge_windows':
>> drivers/pci/probe.c:484:45: error: 'dev' undeclared (first use in this function); did you mean 'cdev'?
484 | list_for_each_entry(dev, &bridge->bus->devices, bus_list) {
| ^~~
include/linux/list.h:778:14: note: in definition of macro 'list_for_each_entry'
778 | for (pos = list_first_entry(head, typeof(*pos), member); \
| ^~~
drivers/pci/probe.c:484:45: note: each undeclared identifier is reported only once for each function it appears in
484 | list_for_each_entry(dev, &bridge->bus->devices, bus_list) {
| ^~~
include/linux/list.h:778:14: note: in definition of macro 'list_for_each_entry'
778 | for (pos = list_first_entry(head, typeof(*pos), member); \
| ^~~
In file included from include/linux/container_of.h:5,
from include/linux/kernel.h:22,
from drivers/pci/probe.c:6:
include/linux/compiler_types.h:428:27: error: expression in static assertion is not an integer
428 | #define __same_type(a, b) __builtin_types_compatible_p(typeof(a), typeof(b))
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
include/linux/build_bug.h:78:56: note: in definition of macro '__static_assert'
78 | #define __static_assert(expr, msg, ...) _Static_assert(expr, msg)
| ^~~~
include/linux/container_of.h:20:9: note: in expansion of macro 'static_assert'
20 | static_assert(__same_type(*(ptr), ((type *)0)->member) || \
| ^~~~~~~~~~~~~
include/linux/container_of.h:20:23: note: in expansion of macro '__same_type'
20 | static_assert(__same_type(*(ptr), ((type *)0)->member) || \
| ^~~~~~~~~~~
include/linux/list.h:601:9: note: in expansion of macro 'container_of'
601 | container_of(ptr, type, member)
| ^~~~~~~~~~~~
include/linux/list.h:612:9: note: in expansion of macro 'list_entry'
612 | list_entry((ptr)->next, type, member)
| ^~~~~~~~~~
include/linux/list.h:778:20: note: in expansion of macro 'list_first_entry'
778 | for (pos = list_first_entry(head, typeof(*pos), member); \
| ^~~~~~~~~~~~~~~~
drivers/pci/probe.c:484:25: note: in expansion of macro 'list_for_each_entry'
484 | list_for_each_entry(dev, &bridge->bus->devices, bus_list) {
| ^~~~~~~~~~~~~~~~~~~
include/linux/compiler_types.h:428:27: error: expression in static assertion is not an integer
428 | #define __same_type(a, b) __builtin_types_compatible_p(typeof(a), typeof(b))
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
include/linux/build_bug.h:78:56: note: in definition of macro '__static_assert'
78 | #define __static_assert(expr, msg, ...) _Static_assert(expr, msg)
| ^~~~
include/linux/container_of.h:20:9: note: in expansion of macro 'static_assert'
20 | static_assert(__same_type(*(ptr), ((type *)0)->member) || \
| ^~~~~~~~~~~~~
include/linux/container_of.h:20:23: note: in expansion of macro '__same_type'
20 | static_assert(__same_type(*(ptr), ((type *)0)->member) || \
| ^~~~~~~~~~~
include/linux/list.h:601:9: note: in expansion of macro 'container_of'
601 | container_of(ptr, type, member)
| ^~~~~~~~~~~~
include/linux/list.h:645:9: note: in expansion of macro 'list_entry'
645 | list_entry((pos)->member.next, typeof(*(pos)), member)
| ^~~~~~~~~~
include/linux/list.h:780:20: note: in expansion of macro 'list_next_entry'
780 | pos = list_next_entry(pos, member))
| ^~~~~~~~~~~~~~~
drivers/pci/probe.c:484:25: note: in expansion of macro 'list_for_each_entry'
484 | list_for_each_entry(dev, &bridge->bus->devices, bus_list) {
| ^~~~~~~~~~~~~~~~~~~
vim +484 drivers/pci/probe.c
458
459 static void pci_read_bridge_windows(struct pci_dev *bridge)
460 {
461 u32 buses;
462 u16 io;
463 u32 pmem, tmp;
464 u16 ven_id, dev_id;
465 u16 en1k = 0;
466 struct resource res;
467
468 pci_read_config_dword(bridge, PCI_PRIMARY_BUS, &buses);
469 res.flags = IORESOURCE_BUS;
470 res.start = (buses >> 8) & 0xff;
471 res.end = (buses >> 16) & 0xff;
472 pci_info(bridge, "PCI bridge to %pR%s\n", &res,
473 bridge->transparent ? " (subtractive decode)" : "");
474
475 pci_read_config_word(bridge, PCI_IO_BASE, &io);
476 if (!io) {
477 pci_write_config_word(bridge, PCI_IO_BASE, 0xe0f0);
478 pci_read_config_word(bridge, PCI_IO_BASE, &io);
479 pci_write_config_word(bridge, PCI_IO_BASE, 0x0);
480 }
481 if (io) {
482 bridge->io_window = 1;
483 if (pci_is_root_bus(bridge->bus)) {
> 484 list_for_each_entry(dev, &bridge->bus->devices, bus_list) {
485 pci_read_config_word(dev, PCI_VENDOR_ID, &ven_id);
486 pci_read_config_word(dev, PCI_DEVICE_ID, &dev_id);
487 if (ven_id == PCI_VENDOR_ID_INTEL && dev_id == 0x09a2) {
488 /*IIO MISC Control offset 0x1c0*/
489 pci_read_config_word(dev, 0x1c0, &en1k);
490 }
491 }
492 /*
493 *Intel ICX SPR EMR GNR
494 *IIO MISC Control (IIOMISCCTRL_1_5_0_CFG) — Offset 1C0h
495 *bit 2:Enable 1K (EN1K)
496 *This bit when set, enables 1K granularity for I/O space decode
497 *in each of the virtual P2P bridges
498 *corresponding to root ports, and DMI ports.
499 */
500 if (en1k & 0x4)
501 bridge->io_window_1k = 1;
502 }
503 pci_read_bridge_io(bridge, &res, true);
504 }
505
506 pci_read_bridge_mmio(bridge, &res, true);
507
508 /*
509 * DECchip 21050 pass 2 errata: the bridge may miss an address
510 * disconnect boundary by one PCI data phase. Workaround: do not
511 * use prefetching on this device.
512 */
513 if (bridge->vendor == PCI_VENDOR_ID_DEC && bridge->device == 0x0001)
514 return;
515
516 pci_read_config_dword(bridge, PCI_PREF_MEMORY_BASE, &pmem);
517 if (!pmem) {
518 pci_write_config_dword(bridge, PCI_PREF_MEMORY_BASE,
519 0xffe0fff0);
520 pci_read_config_dword(bridge, PCI_PREF_MEMORY_BASE, &pmem);
521 pci_write_config_dword(bridge, PCI_PREF_MEMORY_BASE, 0x0);
522 }
523 if (!pmem)
524 return;
525
526 bridge->pref_window = 1;
527
528 if ((pmem & PCI_PREF_RANGE_TYPE_MASK) == PCI_PREF_RANGE_TYPE_64) {
529
530 /*
531 * Bridge claims to have a 64-bit prefetchable memory
532 * window; verify that the upper bits are actually
533 * writable.
534 */
535 pci_read_config_dword(bridge, PCI_PREF_BASE_UPPER32, &pmem);
536 pci_write_config_dword(bridge, PCI_PREF_BASE_UPPER32,
537 0xffffffff);
538 pci_read_config_dword(bridge, PCI_PREF_BASE_UPPER32, &tmp);
539 pci_write_config_dword(bridge, PCI_PREF_BASE_UPPER32, pmem);
540 if (tmp)
541 bridge->pref_64_window = 1;
542 }
543
544 pci_read_bridge_mmio_pref(bridge, &res, true);
545 }
546
Hi Zhou, kernel test robot noticed the following build errors: [auto build test ERROR on pci/next] [also build test ERROR on pci/for-linus linus/master v6.10-rc5 next-20240625] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in https://git-scm.com/docs/git-format-patch#_base_tree_information] url: https://github.com/intel-lab-lkp/linux/commits/Zhou-Shengqing/PCI-Enable-io-space-1k-granularity-for-intel-cpu-root-port/20240625-161818 base: https://git.kernel.org/pub/scm/linux/kernel/git/pci/pci.git next patch link: https://lore.kernel.org/r/20240621020608.28964-1-zhoushengqing%40ttyinfo.com patch subject: [PATCH] PCI: Enable io space 1k granularity for intel cpu root port config: x86_64-rhel-8.3-rust (https://download.01.org/0day-ci/archive/20240626/202406261735.9Fu2z2ic-lkp@intel.com/config) compiler: clang version 18.1.5 (https://github.com/llvm/llvm-project 617a15a9eac96088ae5e9134248d8236e34b91b1) reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20240626/202406261735.9Fu2z2ic-lkp@intel.com/reproduce) If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags | Reported-by: kernel test robot <lkp@intel.com> | Closes: https://lore.kernel.org/oe-kbuild-all/202406261735.9Fu2z2ic-lkp@intel.com/ All errors (new ones prefixed by >>): >> drivers/pci/probe.c:484:24: error: use of undeclared identifier 'dev' 484 | list_for_each_entry(dev, &bridge->bus->devices, bus_list) { | ^ >> drivers/pci/probe.c:484:24: error: use of undeclared identifier 'dev' >> drivers/pci/probe.c:484:24: error: use of undeclared identifier 'dev' >> drivers/pci/probe.c:484:24: error: use of undeclared identifier 'dev' >> drivers/pci/probe.c:484:24: error: use of undeclared identifier 'dev' >> drivers/pci/probe.c:484:24: error: use of undeclared identifier 'dev' >> drivers/pci/probe.c:484:24: error: use of undeclared identifier 'dev' >> drivers/pci/probe.c:484:24: error: use of undeclared identifier 'dev' >> drivers/pci/probe.c:484:24: error: use of undeclared identifier 'dev' drivers/pci/probe.c:485:26: error: use of undeclared identifier 'dev' 485 | pci_read_config_word(dev, PCI_VENDOR_ID, &ven_id); | ^ drivers/pci/probe.c:486:26: error: use of undeclared identifier 'dev' 486 | pci_read_config_word(dev, PCI_DEVICE_ID, &dev_id); | ^ drivers/pci/probe.c:489:27: error: use of undeclared identifier 'dev' 489 | pci_read_config_word(dev, 0x1c0, &en1k); | ^ 12 errors generated. vim +/dev +484 drivers/pci/probe.c 458 459 static void pci_read_bridge_windows(struct pci_dev *bridge) 460 { 461 u32 buses; 462 u16 io; 463 u32 pmem, tmp; 464 u16 ven_id, dev_id; 465 u16 en1k = 0; 466 struct resource res; 467 468 pci_read_config_dword(bridge, PCI_PRIMARY_BUS, &buses); 469 res.flags = IORESOURCE_BUS; 470 res.start = (buses >> 8) & 0xff; 471 res.end = (buses >> 16) & 0xff; 472 pci_info(bridge, "PCI bridge to %pR%s\n", &res, 473 bridge->transparent ? " (subtractive decode)" : ""); 474 475 pci_read_config_word(bridge, PCI_IO_BASE, &io); 476 if (!io) { 477 pci_write_config_word(bridge, PCI_IO_BASE, 0xe0f0); 478 pci_read_config_word(bridge, PCI_IO_BASE, &io); 479 pci_write_config_word(bridge, PCI_IO_BASE, 0x0); 480 } 481 if (io) { 482 bridge->io_window = 1; 483 if (pci_is_root_bus(bridge->bus)) { > 484 list_for_each_entry(dev, &bridge->bus->devices, bus_list) { 485 pci_read_config_word(dev, PCI_VENDOR_ID, &ven_id); 486 pci_read_config_word(dev, PCI_DEVICE_ID, &dev_id); 487 if (ven_id == PCI_VENDOR_ID_INTEL && dev_id == 0x09a2) { 488 /*IIO MISC Control offset 0x1c0*/ 489 pci_read_config_word(dev, 0x1c0, &en1k); 490 } 491 } 492 /* 493 *Intel ICX SPR EMR GNR 494 *IIO MISC Control (IIOMISCCTRL_1_5_0_CFG) — Offset 1C0h 495 *bit 2:Enable 1K (EN1K) 496 *This bit when set, enables 1K granularity for I/O space decode 497 *in each of the virtual P2P bridges 498 *corresponding to root ports, and DMI ports. 499 */ 500 if (en1k & 0x4) 501 bridge->io_window_1k = 1; 502 } 503 pci_read_bridge_io(bridge, &res, true); 504 } 505 506 pci_read_bridge_mmio(bridge, &res, true); 507 508 /* 509 * DECchip 21050 pass 2 errata: the bridge may miss an address 510 * disconnect boundary by one PCI data phase. Workaround: do not 511 * use prefetching on this device. 512 */ 513 if (bridge->vendor == PCI_VENDOR_ID_DEC && bridge->device == 0x0001) 514 return; 515 516 pci_read_config_dword(bridge, PCI_PREF_MEMORY_BASE, &pmem); 517 if (!pmem) { 518 pci_write_config_dword(bridge, PCI_PREF_MEMORY_BASE, 519 0xffe0fff0); 520 pci_read_config_dword(bridge, PCI_PREF_MEMORY_BASE, &pmem); 521 pci_write_config_dword(bridge, PCI_PREF_MEMORY_BASE, 0x0); 522 } 523 if (!pmem) 524 return; 525 526 bridge->pref_window = 1; 527 528 if ((pmem & PCI_PREF_RANGE_TYPE_MASK) == PCI_PREF_RANGE_TYPE_64) { 529 530 /* 531 * Bridge claims to have a 64-bit prefetchable memory 532 * window; verify that the upper bits are actually 533 * writable. 534 */ 535 pci_read_config_dword(bridge, PCI_PREF_BASE_UPPER32, &pmem); 536 pci_write_config_dword(bridge, PCI_PREF_BASE_UPPER32, 537 0xffffffff); 538 pci_read_config_dword(bridge, PCI_PREF_BASE_UPPER32, &tmp); 539 pci_write_config_dword(bridge, PCI_PREF_BASE_UPPER32, pmem); 540 if (tmp) 541 bridge->pref_64_window = 1; 542 } 543 544 pci_read_bridge_mmio_pref(bridge, &res, true); 545 } 546
diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c index 5fbabb4e3425..3f0c901c6653 100644 --- a/drivers/pci/probe.c +++ b/drivers/pci/probe.c @@ -461,6 +461,8 @@ static void pci_read_bridge_windows(struct pci_dev *bridge) u32 buses; u16 io; u32 pmem, tmp; + u16 ven_id, dev_id; + u16 en1k = 0; struct resource res; pci_read_config_dword(bridge, PCI_PRIMARY_BUS, &buses); @@ -478,6 +480,26 @@ static void pci_read_bridge_windows(struct pci_dev *bridge) } if (io) { bridge->io_window = 1; + if (pci_is_root_bus(bridge->bus)) { + list_for_each_entry(dev, &bridge->bus->devices, bus_list) { + pci_read_config_word(dev, PCI_VENDOR_ID, &ven_id); + pci_read_config_word(dev, PCI_DEVICE_ID, &dev_id); + if (ven_id == PCI_VENDOR_ID_INTEL && dev_id == 0x09a2) { + /*IIO MISC Control offset 0x1c0*/ + pci_read_config_word(dev, 0x1c0, &en1k); + } + } + /* + *Intel ICX SPR EMR GNR + *IIO MISC Control (IIOMISCCTRL_1_5_0_CFG) — Offset 1C0h + *bit 2:Enable 1K (EN1K) + *This bit when set, enables 1K granularity for I/O space decode + *in each of the virtual P2P bridges + *corresponding to root ports, and DMI ports. + */ + if (en1k & 0x4) + bridge->io_window_1k = 1; + } pci_read_bridge_io(bridge, &res, true); }
This patch add 1k granularity for intel root port bridge.Intel latest server CPU support 1K granularity,And there is an BIOS setup item named "EN1K",but linux doesn't support it. if an IIO has 5 IOU (SPR has 5 IOUs) all are bifurcated 2x8.In a 2P server system,There are 20 P2P bridges present.if keep 4K granularity allocation,it need 20*4=80k io space, exceeding 64k.I test it in a 16*nvidia 4090s system under intel eaglestrem platform.There are six 4090s that cannot be allocated I/O resources. So I applied this patch.And I found a similar implementation in quirks.c, but it only targets the Intel P64H2 platform. Signed-off-by: Zhou Shengqing <zhoushengqing@ttyinfo.com> --- drivers/pci/probe.c | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+)