| Message ID | 20240703091949.206069-2-frank.heimes@canonical.com |
|---|---|
| State | New |
| Headers | show |
| Series | IOMMU DMA mode changed in kernel config causes massive throughput degradation for PCI-related network workloads (LP: 2071471) | expand |
On 03.07.24 11:19, frank.heimes@canonical.com wrote: > From: Frank Heimes <frank.heimes@canonical.com> > > BugLink: https://bugs.launchpad.net/bugs/2071471 > > Align the Ubuntu kernel config options to the modified defaults from upstream > b2b97a62f055 "Revert "s390: update defconfigs"" > and set IOMMU_DEFAULT_DMA_STRICT=n and IOMMU_DEFAULT_DMA_LAZY=yes for s390x. > This is required to avoid significant PCIe network throughput degradation > with the changed dma-iommu implementation in c76c067e488c > "s390/pci: Use dma-iommu layer" since v6.7. > > Signed-off-by: Frank Heimes <frank.heimes@canonical.com> LGTM. Performance improvement on s390x confirmed by the vendor. Acked-by: Kleber Sacilotto de Souza <kleber.souza@canonical.com> Thanks > --- > debian.master/config/annotations | 8 ++++++-- > 1 file changed, 6 insertions(+), 2 deletions(-) > > diff --git a/debian.master/config/annotations b/debian.master/config/annotations > index dbecd432d4a6..6b039eb3b264 100644 > --- a/debian.master/config/annotations > +++ b/debian.master/config/annotations > @@ -345,6 +345,12 @@ CONFIG_INTEL_IOMMU_SCALABLE_MODE_DEFAULT_ON note<'LP: #1951440'> > CONFIG_IOMMU_DEBUGFS policy<{'amd64': 'n', 'arm64': 'n', 'armhf': 'n', 'ppc64el': 'n', 'riscv64': 'n', 's390x': 'n'}> > CONFIG_IOMMU_DEBUGFS note<'LP: #1861057'> > > +CONFIG_IOMMU_DEFAULT_DMA_LAZY policy<{'amd64': 'y', 'arm64': 'n', 'armhf': 'n', 'ppc64el': 'n', 'riscv64': 'n', 's390x': 'y'}> > +CONFIG_IOMMU_DEFAULT_DMA_LAZY note<'LP: #2071471'> > + > +CONFIG_IOMMU_DEFAULT_DMA_STRICT policy<{'amd64': 'n', 'arm64': 'y', 'armhf': 'y', 'ppc64el': 'y', 'riscv64': 'y', 's390x': 'n'}> > +CONFIG_IOMMU_DEFAULT_DMA_STRICT note<'LP: #2071471'> > + > CONFIG_IPMMU_VMSA policy<{'arm64': 'n', 'armhf': 'n', 'riscv64': '-'}> > CONFIG_IPMMU_VMSA note<'LP: #1718734'> > > @@ -6734,8 +6740,6 @@ CONFIG_IOMMUFD policy<{'amd64': 'm', 'arm64': ' > CONFIG_IOMMUFD_DRIVER policy<{'amd64': 'y', 'arm64': 'y', 'armhf': 'y', 'ppc64el': 'y', 'riscv64': 'y'}> > CONFIG_IOMMU_API policy<{'amd64': 'y', 'arm64': 'y', 'armhf': 'y', 'ppc64el': 'y', 'riscv64': 'y', 's390x': 'y'}> > CONFIG_IOMMU_DEBUG policy<{'amd64': 'n'}> > -CONFIG_IOMMU_DEFAULT_DMA_LAZY policy<{'amd64': 'y', 'arm64': 'n', 'armhf': 'n', 'ppc64el': 'n', 'riscv64': 'n', 's390x': 'n'}> > -CONFIG_IOMMU_DEFAULT_DMA_STRICT policy<{'amd64': 'n', 'arm64': 'y', 'armhf': 'y', 'ppc64el': 'y', 'riscv64': 'y', 's390x': 'y'}> > CONFIG_IOMMU_DEFAULT_PASSTHROUGH policy<{'amd64': 'n', 'arm64': 'n', 'armhf': 'n', 'ppc64el': 'n', 'riscv64': 'n', 's390x': 'n'}> > CONFIG_IOMMU_DMA policy<{'amd64': 'y', 'arm64': 'y', 's390x': 'y'}> > CONFIG_IOMMU_HELPER policy<{'amd64': 'y', 'ppc64el': 'y', 's390x': 'y'}>
(now with the correct subject tag) On 03.07.24 11:19, frank.heimes@canonical.com wrote: > From: Frank Heimes <frank.heimes@canonical.com> > > BugLink: https://bugs.launchpad.net/bugs/2071471 > > Align the Ubuntu kernel config options to the modified defaults from upstream > b2b97a62f055 "Revert "s390: update defconfigs"" > and set IOMMU_DEFAULT_DMA_STRICT=n and IOMMU_DEFAULT_DMA_LAZY=yes for s390x. > This is required to avoid significant PCIe network throughput degradation > with the changed dma-iommu implementation in c76c067e488c > "s390/pci: Use dma-iommu layer" since v6.7. > > Signed-off-by: Frank Heimes <frank.heimes@canonical.com> LGTM. Performance improvement on s390x confirmed by the vendor. Acked-by: Kleber Sacilotto de Souza <kleber.souza@canonical.com> Thanks > --- > debian.master/config/annotations | 8 ++++++-- > 1 file changed, 6 insertions(+), 2 deletions(-) > > diff --git a/debian.master/config/annotations b/debian.master/config/annotations > index dbecd432d4a6..6b039eb3b264 100644 > --- a/debian.master/config/annotations > +++ b/debian.master/config/annotations > @@ -345,6 +345,12 @@ CONFIG_INTEL_IOMMU_SCALABLE_MODE_DEFAULT_ON note<'LP: #1951440'> > CONFIG_IOMMU_DEBUGFS policy<{'amd64': 'n', 'arm64': 'n', 'armhf': 'n', 'ppc64el': 'n', 'riscv64': 'n', 's390x': 'n'}> > CONFIG_IOMMU_DEBUGFS note<'LP: #1861057'> > > +CONFIG_IOMMU_DEFAULT_DMA_LAZY policy<{'amd64': 'y', 'arm64': 'n', 'armhf': 'n', 'ppc64el': 'n', 'riscv64': 'n', 's390x': 'y'}> > +CONFIG_IOMMU_DEFAULT_DMA_LAZY note<'LP: #2071471'> > + > +CONFIG_IOMMU_DEFAULT_DMA_STRICT policy<{'amd64': 'n', 'arm64': 'y', 'armhf': 'y', 'ppc64el': 'y', 'riscv64': 'y', 's390x': 'n'}> > +CONFIG_IOMMU_DEFAULT_DMA_STRICT note<'LP: #2071471'> > + > CONFIG_IPMMU_VMSA policy<{'arm64': 'n', 'armhf': 'n', 'riscv64': '-'}> > CONFIG_IPMMU_VMSA note<'LP: #1718734'> > > @@ -6734,8 +6740,6 @@ CONFIG_IOMMUFD policy<{'amd64': 'm', 'arm64': ' > CONFIG_IOMMUFD_DRIVER policy<{'amd64': 'y', 'arm64': 'y', 'armhf': 'y', 'ppc64el': 'y', 'riscv64': 'y'}> > CONFIG_IOMMU_API policy<{'amd64': 'y', 'arm64': 'y', 'armhf': 'y', 'ppc64el': 'y', 'riscv64': 'y', 's390x': 'y'}> > CONFIG_IOMMU_DEBUG policy<{'amd64': 'n'}> > -CONFIG_IOMMU_DEFAULT_DMA_LAZY policy<{'amd64': 'y', 'arm64': 'n', 'armhf': 'n', 'ppc64el': 'n', 'riscv64': 'n', 's390x': 'n'}> > -CONFIG_IOMMU_DEFAULT_DMA_STRICT policy<{'amd64': 'n', 'arm64': 'y', 'armhf': 'y', 'ppc64el': 'y', 'riscv64': 'y', 's390x': 'y'}> > CONFIG_IOMMU_DEFAULT_PASSTHROUGH policy<{'amd64': 'n', 'arm64': 'n', 'armhf': 'n', 'ppc64el': 'n', 'riscv64': 'n', 's390x': 'n'}> > CONFIG_IOMMU_DMA policy<{'amd64': 'y', 'arm64': 'y', 's390x': 'y'}> > CONFIG_IOMMU_HELPER policy<{'amd64': 'y', 'ppc64el': 'y', 's390x': 'y'}>
diff --git a/debian.master/config/annotations b/debian.master/config/annotations index dbecd432d4a6..6b039eb3b264 100644 --- a/debian.master/config/annotations +++ b/debian.master/config/annotations @@ -345,6 +345,12 @@ CONFIG_INTEL_IOMMU_SCALABLE_MODE_DEFAULT_ON note<'LP: #1951440'> CONFIG_IOMMU_DEBUGFS policy<{'amd64': 'n', 'arm64': 'n', 'armhf': 'n', 'ppc64el': 'n', 'riscv64': 'n', 's390x': 'n'}> CONFIG_IOMMU_DEBUGFS note<'LP: #1861057'> +CONFIG_IOMMU_DEFAULT_DMA_LAZY policy<{'amd64': 'y', 'arm64': 'n', 'armhf': 'n', 'ppc64el': 'n', 'riscv64': 'n', 's390x': 'y'}> +CONFIG_IOMMU_DEFAULT_DMA_LAZY note<'LP: #2071471'> + +CONFIG_IOMMU_DEFAULT_DMA_STRICT policy<{'amd64': 'n', 'arm64': 'y', 'armhf': 'y', 'ppc64el': 'y', 'riscv64': 'y', 's390x': 'n'}> +CONFIG_IOMMU_DEFAULT_DMA_STRICT note<'LP: #2071471'> + CONFIG_IPMMU_VMSA policy<{'arm64': 'n', 'armhf': 'n', 'riscv64': '-'}> CONFIG_IPMMU_VMSA note<'LP: #1718734'> @@ -6734,8 +6740,6 @@ CONFIG_IOMMUFD policy<{'amd64': 'm', 'arm64': ' CONFIG_IOMMUFD_DRIVER policy<{'amd64': 'y', 'arm64': 'y', 'armhf': 'y', 'ppc64el': 'y', 'riscv64': 'y'}> CONFIG_IOMMU_API policy<{'amd64': 'y', 'arm64': 'y', 'armhf': 'y', 'ppc64el': 'y', 'riscv64': 'y', 's390x': 'y'}> CONFIG_IOMMU_DEBUG policy<{'amd64': 'n'}> -CONFIG_IOMMU_DEFAULT_DMA_LAZY policy<{'amd64': 'y', 'arm64': 'n', 'armhf': 'n', 'ppc64el': 'n', 'riscv64': 'n', 's390x': 'n'}> -CONFIG_IOMMU_DEFAULT_DMA_STRICT policy<{'amd64': 'n', 'arm64': 'y', 'armhf': 'y', 'ppc64el': 'y', 'riscv64': 'y', 's390x': 'y'}> CONFIG_IOMMU_DEFAULT_PASSTHROUGH policy<{'amd64': 'n', 'arm64': 'n', 'armhf': 'n', 'ppc64el': 'n', 'riscv64': 'n', 's390x': 'n'}> CONFIG_IOMMU_DMA policy<{'amd64': 'y', 'arm64': 'y', 's390x': 'y'}> CONFIG_IOMMU_HELPER policy<{'amd64': 'y', 'ppc64el': 'y', 's390x': 'y'}>