mbox series

[v4,0/5] PCI Data Object Exchange support + CXL CDAT

Message ID 20210524133938.2815206-1-Jonathan.Cameron@huawei.com
Headers show
Series PCI Data Object Exchange support + CXL CDAT | expand

Message

Jonathan Cameron May 24, 2021, 1:39 p.m. UTC
Series first introduces generic support for DOE mailboxes as defined
in the ECN to the PCIe r5.0 specification available from the PCI SIG [0]

A user is then introduced in the form of the table access protocol defined
in the CXL 2.0 specification [1] used to access the
Coherent Device Attribute Table (CDAT) defined in [2]

Finally, in two patches that are not intended for merging, an example
of a generic IOCTL interface to perform synchronous exchanges from user
space in a kernel mediated fashion is introduced. The current consensus
seems in favour of not introducing such an interface, but instead
providing per protocol interfaces where appropriate. As this code was
developed in parallel with that discussion and may be of use to someone it
is included here.

Open questions.
* Do we need to prevent userspace access to the DOE when the kernel is
  managing it (registered by driver)? Discussion in thread:
  https://lore.kernel.org/linux-pci/20210419165451.2176200-1-Jonathan.Cameron@huawei.com/
* Does a generic userspace mediation interface make sense?
  Currently it seems not, but as it was nearly ready, I've included
  it in this patch set anyway.
* Move CXL cdat handling to a separate file, or wait until we can see
  how it is going to be used?
  
Changes since v3
Thanks to Ira, Bjorn and Dan for feedback.
* Addition of a generic IOCTL interface and demo program for discussion.
* Fixed an issue with accidentally disabling interrupts.
* Ensure IRQ_HANDLED always returned as clearing of BUSY can result
  in an interrupt, but there is no means of identifying this.
  The best that can be done is to eat the interrupt.
* Edited comments to more directly tie them to the code the referred to.
* Lock scope documentation update.
* Dropped the CDAT dump to log patch as we are hopefully getting closer
  to this being applied.
* Fixed the references to pcie_ missed when renaming to pci_ in v3.

Set based on cxl/next

All testing conducted against QEMU emulation of a CXL type 3 device
in conjunction with DOE mailbox patches v5 [3, 4]

[0] https://pcisig.com/specifications
[1] https://www.computeexpresslink.org/download-the-specification
[2] https://uefi.org/node/4093
[3] https://lore.kernel.org/qemu-devel/20210202005948.241655-1-ben.widawsky@intel.com/
[4] https://lore.kernel.org/qemu-devel/1619454964-10190-1-git-send-email-cbrowy@avery-design.com/

Jonathan Cameron (5):
  PCI: Add vendor ID for the PCI SIG
  PCI/DOE: Add Data Object Exchange support
  cxl/mem: Add CDAT table reading from DOE
  DONOTMERGE PCI/DOE Add per DOE chrdev for ioctl based access
  DONOTMERGE: PCI/DOE: Add userspace example program to tools/pci

 drivers/cxl/Kconfig           |   1 +
 drivers/cxl/cxl.h             |  21 +
 drivers/cxl/mem.c             | 174 ++++++++
 drivers/cxl/mem.h             |   6 +
 drivers/pci/Kconfig           |   8 +
 drivers/pci/Makefile          |   1 +
 drivers/pci/doe.c             | 794 ++++++++++++++++++++++++++++++++++
 drivers/pci/pci-driver.c      |   3 +-
 include/linux/pci-doe.h       | 100 +++++
 include/linux/pci.h           |   3 +
 include/linux/pci_ids.h       |   1 +
 include/uapi/linux/pci_doe.h  |  32 ++
 include/uapi/linux/pci_regs.h |  29 +-
 tools/pci/Build               |   1 +
 tools/pci/Makefile            |   9 +-
 tools/pci/doetest.c           | 131 ++++++
 16 files changed, 1311 insertions(+), 3 deletions(-)
 create mode 100644 drivers/pci/doe.c
 create mode 100644 include/linux/pci-doe.h
 create mode 100644 include/uapi/linux/pci_doe.h
 create mode 100644 tools/pci/doetest.c


base-commit: 35c32e3095d396c750f5cdfdaa94cba83d9b23c6

Comments

Jonathan Cameron June 10, 2021, 2:30 p.m. UTC | #1
On Mon, 24 May 2021 21:39:33 +0800
Jonathan Cameron <Jonathan.Cameron@huawei.com> wrote:

> Series first introduces generic support for DOE mailboxes as defined
> in the ECN to the PCIe r5.0 specification available from the PCI SIG [0]
> 
> A user is then introduced in the form of the table access protocol defined
> in the CXL 2.0 specification [1] used to access the
> Coherent Device Attribute Table (CDAT) defined in [2]
> 
> Finally, in two patches that are not intended for merging, an example
> of a generic IOCTL interface to perform synchronous exchanges from user
> space in a kernel mediated fashion is introduced. The current consensus
> seems in favour of not introducing such an interface, but instead
> providing per protocol interfaces where appropriate. As this code was
> developed in parallel with that discussion and may be of use to someone it
> is included here.
>

Gentle, not particularly urgent, request for review on this set.

I am currently missing ABI docs for the CDAT binary attribute, but
that could be handled in a follow up patch.

Thanks,

Jonathan
 
> Open questions.
> * Do we need to prevent userspace access to the DOE when the kernel is
>   managing it (registered by driver)? Discussion in thread:
>   https://lore.kernel.org/linux-pci/20210419165451.2176200-1-Jonathan.Cameron@huawei.com/
> * Does a generic userspace mediation interface make sense?
>   Currently it seems not, but as it was nearly ready, I've included
>   it in this patch set anyway.
> * Move CXL cdat handling to a separate file, or wait until we can see
>   how it is going to be used?
>   
> Changes since v3
> Thanks to Ira, Bjorn and Dan for feedback.
> * Addition of a generic IOCTL interface and demo program for discussion.
> * Fixed an issue with accidentally disabling interrupts.
> * Ensure IRQ_HANDLED always returned as clearing of BUSY can result
>   in an interrupt, but there is no means of identifying this.
>   The best that can be done is to eat the interrupt.
> * Edited comments to more directly tie them to the code the referred to.
> * Lock scope documentation update.
> * Dropped the CDAT dump to log patch as we are hopefully getting closer
>   to this being applied.
> * Fixed the references to pcie_ missed when renaming to pci_ in v3.
> 
> Set based on cxl/next
> 
> All testing conducted against QEMU emulation of a CXL type 3 device
> in conjunction with DOE mailbox patches v5 [3, 4]
> 
> [0] https://pcisig.com/specifications
> [1] https://www.computeexpresslink.org/download-the-specification
> [2] https://uefi.org/node/4093
> [3] https://lore.kernel.org/qemu-devel/20210202005948.241655-1-ben.widawsky@intel.com/
> [4] https://lore.kernel.org/qemu-devel/1619454964-10190-1-git-send-email-cbrowy@avery-design.com/
> 
> Jonathan Cameron (5):
>   PCI: Add vendor ID for the PCI SIG
>   PCI/DOE: Add Data Object Exchange support
>   cxl/mem: Add CDAT table reading from DOE
>   DONOTMERGE PCI/DOE Add per DOE chrdev for ioctl based access
>   DONOTMERGE: PCI/DOE: Add userspace example program to tools/pci
> 
>  drivers/cxl/Kconfig           |   1 +
>  drivers/cxl/cxl.h             |  21 +
>  drivers/cxl/mem.c             | 174 ++++++++
>  drivers/cxl/mem.h             |   6 +
>  drivers/pci/Kconfig           |   8 +
>  drivers/pci/Makefile          |   1 +
>  drivers/pci/doe.c             | 794 ++++++++++++++++++++++++++++++++++
>  drivers/pci/pci-driver.c      |   3 +-
>  include/linux/pci-doe.h       | 100 +++++
>  include/linux/pci.h           |   3 +
>  include/linux/pci_ids.h       |   1 +
>  include/uapi/linux/pci_doe.h  |  32 ++
>  include/uapi/linux/pci_regs.h |  29 +-
>  tools/pci/Build               |   1 +
>  tools/pci/Makefile            |   9 +-
>  tools/pci/doetest.c           | 131 ++++++
>  16 files changed, 1311 insertions(+), 3 deletions(-)
>  create mode 100644 drivers/pci/doe.c
>  create mode 100644 include/linux/pci-doe.h
>  create mode 100644 include/uapi/linux/pci_doe.h
>  create mode 100644 tools/pci/doetest.c
> 
> 
> base-commit: 35c32e3095d396c750f5cdfdaa94cba83d9b23c6