From patchwork Fri Feb 17 08:29:30 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dong Jia Shi X-Patchwork-Id: 729075 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3vPml26y61z9s7x for ; Fri, 17 Feb 2017 19:43:26 +1100 (AEDT) Received: from localhost ([::1]:52168 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cee8S-0008RH-9u for incoming@patchwork.ozlabs.org; Fri, 17 Feb 2017 03:43:24 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:45046) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cedvd-0004Cz-Ly for qemu-devel@nongnu.org; Fri, 17 Feb 2017 03:30:14 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cedvY-0001Dz-N8 for qemu-devel@nongnu.org; Fri, 17 Feb 2017 03:30:09 -0500 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:58334) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cedvY-0001Dd-Cy for qemu-devel@nongnu.org; Fri, 17 Feb 2017 03:30:04 -0500 Received: from pps.filterd (m0098399.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.20/8.16.0.20) with SMTP id v1H8TH5W139512 for ; Fri, 17 Feb 2017 03:30:03 -0500 Received: from e31.co.us.ibm.com (e31.co.us.ibm.com [32.97.110.149]) by mx0a-001b2d01.pphosted.com with ESMTP id 28nrgkj5r0-1 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=NOT) for ; Fri, 17 Feb 2017 03:30:03 -0500 Received: from localhost by e31.co.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Fri, 17 Feb 2017 01:30:02 -0700 Received: from d03dlp03.boulder.ibm.com (9.17.202.179) by e31.co.us.ibm.com (192.168.1.131) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Fri, 17 Feb 2017 01:30:00 -0700 Received: from b01cxnp22036.gho.pok.ibm.com (b01cxnp22036.gho.pok.ibm.com [9.57.198.26]) by d03dlp03.boulder.ibm.com (Postfix) with ESMTP id 0D73D19D801C; Fri, 17 Feb 2017 01:29:11 -0700 (MST) Received: from b01ledav03.gho.pok.ibm.com (b01ledav003.gho.pok.ibm.com [9.57.199.108]) by b01cxnp22036.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id v1H8Tw6B54526044; Fri, 17 Feb 2017 08:29:58 GMT Received: from b01ledav03.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 7B487B204D; Fri, 17 Feb 2017 03:29:57 -0500 (EST) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.152.85.9]) by b01ledav03.gho.pok.ibm.com (Postfix) with ESMTP id 42071B2046; Fri, 17 Feb 2017 03:29:56 -0500 (EST) From: Dong Jia Shi To: kvm@vger.kernel.org, linux-s390@vger.kernel.org, qemu-devel@nongnu.org Date: Fri, 17 Feb 2017 09:29:30 +0100 X-Mailer: git-send-email 2.8.4 In-Reply-To: <20170217082939.33208-1-bjsdjshi@linux.vnet.ibm.com> References: <20170217082939.33208-1-bjsdjshi@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 X-Content-Scanned: Fidelis XPS MAILER x-cbid: 17021708-8235-0000-0000-00000AFCD59D X-IBM-SpamModules-Scores: X-IBM-SpamModules-Versions: BY=3.00006631; HX=3.00000240; KW=3.00000007; PH=3.00000004; SC=3.00000203; SDB=6.00823158; UDB=6.00402781; IPR=6.00600605; BA=6.00005146; NDR=6.00000001; ZLA=6.00000005; ZF=6.00000009; ZB=6.00000000; ZP=6.00000000; ZH=6.00000000; ZU=6.00000002; MB=3.00014319; XFM=3.00000011; UTC=2017-02-17 08:30:02 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 17021708-8236-0000-0000-000039ACE610 Message-Id: <20170217082939.33208-7-bjsdjshi@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-02-17_07:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=2 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1612050000 definitions=main-1702170081 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.156.1 Subject: [Qemu-devel] [PATCH RFC v3 06/15] vfio: ccw: register vfio_ccw to the mediated device framework X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: pasic@linux.vnet.ibm.com, pmorel@linux.vnet.ibm.com, borntraeger@de.ibm.com, alex.williamson@redhat.com, renxiaof@linux.vnet.ibm.com, wkywang@linux.vnet.ibm.com, cornelia.huck@de.ibm.com, bjsdjshi@linux.vnet.ibm.com, agraf@suse.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" To make vfio support subchannel devices, we need to leverage the mediated device framework to create a mediated device for the subchannel device. This registers the subchannel device to the mediated device framework during probe to enable mediated device creation. Signed-off-by: Dong Jia Shi Reviewed-by: Pierre Morel --- arch/s390/Kconfig | 2 +- drivers/s390/cio/Makefile | 2 +- drivers/s390/cio/vfio_ccw_drv.c | 12 ++- drivers/s390/cio/vfio_ccw_ops.c | 147 ++++++++++++++++++++++++++++++++++++ drivers/s390/cio/vfio_ccw_private.h | 11 +++ 5 files changed, 171 insertions(+), 3 deletions(-) create mode 100644 drivers/s390/cio/vfio_ccw_ops.c diff --git a/arch/s390/Kconfig b/arch/s390/Kconfig index b920df8..32008b8 100644 --- a/arch/s390/Kconfig +++ b/arch/s390/Kconfig @@ -673,7 +673,7 @@ config EADM_SCH config VFIO_CCW def_tristate n prompt "Support for VFIO-CCW subchannels" - depends on S390_CCW_IOMMU && VFIO + depends on S390_CCW_IOMMU && VFIO_MDEV help This driver allows usage of VFIO-CCW subchannels. diff --git a/drivers/s390/cio/Makefile b/drivers/s390/cio/Makefile index 1bec279..b0586b2 100644 --- a/drivers/s390/cio/Makefile +++ b/drivers/s390/cio/Makefile @@ -18,5 +18,5 @@ obj-$(CONFIG_CCWGROUP) += ccwgroup.o qdio-objs := qdio_main.o qdio_thinint.o qdio_debug.o qdio_setup.o obj-$(CONFIG_QDIO) += qdio.o -vfio_ccw-objs += vfio_ccw_drv.o vfio_ccw_cp.o +vfio_ccw-objs += vfio_ccw_drv.o vfio_ccw_cp.o vfio_ccw_ops.o obj-$(CONFIG_VFIO_CCW) += vfio_ccw.o diff --git a/drivers/s390/cio/vfio_ccw_drv.c b/drivers/s390/cio/vfio_ccw_drv.c index b068207..80d20e3 100644 --- a/drivers/s390/cio/vfio_ccw_drv.c +++ b/drivers/s390/cio/vfio_ccw_drv.c @@ -19,7 +19,7 @@ /* * Helpers */ -static int vfio_ccw_sch_quiesce(struct subchannel *sch) +int vfio_ccw_sch_quiesce(struct subchannel *sch) { struct vfio_ccw_private *private = dev_get_drvdata(&sch->dev); DECLARE_COMPLETION_ONSTACK(completion); @@ -152,8 +152,16 @@ static int vfio_ccw_sch_probe(struct subchannel *sch) if (ret) goto out_disable; + ret = vfio_ccw_mdev_reg(sch); + if (ret) + goto out_rm_group; + + atomic_set(&private->avail, 1); + return 0; +out_rm_group: + sysfs_remove_group(&sch->dev.kobj, &vfio_subchannel_attr_group); out_disable: cio_disable_subchannel(sch); out_free: @@ -168,6 +176,8 @@ static int vfio_ccw_sch_remove(struct subchannel *sch) vfio_ccw_sch_quiesce(sch); + vfio_ccw_mdev_unreg(sch); + sysfs_remove_group(&sch->dev.kobj, &vfio_subchannel_attr_group); dev_set_drvdata(&sch->dev, NULL); diff --git a/drivers/s390/cio/vfio_ccw_ops.c b/drivers/s390/cio/vfio_ccw_ops.c new file mode 100644 index 0000000..b8a2fed --- /dev/null +++ b/drivers/s390/cio/vfio_ccw_ops.c @@ -0,0 +1,147 @@ +/* + * Physical device callbacks for vfio_ccw + * + * Copyright IBM Corp. 2017 + * + * Author(s): Dong Jia Shi + * Xiao Feng Ren + */ + +#include +#include + +#include "vfio_ccw_private.h" + +static int vfio_ccw_mdev_notifier(struct notifier_block *nb, + unsigned long action, + void *data) +{ + struct vfio_ccw_private *private = + container_of(nb, struct vfio_ccw_private, nb); + + if (!private) + return NOTIFY_STOP; + + /* + * TODO: + * Vendor drivers MUST unpin pages in response to an + * invalidation. + */ + if (action == VFIO_IOMMU_NOTIFY_DMA_UNMAP) + return NOTIFY_BAD; + + return NOTIFY_DONE; +} + +static ssize_t name_show(struct kobject *kobj, struct device *dev, char *buf) +{ + return sprintf(buf, "I/O subchannel (Non-QDIO)\n"); +} +MDEV_TYPE_ATTR_RO(name); + +static ssize_t device_api_show(struct kobject *kobj, struct device *dev, + char *buf) +{ + return sprintf(buf, "%s\n", VFIO_DEVICE_API_CCW_STRING); +} +MDEV_TYPE_ATTR_RO(device_api); + +static ssize_t available_instances_show(struct kobject *kobj, + struct device *dev, char *buf) +{ + struct vfio_ccw_private *private = dev_get_drvdata(dev); + + return sprintf(buf, "%d\n", atomic_read(&private->avail)); +} +MDEV_TYPE_ATTR_RO(available_instances); + +static struct attribute *mdev_types_attrs[] = { + &mdev_type_attr_name.attr, + &mdev_type_attr_device_api.attr, + &mdev_type_attr_available_instances.attr, + NULL, +}; + +static struct attribute_group mdev_type_group = { + .name = "io", + .attrs = mdev_types_attrs, +}; + +struct attribute_group *mdev_type_groups[] = { + &mdev_type_group, + NULL, +}; + +static int vfio_ccw_mdev_create(struct kobject *kobj, struct mdev_device *mdev) +{ + struct vfio_ccw_private *private = + dev_get_drvdata(mdev_parent_dev(mdev)); + + if (atomic_dec_if_positive(&private->avail) < 0) + return -EPERM; + + private->mdev = mdev; + + return 0; +} + +static int vfio_ccw_mdev_remove(struct mdev_device *mdev) +{ + struct vfio_ccw_private *private; + struct subchannel *sch; + int ret; + + private = dev_get_drvdata(mdev_parent_dev(mdev)); + sch = private->sch; + ret = vfio_ccw_sch_quiesce(sch); + if (ret) + return ret; + ret = cio_enable_subchannel(sch, (u32)(unsigned long)sch); + if (ret) + return ret; + + private->mdev = NULL; + atomic_inc(&private->avail); + + return 0; +} + +static int vfio_ccw_mdev_open(struct mdev_device *mdev) +{ + struct vfio_ccw_private *private = + dev_get_drvdata(mdev_parent_dev(mdev)); + unsigned long events = VFIO_IOMMU_NOTIFY_DMA_UNMAP; + + private->nb.notifier_call = vfio_ccw_mdev_notifier; + + return vfio_register_notifier(mdev_dev(mdev), VFIO_IOMMU_NOTIFY, + &events, &private->nb); +} + +void vfio_ccw_mdev_release(struct mdev_device *mdev) +{ + struct vfio_ccw_private *private = + dev_get_drvdata(mdev_parent_dev(mdev)); + + vfio_unregister_notifier(mdev_dev(mdev), VFIO_IOMMU_NOTIFY, + &private->nb); +} + +static const struct mdev_parent_ops vfio_ccw_mdev_ops = { + .owner = THIS_MODULE, + .supported_type_groups = mdev_type_groups, + .create = vfio_ccw_mdev_create, + .remove = vfio_ccw_mdev_remove, + .open = vfio_ccw_mdev_open, + .release = vfio_ccw_mdev_release, +}; + +int vfio_ccw_mdev_reg(struct subchannel *sch) +{ + return mdev_register_device(&sch->dev, &vfio_ccw_mdev_ops); +} + +void vfio_ccw_mdev_unreg(struct subchannel *sch) +{ + mdev_unregister_device(&sch->dev); +} diff --git a/drivers/s390/cio/vfio_ccw_private.h b/drivers/s390/cio/vfio_ccw_private.h index 38d69a5..5afb3ba 100644 --- a/drivers/s390/cio/vfio_ccw_private.h +++ b/drivers/s390/cio/vfio_ccw_private.h @@ -16,10 +16,21 @@ * struct vfio_ccw_private * @sch: pointer to the subchannel * @completion: synchronization helper of the I/O completion + * @avail: available for creating a mediated device + * @mdev: pointer to the mediated device + * @nb: notifier for vfio events */ struct vfio_ccw_private { struct subchannel *sch; struct completion *completion; + atomic_t avail; + struct mdev_device *mdev; + struct notifier_block nb; } __aligned(8); +extern int vfio_ccw_mdev_reg(struct subchannel *sch); +extern void vfio_ccw_mdev_unreg(struct subchannel *sch); + +extern int vfio_ccw_sch_quiesce(struct subchannel *sch); + #endif