Message ID | 20130612134504.GG2918@phenom.dumpdata.com |
---|---|
State | Not Applicable |
Headers | show |
On 12/06/13 14:45, Konrad Rzeszutek Wilk wrote: > On Tue, Jun 11, 2013 at 05:17:45PM +0100, George Dunlap wrote: >> On 06/11/2013 05:08 PM, konrad wilk wrote: >>> On 6/11/2013 11:36 AM, George Dunlap wrote: >>>> On 06/10/2013 10:06 PM, Konrad Rzeszutek Wilk wrote: >>>>> There are two tool-stack that can instruct the Xen PCI frontend >>>>> and backend to change states: 'xm' (Python code with a daemon), >>>>> and 'xl' (C library - does not keep state changes). >>>>> >>>>> With the 'xm', the path to disconnect a PCI device (xm pci-detach >>>>> <guest> <BDF>)is: >>>>> >>>>> 4(Connected)->7(Reconfiguring*)-> 8(Reconfigured)-> >>>>> 4(Connected)->5(Closing*). >>>>> >>>>> The * is for states that the tool-stack sets. For 'xl', it is similar: >>>>> >>>>> 4(Connected)->7(Reconfiguring*)-> 8(Reconfigured)-> 4(Connected) >>>>> >>>>> Both of them also tear down the XenBus structure, so the backend >>>>> state ends up going in the 3(Initialised) and calls >>>>> pcifront_xenbus_remove. >>>> So I looked a little bit into this; there are actually two different >>>> states that happen as part of this handshake. In order to disonnect a >>>> *device*, xl signals using the *bus* state, like this: >>>> * Wait for the *bus* to be in state 4(Connected) >>>> * Set the *device* state to 5(Closing) >>>> * Set the *bus* state to 7(Reconfiguring) >>>> * Wait for the *bus* state to return to 4(Connected) >>>> >>>> So are all of these states you see the *bus* state? And why would you >>>> disconnect the whole pci bus if you're only removing one device? >>> Correct. The stats I enumerated are *bus* states. Not per-device states. >>> I presume (and I hadn't checked xm) that Xend has some logic to only >>> disconnect the bus if all of the PCI devices have been disconnected. In >>> 'xl' it does not do that. >>> >>> The testing I did was just with one PCI device. >> Ah, OK -- I see now. The problem is that the code in the Linux side >> didn't know about the whole "4->7->8->4" thing to unplug a device. >> In all likelihood, if you had used xm with two devices (so that the >> bus didn't get disconnected), then you would have run across the >> same error. >> >> So at least part of the problem *is* a bug in Linux. > Good! Bjorn, would you be OK Ack-ing the patch I sent (attached here > for reference) or putting it in your queue for Linus? > > My plan would be to send it to Linus in the 3.11 merge window. One nit -- "to work with the 'xl' toolstack" -- didn't we theorize this would also be broken with xm if you had two devices passed through? -George -- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Wed, Jun 12, 2013 at 02:47:11PM +0100, George Dunlap wrote: > On 12/06/13 14:45, Konrad Rzeszutek Wilk wrote: > >On Tue, Jun 11, 2013 at 05:17:45PM +0100, George Dunlap wrote: > >>On 06/11/2013 05:08 PM, konrad wilk wrote: > >>>On 6/11/2013 11:36 AM, George Dunlap wrote: > >>>>On 06/10/2013 10:06 PM, Konrad Rzeszutek Wilk wrote: > >>>>>There are two tool-stack that can instruct the Xen PCI frontend > >>>>>and backend to change states: 'xm' (Python code with a daemon), > >>>>>and 'xl' (C library - does not keep state changes). > >>>>> > >>>>>With the 'xm', the path to disconnect a PCI device (xm pci-detach > >>>>><guest> <BDF>)is: > >>>>> > >>>>>4(Connected)->7(Reconfiguring*)-> 8(Reconfigured)-> > >>>>>4(Connected)->5(Closing*). > >>>>> > >>>>>The * is for states that the tool-stack sets. For 'xl', it is similar: > >>>>> > >>>>>4(Connected)->7(Reconfiguring*)-> 8(Reconfigured)-> 4(Connected) > >>>>> > >>>>>Both of them also tear down the XenBus structure, so the backend > >>>>>state ends up going in the 3(Initialised) and calls > >>>>>pcifront_xenbus_remove. > >>>>So I looked a little bit into this; there are actually two different > >>>>states that happen as part of this handshake. In order to disonnect a > >>>>*device*, xl signals using the *bus* state, like this: > >>>>* Wait for the *bus* to be in state 4(Connected) > >>>>* Set the *device* state to 5(Closing) > >>>>* Set the *bus* state to 7(Reconfiguring) > >>>>* Wait for the *bus* state to return to 4(Connected) > >>>> > >>>>So are all of these states you see the *bus* state? And why would you > >>>>disconnect the whole pci bus if you're only removing one device? > >>>Correct. The stats I enumerated are *bus* states. Not per-device states. > >>>I presume (and I hadn't checked xm) that Xend has some logic to only > >>>disconnect the bus if all of the PCI devices have been disconnected. In > >>>'xl' it does not do that. > >>> > >>>The testing I did was just with one PCI device. > >>Ah, OK -- I see now. The problem is that the code in the Linux side > >>didn't know about the whole "4->7->8->4" thing to unplug a device. > >>In all likelihood, if you had used xm with two devices (so that the > >>bus didn't get disconnected), then you would have run across the > >>same error. > >> > >>So at least part of the problem *is* a bug in Linux. > >Good! Bjorn, would you be OK Ack-ing the patch I sent (attached here > >for reference) or putting it in your queue for Linus? > > > >My plan would be to send it to Linus in the 3.11 merge window. > > One nit -- "to work with the 'xl' toolstack" -- didn't we theorize > this would also be broken with xm if you had two devices passed > through? Yes. I will fix up the title to reflect that shortly (say Friday?) Thanks for your sharp eyes. -- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Wed, Jun 12, 2013 at 7:45 AM, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote: > On Tue, Jun 11, 2013 at 05:17:45PM +0100, George Dunlap wrote: >> On 06/11/2013 05:08 PM, konrad wilk wrote: >> > >> >On 6/11/2013 11:36 AM, George Dunlap wrote: >> >>On 06/10/2013 10:06 PM, Konrad Rzeszutek Wilk wrote: >> >>>There are two tool-stack that can instruct the Xen PCI frontend >> >>>and backend to change states: 'xm' (Python code with a daemon), >> >>>and 'xl' (C library - does not keep state changes). >> >>> >> >>>With the 'xm', the path to disconnect a PCI device (xm pci-detach >> >>><guest> <BDF>)is: >> >>> >> >>>4(Connected)->7(Reconfiguring*)-> 8(Reconfigured)-> >> >>>4(Connected)->5(Closing*). >> >>> >> >>>The * is for states that the tool-stack sets. For 'xl', it is similar: >> >>> >> >>>4(Connected)->7(Reconfiguring*)-> 8(Reconfigured)-> 4(Connected) >> >>> >> >>>Both of them also tear down the XenBus structure, so the backend >> >>>state ends up going in the 3(Initialised) and calls >> >>>pcifront_xenbus_remove. >> >> >> >>So I looked a little bit into this; there are actually two different >> >>states that happen as part of this handshake. In order to disonnect a >> >>*device*, xl signals using the *bus* state, like this: >> >>* Wait for the *bus* to be in state 4(Connected) >> >>* Set the *device* state to 5(Closing) >> >>* Set the *bus* state to 7(Reconfiguring) >> >>* Wait for the *bus* state to return to 4(Connected) >> >> >> >>So are all of these states you see the *bus* state? And why would you >> >>disconnect the whole pci bus if you're only removing one device? >> > >> >Correct. The stats I enumerated are *bus* states. Not per-device states. >> >I presume (and I hadn't checked xm) that Xend has some logic to only >> >disconnect the bus if all of the PCI devices have been disconnected. In >> >'xl' it does not do that. >> > >> >The testing I did was just with one PCI device. >> >> Ah, OK -- I see now. The problem is that the code in the Linux side >> didn't know about the whole "4->7->8->4" thing to unplug a device. >> In all likelihood, if you had used xm with two devices (so that the >> bus didn't get disconnected), then you would have run across the >> same error. >> >> So at least part of the problem *is* a bug in Linux. > > Good! Bjorn, would you be OK Ack-ing the patch I sent (attached here > for reference) or putting it in your queue for Linus? > > My plan would be to send it to Linus in the 3.11 merge window. Sure; this is your baby :) Why don't you handle it via your tree, since it's more related to xen than any PCI core stuff. Acked-by: Bjorn Helgaas <bhelgaas@google.com> -- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
> >> So at least part of the problem *is* a bug in Linux. > > > > Good! Bjorn, would you be OK Ack-ing the patch I sent (attached here > > for reference) or putting it in your queue for Linus? > > > > My plan would be to send it to Linus in the 3.11 merge window. > > Sure; this is your baby :) Why don't you handle it via your tree, > since it's more related to xen than any PCI core stuff. OK. Thanks! -- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
> Sure; this is your baby :) Why don't you handle it via your tree, > since it's more related to xen than any PCI core stuff. > > Acked-by: Bjorn Helgaas <bhelgaas@google.com> Definitly fixed in v3.12. Just tested it and it works. George, Ian, how do I "close" a bug in http://bugs.xenproject.org/xen/bug/12 ? -- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
diff --git a/drivers/pci/xen-pcifront.c b/drivers/pci/xen-pcifront.c index ac99515..cc46e253 100644 --- a/drivers/pci/xen-pcifront.c +++ b/drivers/pci/xen-pcifront.c @@ -675,10 +675,9 @@ static int pcifront_connect_and_init_dma(struct pcifront_device *pdev) if (!pcifront_dev) { dev_info(&pdev->xdev->dev, "Installing PCI frontend\n"); pcifront_dev = pdev; - } else { - dev_err(&pdev->xdev->dev, "PCI frontend already installed!\n"); + } else err = -EEXIST; - } + spin_unlock(&pcifront_dev_lock); if (!err && !swiotlb_nr_tbl()) { @@ -846,7 +845,7 @@ static int pcifront_try_connect(struct pcifront_device *pdev) goto out; err = pcifront_connect_and_init_dma(pdev); - if (err) { + if (err && err != -EEXIST) { xenbus_dev_fatal(pdev->xdev, err, "Error setting up PCI Frontend"); goto out;