| Message ID | 1249836296-13288-3-git-send-email-avi@redhat.com |
|---|---|
| State | Superseded |
| Headers | show |
Hi, > typedef struct isa_irq_state { > qemu_irq *i8259; > + qemu_irq *ioapic; > } IsaIrqState; Well. Now it would probably better named "pc_irq_state". While that cleanup certainly makes sense it isn't a isa-bus thing any more. cheers, Gerd
On 08/09/09 18:44, Avi Kivity wrote: > Instead of calling the IOAPIC from the PIC, raise IOAPIC irqs via the ISA bus. > As a side effect, IOAPIC lines 16-23 are enabled. Now we probably need some acpi magic to tell the guest OS that there are a few more IRQ lines? cheers, Gerd
On 08/26/2009 07:03 PM, Gerd Hoffmann wrote: > On 08/09/09 18:44, Avi Kivity wrote: >> Instead of calling the IOAPIC from the PIC, raise IOAPIC irqs via the >> ISA bus. >> As a side effect, IOAPIC lines 16-23 are enabled. > > Now we probably need some acpi magic to tell the guest OS that there > are a few more IRQ lines? > I wanted to route the PCI IRQs to those lines (and have eight links instead of four). Now I don't think it's worthwhile, as every guest that is interesting from a performance point of view has MSI support. So I think we should just leave these lines as is, terminated with a 4.7K resistor to avoid picking up noise.
On 08/26/09 18:06, Avi Kivity wrote: > On 08/26/2009 07:03 PM, Gerd Hoffmann wrote: >> Now we probably need some acpi magic to tell the guest OS that there >> are a few more IRQ lines? > > I wanted to route the PCI IRQs to those lines (and have eight links > instead of four). Right now we have IRQs 5,10,11 for PCI. Having one more IRQ (so we have one for each link) would be useful IMHO. eight links + eight irqs would be even more useful. What needs to be done for that? BTW: Seems linux doesn't use IRQ 5 even with lots of PCI devices, instead it makes them share 10+11 ... > Now I don't think it's worthwhile, as every guest that > is interesting from a performance point of view has MSI support. Not all emulated devices have MSI support though ... > So I think we should just leave these lines as is, terminated with a > 4.7K resistor to avoid picking up noise. Oh, nice, didn't know kvm has virtual resistor support. cheers, Gerd
On Wed, Aug 26, 2009 at 06:30:54PM +0200, Gerd Hoffmann wrote: > On 08/26/09 18:06, Avi Kivity wrote: > >On 08/26/2009 07:03 PM, Gerd Hoffmann wrote: > >>Now we probably need some acpi magic to tell the guest OS that there > >>are a few more IRQ lines? > > > >I wanted to route the PCI IRQs to those lines (and have eight links > >instead of four). > > Right now we have IRQs 5,10,11 for PCI. Having one more IRQ (so we > have one for each link) would be useful IMHO. eight links + eight > irqs would be even more useful. What needs to be done for that? > Current code uses piix3 irq router to route pci interrupts to pic _and_ ioapic and piix3 irq router supports only 16 interrupts. -- Gleb.
On 08/26/2009 07:30 PM, Gerd Hoffmann wrote: > On 08/26/09 18:06, Avi Kivity wrote: >> On 08/26/2009 07:03 PM, Gerd Hoffmann wrote: >>> Now we probably need some acpi magic to tell the guest OS that there >>> are a few more IRQ lines? >> >> I wanted to route the PCI IRQs to those lines (and have eight links >> instead of four). > > Right now we have IRQs 5,10,11 for PCI. Having one more IRQ (so we > have one for each link) would be useful IMHO. eight links + eight > irqs would be even more useful. What needs to be done for that? > > BTW: Seems linux doesn't use IRQ 5 even with lots of PCI devices, > instead it makes them share 10+11 ... We could change the defaults to include 5, but maybe it makes more sense to fix Linux to distribute active PCI IRQs across the resources it has at its disposal. > >> Now I don't think it's worthwhile, as every guest that >> is interesting from a performance point of view has MSI support. > > Not all emulated devices have MSI support though ... They'll be slow regardless. I should be easy to support msi on e1000 though.
>> BTW: Seems linux doesn't use IRQ 5 even with lots of PCI devices, >> instead it makes them share 10+11 ... > > We could change the defaults to include 5, but maybe it makes more sense > to fix Linux to distribute active PCI IRQs across the resources it has > at its disposal. i.e. Linux decides to stick with the defaults (starred) here ... ACPI: PCI Interrupt Routing Table [\_SB_.PCI0._PRT] ACPI: PCI Interrupt Link [LNKA] (IRQs 5 *10 11) ACPI: PCI Interrupt Link [LNKB] (IRQs 5 *10 11) ACPI: PCI Interrupt Link [LNKC] (IRQs 5 10 *11) ACPI: PCI Interrupt Link [LNKD] (IRQs 5 10 *11) ... instead of trying to minimize IRQ sharing by using IRQ 5? > They'll be slow regardless. I should be easy to support msi on e1000 > though. What is needed on the guest side? Looks like even 2.6.30 doesn't use MSI for virtio-net ... cheers, Gerd
On 08/26/09 21:09, Gleb Natapov wrote: > On Wed, Aug 26, 2009 at 06:30:54PM +0200, Gerd Hoffmann wrote: >> Right now we have IRQs 5,10,11 for PCI. Having one more IRQ (so we >> have one for each link) would be useful IMHO. eight links + eight >> irqs would be even more useful. What needs to be done for that? >> > Current code uses piix3 irq router to route pci interrupts to pic _and_ > ioapic and piix3 irq router supports only 16 interrupts. That means? We could add four more PCI links which have IRQs routed through another IRQ router chip and link them to ioapic lines 17-23 that way? Or does it mean we must emulate a more recent chipset? cheers, Gerd
On 08/27/2009 10:35 AM, Gerd Hoffmann wrote: >>> BTW: Seems linux doesn't use IRQ 5 even with lots of PCI devices, >>> instead it makes them share 10+11 ... >> >> We could change the defaults to include 5, but maybe it makes more sense >> to fix Linux to distribute active PCI IRQs across the resources it has >> at its disposal. > > i.e. Linux decides to stick with the defaults (starred) here ... > > ACPI: PCI Interrupt Routing Table [\_SB_.PCI0._PRT] > ACPI: PCI Interrupt Link [LNKA] (IRQs 5 *10 11) > ACPI: PCI Interrupt Link [LNKB] (IRQs 5 *10 11) > ACPI: PCI Interrupt Link [LNKC] (IRQs 5 10 *11) > ACPI: PCI Interrupt Link [LNKD] (IRQs 5 10 *11) > > ... instead of trying to minimize IRQ sharing by using IRQ 5? Yes. And if it turns out you have two active devices and 6 inactive devices, move the two active devices to private lines and pile up the inactive ones on the leftover. > >> They'll be slow regardless. I should be easy to support msi on e1000 >> though. > > What is needed on the guest side? Looks like even 2.6.30 doesn't use > MSI for virtio-net ... virtio msi support was merged in 2.6.31. For e1000, guest support is presumably there, just need to wire it into the device.
On Thu, Aug 27, 2009 at 09:40:06AM +0200, Gerd Hoffmann wrote: > On 08/26/09 21:09, Gleb Natapov wrote: > >On Wed, Aug 26, 2009 at 06:30:54PM +0200, Gerd Hoffmann wrote: > >>Right now we have IRQs 5,10,11 for PCI. Having one more IRQ (so we > >>have one for each link) would be useful IMHO. eight links + eight > >>irqs would be even more useful. What needs to be done for that? > >> > >Current code uses piix3 irq router to route pci interrupts to pic _and_ > >ioapic and piix3 irq router supports only 16 interrupts. > > That means? We could add four more PCI links which have IRQs routed That means changing acpi/bios is unfortunately not enough. > through another IRQ router chip and link them to ioapic lines 17-23 > that way? Something like that. We can invent our own irq router and write driver for it in DSDT. > Or does it mean we must emulate a more recent chipset? > That too will work. -- Gleb.
On 08/27/09 09:57, Avi Kivity wrote: >> What is needed on the guest side? Looks like even 2.6.30 doesn't use >> MSI for virtio-net ... > > virtio msi support was merged in 2.6.31. For e1000, guest support is > presumably there, just need to wire it into the device. lspci shows MSI (not MSI-X) capability for the e1000 in my laptop. It is not enabled. dmesg has this line though: 0000:02:00.0: 0000:02:00.0: Failed to initialize MSI interrupts. Falling back to legacy interrupts. so the driver at least tried to enable MSI ... cheers Gerd
Gleb Natapov wrote: > On Thu, Aug 27, 2009 at 09:40:06AM +0200, Gerd Hoffmann wrote: > > Or does it mean we must emulate a more recent chipset? > > > That too will work. Note that if you change the chipset, it'll break some existing Windows VMs. I had this problem when porting a Windows Server 2003 VM from Virtual PC to QEMU: Virtual PC emulates a PIIX4, while QEMU provides a PIIX3 (even though there's a PIIX4 in the source code, it's not used for PC emulation). The ported image would not boot because of the change of chipset, until I patched the registry to accomodate the change. It will be the same the other way: When some Windows VMs see a change from PIIX3 to a later chipset, they will not boot. Some Windows VMs don't care as much. XP seems to be more like Linux, in that it adapts to different devices at boot time. -- Jamie > > -- > Gleb. > >
On Thu, Aug 27, 2009 at 09:18:32AM +0100, Jamie Lokier wrote: > Gleb Natapov wrote: > > On Thu, Aug 27, 2009 at 09:40:06AM +0200, Gerd Hoffmann wrote: > > > Or does it mean we must emulate a more recent chipset? > > > > > That too will work. > > > Note that if you change the chipset, it'll break some existing Windows VMs. > I don't think it worth changing chipset just to solve irq routing problem. And PIIX4 is the same as PIIX3 in regards to pci irq routing. > I had this problem when porting a Windows Server 2003 VM from Virtual > PC to QEMU: Virtual PC emulates a PIIX4, while QEMU provides a PIIX3 > (even though there's a PIIX4 in the source code, it's not used for PC > emulation). The ported image would not boot because of the change of > chipset, until I patched the registry to accomodate the change. > > It will be the same the other way: When some Windows VMs see a change > from PIIX3 to a later chipset, they will not boot. > > Some Windows VMs don't care as much. XP seems to be more like Linux, > in that it adapts to different devices at boot time. > > -- Jamie > > > > > > > > -- > > Gleb. > > > > -- Gleb.
On 08/27/2009 11:13 AM, Gerd Hoffmann wrote: > On 08/27/09 09:57, Avi Kivity wrote: >>> What is needed on the guest side? Looks like even 2.6.30 doesn't use >>> MSI for virtio-net ... >> >> virtio msi support was merged in 2.6.31. For e1000, guest support is >> presumably there, just need to wire it into the device. > > lspci shows MSI (not MSI-X) capability for the e1000 in my laptop. It > is not enabled. dmesg has this line though: > > 0000:02:00.0: 0000:02:00.0: Failed to initialize MSI interrupts. > Falling back to legacy interrupts. > > so the driver at least tried to enable MSI ... You probably need chipset support. I recommend drop that laptop and running exclusively in VMs.
On 08/27/09 10:18, Jamie Lokier wrote: > Gleb Natapov wrote: >> On Thu, Aug 27, 2009 at 09:40:06AM +0200, Gerd Hoffmann wrote: >>> Or does it mean we must emulate a more recent chipset? >>> >> That too will work. > > > Note that if you change the chipset, it'll break some existing Windows VMs. I think we would rather *add* a new machine type with a new chipset, not *replace* the piix3. IIRC someone is already working on emulation something more recent (ICH9?) to get some modern stuff, so that will probably get us some more ioapic lines. > I had this problem when porting a Windows Server 2003 VM from Virtual > PC to QEMU: Virtual PC emulates a PIIX4, while QEMU provides a PIIX3 > (even though there's a PIIX4 in the source code, it's not used for PC > emulation). The ported image would not boot because of the change of > chipset, until I patched the registry to accomodate the change. Any hints where do dig in the registy? I have a dead notebook with xp (disk still ok), trying to boot the disk image in kvm, and of course it doesn't work due to the hardware being different. Googleing the STOP code printed on the blue screen hinted that xp fails to access the hard drive ... It is a 5-year old intel laptop, so it probably is something like moving from ICH6 to PIIX3 ... thanks, Gerd
On Thu, Aug 27, 2009 at 10:55:54AM +0200, Gerd Hoffmann wrote: > On 08/27/09 10:18, Jamie Lokier wrote: > >Gleb Natapov wrote: > >>On Thu, Aug 27, 2009 at 09:40:06AM +0200, Gerd Hoffmann wrote: > >>> Or does it mean we must emulate a more recent chipset? > >>> > >>That too will work. > > > > > >Note that if you change the chipset, it'll break some existing Windows VMs. > > I think we would rather *add* a new machine type with a new chipset, not > *replace* the piix3. IIRC someone is already working on emulation > something more recent (ICH9?) to get some modern stuff, so that will > probably get us some more ioapic lines. Yes, I'm working on q35 (gmch, ich9) chipset emulators. I'm planning to post the (unfinished) updated patches soon for review once I finish debugging MMCONFIG access. I also wanted IOAPIC and more IRQs as ich9 supports 8 IRQs (IRQ A-H). Possibly I will switch from q35 to x58(ICH10) for PCIe hotplug however I don't have x58 based real machine at hand...
Gerd Hoffmann <kraxel@redhat.com> writes: > Any hints where do dig in the registy? I have a dead notebook with xp > (disk still ok), trying to boot the disk image in kvm, and of course > it doesn't work due to the hardware being different. Googleing the > STOP code printed on the blue screen hinted that xp fails to access > the hard drive ... Is this the info you are looking for? http://support.microsoft.com/kb/314082 I recently went through this, using chntpw to edit the registry on the disk image and eventually getting it to boot. Also had to add the intelide.sys driver as my image lacked this for some reason. Bjørn
Avi Kivity wrote: > On 08/26/2009 07:03 PM, Gerd Hoffmann wrote: >> On 08/09/09 18:44, Avi Kivity wrote: >>> Instead of calling the IOAPIC from the PIC, raise IOAPIC irqs via >>> the ISA bus. >>> As a side effect, IOAPIC lines 16-23 are enabled. >> >> Now we probably need some acpi magic to tell the guest OS that there >> are a few more IRQ lines? >> > > I wanted to route the PCI IRQs to those lines (and have eight links > instead of four). Now I don't think it's worthwhile, as every guest > that is interesting from a performance point of view has MSI support. > > So I think we should just leave these lines as is, terminated with a > 4.7K resistor to avoid picking up noise. > I was thinking of the HPET advertising one or more of these new ioapic interrupts so that it could support non-legacy operation. It can't at the moment because there are no interrupt lines available. I haven't looked into the details of what would remain to be done to make this happen, but does it sound reasonable? Sounds like at a minimum acpi needs some work.
On 08/28/2009 05:20 AM, Beth Kon wrote: > I was thinking of the HPET advertising one or more of these new ioapic > interrupts so that it could support non-legacy operation. It can't at > the moment because there are no interrupt lines available. I haven't > looked into the details of what would remain to be done to make this > happen, but does it sound reasonable? Sounds like at a minimum acpi > needs some work. What does non-legacy HPET buy us?
Avi Kivity wrote: > On 08/28/2009 05:20 AM, Beth Kon wrote: >> I was thinking of the HPET advertising one or more of these new >> ioapic interrupts so that it could support non-legacy operation. It >> can't at the moment because there are no interrupt lines available. I >> haven't looked into the details of what would remain to be done to >> make this happen, but does it sound reasonable? Sounds like at a >> minimum acpi needs some work. > > What does non-legacy HPET buy us? > Actually, it looks like not much. I was thinking about users of /dev/hpet, but there really don't seem to be any. I found posts talking about deprecating the hpet ioctl interface due to availability of POSIX timers, so it doesn't look like anything is on the horizon. Right now, linux has no HPET requirements that aren't handled by timers 0 and 1 that are currently implemented for qemu/kvm, and as far as I can tell, Windows has no such need either. But, to be precise, even if the HPET runs in legacy mode, the standard hardware implementation as I understand it has a third timer available that is not involved with legacy operation. Because there were no spare interrupts around, I recently submitted patches to remove that third timer (that doesn't appear to be used anywhere). When I saw the possible availability of more APIC interrupt lines, I thought about using that for at least this third timer. The above thoughts came about after Andriy Gapon brought up some issues with available HPET interrupts. Andriy, do you have anything to add here about potential use of a third timer or non-legacy mode? Right now I'm inclined to think there isn't much use, and limiting HPET to legacy mode would be fine.
diff --git a/hw/i8259.c b/hw/i8259.c index 0b9fab5..74acc39 100644 --- a/hw/i8259.c +++ b/hw/i8259.c @@ -60,9 +60,6 @@ struct PicState2 { PicState pics[2]; qemu_irq parent_irq; void *irq_request_opaque; - /* IOAPIC callback support */ - SetIRQFunc *alt_irq_func; - void *alt_irq_opaque; }; #if defined(DEBUG_PIC) || defined (DEBUG_IRQ_COUNT) @@ -203,9 +200,6 @@ static void i8259_set_irq(void *opaque, int irq, int level) } #endif pic_set_irq1(&s->pics[irq >> 3], irq & 7, level); - /* used for IOAPIC irqs */ - if (s->alt_irq_func) - s->alt_irq_func(s->alt_irq_opaque, irq, level); pic_update_irq(s); } @@ -562,10 +556,3 @@ qemu_irq *i8259_init(qemu_irq parent_irq) isa_pic = s; return qemu_allocate_irqs(i8259_set_irq, s, 16); } - -void pic_set_alt_irq_func(PicState2 *s, SetIRQFunc *alt_irq_func, - void *alt_irq_opaque) -{ - s->alt_irq_func = alt_irq_func; - s->alt_irq_opaque = alt_irq_opaque; -} diff --git a/hw/ioapic.c b/hw/ioapic.c index a5cdd5d..998894d 100644 --- a/hw/ioapic.c +++ b/hw/ioapic.c @@ -241,9 +241,10 @@ static CPUWriteMemoryFunc *ioapic_mem_write[3] = { ioapic_mem_writel, }; -IOAPICState *ioapic_init(void) +qemu_irq *ioapic_init(void) { IOAPICState *s; + qemu_irq *irq; int io_memory; s = qemu_mallocz(sizeof(IOAPICState)); @@ -255,6 +256,7 @@ IOAPICState *ioapic_init(void) register_savevm("ioapic", 0, 1, ioapic_save, ioapic_load, s); qemu_register_reset(ioapic_reset, s); + irq = qemu_allocate_irqs(ioapic_set_irq, s, IOAPIC_NUM_PINS); - return s; + return irq; } diff --git a/hw/pc.c b/hw/pc.c index a6be4a8..5182a57 100644 --- a/hw/pc.c +++ b/hw/pc.c @@ -60,7 +60,6 @@ static fdctrl_t *floppy_controller; static RTCState *rtc_state; static PITState *pit; -static IOAPICState *ioapic; static PCIDevice *i440fx_state; typedef struct rom_reset_data { @@ -89,14 +88,18 @@ static void option_rom_setup_reset(target_phys_addr_t addr, unsigned size) typedef struct isa_irq_state { qemu_irq *i8259; + qemu_irq *ioapic; } IsaIrqState; static void isa_irq_handler(void *opaque, int n, int level) { IsaIrqState *isa = (IsaIrqState *)opaque; - qemu_set_irq(isa->i8259[n], level); -} + if (n < 16) { + qemu_set_irq(isa->i8259[n], level); + } + qemu_set_irq(isa->ioapic[n], level); +}; static void ioport80_write(void *opaque, uint32_t addr, uint32_t data) { @@ -1279,7 +1282,7 @@ static void pc_init1(ram_addr_t ram_size, i8259 = i8259_init(cpu_irq[0]); isa_irq_state = qemu_mallocz(sizeof(*isa_irq_state)); isa_irq_state->i8259 = i8259; - isa_irq = qemu_allocate_irqs(isa_irq_handler, isa_irq_state, 16); + isa_irq = qemu_allocate_irqs(isa_irq_handler, isa_irq_state, 24); ferr_irq = isa_irq[13]; if (pci_enabled) { @@ -1321,16 +1324,13 @@ static void pc_init1(ram_addr_t ram_size, register_ioport_write(0x92, 1, 1, ioport92_write, NULL); if (pci_enabled) { - ioapic = ioapic_init(); + isa_irq_state->ioapic = ioapic_init(); } pit = pit_init(0x40, isa_irq[0]); pcspk_init(pit); if (!no_hpet) { hpet_init(isa_irq); } - if (pci_enabled) { - pic_set_alt_irq_func(isa_pic, ioapic_set_irq, ioapic); - } for(i = 0; i < MAX_SERIAL_PORTS; i++) { if (serial_hds[i]) { diff --git a/hw/pc.h b/hw/pc.h index 9fbae20..043c216 100644 --- a/hw/pc.h +++ b/hw/pc.h @@ -32,8 +32,6 @@ extern PicState2 *isa_pic; void pic_set_irq(int irq, int level); void pic_set_irq_new(void *opaque, int irq, int level); qemu_irq *i8259_init(qemu_irq parent_irq); -void pic_set_alt_irq_func(PicState2 *s, SetIRQFunc *alt_irq_func, - void *alt_irq_opaque); int pic_read_irq(PicState2 *s); void pic_update_irq(PicState2 *s); uint32_t pic_intack_read(PicState2 *s); @@ -50,7 +48,7 @@ int apic_init(CPUState *env); int apic_accept_pic_intr(CPUState *env); void apic_deliver_pic_intr(CPUState *env, int level); int apic_get_interrupt(CPUState *env); -IOAPICState *ioapic_init(void); +qemu_irq *ioapic_init(void); void ioapic_set_irq(void *opaque, int vector, int level); void apic_reset_irq_delivered(void); int apic_get_irq_delivered(void);
Instead of calling the IOAPIC from the PIC, raise IOAPIC irqs via the ISA bus. As a side effect, IOAPIC lines 16-23 are enabled. Signed-off-by: Avi Kivity <avi@redhat.com> --- hw/i8259.c | 13 ------------- hw/ioapic.c | 6 ++++-- hw/pc.c | 16 ++++++++-------- hw/pc.h | 4 +--- 4 files changed, 13 insertions(+), 26 deletions(-)