From patchwork Mon Apr 11 09:19:22 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 608716 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3qk4Tv0p5Dz9ssM for ; Mon, 11 Apr 2016 19:27:31 +1000 (AEST) Received: from localhost ([::1]:51003 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1apY81-0007Te-B7 for incoming@patchwork.ozlabs.org; Mon, 11 Apr 2016 05:27:29 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:53364) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1apY27-0007uV-Cw for qemu-devel@nongnu.org; Mon, 11 Apr 2016 05:21:24 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1apY22-0007R7-Ae for qemu-devel@nongnu.org; Mon, 11 Apr 2016 05:21:23 -0400 Received: from mx1.redhat.com ([209.132.183.28]:50144) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1apY22-0007R3-3H for qemu-devel@nongnu.org; Mon, 11 Apr 2016 05:21:18 -0400 Received: from int-mx11.intmail.prod.int.phx2.redhat.com (int-mx11.intmail.prod.int.phx2.redhat.com [10.5.11.24]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 4A5EDD7FC3; Mon, 11 Apr 2016 09:21:17 +0000 (UTC) Received: from pxdev.xzpeter.org.com (vpn1-4-150.pek2.redhat.com [10.72.4.150]) by int-mx11.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id u3B9JTU0007357; Mon, 11 Apr 2016 05:21:08 -0400 From: Peter Xu To: qemu-devel@nongnu.org Date: Mon, 11 Apr 2016 17:19:22 +0800 Message-Id: <1460366363-4589-13-git-send-email-peterx@redhat.com> In-Reply-To: <1460366363-4589-1-git-send-email-peterx@redhat.com> References: <1460366363-4589-1-git-send-email-peterx@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.24 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.39]); Mon, 11 Apr 2016 09:21:17 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v2 12/13] intel_iommu: ioapic: IR support for emulated IOAPIC X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: ehabkost@redhat.com, mst@redhat.com, jasowang@redhat.com, rkrcmar@redhat.com, peterx@redhat.com, jan.kiszka@web.de, pbonzini@redhat.com, marcel@redhat.com, imammedo@redhat.com, rth@twiddle.net Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" This patch add the first device support for Intel IOMMU interrupt remapping, which is the default IOAPIC device created alongside with Q35 platform. This will be the first step along the way to fully enable IOMMU IR on x86 systems. Currently, only emluated IOAPIC is supported. This requires "kernel_irqchip=off" parameter specified. Originally, IOAPIC has its own table to maintain IRQ information. When IOMMU IR is enabled, guest OS will fill in the real IRQ data into IRTE entries of IOMMU IR root table, while in IOAPIC only the index information is maintained (with several legacy bits which might not be covered by VT-d IR). If so, we need to talk to IOMMU to get the real IRQ information to deliver. Please refer to VT-d spec 5.1.5.1 for more information. Signed-off-by: Peter Xu --- hw/i386/intel_iommu.c | 127 ++++++++++++++++++++++++++++++++++++++++++ hw/intc/ioapic.c | 36 +++++++++--- include/hw/i386/intel_iommu.h | 17 ++++++ 3 files changed, 173 insertions(+), 7 deletions(-) diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c index a44289f..1dcdc7b 100644 --- a/hw/i386/intel_iommu.c +++ b/hw/i386/intel_iommu.c @@ -2014,6 +2014,133 @@ IntelIOMMUState *vtd_iommu_get(void) return (IntelIOMMUState *)intel_iommu; } +/* Read IRTE entry with specific index */ +static int vtd_irte_get(IntelIOMMUState *iommu, uint16_t index, + VTD_IRTE *entry) +{ + dma_addr_t addr = 0x00; + + addr = iommu->intr_root + index * sizeof(*entry); + if (dma_memory_read(&address_space_memory, addr, entry, + sizeof(*entry))) { + VTD_DPRINTF(GENERAL, "error: fail to access IR root at 0x%"PRIx64 + " + %"PRIu16, iommu->intr_root, index); + return -VTD_FR_IR_ROOT_INVAL; + } + + if (!entry->present) { + VTD_DPRINTF(GENERAL, "error: present flag not set in IRTE" + " entry index %u value 0x%"PRIx64 " 0x%"PRIx64, + index, le64_to_cpu(entry->data[1]), + le64_to_cpu(entry->data[0])); + return -VTD_FR_IR_ENTRY_P; + } + + if (entry->__reserved_0 || entry->__reserved_1 || \ + entry->__reserved_2) { + VTD_DPRINTF(GENERAL, "error: IRTE entry index %"PRIu16 + " reserved fields non-zero: 0x%"PRIx64 " 0x%"PRIx64, + index, le64_to_cpu(entry->data[1]), + le64_to_cpu(entry->data[0])); + return -VTD_FR_IR_IRTE_RSVD; + } + + /* + * TODO: Check Source-ID corresponds to SVT (Source Validation + * Type) bits + */ + + return 0; +} + +/* Fetch IRQ information of specific IR index */ +static int vtd_remap_irq_get(IntelIOMMUState *iommu, uint16_t index, VTDIrq *irq) +{ + VTD_IRTE irte; + int ret = 0; + + bzero(&irte, sizeof(irte)); + + ret = vtd_irte_get(iommu, index, &irte); + if (ret) { + return ret; + } + + irq->trigger_mode = irte.trigger_mode; + irq->vector = irte.vector; + irq->delivery_mode = irte.delivery_mode; + /* Not support EIM yet: please refer to vt-d 9.10 DST bits */ +#define VTD_IR_APIC_DEST_MASK (0xff00ULL) +#define VTD_IR_APIC_DEST_SHIFT (8) + irq->dest = (irte.dest_id & VTD_IR_APIC_DEST_MASK) >> \ + VTD_IR_APIC_DEST_SHIFT; + irq->dest_mode = irte.dest_mode; + + VTD_DPRINTF(IR, "remapping interrupt index %d: trig:%u,vec:%u," + "deliver:%u,dest:%u,dest_mode:%u", index, + irq->trigger_mode, irq->vector, irq->delivery_mode, + irq->dest, irq->dest_mode); + + return 0; +} + +/* Interrupt remapping for IOAPIC IRQ entry */ +int vtd_interrupt_remap_ioapic(IntelIOMMUState *iommu, + uint64_t *ioapic_entry, VTDIrq *irq) +{ + int ret = 0; + uint16_t index = 0; + VTD_IR_IOAPICEntry *entry = (VTD_IR_IOAPICEntry *)ioapic_entry; + + assert(iommu && entry && irq); + assert(iommu->intr_enabled); + + /* TODO: Currently we still do not support compatible mode */ + if (entry->int_mode != VTD_IR_INT_FORMAT_REMAP) { + VTD_DPRINTF(GENERAL, "error: trying to remap IOAPIC entry" + " with compatible format: 0x%"PRIx64, + le64_to_cpu(entry->data)); + return -VTD_FR_IR_REQ_COMPAT; + } + + if (entry->__zeros || entry->__reserved) { + VTD_DPRINTF(GENERAL, "error: reserved not empty for IOAPIC" + "entry 0x%"PRIx64, le64_to_cpu(entry->data)); + return -VTD_FR_IR_REQ_RSVD; + } + + index = entry->index_h << 15 | entry->index_l; + ret = vtd_remap_irq_get(iommu, index, irq); + if (ret) { + return ret; + } + + /* Trigger mode should be aligned between IOAPIC entry and IRTE + * entry */ + if (irq->trigger_mode != entry->trigger_mode) { + /* This is possibly guest OS bug?! */ + VTD_DPRINTF(GENERAL, "error: IOAPIC trigger mode inconsistent: " + "0x%"PRIx64 " with IR table index %d", + le64_to_cpu(entry->data), index); + /* Currently no such error defined */ + return -VTD_FR_RESERVED_ERR; + } + + /* Vector should be aligned too */ + if (irq->vector != entry->vector) { + /* + * Latest linux kernel will not provide consistent + * vectors. Need some more digging to know why. Whatever, + * the one in IRTE is always correct. So directly use it. + */ + VTD_DPRINTF(IR, "warn: IOAPIC vector inconsistent: " + "index %d: entry=%d, IRTE=%d", index, + entry->vector, irq->vector); + } + + return 0; +} + /* Do the initialization. It will also be called when reset, so pay * attention when adding new initialization stuff. */ diff --git a/hw/intc/ioapic.c b/hw/intc/ioapic.c index 378e663..d963d45 100644 --- a/hw/intc/ioapic.c +++ b/hw/intc/ioapic.c @@ -57,6 +57,8 @@ static void ioapic_service(IOAPICCommonState *s) uint64_t entry; uint8_t dest; uint8_t dest_mode; + IntelIOMMUState *iommu = s->iommu; + VTDIrq irq = {0}; for (i = 0; i < IOAPIC_NUM_PINS; i++) { mask = 1 << i; @@ -65,11 +67,33 @@ static void ioapic_service(IOAPICCommonState *s) entry = s->ioredtbl[i]; if (!(entry & IOAPIC_LVT_MASKED)) { - trig_mode = ((entry >> IOAPIC_LVT_TRIGGER_MODE_SHIFT) & 1); - dest = entry >> IOAPIC_LVT_DEST_SHIFT; - dest_mode = (entry >> IOAPIC_LVT_DEST_MODE_SHIFT) & 1; - delivery_mode = - (entry >> IOAPIC_LVT_DELIV_MODE_SHIFT) & IOAPIC_DM_MASK; + + if (iommu && iommu->intr_enabled) { + /* + * Interrupt remapping is enabled in owner IOMMU, + * we need to fetch the real IRQ information via + * IRTE of the root mapping table + */ + if (vtd_interrupt_remap_ioapic(iommu, &entry, &irq)) { + DPRINTF("%s: IOAPIC remap fail on index %d " + "entry 0x%lx, drop it for now\n", + __func__, index, entry); + return; + } + trig_mode = irq.trigger_mode; + dest = irq.dest; + dest_mode = irq.dest_mode; + delivery_mode = irq.delivery_mode; + vector = irq.vector; + } else { + /* This is generic IOAPIC entry */ + trig_mode = ((entry >> IOAPIC_LVT_TRIGGER_MODE_SHIFT) & 1); + dest = entry >> IOAPIC_LVT_DEST_SHIFT; + dest_mode = (entry >> IOAPIC_LVT_DEST_MODE_SHIFT) & 1; + delivery_mode = + (entry >> IOAPIC_LVT_DELIV_MODE_SHIFT) & IOAPIC_DM_MASK; + vector = entry & IOAPIC_VECTOR_MASK; + } if (trig_mode == IOAPIC_TRIGGER_EDGE) { s->irr &= ~mask; } else { @@ -78,8 +102,6 @@ static void ioapic_service(IOAPICCommonState *s) } if (delivery_mode == IOAPIC_DM_EXTINT) { vector = pic_read_irq(isa_pic); - } else { - vector = entry & IOAPIC_VECTOR_MASK; } #ifdef CONFIG_KVM if (kvm_irqchip_is_split()) { diff --git a/include/hw/i386/intel_iommu.h b/include/hw/i386/intel_iommu.h index 9ee84f7..6a79207 100644 --- a/include/hw/i386/intel_iommu.h +++ b/include/hw/i386/intel_iommu.h @@ -23,6 +23,7 @@ #define INTEL_IOMMU_H #include "hw/qdev.h" #include "sysemu/dma.h" +#include "hw/i386/ioapic.h" #define TYPE_INTEL_IOMMU_DEVICE "intel-iommu" #define INTEL_IOMMU_DEVICE(obj) \ @@ -55,6 +56,7 @@ typedef struct VTDBus VTDBus; typedef union VTD_IRTE VTD_IRTE; typedef union VTD_IR_IOAPICEntry VTD_IR_IOAPICEntry; typedef union VTD_IR_MSIAddress VTD_IR_MSIAddress; +typedef struct VTDIrq VTDIrq; /* Context-Entry */ struct VTDContextEntry { @@ -116,6 +118,9 @@ union VTD_IRTE { uint64_t data[2]; }; +#define VTD_IR_INT_FORMAT_COMPAT (0) /* Compatible Interrupt */ +#define VTD_IR_INT_FORMAT_REMAP (1) /* Remappable Interrupt */ + /* Programming format for IOAPIC table entries */ union VTD_IR_IOAPICEntry { struct { @@ -147,6 +152,15 @@ union VTD_IR_MSIAddress { uint32_t data; }; +/* Generic IRQ entry information */ +struct VTDIrq { + uint8_t trigger_mode; + uint8_t vector; + uint8_t delivery_mode; + uint32_t dest; + uint8_t dest_mode; +}; + /* When IR is enabled, all MSI/MSI-X data bits should be zero */ #define VTD_IR_MSI_DATA (0) @@ -198,5 +212,8 @@ struct IntelIOMMUState { VTDAddressSpace *vtd_find_add_as(IntelIOMMUState *s, PCIBus *bus, int devfn); /* Get default IOMMU object */ IntelIOMMUState *vtd_iommu_get(void); +/* Interrupt remapping for IOAPIC IRQ entry */ +int vtd_interrupt_remap_ioapic(IntelIOMMUState *iommu, + uint64_t *ioapic_entry, VTDIrq *irq); #endif