From patchwork Tue Jan 24 14:07:56 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Srinivas Kandagatla X-Patchwork-Id: 719145 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 3v794f1bbfz9s9Y for ; Wed, 25 Jan 2017 01:08:02 +1100 (AEDT) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="iUH1CosQ"; dkim-atps=neutral Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1750977AbdAXOIA (ORCPT ); Tue, 24 Jan 2017 09:08:00 -0500 Received: from mail-wm0-f49.google.com ([74.125.82.49]:38784 "EHLO mail-wm0-f49.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750969AbdAXOH7 (ORCPT ); Tue, 24 Jan 2017 09:07:59 -0500 Received: by mail-wm0-f49.google.com with SMTP id r144so210811805wme.1 for ; Tue, 24 Jan 2017 06:07:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:subject:to:cc:message-id:date:user-agent:mime-version :content-transfer-encoding; bh=Tx75uVq4gJ7pKERdFoIkKmjp0dUwFf8znPr0TBD3FTI=; b=iUH1CosQ+2IVU+7g+fMtLUZx1nFfmocCxqU4h5J3TVEbiBB3AeQIu8Gsjc2uedvpZz 9w06j4ZCnQ5G6gGe9XB+yyi3r3SKgaFjoOO8vvnYzyko7kgHn3XSqPiwP3FpmChXryIg SmbrtfHNWI9V3ZrishzeM5Xvypk3v+CKQFvXo= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:subject:to:cc:message-id:date:user-agent :mime-version:content-transfer-encoding; bh=Tx75uVq4gJ7pKERdFoIkKmjp0dUwFf8znPr0TBD3FTI=; b=EG8QunMjdRF8TUiOt+HqAajwvaWs152RseJpqTm/5k/gZEz56d2Bu4UKpoqvAl76MI gSU7mHQudi6VDd+uGSPhWb3X3FJeKRhsYqNgrARUy0sTibcmn7HrFH5HhBn+bXEdllo8 RetgEs0fhgrq7qNPx67gEGgSQ+TbRyRZ4ipj0ZMwYUlJtIXe2QmhoNG7TB8pVmuwV+51 hCFjWoLzUwRQDy1NT5LqLEFNF/q8e/UsgKyp3JdQnWqtf0DVAHXmtXbG5dUSQ5js0Qy+ udfnQvPzk57cNAU09f/gb7afOuuzpLzxt/m56gI81xLBqMdskp+6Pp6mnUpKKOwl93T2 sy8g== X-Gm-Message-State: AIkVDXJMTbt5eOUazTMkKHAgPyP+p6Wgv2CGp4IMI8tcYGf8L3pSpHgCoqK2yb7Lv6RNwFB8 X-Received: by 10.28.46.74 with SMTP id u71mr17041918wmu.136.1485266877686; Tue, 24 Jan 2017 06:07:57 -0800 (PST) Received: from [192.168.1.180] (host-2-98-105-38.as13285.net. [2.98.105.38]) by smtp.googlemail.com with ESMTPSA id s26sm20091576wra.26.2017.01.24.06.07.56 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 24 Jan 2017 06:07:57 -0800 (PST) From: Srinivas Kandagatla Subject: pcie-designware: Incorrect programming of msi address register on some setups. To: Jingoo Han , Joao Pinto , Bjorn Helgaas Cc: linux-pci@vger.kernel.org, Bjorn Andersson , Arnd Bergmann , "linux-arm-kernel@lists.infradead.org" Message-ID: <922851d0-a847-762a-5067-88c92d416d58@linaro.org> Date: Tue, 24 Jan 2017 14:07:56 +0000 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org Hi, Recently I hit a bug with pcie-designware driver with triggers a board reboot, this is my analysis so far on the issue. Issue is because designware driver is programming msi address which is above 32 address space eventhough one of the pcie endpoint indicated that it does not support 64 bit msi addresses. My setup is arm64 board DB820c with 3 root complexes, and the system has memory which spans across 4GB address space. First root complex is connected to a WLAN Chip (QCA6174) only supports 32 bit addresses. Second one is connected to a Ethernet Controller (ATL1c) value of MSI control register aka PCI_MSI_FLAGS (0x2) for device QCA6174 is 0x106 value of MSI control register aka PCI_MSI_FLAGS (0x2) for device ATL1C is 0x80 Now the problem is that on Synopsis IP, the MSI address register is programmed at two places 1> one at controller config dbi address space itself in PCIE_MSI_ADDR_LO(0x820) & PCIE_MSI_ADDR_HI (0x824) 2> at PCI_MSI_ADDRESS_LO (0x2) and PCI_MSI_ADDRESS_HI of the device cfg. If step 1 allocates msi address which is above 32 bit address space, then step 2 would truncate it to 32 bit address if it discovers endpoint which does not support 64 bit msi address. Then if endpoint tries to read/write this address as part of MSI mechanism it would fault and crash system. Am not sure how we can solve this correctly because we do not know about endpoints at step 1. On the other hand if we can force step 1 to allocate addresses with in 32 bit address space than it would work for both 32 bit and 64 bit. Has anyone hit this issue before? Do we already have a solution? Any suggestions? Below ugly Hack is what Am using to workaround this issue on my platform but am not sure how this will influence other platforms. ---------------------------->cut<------------------------------- ---------------------------->cut<------------------------------- Thanks, srini --- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/drivers/pci/host/pcie-designware.c b/drivers/pci/host/pcie-designware.c index 4a81b72..d694e66 100644 --- a/drivers/pci/host/pcie-designware.c +++ b/drivers/pci/host/pcie-designware.c @@ -286,9 +286,8 @@ void dw_pcie_msi_init(struct pcie_port *pp) { u64 msi_target; - pp->msi_data = __get_free_pages(GFP_KERNEL, 0); + pp->msi_data = __get_free_pages(GFP_KERNEL | GFP_DMA, 0); msi_target = virt_to_phys((void *)pp->msi_data); /* program the msi_data */ dw_pcie_wr_own_conf(pp, PCIE_MSI_ADDR_LO, 4, (u32)(msi_target & 0xffffffff));