From patchwork Fri Sep 20 08:00:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Neil Armstrong X-Patchwork-Id: 1987763 X-Patchwork-Delegate: narmstrong@baylibre.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.a=rsa-sha256 header.s=google header.b=Q5fBekqA; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=patchwork.ozlabs.org) Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4X94c871zXz1xrD for ; Fri, 20 Sep 2024 18:02:16 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 4EE7A8941F; Fri, 20 Sep 2024 10:00:47 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (2048-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="Q5fBekqA"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 07DD8893FD; Fri, 20 Sep 2024 10:00:44 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE,T_SPF_TEMPERROR autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-wr1-x430.google.com (mail-wr1-x430.google.com [IPv6:2a00:1450:4864:20::430]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id C7A5E88DA0 for ; Fri, 20 Sep 2024 10:00:35 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=neil.armstrong@linaro.org Received: by mail-wr1-x430.google.com with SMTP id ffacd0b85a97d-374c1e5fe79so1162542f8f.1 for ; Fri, 20 Sep 2024 01:00:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1726819235; x=1727424035; darn=lists.denx.de; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=Q3f/dQcwAhnGcS9GQKvNZXgCdNcW1/Vm7vzvFXIm/Mo=; b=Q5fBekqAPxzzJ6JtPCqB/LPjoS7WsCIqGUPCxwgg1WAhIeaWJ8Bn+2LIHy1pwbgEtl kvqUO+H4WVRJ1tzU51Doup+JFpl084t6o7PFX5m0zXTefanqACF8vgtJHQPefe+td2CF rMgQdM9DYCcpkd+kOISqRVuRbpEPivdh1+2jPjPEICHPsEQPiCyT7UvxSuM+NicqWsoQ 42Tj8vooW5BKjUUGfIEvj4mRiOk/PyPoYnpoWwCiCFHsusx9lR7wFo58AshmjzS+shcf nsJKw2rihQr7b7N0X6Hn663VoRH+j0ORyj94TCYzww7RuyI9fkz0CFn4G4dYY38uJLEk INug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1726819235; x=1727424035; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Q3f/dQcwAhnGcS9GQKvNZXgCdNcW1/Vm7vzvFXIm/Mo=; b=os761h/Z5tJYzFihNedQEldahxfzlUDw2grzMgRDZwLuIDhLX1jAFN8yQBvBSLiEjU ou+PvziPTCVIHp+fGB9nl2D64ztNphHzzngxsAKA/ILUyFyfcaci/75ytTtvNBoHqdpZ JL9BWbnfsvyYKCiA++GKtyHj5e1MRASjeEI9B/3yeQpSdiVrazVKOLscV6c08C/6xobc xdZwu+XCMV+lKWsRlmcQ+5JEDROzg3pFXuUiurfUj5snOh1fk10DByQOQPQS5g3g33jR 3LKfL0emuwaJNtlrtRr8e3Jxt4hgfJ3m0/7ZxFGasVCosnvaYhHI7RtXzPW2+jbTT2Y6 lpEw== X-Forwarded-Encrypted: i=1; AJvYcCW81BvrQyRJzrg+PALKJQRtSCwAXwNWV06C1+C35T3ww9Q605i2sbMIdTvBncDnFV9h54hGIo4=@lists.denx.de X-Gm-Message-State: AOJu0YwC8dpxeZW8GzJk+ZUVLYN+y6V7K/JWS8U9zTmbxOhu76gBy1wn XwtRZDi0+00vuCl6qoL06EQ9d+ZhTpfvpSfd9Ea1c2+QAi26pwBR3sa+9RJ7r5Q= X-Google-Smtp-Source: AGHT+IFLtdRo7kbKBybo9LC+9xSYzFqa7HLTfWTObBWoT6ETkhvkdIEEtaArsiR4kJndRTqg4ZC0fw== X-Received: by 2002:a5d:4488:0:b0:377:9867:9028 with SMTP id ffacd0b85a97d-37a431ad4dbmr920962f8f.44.1726819235124; Fri, 20 Sep 2024 01:00:35 -0700 (PDT) Received: from arrakeen.starnux.net ([2a01:e0a:982:cbb0:52eb:f6ff:feb3:451a]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-378e7800313sm17059497f8f.88.2024.09.20.01.00.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 Sep 2024 01:00:34 -0700 (PDT) From: Neil Armstrong Date: Fri, 20 Sep 2024 10:00:24 +0200 Subject: [PATCH v2 03/13] ufs: split flush and invalidate to only invalidate when required MIME-Version: 1.0 Message-Id: <20240920-topic-ufs-enhancements-v2-3-65ae61e73eaa@linaro.org> References: <20240920-topic-ufs-enhancements-v2-0-65ae61e73eaa@linaro.org> In-Reply-To: <20240920-topic-ufs-enhancements-v2-0-65ae61e73eaa@linaro.org> To: Tom Rini , Bhupesh Sharma , Neha Malcom Francis Cc: Michal Simek , Marek Vasut , bmeng.cn@gmail.com, u-boot@lists.denx.de, u-boot-qcom@groups.io, Neil Armstrong , Venkatesh Yadav Abbarapu X-Mailer: b4 0.14.1 X-Developer-Signature: v=1; a=openpgp-sha256; l=5705; i=neil.armstrong@linaro.org; h=from:subject:message-id; bh=dZauaajcNAzB67hUPDnxtn7pEfA386aQNWWZ4IUJBzg=; b=owEBbQKS/ZANAwAKAXfc29rIyEnRAcsmYgBm7Sua4JjCflgT+dtqqBgCrTxtB+yKq/3ZGF4x1KBv 3t7/QV6JAjMEAAEKAB0WIQQ9U8YmyFYF/h30LIt33NvayMhJ0QUCZu0rmgAKCRB33NvayMhJ0S8KEA CT+d3si6kY98GQFHfYpP4tOJbFGkPlWg42mO9tAqB1zcDtAWEbCTZkJHReckPabOsR8ajJSgTedppU zt0HBa1+Fcm2p11a8UWz4ZUQoIYIYfqUk2UYEDJ/2zvfRyQH3/dcuZ4jZ23uF3NkEs82ZabARCtcax SVDDTmaEFCq8a3TDy7eW+tBe7h6Mu0SOB5c+66kUJDlCVqscTFuzJmT881W5mpG3lRCcnx7oPcWAcE sFtIQhUj/3fTkw8s2hAALzE396H3E4iqikk8IgNveHmGHT3GrJQw1nlQMNK41S1GU9Gun02bjAD8KY 0JduV0kygJykP8GfPnvX2KwOegsapAb+qmVwpkftnfd2x8lO9JeW+HQ84KiGvGfcVz7GbbBxE0BNHL BTB0wB3Ij8hcb0vvw7naoYBp8L/Y6cxGugQGG9ahyGoqhH6qVb7b2r1uS/2drqADugzCM9OobxLk4F eLePLfA2vlwONwXYPyh88SSNn2jHk5IpdzSZ8I41IxFUC3utNsJazENsV9BZpyWeueY4JjAz/537SI NSd7RnZAAsAdsb0wksaH21lNS6YyTVgp/FE5yd+4iUAE1dCZJ38MaSt1FjDFgo8XSOeL0JfW1Htdeo aeNlknuO4yYOwOSoXc8lroVeSpyM7ltuaO4tmGlX6Jbetio48IvL/ZlQ5cmQ== X-Developer-Key: i=neil.armstrong@linaro.org; a=openpgp; fpr=89EC3D058446217450F22848169AB7B1A4CFF8AE X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean There is no need to flush and invalidate all data updated by the driver, mainly because on ARM platforms flush also invalidates the cachelines. Split the function in two and add the appropriate cacheline invalidates after the UFS DMA operation finishes to make sure we read from memory. Flushing then invalidating cacheline unaligned data causes data corruption issues on Qualcomm platforms, and is largely unnecessary anyway, so let's cleanup the cache operations. Reviewed-by: Neha Malcom Francis Tested-by: Venkatesh Yadav Abbarapu Signed-off-by: Neil Armstrong --- drivers/ufs/ufs.c | 45 ++++++++++++++++++++++++++++++--------------- 1 file changed, 30 insertions(+), 15 deletions(-) diff --git a/drivers/ufs/ufs.c b/drivers/ufs/ufs.c index 3d9a7d7ee12..5845fd694d3 100644 --- a/drivers/ufs/ufs.c +++ b/drivers/ufs/ufs.c @@ -696,17 +696,28 @@ static inline u8 ufshcd_get_upmcrs(struct ufs_hba *hba) } /** - * ufshcd_cache_flush_and_invalidate - Flush and invalidate cache + * ufshcd_cache_flush - Flush cache * - * Flush and invalidate cache in aligned address..address+size range. - * The invalidation is in place to avoid stale data in cache. + * Flush cache in aligned address..address+size range. */ -static void ufshcd_cache_flush_and_invalidate(void *addr, unsigned long size) +static void ufshcd_cache_flush(void *addr, unsigned long size) { uintptr_t start_addr = (uintptr_t)addr & ~(ARCH_DMA_MINALIGN - 1); uintptr_t end_addr = ALIGN((uintptr_t)addr + size, ARCH_DMA_MINALIGN); flush_dcache_range(start_addr, end_addr); +} + +/** + * ufshcd_cache_invalidate - Invalidate cache + * + * Invalidate cache in aligned address..address+size range. + */ +static void ufshcd_cache_invalidate(void *addr, unsigned long size) +{ + uintptr_t start_addr = (uintptr_t)addr & ~(ARCH_DMA_MINALIGN - 1); + uintptr_t end_addr = ALIGN((uintptr_t)addr + size, ARCH_DMA_MINALIGN); + invalidate_dcache_range(start_addr, end_addr); } @@ -754,7 +765,7 @@ static void ufshcd_prepare_req_desc_hdr(struct ufs_hba *hba, req_desc->prd_table_length = 0; - ufshcd_cache_flush_and_invalidate(req_desc, sizeof(*req_desc)); + ufshcd_cache_flush(req_desc, sizeof(*req_desc)); } static void ufshcd_prepare_utp_query_req_upiu(struct ufs_hba *hba, @@ -785,13 +796,13 @@ static void ufshcd_prepare_utp_query_req_upiu(struct ufs_hba *hba, /* Copy the Descriptor */ if (query->request.upiu_req.opcode == UPIU_QUERY_OPCODE_WRITE_DESC) { memcpy(ucd_req_ptr + 1, query->descriptor, len); - ufshcd_cache_flush_and_invalidate(ucd_req_ptr, 2 * sizeof(*ucd_req_ptr)); + ufshcd_cache_flush(ucd_req_ptr, 2 * sizeof(*ucd_req_ptr)); } else { - ufshcd_cache_flush_and_invalidate(ucd_req_ptr, sizeof(*ucd_req_ptr)); + ufshcd_cache_flush(ucd_req_ptr, sizeof(*ucd_req_ptr)); } memset(hba->ucd_rsp_ptr, 0, sizeof(struct utp_upiu_rsp)); - ufshcd_cache_flush_and_invalidate(hba->ucd_rsp_ptr, sizeof(*hba->ucd_rsp_ptr)); + ufshcd_cache_flush(hba->ucd_rsp_ptr, sizeof(*hba->ucd_rsp_ptr)); } static inline void ufshcd_prepare_utp_nop_upiu(struct ufs_hba *hba) @@ -809,8 +820,8 @@ static inline void ufshcd_prepare_utp_nop_upiu(struct ufs_hba *hba) memset(hba->ucd_rsp_ptr, 0, sizeof(struct utp_upiu_rsp)); - ufshcd_cache_flush_and_invalidate(ucd_req_ptr, sizeof(*ucd_req_ptr)); - ufshcd_cache_flush_and_invalidate(hba->ucd_rsp_ptr, sizeof(*hba->ucd_rsp_ptr)); + ufshcd_cache_flush(ucd_req_ptr, sizeof(*ucd_req_ptr)); + ufshcd_cache_flush(hba->ucd_rsp_ptr, sizeof(*hba->ucd_rsp_ptr)); } /** @@ -877,6 +888,8 @@ static int ufshcd_send_command(struct ufs_hba *hba, unsigned int task_tag) */ static inline int ufshcd_get_req_rsp(struct utp_upiu_rsp *ucd_rsp_ptr) { + ufshcd_cache_invalidate(ucd_rsp_ptr, sizeof(*ucd_rsp_ptr)); + return be32_to_cpu(ucd_rsp_ptr->header.dword_0) >> 24; } @@ -888,6 +901,8 @@ static inline int ufshcd_get_tr_ocs(struct ufs_hba *hba) { struct utp_transfer_req_desc *req_desc = hba->utrdl; + ufshcd_cache_invalidate(req_desc, sizeof(*req_desc)); + return le32_to_cpu(req_desc->header.dword_2) & MASK_OCS; } @@ -1437,8 +1452,8 @@ void ufshcd_prepare_utp_scsi_cmd_upiu(struct ufs_hba *hba, memcpy(ucd_req_ptr->sc.cdb, pccb->cmd, cdb_len); memset(hba->ucd_rsp_ptr, 0, sizeof(struct utp_upiu_rsp)); - ufshcd_cache_flush_and_invalidate(ucd_req_ptr, sizeof(*ucd_req_ptr)); - ufshcd_cache_flush_and_invalidate(hba->ucd_rsp_ptr, sizeof(*hba->ucd_rsp_ptr)); + ufshcd_cache_flush(ucd_req_ptr, sizeof(*ucd_req_ptr)); + ufshcd_cache_flush(hba->ucd_rsp_ptr, sizeof(*hba->ucd_rsp_ptr)); } static inline void prepare_prdt_desc(struct ufshcd_sg_entry *entry, @@ -1461,7 +1476,7 @@ static void prepare_prdt_table(struct ufs_hba *hba, struct scsi_cmd *pccb) if (!datalen) { req_desc->prd_table_length = 0; - ufshcd_cache_flush_and_invalidate(req_desc, sizeof(*req_desc)); + ufshcd_cache_flush(req_desc, sizeof(*req_desc)); return; } @@ -1487,8 +1502,8 @@ static void prepare_prdt_table(struct ufs_hba *hba, struct scsi_cmd *pccb) prepare_prdt_desc(&prd_table[table_length - i - 1], buf, datalen - 1); req_desc->prd_table_length = table_length; - ufshcd_cache_flush_and_invalidate(prd_table, sizeof(*prd_table) * table_length); - ufshcd_cache_flush_and_invalidate(req_desc, sizeof(*req_desc)); + ufshcd_cache_flush(prd_table, sizeof(*prd_table) * table_length); + ufshcd_cache_flush(req_desc, sizeof(*req_desc)); } static int ufs_scsi_exec(struct udevice *scsi_dev, struct scsi_cmd *pccb)