From patchwork Mon Nov 20 15:47:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefano Babic X-Patchwork-Id: 1866107 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=googlegroups.com header.i=@googlegroups.com header.a=rsa-sha256 header.s=20230601 header.b=ihbUCpYB; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=googlegroups.com (client-ip=2a00:1450:4864:20::340; helo=mail-wm1-x340.google.com; envelope-from=swupdate+bncbaabbgh75wvamgqelehifry@googlegroups.com; receiver=patchwork.ozlabs.org) Received: from mail-wm1-x340.google.com (mail-wm1-x340.google.com [IPv6:2a00:1450:4864:20::340]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SYsMw6Xynz1ySR for ; Tue, 21 Nov 2023 02:47:40 +1100 (AEDT) Received: by mail-wm1-x340.google.com with SMTP id 5b1f17b1804b1-40856440f41sf8518555e9.1 for ; Mon, 20 Nov 2023 07:47:40 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1700495257; cv=pass; d=google.com; s=arc-20160816; b=1FxJxZlMjDleMe72qeycgCvp0whNzNL6YS6a4bJlK7LiM5bqXVklJZvPYjEpYGcYbS CfYwmGntPUime37fDSN4uSn/ZQ/4VVDMuNtFi7JGktYJX730MTdNpta3nLVk4tbyWtFT P0D/ogb2xcCx9i3uYzdZBs+7ESrzsQifD3M2rGANbFwjJDxmxnZ6maLyYZcZp2fY45WD 0AUFw2Mc1d1ZcCTJMbNfE5b6OLVOhItMGx/rZh9L1m7IhE6RNiN4Xs7UH8VNJccAH9hi /fCNtt/Wc5821IHDSx8KdqINMckTSo1HpuPSwNpe7f6JB3IVjmp4MdrdgC+REqf86D0z KtDA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:ui-outboundreport:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :dkim-signature; bh=G9Ew4mvvBZFkSJxQj9jAFpFoQlWHN4xNXUEwTRLaC0k=; fh=TiEWcqMcKpHk5s7uErzpntoONrNfOXwKpI5P8bIlggk=; b=nbTmxAlor+eZXkfil92NoZRQ6BdW+i/4ijpw68PHYpzApebx7lr0bHQffmpUDHTeGk o0jjH+KsLTCJqGF3/Xz8SFNuuU1ExdOMXKu0wHKhKBry3goxWYL6IFr9oLd6cnsD1ovd wz66dIx5xVgIfJmLsZq+SXaERb9VxY7slFxXFQ4hpi+GLeIrXfhtWBJcgUseQv+KBDO0 40vAln/y18cVHpI11K1FkhMG2GGqNnhgzYVv1Foi0+TdM8yVt352VyIfY81Guw84JJtE pr/VHgzL2Hc5k1QuW76r8RtNfkHbJvpPOhNnDD4Ss7CZqy1nzWFuSLppxG/+fPH5e9Dh 7VHA== ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of stefano.babic@swupdate.org designates 212.227.17.24 as permitted sender) smtp.mailfrom=stefano.babic@swupdate.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1700495257; x=1701100057; darn=patchwork.ozlabs.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:ui-outboundreport:mime-version:references :in-reply-to:message-id:date:subject:cc:to:from:sender:from:to:cc :subject:date:message-id:reply-to; bh=G9Ew4mvvBZFkSJxQj9jAFpFoQlWHN4xNXUEwTRLaC0k=; b=ihbUCpYBHHe1BwRTnldbS/E8N2qaig+uqkKlef+oaRvIO/5f27tBwIUnJ7kB1CZ3N6 I3elJzCfr2qYXwEXFd6J0xJ23ZQkUMsy1GLU5WLwvlVnmYfKib4SQVLHLMbJPKOthBNg bhJdUXOBEGv58pmkPVpjeb/kvqKh7gLyX+viInTgYtRC1rAQPgLjYXIgHUtgu2XGotQC /X3pLnohJhqyJRtNepGf2jVzow4SQxYILZzS5RHZr6j56dIYUyAP7kyVdOcIdSWmiRug dDYdm5gaboX9crc/wUctPtS21wCHP/FFZFLqCHGY5gUdkiUg2+Zw8wCZp1e020WE1PIW cFSQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700495257; x=1701100057; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-authentication-results:x-original-sender :ui-outboundreport:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:x-beenthere:x-gm-message-state:sender:from :to:cc:subject:date:message-id:reply-to; bh=G9Ew4mvvBZFkSJxQj9jAFpFoQlWHN4xNXUEwTRLaC0k=; b=nzxbE/4PMaEAPiZ5bdDqnFw75QqIvdJUtqm+mvnk38XTA9DjYbGpFulayI4lwPCZ/x tY1grcUabONf4bcAaPM22yIca+PabZygpPTSxJiJXl4exGT9SzNYsdHOCgOlZvEPSO6C yWgBp86WtHNEHG/HzsctotxN9AsKLLno6lMaQAzhAnGiy8suR7/NstYXAFQSFkLeDK8a 5/a5D7i/9jyLiSMAh/x2X9DbDbdR6fWlYmm7wZulgKTD4Ckk1GSXj+3lN7InM2itHeHc rYFSOMiBSs11tF4PHulPhuOjgsE2UUvZ59nCxAxnOyJqkQd1UKlmZ5sdUXQAnsOUDsi7 9Zug== Sender: swupdate@googlegroups.com X-Gm-Message-State: AOJu0YzQKe7YIa9BKPjRY5ruMsQ2wbXH1PqJr/cCxWSTBxb2Vf6u3EsI 7vgn7dfGzBp/Hmy5CSAtN10= X-Google-Smtp-Source: AGHT+IGzea0KmNKM6q+GjhA+UJtqTD3Wiy9HdWihDSdHBm6LtKLRgA9EVQJFZFjAuSCQ+aF6kQIqaw== X-Received: by 2002:a05:6000:1107:b0:331:3cec:214f with SMTP id z7-20020a056000110700b003313cec214fmr5520490wrw.7.1700495256777; Mon, 20 Nov 2023 07:47:36 -0800 (PST) X-BeenThere: swupdate@googlegroups.com Received: by 2002:a05:6000:25a:b0:313:f4ee:a4cc with SMTP id m26-20020a056000025a00b00313f4eea4ccls1549748wrz.1.-pod-prod-01-eu; Mon, 20 Nov 2023 07:47:35 -0800 (PST) X-Received: by 2002:a5d:4647:0:b0:32f:7f78:e29c with SMTP id j7-20020a5d4647000000b0032f7f78e29cmr5055785wrs.38.1700495255194; Mon, 20 Nov 2023 07:47:35 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700495255; cv=none; d=google.com; s=arc-20160816; b=emLPACmLEE/aCdkNpGUP78Fo4ARAt56p0SUerTzpA5XmMLKYHfXk16CjPi57QhkRHV qtBwlONG2PUorWGLLE2pXOUst/cmsFJ/nCS3gxgmSIsAPgNH+zWZRcLnQS92a7N3qGqt nDoHV6DA42I7rqONu0v5EegBySyC3Lvfwoa4L+aY+ohQO5vLkXqZme/7IjuU+8ogxUic +vVAHmYQP7N82M0KkG0cTHZ56Ktexlz/QKWV0BvL55IOfg68+ymvPMSc4mukzHqRTKaA BpZBlY6DKHmF1fcS6q4jT0mtERvT3vrG9XOCxttZjkuihrxaGWq7ojW+9AkdxssU5cjg fphQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=ui-outboundreport:content-transfer-encoding:mime-version:references :in-reply-to:message-id:date:subject:cc:to:from; bh=HFuI6g6Vi3xp013LBIDZ7viy7AXQiOwTRCSjNFSWWiM=; fh=TiEWcqMcKpHk5s7uErzpntoONrNfOXwKpI5P8bIlggk=; b=lLQ0NlR6F02GmHJknadsMwQERhbNWxaLZqUlevo+YBMQOCXG3ftNAoZUyrn8PXL5yZ gW3ZOkk1DTtsdP7gT1sD/fL4oB4StqYsEDgMCBAOWxGLklxrm88RJHsc9Ha5LZm8nstO 2KLPdJtdDzZ8O/zBTiSzQimdmRYap/g23hbfD7ZqFB405xrYXw84VSVHKh3QTUAURz08 w4n1JmRn9zMp8WorL8W5/EKkk/aolGSgZ43atVyBkbZlHFDsQLDEnZVI67rjJELVA4pl 0QErDlyt/XB+sb/12aiEXT4PI7W+pJ5Sk6XHX7kyXdh1z1PIHiv8XC+9rTQTZXD/Fv3B 8BDQ== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of stefano.babic@swupdate.org designates 212.227.17.24 as permitted sender) smtp.mailfrom=stefano.babic@swupdate.org Received: from mout.kundenserver.de (mout.kundenserver.de. [212.227.17.24]) by gmr-mx.google.com with ESMTPS id k2-20020a5d6e82000000b0032fccc15b58si366904wrz.2.2023.11.20.07.47.35 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Nov 2023 07:47:35 -0800 (PST) Received-SPF: pass (google.com: domain of stefano.babic@swupdate.org designates 212.227.17.24 as permitted sender) client-ip=212.227.17.24; Received: from paperino.fritz.box ([88.217.136.221]) by mrelayeu.kundenserver.de (mreue109 [213.165.67.113]) with ESMTPSA (Nemesis) id 1MekvT-1rfG3n3dKf-00akmv; Mon, 20 Nov 2023 16:47:34 +0100 From: Stefano Babic To: swupdate@googlegroups.com Cc: Stefano Babic Subject: [swupdate] [PATCH 09/13] channel_curl: implement put_file entry point Date: Mon, 20 Nov 2023 16:47:31 +0100 Message-Id: <20231120154731.44988-10-stefano.babic@swupdate.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231120154731.44988-1-stefano.babic@swupdate.org> References: <20231120154731.44988-1-stefano.babic@swupdate.org> MIME-Version: 1.0 X-Provags-ID: V03:K1:fmhMe6CLQPsm1e4jB2y5NfgpYbRDv7V0xlm7lyUa2fB+KtUClhZ 0LV8soQvwrYp5Zv3UMkYN1QuAOGy4oGtcw+K1K1jfG4wgdoPKL/MaUEAhny0lVL75yWgzFZ CKntg6Nfzu3mHDTU+fdNgv6nqR+j2VZD9vfqelFDQgoRyVIK3OODSs91BPyq/DgtuNmB2zh JQzp32vJD6+hRld6p69/g== X-Spam-Flag: NO UI-OutboundReport: notjunk:1;M01:P0:kK8C56llMJk=;qPzoytghD3U3xmjb6IVytZg9YDj jq+MKs1CBTqC+sFP4Oyoco8SYsyBu0hJqwR46iLFrWeQSZUQ5cCd1FuzmmXzrKmB9MlPB0HAU +iZU23mdgE2aTnoyiqgYLBBOlGqEDlkuvqGl4D2yGAQ6LpwpnWIviWLWmotc0cPUBbuP7dDhe mv51wIO5RRBnLwRMgdWnc9QLxEWklaqwj8Fbl4RAxBzVA5CJb/1PxyOkTzeram7UXFAnW6tMv lY9HBkHnTAsBNUiCocdnG7iKsb6uJ/4SEdYZ89OEwBRv/TyOVUmfLddBohtDLmUb9WeVuKKG+ xRBFldD08/mvgWHB1Y+t7FL+05B4+Hv5J6OkavBzXcXW3LEgn6ztVbc8uZxfQJoqwgjLbHe5n /WTmYnW9tU/ckHGQlagOXUCUUQY4eMowK3aly8x+DR8NDegSfZqiZocd63YYtqZT5dV1mXvxu u1mBhBvg/s/QtN+3+thl2a3OGbZLCqUYrPOGqQ8Z6MGkKsP3G3QbjMKpN5vukjbNOYPzMCvVl SYNUGXdgXmOSDK2Bhvf4bnSQhvgIIdI/oNbA7o+dXHvlVs2XUBbsSkqavnFAePvqMiQgPXNDf UGHW6ZNrT58DYbppWLP3NNN8cvZuS0jB4pmJ8wGDlUgYxPRm9t9eFKFXHiBve9KiAKx/hKWzX Zg7HXc2fTveSWWV5d6NwlXLg/G85SgPE3fAAwfnWuA6gPPUw3Q1cFYnj2OB0qEXIauaFK+IzI SRaL+NYbSPisd3ecUu1F74Nj/Phz8H3/N76As/iQ2hHs0X4yZrqBBhJqWM1zX0YOt0IYqJcmO O/kmzG0t4at+33x4tKo8ejwOtzvlqPEq5xaQmICm4e52jap5Z0vPgsw7Q7gFFyPn6IQWD8No4 X7cPAyHP0HLb7mP0ZGw+85dw5b2g4/qaozKk= X-Original-Sender: stefano.babic@swupdate.org X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of stefano.babic@swupdate.org designates 212.227.17.24 as permitted sender) smtp.mailfrom=stefano.babic@swupdate.org Precedence: list Mailing-list: list swupdate@googlegroups.com; contact swupdate+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: swupdate@googlegroups.com X-Google-Group-Id: 605343134186 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , Use put_file to upload a file from a stream. Data is read from a fd that must be set by the caller into channel_data_t structure. Caller must set the total amoun t of data that must be uploaded, else libcurl cannot know. Signed-off-by: Stefano Babic --- corelib/channel_curl.c | 125 +++++++++++++++++++++++++++++++++++------ include/channel_curl.h | 1 + 2 files changed, 110 insertions(+), 16 deletions(-) diff --git a/corelib/channel_curl.c b/corelib/channel_curl.c index 4a209bcd..27b31247 100644 --- a/corelib/channel_curl.c +++ b/corelib/channel_curl.c @@ -86,6 +86,7 @@ static channel_op_res_t channel_open(channel_t *this, void *cfg); static channel_op_res_t channel_get(channel_t *this, void *data); static channel_op_res_t channel_get_file(channel_t *this, void *data); static channel_op_res_t channel_put(channel_t *this, void *data); +static channel_op_res_t channel_put_file(channel_t *this, void *data); channel_op_res_t channel_curl_init(void); channel_t *channel_new(void); @@ -119,6 +120,7 @@ channel_t *channel_new(void) newchan->get = &channel_get; newchan->get_file = &channel_get_file; newchan->put = &channel_put; + newchan->put_file = &channel_put_file; newchan->get_redirect_url = &channel_get_redirect_url; } @@ -888,12 +890,11 @@ cleanup: return result; } -static size_t read_callback(void *ptr, size_t size, size_t nmemb, void *data) +static size_t read_callback(char *ptr, size_t size, size_t nmemb, void *data) { channel_data_t *channel_data = (channel_data_t *)data; - ssize_t bytes; + ssize_t nbytes; size_t n = 0; - int ret; /* * Check if data is stored in a buffer or should be read @@ -901,30 +902,34 @@ static size_t read_callback(void *ptr, size_t size, size_t nmemb, void *data) */ if (channel_data->request_body) { /* Check data to be sent */ - bytes = strlen(channel_data->request_body) - channel_data->offs; + nbytes = strlen(channel_data->request_body) - channel_data->offs; - if (!bytes) + if (!nbytes) return 0; - n = min(bytes, size * nmemb); + n = min(nbytes, size * nmemb); memcpy(ptr, &channel_data->request_body[channel_data->offs], n); channel_data->offs += n; } else { - bytes = nmemb * size; - ret = read(channel_data->read_fifo, ptr, bytes); - if (ret < 0) { + if (nmemb * size > channel_data->upload_filesize) + nbytes = channel_data->upload_filesize; + else + nbytes = nmemb * size; + + nbytes = read(channel_data->read_fifo, ptr, nbytes); + if (nbytes < 0) { if (errno == EAGAIN) { TRACE("READ EAGAIN"); - bytes = 0; + nbytes = 0; } else { ERROR("Cannot read from FIFO"); return CURL_READFUNC_ABORT; } - } else - bytes = ret; + } - n = bytes / nmemb; + n = nbytes / size; + channel_data->upload_filesize -= nbytes; } return n; @@ -1018,9 +1023,8 @@ static CURLcode channel_set_read_callback(channel_curl_t *handle, channel_data_t { return curl_easy_setopt(handle, CURLOPT_READFUNCTION, read_callback) || - (channel_data->request_body ? - curl_easy_setopt(handle, CURLOPT_INFILESIZE_LARGE, - (curl_off_t)strlen(channel_data->request_body)) : 0) || + curl_easy_setopt(handle, CURLOPT_INFILESIZE_LARGE, + channel_data->request_body ? (curl_off_t)strlen(channel_data->request_body) : (curl_off_t)channel_data->upload_filesize) || curl_easy_setopt(handle, CURLOPT_READDATA, channel_data); } @@ -1136,6 +1140,95 @@ channel_op_res_t channel_put(channel_t *this, void *data) } } +channel_op_res_t channel_put_file(channel_t *this, void *data) +{ + CURLcode curl_result = CURLE_OK; + channel_curl_t *channel_curl = this->priv; + assert(data != NULL); + assert(channel_curl->handle != NULL); + + channel_op_res_t result = CHANNEL_OK; + channel_data_t *channel_data = (channel_data_t *)data; + channel_data->offs = 0; + output_data_t outdata = {}; + write_callback_t wrdata = { .this = this, .channel_data = channel_data, .outdata = &outdata }; + + if ((result = channel_set_content_type(this, channel_data)) != + CHANNEL_OK) { + ERROR("Set content-type option failed."); + goto cleanup_header; + } + + if ((result = channel_set_options(this, channel_data)) != CHANNEL_OK) { + ERROR("Set channel option failed."); + goto cleanup_header; + } + + if ((result = setup_reply_buffer(channel_curl->handle, &wrdata)) != CHANNEL_OK) { + goto cleanup_header; + } + + if (!channel_data->method) + channel_data->method = CHANNEL_POST; + + switch (channel_data->method) { + case CHANNEL_PATCH: + case CHANNEL_POST: + if (channel_data->method == CHANNEL_PATCH) + curl_result |= curl_easy_setopt(channel_curl->handle, CURLOPT_CUSTOMREQUEST, "PATCH"); + else + curl_result |= curl_easy_setopt(channel_curl->handle, CURLOPT_POST, 1L); + + break; + + case CHANNEL_PUT: + curl_result |= curl_easy_setopt(channel_curl->handle, + #if LIBCURL_VERSION_NUM >= 0x70C01 + CURLOPT_UPLOAD, + #else + CURLOPT_PUT, + #endif + 1L); + break; + } + + curl_result |= channel_set_read_callback(channel_curl->handle, channel_data); + if (curl_result != CURLE_OK) { + result = CHANNEL_EINIT; + ERROR("Set %s channel method option failed.", method_desc[channel_data->method]); + goto cleanup_header; + } + + CURLcode curlrc = curl_easy_perform(channel_curl->handle); + if (curlrc != CURLE_OK) { + ERROR("Channel %s operation failed (%d): '%s'", method_desc[channel_data->method], curlrc, + curl_easy_strerror(curlrc)); + result = channel_map_curl_error(curlrc); + goto cleanup_header; + } + + channel_log_effective_url(this); + + result = channel_map_http_code(this, &channel_data->http_response_code); + + if (channel_data->nocheckanswer) + goto cleanup_header; + + channel_log_reply(result, channel_data, &outdata); + + if (result == CHANNEL_OK) { + result = parse_reply(channel_data, &outdata); + } + +cleanup_header: + outdata.memory != NULL ? free(outdata.memory) : (void)0; + curl_easy_reset(channel_curl->handle); + curl_slist_free_all(channel_curl->header); + channel_curl->header = NULL; + + return result; +} + channel_op_res_t channel_get_file(channel_t *this, void *data) { channel_curl_t *channel_curl = this->priv; diff --git a/include/channel_curl.h b/include/channel_curl.h index b1fc65c2..b3b51e85 100644 --- a/include/channel_curl.h +++ b/include/channel_curl.h @@ -90,6 +90,7 @@ typedef struct { struct dict *headers_to_send; struct dict *received_headers; unsigned int max_download_speed; + size_t upload_filesize; char *range; /* Range request for get_file in any */ void *user; } channel_data_t;