From nobody Mon Mar 2 08:45:34 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=scaleway.com ARC-Seal: i=1; a=rsa-sha256; t=1772196485; cv=none; d=zohomail.com; s=zohoarc; b=PMYZBswkEGHF1XZ+sB8+MIo24icUKwuhAVODMESsKTXKIsOs3+zfd3LY07pvfS7lwt7l9NsrSsfiK7rkP7Kc++1hXvoajduCSet4vNS4kYAaL74bJ/vFwDBV52PqHGz2D+qOFKiv4MGkart6lh3/tvhyebg4Q44bEcyu2fMkoRA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1772196485; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=snnOG+Kv8JUJbX9gCRA+PikPotaKeAEz7HPG/ojeglk=; b=TIzAsRX8BgA9rZTmFaZEo2fHt+oM9m3OlhIk+eohriEAKAQSf9AZ2tmdoNlsHVTqzlfB5fB2lD/E4a7eDULSFLCyjFVllyizha3pU72PYWFUCAmThv4jSewgPzuIY4CY7WLhDurtX6riUsJ6o+L3mjao9hibzzOnH86gc8+u9Rc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1772196485967165.55521291703383; Fri, 27 Feb 2026 04:48:05 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1vvxFV-00072t-Ua; Fri, 27 Feb 2026 07:47:17 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vvxF9-0006t3-4r for qemu-devel@nongnu.org; Fri, 27 Feb 2026 07:46:55 -0500 Received: from mail-wm1-x32f.google.com ([2a00:1450:4864:20::32f]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1vvxF4-0001eZ-NT for qemu-devel@nongnu.org; Fri, 27 Feb 2026 07:46:54 -0500 Received: by mail-wm1-x32f.google.com with SMTP id 5b1f17b1804b1-4806ce0f97bso16306205e9.0 for ; Fri, 27 Feb 2026 04:46:47 -0800 (PST) Received: from localhost (710304585.box.freepro.com. [130.180.219.188]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-483bfccff00sm61962215e9.25.2026.02.27.04.46.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 27 Feb 2026 04:46:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=scaleway.com; s=google; t=1772196407; x=1772801207; darn=nongnu.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=snnOG+Kv8JUJbX9gCRA+PikPotaKeAEz7HPG/ojeglk=; b=TxmnQIqMErdub7CKfVVqyB+5Njj8VkrqldpwOEmzN2OiyD4OBJLx4LalGtx1fiyvqI ADdAulUvk7As0wZr4YIKZE+j7tf776SblFJ5sQkiVMRlW2FNZ44G3qGNsCjDVQO746vK pL3xJO+IWtLRAnDNyMv1QNtKytbbA9GXzPhmH0q61yGVs2mtSahbeyW0AhxQ53tGM/f/ yhvzQtPdqvSnCfkonJ4ovQmlrTPXViy+9rmYpqaQACWDg+Ahg0cq+bF7Ro7ZIh0bjKsu qv46C7ROT5ydYHCIuGohbdwhGtJd6QWObBfT+2LCYzDxxSiUlnNkgmdR+uq+1Sh0oHM7 Wtfw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1772196407; x=1772801207; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-gg:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=snnOG+Kv8JUJbX9gCRA+PikPotaKeAEz7HPG/ojeglk=; b=GCyQ7ocCXoA57YbE1q7Np4L2JRDoLk7i0T111PXP/bgEQt6TVKXCZuRa5QQwh7JppT okIjz7hUC6SMAqPg59vHDdPId0dn4tgbzc7Juh13iwb08ShHJYGKhItAIwWpUkxJP+YX CjI3RneIMPexz4GtQ8MHJOdhxMuXIDOD9U0Q+KkwgIROW1sv14DNHgLoFDsrp05aeWGa 6hFJGdBBvx5tApoehhpPWlIUeFhTpBBc9B9xLQF/gNgtExH84pAfIt7wqvHsk+Bq+Bok t1eq5DN+e0CWsXb/405G5T3ahLqMZEgYGGi4OLxB80x6P00nQnyMe0lE6SzB1Jh0AlLu xrQg== X-Gm-Message-State: AOJu0YyhmwZc7xfvU6p4uuuGxaIzNUKfzalM+QfINSwQqzyHus28Y6nF jSs65c4J3TfPHqBPMeSDOYfkIJwMQnEHazHMtPKTEcQyH/tAhF/aNkiHljg61CsVrQx/MJvUZ7w tIl5g X-Gm-Gg: ATEYQzwuNsb9PxK06v9D0gX3Hw2CIAL+lI5zr5DNOkQWRq2nwPus2u8cmWTZNZn4iAs XCMEDORFti3L/m87/euMDwXHvU0ngkP+cwxPDKFCqI2+D8JwsgsJNjDMjOylG1SlDOcpxcH/oBJ /vhFSfGSlzKzSG2Dimr5slfwGq6uvVCxOW+elQkPbxFHA3NYF43s40yK4cYVt4dsbMQrD5CLZcX BVIdAcSDiPnr/c/8ZFBcWyzq0naGCPTsHNrbxyoujbmFnzXGn89YRsbr7hHe52xeiaaxKAeYRPD xm8t5a72mprJmsdJSlc/coqte2ybAHaFSsW3iF1/yOZQ7DyxqYS84P1S6bwmMENRRPGa5t/FErq kzU4l59zAzKwXfqh+n3Vl/WLw9mCJdKxtNDWCfFN5ADcV2wCFxjpiqw2/HI4mr4zlG4fY+SNSHl o3BX/ivCRrzaWOsU7vWVs9Jglgh2idFMNKtFl7HmABw7EWiuYe X-Received: by 2002:a05:600c:4fc8:b0:47b:e2a9:2bd7 with SMTP id 5b1f17b1804b1-483c9beaca0mr47095325e9.19.1772196406609; Fri, 27 Feb 2026 04:46:46 -0800 (PST) From: Antoine Damhet Date: Fri, 27 Feb 2026 13:45:51 +0100 Subject: [PATCH v3 1/3] block/curl: fix concurrent completion handling MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260227-fix-curl-v3-v3-1-eb8a4d88feef@scaleway.com> References: <20260227-fix-curl-v3-v3-0-eb8a4d88feef@scaleway.com> In-Reply-To: <20260227-fix-curl-v3-v3-0-eb8a4d88feef@scaleway.com> To: qemu-devel@nongnu.org Cc: Kevin Wolf , Hanna Reitz , qemu-block@nongnu.org, Eric Blake , Markus Armbruster , Pierrick Bouvier , qemu-stable@nongnu.org, Antoine Damhet X-Mailer: b4 0.15-dev X-Developer-Signature: v=1; a=openpgp-sha256; l=1787; i=adamhet@scaleway.com; h=from:subject:message-id; bh=0BKn07VDgAZvORo+jFUCQ3gue1nBEaPUdvRcvK5Z3tc=; b=owEBbQGS/pANAwAKAdwJoy885Y8+AcsmYgBpoZIsJAkOPtugoIRekjN8OBEbuPw38hluIhNwv nagoKS2GtqJATMEAAEKAB0WIQQCubVZtDHYaY6x8XzcCaMvPOWPPgUCaaGSLAAKCRDcCaMvPOWP PtvRCACgCRpHHKsDt/E5lznSfG5YnQtX9nSX/W9LZSIEbDx8whODkeCPSshj3kqlwwuJC+KtYZZ UPYDzgP3Y8njsgxYYEQdWYlfnvduD93/qR7mjkWX4Qd0ARleVsEmHotfGP+sbuRVGIJImA2m5fw rRa2owwoQdZsb8PVZ7kqVCjOqMXaO6STMi981mNGBDwRopgqYWb00su4nhIwOQHGa0obv/Aa/yu 1n92jHectRkynRv8ECUUIcuGBw/9z36PLjzE8XQkeB8ye4d002NJVrVUZpky2WOXimucugrLAWz 6J9vxDBKqPUjMcyxRNy69rbvV1HGmj5hJ8zdtdSjnUUtppfv X-Developer-Key: i=adamhet@scaleway.com; a=openpgp; fpr=4C3AEF45D70C65052D3F37E02A5F5236CFB70DEC Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::32f; envelope-from=adamhet@scaleway.com; helo=mail-wm1-x32f.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @scaleway.com) X-ZM-MESSAGEID: 1772196486765158500 curl_multi_check_completion would bail upon the first completed transfer even if more completion messages were available thus leaving some in flight IOs stuck. Rework a bit the loop to make the iterations clearer and drop the breaks. The original hang can be somewhat reproduced with the following command: $ qemu-img convert -p -m 16 -O qcow2 -c --image-opts \ 'file.driver=3Dhttps,file.url=3Dhttps://scaleway.testdebit.info/10G.iso,f= ile.readahead=3D1M' \ /tmp/test.qcow2 Fixes: 1f2cead32443 ("curl: Ensure all informationals are checked for compl= etion") Cc: qemu-stable@nongnu.org Reviewed-by: Kevin Wolf Signed-off-by: Antoine Damhet --- block/curl.c | 11 ++--------- 1 file changed, 2 insertions(+), 9 deletions(-) diff --git a/block/curl.c b/block/curl.c index 4e77c93b46e6..6dccf002564e 100644 --- a/block/curl.c +++ b/block/curl.c @@ -324,17 +324,11 @@ curl_find_buf(BDRVCURLState *s, uint64_t start, uint6= 4_t len, CURLAIOCB *acb) static void curl_multi_check_completion(BDRVCURLState *s) { int msgs_in_queue; + CURLMsg *msg; =20 /* Try to find done transfers, so we can free the easy * handle again. */ - for (;;) { - CURLMsg *msg; - msg =3D curl_multi_info_read(s->multi, &msgs_in_queue); - - /* Quit when there are no more completions */ - if (!msg) - break; - + while ((msg =3D curl_multi_info_read(s->multi, &msgs_in_queue))) { if (msg->msg =3D=3D CURLMSG_DONE) { int i; CURLState *state =3D NULL; @@ -397,7 +391,6 @@ static void curl_multi_check_completion(BDRVCURLState *= s) } =20 curl_clean_state(state); - break; } } } --=20 2.53.0 From nobody Mon Mar 2 08:45:34 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=scaleway.com ARC-Seal: i=1; a=rsa-sha256; t=1772196438; cv=none; d=zohomail.com; s=zohoarc; b=ACqkS02NsV23wFCMvdf8T5rZRChrBZ1Wishd9V9lzMMq/v5W/NW2095CmVebfbzAPF6XFwDLss9jOZyvAK06e4vkKmnMM1T9p9ywHTBHJJuWf024U+xOKhYMN1whuVIjV4Y8MiXfJUH4K8Ly8US0j+Lg8IaH+nZoMx8v55TI72Y= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1772196438; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=6gaYJdfbi1zVa7h6egzBhs1mlyDjTkzG5UEwNdlb5xM=; b=BKUPiqAXrJe4/kZypCgtlwTadUjM5HGUoucPEoloV4oJdPcngz/8trniKODppr791blpkkFI1BVLgKJV6OqEmHqx9YGx8aCMix2gj5NDlAftvgpBu0z43YdQ/5PZzt5GLaycBrCw6+QVo3oWQFriI3tJKoF8QYrN8qanENpAw74= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1772196437782106.82028635065876; Fri, 27 Feb 2026 04:47:17 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1vvxFH-0006ue-FV; Fri, 27 Feb 2026 07:47:03 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vvxF7-0006sM-Oy for qemu-devel@nongnu.org; Fri, 27 Feb 2026 07:46:53 -0500 Received: from mail-wr1-x431.google.com ([2a00:1450:4864:20::431]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1vvxF4-0001ep-Nd for qemu-devel@nongnu.org; Fri, 27 Feb 2026 07:46:53 -0500 Received: by mail-wr1-x431.google.com with SMTP id ffacd0b85a97d-4399dd8d7b3so585124f8f.0 for ; Fri, 27 Feb 2026 04:46:49 -0800 (PST) Received: from localhost (710304585.box.freepro.com. [130.180.219.188]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-4399c75b272sm6498399f8f.24.2026.02.27.04.46.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 27 Feb 2026 04:46:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=scaleway.com; s=google; t=1772196408; x=1772801208; darn=nongnu.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=6gaYJdfbi1zVa7h6egzBhs1mlyDjTkzG5UEwNdlb5xM=; b=GzX+Yj7j+t6kMW5R5FKr9cEaP7pkvtztx8Yez0/oWk2s2KcLauc12T6VQvdKXglxOE rlAXOyEJrUeRK6wYCWuaYytmplja07XH/QB0SRAKmCSBOPAIz3Q/ds4Tea8C9GqKBUBp lC/D1XftDylMaaNmaqDu1LCkBbNKXcs+NqOrFcbv2NMADqD0OeTwbMHks/kgIclomk9P 5qYDsbCn1HsnnP8xqi1L0qLiKAGJcuXjs9MiqqUWxc5gTLESl0lvyVPR9uClFRHHHZiL 1Drc5DvE3rd/e2Qbt4PC2QqERlebIrZh8iQtKTk7Fr4vHjQt0UkNQDQ5ITwnyqc7irYl mUVA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1772196408; x=1772801208; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-gg:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=6gaYJdfbi1zVa7h6egzBhs1mlyDjTkzG5UEwNdlb5xM=; b=j5R5S1xdiTM2mAJ2ZhPoIy4NQL1UgeoCUDMKbRxzWGqCUC36A3/afil6PWYunn5Kus 0fA40sGTR8DL9CezxzjIFaSXnaFL39qo1GsMPakO5jDhKPnRYfJ/pFhIdAGTeHd/KNIt 1U3+NVncyomfB8esS0U/rmECMLrNE37vylb/+cd//84HDInsuoWbnF5NzV7PyQ6USwPo juDeqCGJLQo3gCA+BUEnvp9spTOiQhNSfQ8TQ26zilzkUXaUl/3jlo3bFkbKfIl93ula WCusCkXMCrrf3swuTXq0JTU+hpuIQcKZGVWg7tdk7/1iMnaHUwEWEg3mPH2XSZcSJP3Y P4eQ== X-Gm-Message-State: AOJu0YyXGHq9CqC/3VoPKy38tqqy3QNqbsjnwSv3MlhQvxjKxw3yDRBn 9nKQBvZWYllDW45B3HvBRqDMscWx8/S9ZjyMRq19hAbYPduuYko+0GzdPCyHeMsmfU0= X-Gm-Gg: ATEYQzwoXVbEl5DFTPxbxvHDvnSJiMD7+FxyJaG4Bn4v5FwPlKilL5m660pm3Pvi1FT joAl5sFWC2MRx11ZsMM9ftS/vvsLmor+ENT3PS3zu0Y2Hj0tP5YbnE/Uuc9RWqQKs+J64AJmuYv M9OOSECxYtjgno8kCrl9ngiuuYtUKV4vapnwQRKkSK4WY+yYUPpgUQUeLP5JZrzu3z3m3pbR1Jx RJhWd5ztW9m6JDklYQONvK8vy6ARugMys/pMQq52yQK30zkzDQClxmi06yk1Zoc828+jViftMWj gQiLSiJwyaE4FCoIzebaJZOyPkPbq4OXlsb4Rx+ZNuhSnwCoGCVAb31+m9Qloc3qFepZb7sKxNr zw2OGJUb3WbNThSdJfVq9Gn6bJZk20yJH2HzYSNdPHm/3ywkYiJ1kpeORa+7xSgTd4X2AxkrXMg U2wdj/RYNuQ4XEGvYDm81UKONE2Jkee0ihcYC3w9whxZe1WhgZ X-Received: by 2002:a05:6000:250e:b0:436:14fa:a3fd with SMTP id ffacd0b85a97d-439971fc709mr10542084f8f.24.1772196407804; Fri, 27 Feb 2026 04:46:47 -0800 (PST) From: Antoine Damhet Date: Fri, 27 Feb 2026 13:45:52 +0100 Subject: [PATCH v3 2/3] qapi: block: Refactor HTTP(s) common arguments MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260227-fix-curl-v3-v3-2-eb8a4d88feef@scaleway.com> References: <20260227-fix-curl-v3-v3-0-eb8a4d88feef@scaleway.com> In-Reply-To: <20260227-fix-curl-v3-v3-0-eb8a4d88feef@scaleway.com> To: qemu-devel@nongnu.org Cc: Kevin Wolf , Hanna Reitz , qemu-block@nongnu.org, Eric Blake , Markus Armbruster , Pierrick Bouvier , Antoine Damhet X-Mailer: b4 0.15-dev X-Developer-Signature: v=1; a=openpgp-sha256; l=1402; i=adamhet@scaleway.com; h=from:subject:message-id; bh=ohTVUeEtivMPCvMY4Z1W58yowQAP11aeY+udzyn2cms=; b=owEBbQGS/pANAwAKAdwJoy885Y8+AcsmYgBpoZIv/++BUr+amfRaChsOdBRP5d/awNL6WF3b8 BtfHKiQLUKJATMEAAEKAB0WIQQCubVZtDHYaY6x8XzcCaMvPOWPPgUCaaGSLwAKCRDcCaMvPOWP Pp33B/0V+DhO9ljGe2Rwm19Zc6uyeBa/d7I0Z0gv7PHgSrdBCo1/TWxS5FAb21wi1SjHhwd9hlD dj4FiBXyXuHy1BWsjr/KtEYoKA7FzUiWGMfVmfKiF5hSt+j61wbeimjckyGBNQD6Tncl3JYOEE1 nWEKGd3vjgvoCWxFTilTX/dH85lWSk3BoltQZZNBO3fBZHK1vWC6H3xDGRviybB9vzSQEjC7GqU UeVB6Nf3eZAhHYsdwPQ0hvbtOHvANQtAvWbDNg8a7w66YARMdKTAL5JRkvhzdMOw5T2VgUhV0k0 4DXXcAS3d3p5IvH0o2eMUgmZNvav7mA2Yd4yZ3EwDmu7WBbD X-Developer-Key: i=adamhet@scaleway.com; a=openpgp; fpr=4C3AEF45D70C65052D3F37E02A5F5236CFB70DEC Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::431; envelope-from=adamhet@scaleway.com; helo=mail-wr1-x431.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @scaleway.com) X-ZM-MESSAGEID: 1772196441749158500 The HTTPs curl block driver is a superset of the HTTP driver, reflect that in the QAPI. Suggested-by: Markus Armbruster Reviewed-by: Markus Armbruster Signed-off-by: Antoine Damhet --- qapi/block-core.json | 13 ++----------- 1 file changed, 2 insertions(+), 11 deletions(-) diff --git a/qapi/block-core.json b/qapi/block-core.json index b82af7425614..a7871705fa69 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -4595,23 +4595,14 @@ # Driver specific block device options for HTTPS connections over the # curl backend. URLs must start with "https://". # -# @cookie: List of cookies to set; format is "name1=3Dcontent1; -# name2=3Dcontent2;" as explained by CURLOPT_COOKIE(3). Defaults to -# no cookies. -# # @sslverify: Whether to verify the SSL certificate's validity # (defaults to true) # -# @cookie-secret: ID of a QCryptoSecret object providing the cookie -# data in a secure way. See @cookie for the format. (since 2.10) -# # Since: 2.9 ## { 'struct': 'BlockdevOptionsCurlHttps', - 'base': 'BlockdevOptionsCurlBase', - 'data': { '*cookie': 'str', - '*sslverify': 'bool', - '*cookie-secret': 'str'} } + 'base': 'BlockdevOptionsCurlHttp', + 'data': { '*sslverify': 'bool'} } =20 ## # @BlockdevOptionsCurlFtp: --=20 2.53.0 From nobody Mon Mar 2 08:45:34 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=scaleway.com ARC-Seal: i=1; a=rsa-sha256; t=1772196484; cv=none; d=zohomail.com; s=zohoarc; b=UlubayOOS2ABTw5J6vz2bvH4QOFFbh4HFZPJGKWbrAfwLT7wM66lsXwloTECLCIS+iI1j80csJaXeT6ZMbbutrNNEEpZYo+qP+5o7Urh+5ZCFTWAOr1iMf8XmJ8ZzMxjlQ8avv8eYm/0v20rrdxDec5bUjgR/rO8KjbNqzxeyPU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1772196484; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=utum6+s33iaXOZ1VCoUahsk6lJbB3MentN70PBUsR4c=; b=IMuzpKHe3IoOiyl3qBcGCZYPwxw/dvp/Vaf5y/AXGL/p2zafSlo5Jelzrzs9hUB7Awyqr3MsupqGEsTcF1YvWF7MTBjxQKk3pP69cV6DDSj9LmrUOrI5K9y6mGwYXKad7TL4NtBOsqx2yTJ90AFI20u/zMs3G+SB2wzN3pv8zmk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1772196484200967.7734803417014; Fri, 27 Feb 2026 04:48:04 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1vvxFY-00079C-RZ; Fri, 27 Feb 2026 07:47:20 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vvxFB-0006tv-Al for qemu-devel@nongnu.org; Fri, 27 Feb 2026 07:46:57 -0500 Received: from mail-wm1-x32c.google.com ([2a00:1450:4864:20::32c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1vvxF4-0001f2-OD for qemu-devel@nongnu.org; Fri, 27 Feb 2026 07:46:55 -0500 Received: by mail-wm1-x32c.google.com with SMTP id 5b1f17b1804b1-4836d4c26d3so13307075e9.2 for ; Fri, 27 Feb 2026 04:46:49 -0800 (PST) Received: from localhost (710304585.box.freepro.com. [130.180.219.188]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-483bd68826asm346099895e9.0.2026.02.27.04.46.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 27 Feb 2026 04:46:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=scaleway.com; s=google; t=1772196409; x=1772801209; darn=nongnu.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=utum6+s33iaXOZ1VCoUahsk6lJbB3MentN70PBUsR4c=; b=LPoOGpq9GuZnDe/PsOBwSgdeOXVgq4vYY2xcovo0IVK/ToyyLGi+b60+KpPLuzWj8R jmVzlWXb4ot6v54oaeS3Z3OSBlCSM/ua+fCbKAljgw92gHztFyi51TocQmlNK17OJ56y lPk+1o69BcqBAvpVDojLtXz4B3fSkP45rMvdllps7D9ErtDW33Bpljz06VWVmEGRXlBf Ye4WtCkJwIqj9xXg/cKe5hivYbQ3cxGyZ+8qmOqQCQN4tQ4xPSTPCAOjAFMiOJivZLgU ab1aYYzJ+6JgU1B84pI1JK6KUUwoaWZ7vMkwV0W8fCBapgZ+YV/cUMpFANuX6+8zaTUU WHSg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1772196409; x=1772801209; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-gg:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=utum6+s33iaXOZ1VCoUahsk6lJbB3MentN70PBUsR4c=; b=PdBKRODrz6Um9GvntXqkC5vDcl/GejHkwxnhJJCrgtiyLYvZdOTc0Z6y8+7ZSs0wuc CD24vMyTj9PrbLbdLqbA6YVn6cVXgzTDoDXnib1XZvn3oTLRtsgmm4TpEvX+L8rVqX4V 3MbtEQUjmZFRqjAxz2kbYaEqPYt4yC1FqJfBTeSRAQj1GPw8hWP2Fb13ESESlltq/ujH ZlsnnrkHbNd3OyU+QdEK0FP0t2M0WoEIqkUTXJJ4r6E+AkiLFE+DlM08JLBiOndP6VwS NX/nvG7yOQxROht3ArZPZIfKyYlhkYrKZ82YBv7ztr/5kYqhHNIDd3EQRdynMOa2MSZY byIg== X-Gm-Message-State: AOJu0YzlsDm08J9MjcprPwqZoAa9cIThTsSnUxtPctKq1j7U1hOqum/V GKh0POkB762i6BTHBPFNFuJm0+Ywn5TxNgFXlQBHxs075rIdOkbNC22Jj8HSdLjWhfM= X-Gm-Gg: ATEYQzyeswweVdQJGZca9aEfycjBu0fcVcXuRsbXi/4KAbirSOFe7Dt5g/zlC/qNKMG 3ZaC4ba3JjUK1xDOFzNphmww3Oa13hRLpkDMA4UaPESjpl8dHnymrQumhTI1q0JaqDs5s5PovJp 0VDIE+8xshBhalJBowEcQbqPRo+DlKfxkj1bn+3KvvYAhN/g43u/VPuZdiy9l/6n/lD9jhlyChk seVfBK+i/EUi7Ezywx7qArJeg93iMWo5T74HKdCKs0zn4awobsFTuM6o62ZgDOWan8OPaybB7hf kWb5LIjQNObohp34YannOJ7dbIMDb7B3mYobyl2BZQgqC+dYTohXfgapPQhB/Y62OB88RnYBV4Y Sndueq655OgsD/eoNydz0AD6/uaf7APaBJHmtK3bdgEc7yqYkIfqgItZGSnJPkPmE3LDmEM0v+k OPpYXEN8lpNjGhWVz8JIo3lpq2yuHXOzjD/8/5RZdDGD85lwvo X-Received: by 2002:a05:600c:a16:b0:483:2c98:4368 with SMTP id 5b1f17b1804b1-483c9c02ef1mr36960715e9.18.1772196408647; Fri, 27 Feb 2026 04:46:48 -0800 (PST) From: Antoine Damhet Date: Fri, 27 Feb 2026 13:45:53 +0100 Subject: [PATCH v3 3/3] block/curl: add support for S3 presigned URLs MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260227-fix-curl-v3-v3-3-eb8a4d88feef@scaleway.com> References: <20260227-fix-curl-v3-v3-0-eb8a4d88feef@scaleway.com> In-Reply-To: <20260227-fix-curl-v3-v3-0-eb8a4d88feef@scaleway.com> To: qemu-devel@nongnu.org Cc: Kevin Wolf , Hanna Reitz , qemu-block@nongnu.org, Eric Blake , Markus Armbruster , Pierrick Bouvier , Antoine Damhet X-Mailer: b4 0.15-dev X-Developer-Signature: v=1; a=openpgp-sha256; l=9839; i=adamhet@scaleway.com; h=from:subject:message-id; bh=z8OIN6LEUGyjWk5XyYZrkXCTubPYiO201/qNmV0EHv4=; b=owEBbQGS/pANAwAKAdwJoy885Y8+AcsmYgBpoZIxqcwIqjRH62r70KCwS2sTiY/L0jVx/vOG3 b5q2HqGODaJATMEAAEKAB0WIQQCubVZtDHYaY6x8XzcCaMvPOWPPgUCaaGSMQAKCRDcCaMvPOWP PiS7CACrQly84n0XgJlkzdlRqTW5Vz+M2/0vtTD915xqVJHGSCnI+jOUNG72SncftpOjO2EGMvv /V+KKhZ6Ph36TPp9XqwR2gxjYkX6SfwmTeuet0HE8HoQY9PBJy50dZbLA8NDJ914lmNh2Zwisos U3XYyGD7bI15gKgagbt9GasEFEaqFxJ+uXoOjRl/Cpj551aNsZADlkg75KYlKFfk8ipGVRs1mqt sM/bdVIWch1OVE3Cn+3XrXW4inYkZVqAHRtLAyr+6v43XIurEJ8K+ffcXUb3LzJAyREU1Im8fYg q9aoHOVo1YNbblIO/htaNNrWE+eH/6CvXSBeYugcb6p7TCGp X-Developer-Key: i=adamhet@scaleway.com; a=openpgp; fpr=4C3AEF45D70C65052D3F37E02A5F5236CFB70DEC Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::32c; envelope-from=adamhet@scaleway.com; helo=mail-wm1-x32c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @scaleway.com) X-ZM-MESSAGEID: 1772196486798158500 S3 presigned URLs are signed for a specific HTTP method (typically GET for our use cases). The curl block driver currently issues a HEAD request to discover the web server features and the file size, which fails with 'HTTP 403' (forbidden). Add a 'force-range' option that skips the HEAD request and instead issues a minimal GET request (querying 1 byte from the server) to extract the file size from the 'Content-Range' response header. To achieve this the 'curl_header_cb' is redesigned to generically parse HTTP headers. $ $QEMU -drive driver=3Dhttps,\ 'url=3Dhttps://s3.example.com/some.img?X-Amz-Security-Token=3D= XXX', force-range=3Dtrue Enabling the 'force-range' option without the web server specified with @url supporting it might cause the server to respond successfully with 'HTTP 200' and attempt to send the whole file body. With the 'CURLOPT_NOBODY' option set the libcurl will skip reading after the headers and close the connection. QEMU still gracefully detects the missing feature. This might waste a small number of TCP packets but is otherwise transparent to the user. Acked-by: Markus Armbruster Signed-off-by: Antoine Damhet --- block/curl.c | 104 +++++++++++++++++++++++-------= ---- block/trace-events | 1 + docs/system/device-url-syntax.rst.inc | 6 ++ qapi/block-core.json | 8 ++- 4 files changed, 85 insertions(+), 34 deletions(-) diff --git a/block/curl.c b/block/curl.c index 6dccf002564e..66aecfb20ec6 100644 --- a/block/curl.c +++ b/block/curl.c @@ -62,10 +62,12 @@ #define CURL_BLOCK_OPT_PASSWORD_SECRET "password-secret" #define CURL_BLOCK_OPT_PROXY_USERNAME "proxy-username" #define CURL_BLOCK_OPT_PROXY_PASSWORD_SECRET "proxy-password-secret" +#define CURL_BLOCK_OPT_FORCE_RANGE "force-range" =20 #define CURL_BLOCK_OPT_READAHEAD_DEFAULT (256 * 1024) #define CURL_BLOCK_OPT_SSLVERIFY_DEFAULT true #define CURL_BLOCK_OPT_TIMEOUT_DEFAULT 5 +#define CURL_BLOCK_OPT_FORCE_RANGE_DEFAULT false =20 struct BDRVCURLState; struct CURLState; @@ -206,27 +208,33 @@ static size_t curl_header_cb(void *ptr, size_t size, = size_t nmemb, void *opaque) { BDRVCURLState *s =3D opaque; size_t realsize =3D size * nmemb; - const char *p =3D ptr; - const char *end =3D p + realsize; - const char *t =3D "accept-ranges : bytes "; /* A lowercase template */ + g_autofree char *header =3D g_strstrip(g_strndup(ptr, realsize)); + char *val =3D strchr(header, ':'); =20 - /* check if header matches the "t" template */ - for (;;) { - if (*t =3D=3D ' ') { /* space in t matches any amount of isspace i= n p */ - if (p < end && g_ascii_isspace(*p)) { - ++p; - } else { - ++t; - } - } else if (*t && p < end && *t =3D=3D g_ascii_tolower(*p)) { - ++p, ++t; - } else { - break; - } + if (!val) { + return realsize; } =20 - if (!*t && p =3D=3D end) { /* if we managed to reach ends of both stri= ngs */ - s->accept_range =3D true; + *val++ =3D '\0'; + g_strchomp(header); + while (g_ascii_isspace(*val)) { + ++val; + } + + trace_curl_header_cb(header, val); + + if (!g_ascii_strcasecmp(header, "accept-ranges")) { + if (!g_ascii_strcasecmp(val, "bytes")) { + s->accept_range =3D true; + } + } else if (!g_ascii_strcasecmp(header, "Content-Range")) { + /* Content-Range fmt is `bytes begin-end/full_size` */ + val =3D strchr(val, '/'); + if (val) { + if (qemu_strtou64(val + 1, NULL, 10, &s->len) < 0) { + s->len =3D UINT64_MAX; + } + } } =20 return realsize; @@ -668,6 +676,11 @@ static QemuOptsList runtime_opts =3D { .type =3D QEMU_OPT_STRING, .help =3D "ID of secret used as password for HTTP proxy auth", }, + { + .name =3D CURL_BLOCK_OPT_FORCE_RANGE, + .type =3D QEMU_OPT_BOOL, + .help =3D "Assume HTTP range requests are supported", + }, { /* end of list */ } }, }; @@ -690,6 +703,7 @@ static int curl_open(BlockDriverState *bs, QDict *optio= ns, int flags, #endif const char *secretid; const char *protocol_delimiter; + bool force_range; int ret; =20 bdrv_graph_rdlock_main_loop(); @@ -807,35 +821,56 @@ static int curl_open(BlockDriverState *bs, QDict *opt= ions, int flags, } =20 s->accept_range =3D false; + s->len =3D UINT64_MAX; + force_range =3D qemu_opt_get_bool(opts, CURL_BLOCK_OPT_FORCE_RANGE, + CURL_BLOCK_OPT_FORCE_RANGE_DEFAULT); + /* + * When minimal CURL will be bumped to `7.83`, the header callback + m= anual + * parsing can be replaced by `curl_easy_header` calls + */ if (curl_easy_setopt(state->curl, CURLOPT_NOBODY, 1L) || curl_easy_setopt(state->curl, CURLOPT_HEADERFUNCTION, curl_header_= cb) || curl_easy_setopt(state->curl, CURLOPT_HEADERDATA, s)) { - pstrcpy(state->errmsg, CURL_ERROR_SIZE, - "curl library initialization failed."); - goto out; + goto out_init; + } + if (force_range) { + if (curl_easy_setopt(state->curl, CURLOPT_CUSTOMREQUEST, "GET") || + curl_easy_setopt(state->curl, CURLOPT_RANGE, "0-0")) { + goto out_init; + } } + if (curl_easy_perform(state->curl)) goto out; - /* CURL 7.55.0 deprecates CURLINFO_CONTENT_LENGTH_DOWNLOAD in favour of - * the *_T version which returns a more sensible type for content leng= th. - */ + + if (!force_range) { + /* + * CURL 7.55.0 deprecates CURLINFO_CONTENT_LENGTH_DOWNLOAD in favo= ur of + * the *_T version which returns a more sensible type for content + * length. + */ #if LIBCURL_VERSION_NUM >=3D 0x073700 - if (curl_easy_getinfo(state->curl, CURLINFO_CONTENT_LENGTH_DOWNLOAD_T,= &cl)) { - goto out; - } + if (curl_easy_getinfo(state->curl, CURLINFO_CONTENT_LENGTH_DOWNLOA= D_T, + &cl)) { + goto out; + } #else - if (curl_easy_getinfo(state->curl, CURLINFO_CONTENT_LENGTH_DOWNLOAD, &= cl)) { - goto out; - } + if (curl_easy_getinfo(state->curl, CURLINFO_CONTENT_LENGTH_DOWNLOA= D, + &cl)) { + goto out; + } #endif - if (cl < 0) { + if (cl >=3D 0) { + s->len =3D cl; + } + } + + if (s->len =3D=3D UINT64_MAX) { pstrcpy(state->errmsg, CURL_ERROR_SIZE, "Server didn't report file size."); goto out; } =20 - s->len =3D cl; - if ((!strncasecmp(s->url, "http://", strlen("http://")) || !strncasecmp(s->url, "https://", strlen("https://"))) && !s->accept_range) { @@ -856,6 +891,9 @@ static int curl_open(BlockDriverState *bs, QDict *optio= ns, int flags, qemu_opts_del(opts); return 0; =20 +out_init: + pstrcpy(state->errmsg, CURL_ERROR_SIZE, + "curl library initialization failed."); out: error_setg(errp, "CURL: Error opening file: %s", state->errmsg); curl_easy_cleanup(state->curl); diff --git a/block/trace-events b/block/trace-events index c9b4736ff884..d170fc96f15f 100644 --- a/block/trace-events +++ b/block/trace-events @@ -191,6 +191,7 @@ ssh_server_status(int status) "server status=3D%d" curl_timer_cb(long timeout_ms) "timer callback timeout_ms %ld" curl_sock_cb(int action, int fd) "sock action %d on fd %d" curl_read_cb(size_t realsize) "just reading %zu bytes" +curl_header_cb(const char *key, const char *val) "looking at %s: %s" curl_open(const char *file) "opening %s" curl_open_size(uint64_t size) "size =3D %" PRIu64 curl_setup_preadv(uint64_t bytes, uint64_t start, const char *range) "read= ing %" PRIu64 " at %" PRIu64 " (%s)" diff --git a/docs/system/device-url-syntax.rst.inc b/docs/system/device-url= -syntax.rst.inc index aae65d138c00..996ce5418ffb 100644 --- a/docs/system/device-url-syntax.rst.inc +++ b/docs/system/device-url-syntax.rst.inc @@ -179,6 +179,12 @@ These are specified using a special URL syntax. get the size of the image to be downloaded. If not set, the default timeout of 5 seconds is used. =20 + ``force-range`` + Don't issue a HEAD HTTP request to discover if the http server + server supports range requests and rely only on GET requests. This + is especially useful for S3 presigned URLs where HEAD requests + are unauthorized. It defaults to 'false'. + Note that when passing options to qemu explicitly, ``driver`` is the value of . =20 diff --git a/qapi/block-core.json b/qapi/block-core.json index a7871705fa69..18ba3b732ff4 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -4582,12 +4582,18 @@ # @cookie-secret: ID of a QCryptoSecret object providing the cookie # data in a secure way. See @cookie for the format. (since 2.10) # +# @force-range: Don't issue a HEAD HTTP request to discover if the +# http server supports range requests and rely only on GET +# requests. This is especially useful for S3 presigned URLs where +# HEAD requests are unauthorized. (default: false; since 11.0) +# # Since: 2.9 ## { 'struct': 'BlockdevOptionsCurlHttp', 'base': 'BlockdevOptionsCurlBase', 'data': { '*cookie': 'str', - '*cookie-secret': 'str'} } + '*cookie-secret': 'str', + '*force-range': 'bool'} } =20 ## # @BlockdevOptionsCurlHttps: --=20 2.53.0