From nobody Mon Apr 29 04:30:40 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1652718584; cv=none; d=zohomail.com; s=zohoarc; b=fPxdSeJ+X4CNH+/UI4sgbrCKx+IIN4AN0isooM2Fx+CXHlfeq1Nj1k4+tS1ISR9QvAE1qvR5z/akgQpH+Ia9obgqfR21BURrhH5sjYw8XjoWSAUfN/Xf+94L1gCdjKIzWPHrqYwVsn8Tr2BHSvAWtiXrghOe9YfI1O2Q3bmKRZY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1652718584; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=eXQvHdcbDNXDS2GzdfvHcEmF0uMa7RMOBMpp3obdwqk=; b=FH1KpEzJwcwlK5FhgrX8bpzc4cmFUBt6OduUJeHgSFi51yuLvbVsN5fTAUUBFrbAcgXStbmUrASYPExZbKPeF8lottV1jYzfYf0cW7xnrvd2G0fVf+A99ni8QLzU33y99dtJeu0CYBKbL2iM5i4/BSsBl8L+I4AQNUCJFfU6qHw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 165271858413934.234979467247854; Mon, 16 May 2022 09:29:44 -0700 (PDT) Received: from localhost ([::1]:45484 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nqdbL-0002Zx-5n for importer@patchew.org; Mon, 16 May 2022 12:29:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35550) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqcng-0001xe-Ce for qemu-devel@nongnu.org; Mon, 16 May 2022 11:38:24 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:41913) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqcne-0008DZ-0d for qemu-devel@nongnu.org; Mon, 16 May 2022 11:38:23 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-571-m7fIbi7xOPe_JPHbXBDIKg-1; Mon, 16 May 2022 11:38:19 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 74BE8804184 for ; Mon, 16 May 2022 15:38:19 +0000 (UTC) Received: from dgilbert-t580.localhost (unknown [10.39.194.215]) by smtp.corp.redhat.com (Postfix) with ESMTP id 835A315228C6; Mon, 16 May 2022 15:38:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1652715501; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=eXQvHdcbDNXDS2GzdfvHcEmF0uMa7RMOBMpp3obdwqk=; b=IkIL48dEUUCmmiDMZQh1bvuPu687vcoAVq4rCW7QMqW6/qFwNLGdDFrGvAQUHU01/cpXHB UxWTY0KuLLy3tYEhI9xBaovLgc9ZHIrSU50jm7DeqxZ6IFKViEkOh+NfnoaGtp0lOTCfVS OvfNnGphJN5vGWNwvXcYJSMmOMR1apc= X-MC-Unique: m7fIbi7xOPe_JPHbXBDIKg-1 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, quintela@redhat.com, peterx@redhat.com, leobras@redhat.com, berrange@redhat.com Subject: [PULL 01/17] tests: fix encoding of IP addresses in x509 certs Date: Mon, 16 May 2022 16:37:56 +0100 Message-Id: <20220516153812.127155-2-dgilbert@redhat.com> In-Reply-To: <20220516153812.127155-1-dgilbert@redhat.com> References: <20220516153812.127155-1-dgilbert@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.85 on 10.11.54.7 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=dgilbert@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1652718586438100001 From: Daniel P. Berrang=C3=A9 We need to encode just the address bytes, not the whole struct sockaddr data. Add a test case to validate that we're matching on SAN IP addresses correctly. Signed-off-by: Daniel P. Berrang=C3=A9 Message-Id: <20220426160048.812266-2-berrange@redhat.com> Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Dr. David Alan Gilbert --- tests/unit/crypto-tls-x509-helpers.c | 16 +++++++++++++--- tests/unit/test-crypto-tlssession.c | 11 +++++++++-- 2 files changed, 22 insertions(+), 5 deletions(-) diff --git a/tests/unit/crypto-tls-x509-helpers.c b/tests/unit/crypto-tls-x= 509-helpers.c index fc609b3fd4..e9937f60d8 100644 --- a/tests/unit/crypto-tls-x509-helpers.c +++ b/tests/unit/crypto-tls-x509-helpers.c @@ -168,9 +168,19 @@ test_tls_get_ipaddr(const char *addrstr, hints.ai_flags =3D AI_NUMERICHOST; g_assert(getaddrinfo(addrstr, NULL, &hints, &res) =3D=3D 0); =20 - *datalen =3D res->ai_addrlen; - *data =3D g_new(char, *datalen); - memcpy(*data, res->ai_addr, *datalen); + if (res->ai_family =3D=3D AF_INET) { + struct sockaddr_in *in =3D (struct sockaddr_in *)res->ai_addr; + *datalen =3D sizeof(in->sin_addr); + *data =3D g_new(char, *datalen); + memcpy(*data, &in->sin_addr, *datalen); + } else if (res->ai_family =3D=3D AF_INET6) { + struct sockaddr_in6 *in =3D (struct sockaddr_in6 *)res->ai_addr; + *datalen =3D sizeof(in->sin6_addr); + *data =3D g_new(char, *datalen); + memcpy(*data, &in->sin6_addr, *datalen); + } else { + g_assert_not_reached(); + } freeaddrinfo(res); } =20 diff --git a/tests/unit/test-crypto-tlssession.c b/tests/unit/test-crypto-t= lssession.c index a266dc32da..f222959d36 100644 --- a/tests/unit/test-crypto-tlssession.c +++ b/tests/unit/test-crypto-tlssession.c @@ -512,12 +512,19 @@ int main(int argc, char **argv) false, true, "wiki.qemu.org", NULL); =20 TEST_SESS_REG(altname4, cacertreq.filename, + servercertalt1req.filename, clientcertreq.filename, + false, false, "192.168.122.1", NULL); + TEST_SESS_REG(altname5, cacertreq.filename, + servercertalt1req.filename, clientcertreq.filename, + false, false, "fec0::dead:beaf", NULL); + + TEST_SESS_REG(altname6, cacertreq.filename, servercertalt2req.filename, clientcertreq.filename, false, true, "qemu.org", NULL); - TEST_SESS_REG(altname5, cacertreq.filename, + TEST_SESS_REG(altname7, cacertreq.filename, servercertalt2req.filename, clientcertreq.filename, false, false, "www.qemu.org", NULL); - TEST_SESS_REG(altname6, cacertreq.filename, + TEST_SESS_REG(altname8, cacertreq.filename, servercertalt2req.filename, clientcertreq.filename, false, false, "wiki.qemu.org", NULL); =20 --=20 2.36.1 From nobody Mon Apr 29 04:30:41 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1652718744; cv=none; d=zohomail.com; s=zohoarc; b=XAfA+dgWffWdmAripsQgXn++JwGszafhs+1L8e8NoVk7a1YsnA2yPYcE4LKbGvvAARSza6mXwKZALhleWLUlh3O8oOGOGor1F+dUh7e/Ts82vQlFf2e5IjP1zUPgkqMZRkb8+pxQSum4/8sGxTmiw558Ys5nNalsW8DjzagUrtI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1652718744; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=rMExqTuCRGm8E2egRl0yW+CCn6HFDK/Gkhm3uQXiRQo=; b=S4ddhBRcywNv0iaIM3MztGWZW2kSyRUfPpYbmgzaboy1qZoRnCSrKojHxnsakFgQ0tlh4f3x1YrEV+LwtejrZFJt+alzCh8coBOF6w386QXo42XphXrj04OXJVKEnkkP5eEduzCv+WauW0gPuNMaDMYibKyK1S4N9547xBI/JaE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1652718744413739.6996272230509; Mon, 16 May 2022 09:32:24 -0700 (PDT) Received: from localhost ([::1]:53118 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nqddu-0007sk-UT for importer@patchew.org; Mon, 16 May 2022 12:32:22 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35572) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqcnh-00022g-Vk for qemu-devel@nongnu.org; Mon, 16 May 2022 11:38:26 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:21090) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqcnf-0008Dg-Aq for qemu-devel@nongnu.org; Mon, 16 May 2022 11:38:25 -0400 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-369-_38BA8JrNiO9YxUtY4m9hA-1; Mon, 16 May 2022 11:38:20 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 96D2129ABA13 for ; Mon, 16 May 2022 15:38:20 +0000 (UTC) Received: from dgilbert-t580.localhost (unknown [10.39.194.215]) by smtp.corp.redhat.com (Postfix) with ESMTP id B0FEA15228C6; Mon, 16 May 2022 15:38:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1652715502; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=rMExqTuCRGm8E2egRl0yW+CCn6HFDK/Gkhm3uQXiRQo=; b=JPssAW+CZeK1anvLYNElOymNKV/khAj3TA/3w4ISuC01z2437BzVVFO8vsvN3PbCe7XE5r UnDhFgPVk+VDWmQLMaHUot7bE8bTueHMfOxybbySmR/s5IJArqcXqacAjxoDTqXx1dEPSz yu2xfc+OdriQ0Pg8Gl68bCxNIwkLxcw= X-MC-Unique: _38BA8JrNiO9YxUtY4m9hA-1 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, quintela@redhat.com, peterx@redhat.com, leobras@redhat.com, berrange@redhat.com Subject: [PULL 02/17] tests: add more helper macros for creating TLS x509 certs Date: Mon, 16 May 2022 16:37:57 +0100 Message-Id: <20220516153812.127155-3-dgilbert@redhat.com> In-Reply-To: <20220516153812.127155-1-dgilbert@redhat.com> References: <20220516153812.127155-1-dgilbert@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.85 on 10.11.54.7 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=dgilbert@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1652718745458100003 From: Daniel P. Berrang=C3=A9 These macros are more suited to the general consumers of certs in the test suite, where we don't need to exercise every single possible permutation. Signed-off-by: Daniel P. Berrang=C3=A9 Message-Id: <20220426160048.812266-3-berrange@redhat.com> Reviewed-by: Eric Blake Signed-off-by: Dr. David Alan Gilbert --- tests/unit/crypto-tls-x509-helpers.h | 53 ++++++++++++++++++++++++++++ 1 file changed, 53 insertions(+) diff --git a/tests/unit/crypto-tls-x509-helpers.h b/tests/unit/crypto-tls-x= 509-helpers.h index cf6329e653..247e7160eb 100644 --- a/tests/unit/crypto-tls-x509-helpers.h +++ b/tests/unit/crypto-tls-x509-helpers.h @@ -26,6 +26,9 @@ #include =20 =20 +#define QCRYPTO_TLS_TEST_CLIENT_NAME "ACME QEMU Client" +#define QCRYPTO_TLS_TEST_CLIENT_HOSTILE_NAME "ACME Hostile Client" + /* * This contains parameter about how to generate * certificates. @@ -118,6 +121,56 @@ void test_tls_cleanup(const char *keyfile); }; \ test_tls_generate_cert(&varname, NULL) =20 +# define TLS_ROOT_REQ_SIMPLE(varname, fname) \ + QCryptoTLSTestCertReq varname =3D { \ + .filename =3D fname, \ + .cn =3D "qemu-CA", \ + .basicConstraintsEnable =3D true, \ + .basicConstraintsCritical =3D true, \ + .basicConstraintsIsCA =3D true, \ + .keyUsageEnable =3D true, \ + .keyUsageCritical =3D true, \ + .keyUsageValue =3D GNUTLS_KEY_KEY_CERT_SIGN, \ + }; \ + test_tls_generate_cert(&varname, NULL) + +# define TLS_CERT_REQ_SIMPLE_CLIENT(varname, cavarname, cname, fname) \ + QCryptoTLSTestCertReq varname =3D { \ + .filename =3D fname, \ + .cn =3D cname, \ + .basicConstraintsEnable =3D true, \ + .basicConstraintsCritical =3D true, \ + .basicConstraintsIsCA =3D false, \ + .keyUsageEnable =3D true, \ + .keyUsageCritical =3D true, \ + .keyUsageValue =3D \ + GNUTLS_KEY_DIGITAL_SIGNATURE | GNUTLS_KEY_KEY_ENCIPHERMENT, \ + .keyPurposeEnable =3D true, \ + .keyPurposeCritical =3D true, \ + .keyPurposeOID1 =3D GNUTLS_KP_TLS_WWW_CLIENT, \ + }; \ + test_tls_generate_cert(&varname, cavarname.crt) + +# define TLS_CERT_REQ_SIMPLE_SERVER(varname, cavarname, fname, \ + hostname, ipaddr) \ + QCryptoTLSTestCertReq varname =3D { \ + .filename =3D fname, \ + .cn =3D hostname ? hostname : ipaddr, \ + .altname1 =3D hostname, \ + .ipaddr1 =3D ipaddr, \ + .basicConstraintsEnable =3D true, \ + .basicConstraintsCritical =3D true, \ + .basicConstraintsIsCA =3D false, \ + .keyUsageEnable =3D true, \ + .keyUsageCritical =3D true, \ + .keyUsageValue =3D \ + GNUTLS_KEY_DIGITAL_SIGNATURE | GNUTLS_KEY_KEY_ENCIPHERMENT, \ + .keyPurposeEnable =3D true, \ + .keyPurposeCritical =3D true, \ + .keyPurposeOID1 =3D GNUTLS_KP_TLS_WWW_SERVER, \ + }; \ + test_tls_generate_cert(&varname, cavarname.crt) + extern const asn1_static_node pkix_asn1_tab[]; =20 #endif --=20 2.36.1 From nobody Mon Apr 29 04:30:41 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1652719536; cv=none; d=zohomail.com; s=zohoarc; b=PL5j6jWwR7qkzAOy1BqLwKXANljGzyQfM2R9HkPDDKioznmtkjxQ+5efW7VmUL1cgkaeuP3N1HTkThdwNW+iIUYB9oCsZchJ36sOgECTcJ2vNCAD+fdsb755BMBX94nOIOGS7v93czEWYGuSWBZcXKVgbo3za8nYPI30Ld+T8SA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1652719536; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=7bXhEb0SzHDbgdLh6ukB4AVHjtc3l9JpqMnK5i8Moi4=; b=kx3zfVXDkFzp/EBlV3iSCKDS9nA3lkqfOCsFN7tyYv0t/2eb0Rzcak3eslHI3oSTSCeJi61Fukemot0TB6PfiaDNZbxZMWetbzvCe1JELZnVqSqY8EW5tODp1HXqfUi723P3cUs7VkvbLASRt/c9X5E9Le3E2k3vdzPTchrSmeM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1652719536024374.4849078687348; Mon, 16 May 2022 09:45:36 -0700 (PDT) Received: from localhost ([::1]:56498 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nqdqg-0004gv-Oi for importer@patchew.org; Mon, 16 May 2022 12:45:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35574) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqcni-00023O-6y for qemu-devel@nongnu.org; Mon, 16 May 2022 11:38:26 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:52638) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqcng-0008Dt-2n for qemu-devel@nongnu.org; Mon, 16 May 2022 11:38:25 -0400 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-247-ImNODlCTMWmDGUvRsU7Pdw-1; Mon, 16 May 2022 11:38:22 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id BD557395AFE8 for ; Mon, 16 May 2022 15:38:21 +0000 (UTC) Received: from dgilbert-t580.localhost (unknown [10.39.194.215]) by smtp.corp.redhat.com (Postfix) with ESMTP id D1E1815228C6; Mon, 16 May 2022 15:38:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1652715503; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=7bXhEb0SzHDbgdLh6ukB4AVHjtc3l9JpqMnK5i8Moi4=; b=Z2e5rxfPO/j3AMW/dAXG22Ukog9EwQX37uwnMTSuNevO1Do+wEbnD+Kua87/bqgu/avaRE SDH31OPvL8tjgjnJsopuxieeR+kdGzp38MFzvXfsEQzgEWeEUH5mq056Og3316fa2kH8f0 mtR6ImTPNSrYEss3ayygmN8NoaO61sM= X-MC-Unique: ImNODlCTMWmDGUvRsU7Pdw-1 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, quintela@redhat.com, peterx@redhat.com, leobras@redhat.com, berrange@redhat.com Subject: [PULL 03/17] tests: add migration tests of TLS with PSK credentials Date: Mon, 16 May 2022 16:37:58 +0100 Message-Id: <20220516153812.127155-4-dgilbert@redhat.com> In-Reply-To: <20220516153812.127155-1-dgilbert@redhat.com> References: <20220516153812.127155-1-dgilbert@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.85 on 10.11.54.7 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=dgilbert@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1652719536623100001 From: Daniel P. Berrang=C3=A9 This validates that we correctly handle migration success and failure scenarios when using TLS with pre shared keys. Signed-off-by: Daniel P. Berrang=C3=A9 Message-Id: <20220426160048.812266-4-berrange@redhat.com> Reviewed-by: Eric Blake Signed-off-by: Dr. David Alan Gilbert --- tests/qtest/meson.build | 7 +- tests/qtest/migration-test.c | 161 +++++++++++++++++++++++++++- tests/unit/crypto-tls-psk-helpers.c | 18 +++- tests/unit/crypto-tls-psk-helpers.h | 1 + 4 files changed, 179 insertions(+), 8 deletions(-) diff --git a/tests/qtest/meson.build b/tests/qtest/meson.build index 3551b9c946..166450135d 100644 --- a/tests/qtest/meson.build +++ b/tests/qtest/meson.build @@ -273,13 +273,18 @@ endif =20 tpmemu_files =3D ['tpm-emu.c', 'tpm-util.c', 'tpm-tests.c'] =20 +migration_files =3D [files('migration-helpers.c')] +if gnutls.found() + migration_files +=3D [files('../unit/crypto-tls-psk-helpers.c'), gnutls] +endif + qtests =3D { 'bios-tables-test': [io, 'boot-sector.c', 'acpi-utils.c', 'tpm-emu.c'], 'cdrom-test': files('boot-sector.c'), 'dbus-vmstate-test': files('migration-helpers.c') + dbus_vmstate1, 'erst-test': files('erst-test.c'), 'ivshmem-test': [rt, '../../contrib/ivshmem-server/ivshmem-server.c'], - 'migration-test': files('migration-helpers.c'), + 'migration-test': migration_files, 'pxe-test': files('boot-sector.c'), 'qos-test': [chardev, io, qos_test_ss.apply(config_host, strict: false).= sources()], 'tpm-crb-swtpm-test': [io, tpmemu_files], diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c index cba6023eb5..2eefc9c1ff 100644 --- a/tests/qtest/migration-test.c +++ b/tests/qtest/migration-test.c @@ -23,9 +23,13 @@ #include "qapi/qapi-visit-sockets.h" #include "qapi/qobject-input-visitor.h" #include "qapi/qobject-output-visitor.h" +#include "crypto/tlscredspsk.h" =20 #include "migration-helpers.h" #include "tests/migration/migration-test.h" +#ifdef CONFIG_GNUTLS +# include "tests/unit/crypto-tls-psk-helpers.h" +#endif /* CONFIG_GNUTLS */ =20 /* For dirty ring test; so far only x86_64 is supported */ #if defined(__linux__) && defined(HOST_X86_64) @@ -640,6 +644,100 @@ static void test_migrate_end(QTestState *from, QTestS= tate *to, bool test_dest) cleanup("dest_serial"); } =20 +#ifdef CONFIG_GNUTLS +struct TestMigrateTLSPSKData { + char *workdir; + char *workdiralt; + char *pskfile; + char *pskfilealt; +}; + +static void * +test_migrate_tls_psk_start_common(QTestState *from, + QTestState *to, + bool mismatch) +{ + struct TestMigrateTLSPSKData *data =3D + g_new0(struct TestMigrateTLSPSKData, 1); + QDict *rsp; + + data->workdir =3D g_strdup_printf("%s/tlscredspsk0", tmpfs); + data->pskfile =3D g_strdup_printf("%s/%s", data->workdir, + QCRYPTO_TLS_CREDS_PSKFILE); + mkdir(data->workdir, 0700); + test_tls_psk_init(data->pskfile); + + if (mismatch) { + data->workdiralt =3D g_strdup_printf("%s/tlscredspskalt0", tmpfs); + data->pskfilealt =3D g_strdup_printf("%s/%s", data->workdiralt, + QCRYPTO_TLS_CREDS_PSKFILE); + mkdir(data->workdiralt, 0700); + test_tls_psk_init_alt(data->pskfilealt); + } + + rsp =3D wait_command(from, + "{ 'execute': 'object-add'," + " 'arguments': { 'qom-type': 'tls-creds-psk'," + " 'id': 'tlscredspsk0'," + " 'endpoint': 'client'," + " 'dir': %s," + " 'username': 'qemu'} }", + data->workdir); + qobject_unref(rsp); + + rsp =3D wait_command(to, + "{ 'execute': 'object-add'," + " 'arguments': { 'qom-type': 'tls-creds-psk'," + " 'id': 'tlscredspsk0'," + " 'endpoint': 'server'," + " 'dir': %s } }", + mismatch ? data->workdiralt : data->workdir); + qobject_unref(rsp); + + migrate_set_parameter_str(from, "tls-creds", "tlscredspsk0"); + migrate_set_parameter_str(to, "tls-creds", "tlscredspsk0"); + + return data; +} + +static void * +test_migrate_tls_psk_start_match(QTestState *from, + QTestState *to) +{ + return test_migrate_tls_psk_start_common(from, to, false); +} + +static void * +test_migrate_tls_psk_start_mismatch(QTestState *from, + QTestState *to) +{ + return test_migrate_tls_psk_start_common(from, to, true); +} + +static void +test_migrate_tls_psk_finish(QTestState *from, + QTestState *to, + void *opaque) +{ + struct TestMigrateTLSPSKData *data =3D opaque; + + test_tls_psk_cleanup(data->pskfile); + if (data->pskfilealt) { + test_tls_psk_cleanup(data->pskfilealt); + } + rmdir(data->workdir); + if (data->workdiralt) { + rmdir(data->workdiralt); + } + + g_free(data->workdiralt); + g_free(data->pskfilealt); + g_free(data->workdir); + g_free(data->pskfile); + g_free(data); +} +#endif /* CONFIG_GNUTLS */ + static int migrate_postcopy_prepare(QTestState **from_ptr, QTestState **to_ptr, MigrateStart *args) @@ -911,7 +1009,7 @@ static void test_precopy_common(MigrateCommon *args) test_migrate_end(from, to, args->result =3D=3D MIG_TEST_SUCCEED); } =20 -static void test_precopy_unix(void) +static void test_precopy_unix_plain(void) { g_autofree char *uri =3D g_strdup_printf("unix:%s/migsocket", tmpfs); MigrateCommon args =3D { @@ -922,6 +1020,21 @@ static void test_precopy_unix(void) test_precopy_common(&args); } =20 +#ifdef CONFIG_GNUTLS +static void test_precopy_unix_tls_psk(void) +{ + g_autofree char *uri =3D g_strdup_printf("unix:%s/migsocket", tmpfs); + MigrateCommon args =3D { + .connect_uri =3D uri, + .listen_uri =3D uri, + .start_hook =3D test_migrate_tls_psk_start_match, + .finish_hook =3D test_migrate_tls_psk_finish, + }; + + test_precopy_common(&args); +} +#endif + static void test_precopy_unix_dirty_ring(void) { g_autofree char *uri =3D g_strdup_printf("unix:%s/migsocket", tmpfs); @@ -1026,7 +1139,7 @@ static void test_xbzrle_unix(void) test_xbzrle(uri); } =20 -static void test_precopy_tcp(void) +static void test_precopy_tcp_plain(void) { MigrateCommon args =3D { .listen_uri =3D "tcp:127.0.0.1:0", @@ -1035,6 +1148,34 @@ static void test_precopy_tcp(void) test_precopy_common(&args); } =20 +#ifdef CONFIG_GNUTLS +static void test_precopy_tcp_tls_psk_match(void) +{ + MigrateCommon args =3D { + .listen_uri =3D "tcp:127.0.0.1:0", + .start_hook =3D test_migrate_tls_psk_start_match, + .finish_hook =3D test_migrate_tls_psk_finish, + }; + + test_precopy_common(&args); +} + +static void test_precopy_tcp_tls_psk_mismatch(void) +{ + MigrateCommon args =3D { + .start =3D { + .hide_stderr =3D true, + }, + .listen_uri =3D "tcp:127.0.0.1:0", + .start_hook =3D test_migrate_tls_psk_start_mismatch, + .finish_hook =3D test_migrate_tls_psk_finish, + .result =3D MIG_TEST_FAIL, + }; + + test_precopy_common(&args); +} +#endif /* CONFIG_GNUTLS */ + static void *test_migrate_fd_start_hook(QTestState *from, QTestState *to) { @@ -1497,8 +1638,20 @@ int main(int argc, char **argv) qtest_add_func("/migration/postcopy/unix", test_postcopy); qtest_add_func("/migration/postcopy/recovery", test_postcopy_recovery); qtest_add_func("/migration/bad_dest", test_baddest); - qtest_add_func("/migration/precopy/unix", test_precopy_unix); - qtest_add_func("/migration/precopy/tcp", test_precopy_tcp); + qtest_add_func("/migration/precopy/unix/plain", test_precopy_unix_plai= n); +#ifdef CONFIG_GNUTLS + qtest_add_func("/migration/precopy/unix/tls/psk", + test_precopy_unix_tls_psk); +#endif /* CONFIG_GNUTLS */ + + qtest_add_func("/migration/precopy/tcp/plain", test_precopy_tcp_plain); +#ifdef CONFIG_GNUTLS + qtest_add_func("/migration/precopy/tcp/tls/psk/match", + test_precopy_tcp_tls_psk_match); + qtest_add_func("/migration/precopy/tcp/tls/psk/mismatch", + test_precopy_tcp_tls_psk_mismatch); +#endif /* CONFIG_GNUTLS */ + /* qtest_add_func("/migration/ignore_shared", test_ignore_shared); */ qtest_add_func("/migration/xbzrle/unix", test_xbzrle_unix); qtest_add_func("/migration/fd_proto", test_migrate_fd_proto); diff --git a/tests/unit/crypto-tls-psk-helpers.c b/tests/unit/crypto-tls-ps= k-helpers.c index 4bea7c6fa2..511e08cc9c 100644 --- a/tests/unit/crypto-tls-psk-helpers.c +++ b/tests/unit/crypto-tls-psk-helpers.c @@ -24,7 +24,8 @@ #include "crypto-tls-psk-helpers.h" #include "qemu/sockets.h" =20 -void test_tls_psk_init(const char *pskfile) +static void +test_tls_psk_init_common(const char *pskfile, const char *user, const char= *key) { FILE *fp; =20 @@ -33,11 +34,22 @@ void test_tls_psk_init(const char *pskfile) g_critical("Failed to create pskfile %s: %s", pskfile, strerror(er= rno)); abort(); } - /* Don't do this in real applications! Use psktool. */ - fprintf(fp, "qemu:009d5638c40fde0c\n"); + fprintf(fp, "%s:%s\n", user, key); fclose(fp); } =20 +void test_tls_psk_init(const char *pskfile) +{ + /* Don't hard code a key like this in real applications! Use psktool.= */ + test_tls_psk_init_common(pskfile, "qemu", "009d5638c40fde0c"); +} + +void test_tls_psk_init_alt(const char *pskfile) +{ + /* Don't hard code a key like this in real applications! Use psktool.= */ + test_tls_psk_init_common(pskfile, "qemu", "10ffa6a2c42f0388"); +} + void test_tls_psk_cleanup(const char *pskfile) { unlink(pskfile); diff --git a/tests/unit/crypto-tls-psk-helpers.h b/tests/unit/crypto-tls-ps= k-helpers.h index faa645c629..67f8bdda71 100644 --- a/tests/unit/crypto-tls-psk-helpers.h +++ b/tests/unit/crypto-tls-psk-helpers.h @@ -24,6 +24,7 @@ #include =20 void test_tls_psk_init(const char *keyfile); +void test_tls_psk_init_alt(const char *keyfile); void test_tls_psk_cleanup(const char *keyfile); =20 #endif --=20 2.36.1 From nobody Mon Apr 29 04:30:41 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1652718387; cv=none; d=zohomail.com; s=zohoarc; b=JceUpv1pBFjV3Re+aCcKFvAt1jgGfSvZtxYi+TDPbFz4yY8m/OoHPHm/4aqvdASNYwobj2JV94V5XTwmC00diygrLgM/hPDG0YWAvr9mKJ3BsMje3wPoiflwrwbgS2/ydOhvozVsE33l16kuQdcJZM3oUv5Okg/bqWVnWveSv34= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1652718387; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=SkFh7ulzuvTBAN1o08L5PJUj9zdHkaLvCp1I4mj6r1I=; b=IbEolkBlFuRloS+GntjTJD2pMdb0Fm1kREWgDHEjzqsjLPMA9cUwhFVokCMlAWd0b4q++NBG/n2o3BaWgWNgSY+fTbsG2eLtD/vjF/pZheQp0RVL085Nl4VSDOqThH9XhKN5S5vAKA0izFnGHot2jf5Dx0eGRvmewWwKszVEehU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1652718387751340.934160511408; Mon, 16 May 2022 09:26:27 -0700 (PDT) Received: from localhost ([::1]:34246 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nqdYA-0003Ge-Ku for importer@patchew.org; Mon, 16 May 2022 12:26:26 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35608) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqcnk-0002Az-Ne for qemu-devel@nongnu.org; Mon, 16 May 2022 11:38:28 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:32720) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqcni-0008E2-7l for qemu-devel@nongnu.org; Mon, 16 May 2022 11:38:28 -0400 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-428-9RpI40xLNh-NE3gIwQ2OMw-1; Mon, 16 May 2022 11:38:23 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 0F132395AFE5 for ; Mon, 16 May 2022 15:38:23 +0000 (UTC) Received: from dgilbert-t580.localhost (unknown [10.39.194.215]) by smtp.corp.redhat.com (Postfix) with ESMTP id 059BB15228C6; Mon, 16 May 2022 15:38:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1652715504; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=SkFh7ulzuvTBAN1o08L5PJUj9zdHkaLvCp1I4mj6r1I=; b=CI/FkjT45X/Ifod/ly+2HAcl2H6x63vMqqm/2Vkr78U8alDmnwzfk1R5G8iz4X8W0gfLUC bL+SVVfExOZQilUijG6bfnO+EtHlAtiQPVqv+QmqmD3D5lVzmePPujEObL7dobC6D0igiH FYDU5xglU1tCanInB2rwKng3hJ2yPV0= X-MC-Unique: 9RpI40xLNh-NE3gIwQ2OMw-1 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, quintela@redhat.com, peterx@redhat.com, leobras@redhat.com, berrange@redhat.com Subject: [PULL 04/17] tests: add migration tests of TLS with x509 credentials Date: Mon, 16 May 2022 16:37:59 +0100 Message-Id: <20220516153812.127155-5-dgilbert@redhat.com> In-Reply-To: <20220516153812.127155-1-dgilbert@redhat.com> References: <20220516153812.127155-1-dgilbert@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.85 on 10.11.54.7 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=dgilbert@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1652718388910100001 From: Daniel P. Berrang=C3=A9 This validates that we correctly handle migration success and failure scenarios when using TLS with x509 certificates. There are quite a few different scenarios that matter in relation to hostname validation. Signed-off-by: Daniel P. Berrang=C3=A9 Message-Id: <20220426160048.812266-5-berrange@redhat.com> Reviewed-by: Eric Blake Signed-off-by: Dr. David Alan Gilbert dgilbert: Manual merge due to ifdef change in 3 --- meson.build | 1 + tests/qtest/meson.build | 5 + tests/qtest/migration-test.c | 383 ++++++++++++++++++++++++++++++++++- 3 files changed, 386 insertions(+), 3 deletions(-) diff --git a/meson.build b/meson.build index 9b20dcd143..93aa31a9e4 100644 --- a/meson.build +++ b/meson.build @@ -1742,6 +1742,7 @@ config_host_data.set('CONFIG_KEYUTILS', keyutils.foun= d()) config_host_data.set('CONFIG_GETTID', has_gettid) config_host_data.set('CONFIG_GNUTLS', gnutls.found()) config_host_data.set('CONFIG_GNUTLS_CRYPTO', gnutls_crypto.found()) +config_host_data.set('CONFIG_TASN1', tasn1.found()) config_host_data.set('CONFIG_GCRYPT', gcrypt.found()) config_host_data.set('CONFIG_NETTLE', nettle.found()) config_host_data.set('CONFIG_QEMU_PRIVATE_XTS', xts =3D=3D 'private') diff --git a/tests/qtest/meson.build b/tests/qtest/meson.build index 166450135d..b425484920 100644 --- a/tests/qtest/meson.build +++ b/tests/qtest/meson.build @@ -276,6 +276,11 @@ tpmemu_files =3D ['tpm-emu.c', 'tpm-util.c', 'tpm-test= s.c'] migration_files =3D [files('migration-helpers.c')] if gnutls.found() migration_files +=3D [files('../unit/crypto-tls-psk-helpers.c'), gnutls] + + if tasn1.found() + migration_files +=3D [files('../unit/crypto-tls-x509-helpers.c', + '../unit/pkix_asn1_tab.c'), tasn1] + endif endif =20 qtests =3D { diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c index 2eefc9c1ff..5a3edf2da6 100644 --- a/tests/qtest/migration-test.c +++ b/tests/qtest/migration-test.c @@ -29,6 +29,9 @@ #include "tests/migration/migration-test.h" #ifdef CONFIG_GNUTLS # include "tests/unit/crypto-tls-psk-helpers.h" +# ifdef CONFIG_TASN1 +# include "tests/unit/crypto-tls-x509-helpers.h" +# endif /* CONFIG_TASN1 */ #endif /* CONFIG_GNUTLS */ =20 /* For dirty ring test; so far only x86_64 is supported */ @@ -736,6 +739,234 @@ test_migrate_tls_psk_finish(QTestState *from, g_free(data->pskfile); g_free(data); } + +#ifdef CONFIG_TASN1 +typedef struct { + char *workdir; + char *keyfile; + char *cacert; + char *servercert; + char *serverkey; + char *clientcert; + char *clientkey; +} TestMigrateTLSX509Data; + +typedef struct { + bool verifyclient; + bool clientcert; + bool hostileclient; + bool authzclient; + const char *certhostname; + const char *certipaddr; +} TestMigrateTLSX509; + +static void * +test_migrate_tls_x509_start_common(QTestState *from, + QTestState *to, + TestMigrateTLSX509 *args) +{ + TestMigrateTLSX509Data *data =3D g_new0(TestMigrateTLSX509Data, 1); + QDict *rsp; + + data->workdir =3D g_strdup_printf("%s/tlscredsx5090", tmpfs); + data->keyfile =3D g_strdup_printf("%s/key.pem", data->workdir); + + data->cacert =3D g_strdup_printf("%s/ca-cert.pem", data->workdir); + data->serverkey =3D g_strdup_printf("%s/server-key.pem", data->workdir= ); + data->servercert =3D g_strdup_printf("%s/server-cert.pem", data->workd= ir); + if (args->clientcert) { + data->clientkey =3D g_strdup_printf("%s/client-key.pem", data->wor= kdir); + data->clientcert =3D g_strdup_printf("%s/client-cert.pem", data->w= orkdir); + } + + mkdir(data->workdir, 0700); + + test_tls_init(data->keyfile); + g_assert(link(data->keyfile, data->serverkey) =3D=3D 0); + if (args->clientcert) { + g_assert(link(data->keyfile, data->clientkey) =3D=3D 0); + } + + TLS_ROOT_REQ_SIMPLE(cacertreq, data->cacert); + if (args->clientcert) { + TLS_CERT_REQ_SIMPLE_CLIENT(servercertreq, cacertreq, + args->hostileclient ? + QCRYPTO_TLS_TEST_CLIENT_HOSTILE_NAME : + QCRYPTO_TLS_TEST_CLIENT_NAME, + data->clientcert); + } + + TLS_CERT_REQ_SIMPLE_SERVER(clientcertreq, cacertreq, + data->servercert, + args->certhostname, + args->certipaddr); + + rsp =3D wait_command(from, + "{ 'execute': 'object-add'," + " 'arguments': { 'qom-type': 'tls-creds-x509'," + " 'id': 'tlscredsx509client0'," + " 'endpoint': 'client'," + " 'dir': %s," + " 'sanity-check': true," + " 'verify-peer': true} }", + data->workdir); + qobject_unref(rsp); + migrate_set_parameter_str(from, "tls-creds", "tlscredsx509client0"); + if (args->certhostname) { + migrate_set_parameter_str(from, "tls-hostname", args->certhostname= ); + } + + rsp =3D wait_command(to, + "{ 'execute': 'object-add'," + " 'arguments': { 'qom-type': 'tls-creds-x509'," + " 'id': 'tlscredsx509server0'," + " 'endpoint': 'server'," + " 'dir': %s," + " 'sanity-check': true," + " 'verify-peer': %i} }", + data->workdir, args->verifyclient); + qobject_unref(rsp); + migrate_set_parameter_str(to, "tls-creds", "tlscredsx509server0"); + + if (args->authzclient) { + rsp =3D wait_command(to, + "{ 'execute': 'object-add'," + " 'arguments': { 'qom-type': 'authz-simple'," + " 'id': 'tlsauthz0'," + " 'identity': %s} }", + "CN=3D" QCRYPTO_TLS_TEST_CLIENT_NAME); + migrate_set_parameter_str(to, "tls-authz", "tlsauthz0"); + } + + return data; +} + +/* + * The normal case: match server's cert hostname against + * whatever host we were telling QEMU to connect to (if any) + */ +static void * +test_migrate_tls_x509_start_default_host(QTestState *from, + QTestState *to) +{ + TestMigrateTLSX509 args =3D { + .verifyclient =3D true, + .clientcert =3D true, + .certipaddr =3D "127.0.0.1" + }; + return test_migrate_tls_x509_start_common(from, to, &args); +} + +/* + * The unusual case: the server's cert is different from + * the address we're telling QEMU to connect to (if any), + * so we must give QEMU an explicit hostname to validate + */ +static void * +test_migrate_tls_x509_start_override_host(QTestState *from, + QTestState *to) +{ + TestMigrateTLSX509 args =3D { + .verifyclient =3D true, + .clientcert =3D true, + .certhostname =3D "qemu.org", + }; + return test_migrate_tls_x509_start_common(from, to, &args); +} + +/* + * The unusual case: the server's cert is different from + * the address we're telling QEMU to connect to, and so we + * expect the client to reject the server + */ +static void * +test_migrate_tls_x509_start_mismatch_host(QTestState *from, + QTestState *to) +{ + TestMigrateTLSX509 args =3D { + .verifyclient =3D true, + .clientcert =3D true, + .certipaddr =3D "10.0.0.1", + }; + return test_migrate_tls_x509_start_common(from, to, &args); +} + +static void * +test_migrate_tls_x509_start_friendly_client(QTestState *from, + QTestState *to) +{ + TestMigrateTLSX509 args =3D { + .verifyclient =3D true, + .clientcert =3D true, + .authzclient =3D true, + .certipaddr =3D "127.0.0.1", + }; + return test_migrate_tls_x509_start_common(from, to, &args); +} + +static void * +test_migrate_tls_x509_start_hostile_client(QTestState *from, + QTestState *to) +{ + TestMigrateTLSX509 args =3D { + .verifyclient =3D true, + .clientcert =3D true, + .hostileclient =3D true, + .authzclient =3D true, + .certipaddr =3D "127.0.0.1", + }; + return test_migrate_tls_x509_start_common(from, to, &args); +} + +/* + * The case with no client certificate presented, + * and no server verification + */ +static void * +test_migrate_tls_x509_start_allow_anon_client(QTestState *from, + QTestState *to) +{ + TestMigrateTLSX509 args =3D { + .certipaddr =3D "127.0.0.1", + }; + return test_migrate_tls_x509_start_common(from, to, &args); +} + +/* + * The case with no client certificate presented, + * and server verification rejecting + */ +static void * +test_migrate_tls_x509_start_reject_anon_client(QTestState *from, + QTestState *to) +{ + TestMigrateTLSX509 args =3D { + .verifyclient =3D true, + .certipaddr =3D "127.0.0.1", + }; + return test_migrate_tls_x509_start_common(from, to, &args); +} + +static void +test_migrate_tls_x509_finish(QTestState *from, + QTestState *to, + void *opaque) +{ + TestMigrateTLSX509Data *data =3D opaque; + + test_tls_cleanup(data->keyfile); + unlink(data->cacert); + unlink(data->servercert); + unlink(data->serverkey); + unlink(data->clientcert); + unlink(data->clientkey); + rmdir(data->workdir); + + g_free(data->workdir); + g_free(data->keyfile); + g_free(data); +} +#endif /* CONFIG_TASN1 */ #endif /* CONFIG_GNUTLS */ =20 static int migrate_postcopy_prepare(QTestState **from_ptr, @@ -1020,6 +1251,21 @@ static void test_precopy_unix_plain(void) test_precopy_common(&args); } =20 + +static void test_precopy_unix_dirty_ring(void) +{ + g_autofree char *uri =3D g_strdup_printf("unix:%s/migsocket", tmpfs); + MigrateCommon args =3D { + .start =3D { + .use_dirty_ring =3D true, + }, + .listen_uri =3D uri, + .connect_uri =3D uri, + }; + + test_precopy_common(&args); +} + #ifdef CONFIG_GNUTLS static void test_precopy_unix_tls_psk(void) { @@ -1033,21 +1279,39 @@ static void test_precopy_unix_tls_psk(void) =20 test_precopy_common(&args); } -#endif =20 -static void test_precopy_unix_dirty_ring(void) +#ifdef CONFIG_TASN1 +static void test_precopy_unix_tls_x509_default_host(void) { g_autofree char *uri =3D g_strdup_printf("unix:%s/migsocket", tmpfs); MigrateCommon args =3D { .start =3D { - .use_dirty_ring =3D true, + .hide_stderr =3D true, }, + .connect_uri =3D uri, .listen_uri =3D uri, + .start_hook =3D test_migrate_tls_x509_start_default_host, + .finish_hook =3D test_migrate_tls_x509_finish, + .result =3D MIG_TEST_FAIL_DEST_QUIT_ERR, + }; + + test_precopy_common(&args); +} + +static void test_precopy_unix_tls_x509_override_host(void) +{ + g_autofree char *uri =3D g_strdup_printf("unix:%s/migsocket", tmpfs); + MigrateCommon args =3D { .connect_uri =3D uri, + .listen_uri =3D uri, + .start_hook =3D test_migrate_tls_x509_start_override_host, + .finish_hook =3D test_migrate_tls_x509_finish, }; =20 test_precopy_common(&args); } +#endif /* CONFIG_TASN1 */ +#endif /* CONFIG_GNUTLS */ =20 #if 0 /* Currently upset on aarch64 TCG */ @@ -1174,6 +1438,97 @@ static void test_precopy_tcp_tls_psk_mismatch(void) =20 test_precopy_common(&args); } + +#ifdef CONFIG_TASN1 +static void test_precopy_tcp_tls_x509_default_host(void) +{ + MigrateCommon args =3D { + .listen_uri =3D "tcp:127.0.0.1:0", + .start_hook =3D test_migrate_tls_x509_start_default_host, + .finish_hook =3D test_migrate_tls_x509_finish, + }; + + test_precopy_common(&args); +} + +static void test_precopy_tcp_tls_x509_override_host(void) +{ + MigrateCommon args =3D { + .listen_uri =3D "tcp:127.0.0.1:0", + .start_hook =3D test_migrate_tls_x509_start_override_host, + .finish_hook =3D test_migrate_tls_x509_finish, + }; + + test_precopy_common(&args); +} + +static void test_precopy_tcp_tls_x509_mismatch_host(void) +{ + MigrateCommon args =3D { + .start =3D { + .hide_stderr =3D true, + }, + .listen_uri =3D "tcp:127.0.0.1:0", + .start_hook =3D test_migrate_tls_x509_start_mismatch_host, + .finish_hook =3D test_migrate_tls_x509_finish, + .result =3D MIG_TEST_FAIL_DEST_QUIT_ERR, + }; + + test_precopy_common(&args); +} + +static void test_precopy_tcp_tls_x509_friendly_client(void) +{ + MigrateCommon args =3D { + .listen_uri =3D "tcp:127.0.0.1:0", + .start_hook =3D test_migrate_tls_x509_start_friendly_client, + .finish_hook =3D test_migrate_tls_x509_finish, + }; + + test_precopy_common(&args); +} + +static void test_precopy_tcp_tls_x509_hostile_client(void) +{ + MigrateCommon args =3D { + .start =3D { + .hide_stderr =3D true, + }, + .listen_uri =3D "tcp:127.0.0.1:0", + .start_hook =3D test_migrate_tls_x509_start_hostile_client, + .finish_hook =3D test_migrate_tls_x509_finish, + .result =3D MIG_TEST_FAIL, + }; + + test_precopy_common(&args); +} + +static void test_precopy_tcp_tls_x509_allow_anon_client(void) +{ + MigrateCommon args =3D { + .listen_uri =3D "tcp:127.0.0.1:0", + .start_hook =3D test_migrate_tls_x509_start_allow_anon_client, + .finish_hook =3D test_migrate_tls_x509_finish, + }; + + test_precopy_common(&args); +} + +static void test_precopy_tcp_tls_x509_reject_anon_client(void) +{ + MigrateCommon args =3D { + .start =3D { + .hide_stderr =3D true, + }, + .listen_uri =3D "tcp:127.0.0.1:0", + .start_hook =3D test_migrate_tls_x509_start_reject_anon_client, + .finish_hook =3D test_migrate_tls_x509_finish, + .result =3D MIG_TEST_FAIL, + }; + + test_precopy_common(&args); +} +#endif /* CONFIG_TASN1 */ #endif /* CONFIG_GNUTLS */ =20 static void *test_migrate_fd_start_hook(QTestState *from, @@ -1642,6 +1997,12 @@ int main(int argc, char **argv) #ifdef CONFIG_GNUTLS qtest_add_func("/migration/precopy/unix/tls/psk", test_precopy_unix_tls_psk); +#ifdef CONFIG_TASN1 + qtest_add_func("/migration/precopy/unix/tls/x509/default-host", + test_precopy_unix_tls_x509_default_host); + qtest_add_func("/migration/precopy/unix/tls/x509/override-host", + test_precopy_unix_tls_x509_override_host); +#endif /* CONFIG_TASN1 */ #endif /* CONFIG_GNUTLS */ =20 qtest_add_func("/migration/precopy/tcp/plain", test_precopy_tcp_plain); @@ -1650,6 +2011,22 @@ int main(int argc, char **argv) test_precopy_tcp_tls_psk_match); qtest_add_func("/migration/precopy/tcp/tls/psk/mismatch", test_precopy_tcp_tls_psk_mismatch); +#ifdef CONFIG_TASN1 + qtest_add_func("/migration/precopy/tcp/tls/x509/default-host", + test_precopy_tcp_tls_x509_default_host); + qtest_add_func("/migration/precopy/tcp/tls/x509/override-host", + test_precopy_tcp_tls_x509_override_host); + qtest_add_func("/migration/precopy/tcp/tls/x509/mismatch-host", + test_precopy_tcp_tls_x509_mismatch_host); + qtest_add_func("/migration/precopy/tcp/tls/x509/friendly-client", + test_precopy_tcp_tls_x509_friendly_client); + qtest_add_func("/migration/precopy/tcp/tls/x509/hostile-client", + test_precopy_tcp_tls_x509_hostile_client); + qtest_add_func("/migration/precopy/tcp/tls/x509/allow-anon-client", + test_precopy_tcp_tls_x509_allow_anon_client); + qtest_add_func("/migration/precopy/tcp/tls/x509/reject-anon-client", + test_precopy_tcp_tls_x509_reject_anon_client); +#endif /* CONFIG_TASN1 */ #endif /* CONFIG_GNUTLS */ =20 /* qtest_add_func("/migration/ignore_shared", test_ignore_shared); */ --=20 2.36.1 From nobody Mon Apr 29 04:30:41 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1652718211; cv=none; d=zohomail.com; s=zohoarc; b=WUaYwB/kj+DuyUhHN42e79oNjYia/GWD9DF4QvvKjhfkeCagaqazviX6r+Q7FWTD4iJhjujOXxd0ekBsw6AymjTgEkEM7TsMiI96yUdBGrnNP2IuBdyB4u+wUFOnuDKYptmdrqEyFahxCInkY85iCOIMRZcYSOk2fGUBXWacuVQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1652718211; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=KL3nibOyS3OGjHim7mZMZGap+wwWj+6J/2SlAhXpRPE=; b=lPgGBq9m7FTxLK/Fn0l4y/OEkNloVmE7FTrCGgTww+jQPhWqD7ZZXUNJFNMmkHk+MKiELCkY3KCmYuZcfT61EJ6uLtUh4U5nqSZdNzJBqwe+T273lP8q7XoLtG5SA/quWV7e852u+/A+uyEMk8+6yqo0TRs9u481kCRaWusyLew= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 16527182117911.6007064298870546; Mon, 16 May 2022 09:23:31 -0700 (PDT) Received: from localhost ([::1]:53936 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nqdVJ-0005sp-92 for importer@patchew.org; Mon, 16 May 2022 12:23:29 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35604) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqcnk-00029a-87 for qemu-devel@nongnu.org; Mon, 16 May 2022 11:38:28 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:20329) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqcni-0008EB-G5 for qemu-devel@nongnu.org; Mon, 16 May 2022 11:38:27 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-12-LHCD0Ek_Pw6szN1XHDTD2Q-1; Mon, 16 May 2022 11:38:24 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 3B5218015BA for ; Mon, 16 May 2022 15:38:24 +0000 (UTC) Received: from dgilbert-t580.localhost (unknown [10.39.194.215]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4ED2A15156FC; Mon, 16 May 2022 15:38:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1652715505; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=KL3nibOyS3OGjHim7mZMZGap+wwWj+6J/2SlAhXpRPE=; b=TPnQVHjbVJZuXXC9JkxieGCV88irihrxDHPZ0SmZ9rgeyC2qApFUx+o29N1uP2oD2X/8Ku z8GkuiwHv5WTAUGtCagcqUPgXoPb/NG1mF3Ux7SEIhsV4qNP5mOKmdl91vbeQ8wn32SX8+ k1E45yUhe5tfQTSpo2E+Il1WtY90Mtw= X-MC-Unique: LHCD0Ek_Pw6szN1XHDTD2Q-1 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, quintela@redhat.com, peterx@redhat.com, leobras@redhat.com, berrange@redhat.com Subject: [PULL 05/17] tests: convert XBZRLE migration test to use common helper Date: Mon, 16 May 2022 16:38:00 +0100 Message-Id: <20220516153812.127155-6-dgilbert@redhat.com> In-Reply-To: <20220516153812.127155-1-dgilbert@redhat.com> References: <20220516153812.127155-1-dgilbert@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.85 on 10.11.54.7 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=dgilbert@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1652718212208100001 From: Daniel P. Berrang=C3=A9 Most of the XBZRLE migration test logic is common with the rest of the precopy tests, so it can use the helper with just one small tweak. Reviewed-by: Peter Xu Signed-off-by: Daniel P. Berrang=C3=A9 Message-Id: <20220426160048.812266-6-berrange@redhat.com> Signed-off-by: Dr. David Alan Gilbert --- tests/qtest/migration-test.c | 67 ++++++++++++++---------------------- 1 file changed, 25 insertions(+), 42 deletions(-) diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c index 5a3edf2da6..a1dc08a93e 100644 --- a/tests/qtest/migration-test.c +++ b/tests/qtest/migration-test.c @@ -1174,6 +1174,9 @@ typedef struct { /* This test should fail, dest qemu should fail with abnormal stat= us */ MIG_TEST_FAIL_DEST_QUIT_ERR, } result; + + /* Optional: set number of migration passes to wait for */ + unsigned int iterations; } MigrateCommon; =20 static void test_precopy_common(MigrateCommon *args) @@ -1219,7 +1222,13 @@ static void test_precopy_common(MigrateCommon *args) qtest_set_expected_status(to, 1); } } else { - wait_for_migration_pass(from); + if (args->iterations) { + while (args->iterations--) { + wait_for_migration_pass(from); + } + } else { + wait_for_migration_pass(from); + } =20 migrate_set_parameter_int(from, "downtime-limit", CONVERGE_DOWNTIM= E); =20 @@ -1350,57 +1359,31 @@ static void test_ignore_shared(void) } #endif =20 -static void test_xbzrle(const char *uri) +static void * +test_migrate_xbzrle_start(QTestState *from, + QTestState *to) { - MigrateStart args =3D {}; - QTestState *from, *to; - - if (test_migrate_start(&from, &to, uri, &args)) { - return; - } - - /* - * We want to pick a speed slow enough that the test completes - * quickly, but that it doesn't complete precopy even on a slow - * machine, so also set the downtime. - */ - /* 1 ms should make it not converge*/ - migrate_set_parameter_int(from, "downtime-limit", 1); - /* 1GB/s */ - migrate_set_parameter_int(from, "max-bandwidth", 1000000000); - migrate_set_parameter_int(from, "xbzrle-cache-size", 33554432); =20 migrate_set_capability(from, "xbzrle", true); migrate_set_capability(to, "xbzrle", true); - /* Wait for the first serial output from the source */ - wait_for_serial("src_serial"); =20 - migrate_qmp(from, uri, "{}"); - - wait_for_migration_pass(from); - /* Make sure we have 2 passes, so the xbzrle cache gets a workout */ - wait_for_migration_pass(from); - - /* 1000ms should converge */ - migrate_set_parameter_int(from, "downtime-limit", 1000); - - if (!got_stop) { - qtest_qmp_eventwait(from, "STOP"); - } - qtest_qmp_eventwait(to, "RESUME"); - - wait_for_serial("dest_serial"); - wait_for_migration_complete(from); - - test_migrate_end(from, to, true); + return NULL; } =20 -static void test_xbzrle_unix(void) +static void test_precopy_unix_xbzrle(void) { g_autofree char *uri =3D g_strdup_printf("unix:%s/migsocket", tmpfs); + MigrateCommon args =3D { + .connect_uri =3D uri, + .listen_uri =3D uri, + + .start_hook =3D test_migrate_xbzrle_start, =20 - test_xbzrle(uri); + .iterations =3D 2, + }; + + test_precopy_common(&args); } =20 static void test_precopy_tcp_plain(void) @@ -1994,6 +1977,7 @@ int main(int argc, char **argv) qtest_add_func("/migration/postcopy/recovery", test_postcopy_recovery); qtest_add_func("/migration/bad_dest", test_baddest); qtest_add_func("/migration/precopy/unix/plain", test_precopy_unix_plai= n); + qtest_add_func("/migration/precopy/unix/xbzrle", test_precopy_unix_xbz= rle); #ifdef CONFIG_GNUTLS qtest_add_func("/migration/precopy/unix/tls/psk", test_precopy_unix_tls_psk); @@ -2030,7 +2014,6 @@ int main(int argc, char **argv) #endif /* CONFIG_GNUTLS */ =20 /* qtest_add_func("/migration/ignore_shared", test_ignore_shared); */ - qtest_add_func("/migration/xbzrle/unix", test_xbzrle_unix); qtest_add_func("/migration/fd_proto", test_migrate_fd_proto); qtest_add_func("/migration/validate_uuid", test_validate_uuid); qtest_add_func("/migration/validate_uuid_error", test_validate_uuid_er= ror); --=20 2.36.1 From nobody Mon Apr 29 04:30:41 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1652719750; cv=none; d=zohomail.com; s=zohoarc; b=Ix5anenQc8knkhrMUbVz57Ilhrj3ASIXZDznqfphMedJ3REcFFcICt2mZVoBo5fI3/lpcYwojEvuRdWaIrK/s7hy9gvmDtrni+ws8xxyc/y4z/VjcU5U3sjNwAdvTthPFNfukUtAKf+hVRMSJXNv8nhzuoulWrSSyGC2uLGLgzM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1652719750; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=ObzRTNg1+iAsvYaW68+gbeUan3xESPSxcQTkARXStUg=; b=JmUYGmJbbklSXkiRFC+9cjqZAKhc+un1QbL6AwGYqI29cPRGGl76akiXV91OVkmDPmljhW2WaJvGMZW/pgsNhSoiRrq/Rv0cnuLmEJ8+zclMi1qF6nqJDTT09NVHO1d6h/sQhWS10E9IGy6ds1Tow3Qjhha0r8TWSlQNv4yZ9/8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1652719750864498.44438626684416; Mon, 16 May 2022 09:49:10 -0700 (PDT) Received: from localhost ([::1]:35840 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nqdu9-0001aP-3i for importer@patchew.org; Mon, 16 May 2022 12:49:09 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35620) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqcnl-0002Ca-LR for qemu-devel@nongnu.org; Mon, 16 May 2022 11:38:29 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:45856) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqcnj-0008ER-Om for qemu-devel@nongnu.org; Mon, 16 May 2022 11:38:29 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-577-J8otSle0N16kdOuSeHebPA-1; Mon, 16 May 2022 11:38:25 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 859508015BA for ; Mon, 16 May 2022 15:38:25 +0000 (UTC) Received: from dgilbert-t580.localhost (unknown [10.39.194.215]) by smtp.corp.redhat.com (Postfix) with ESMTP id 76C9915156FC; Mon, 16 May 2022 15:38:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1652715507; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ObzRTNg1+iAsvYaW68+gbeUan3xESPSxcQTkARXStUg=; b=YQIfkVDYPqc7rE3y11LnpfCA84p0OUMWL5XZv1C01H6CpaRUaPm1k1bUaO331agpCJwGoW dMpcUIakhj6RsFKRugVI+nSluIYucDvl91ZtijqM1fb2YY5SLPMn8+8BI1GvLNvk0YnnLS y79qB47k9nZbDnPI5eN8oxXY5p3K+pc= X-MC-Unique: J8otSle0N16kdOuSeHebPA-1 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, quintela@redhat.com, peterx@redhat.com, leobras@redhat.com, berrange@redhat.com Subject: [PULL 06/17] tests: convert multifd migration tests to use common helper Date: Mon, 16 May 2022 16:38:01 +0100 Message-Id: <20220516153812.127155-7-dgilbert@redhat.com> In-Reply-To: <20220516153812.127155-1-dgilbert@redhat.com> References: <20220516153812.127155-1-dgilbert@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.85 on 10.11.54.7 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=dgilbert@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1652719752038100001 From: Daniel P. Berrang=C3=A9 Most of the multifd migration test logic is common with the rest of the precopy tests, so it can use the helper without difficulty. The only exception of the multifd cancellation test which tries to run multiple migrations in a row. Reviewed-by: Peter Xu Signed-off-by: Daniel P. Berrang=C3=A9 Message-Id: <20220426160048.812266-7-berrange@redhat.com> Signed-off-by: Dr. David Alan Gilbert --- tests/qtest/migration-test.c | 77 +++++++++++++++++++----------------- 1 file changed, 40 insertions(+), 37 deletions(-) diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c index a1dc08a93e..f551c8d030 100644 --- a/tests/qtest/migration-test.c +++ b/tests/qtest/migration-test.c @@ -1740,26 +1740,12 @@ static void test_migrate_auto_converge(void) test_migrate_end(from, to, true); } =20 -static void test_multifd_tcp(const char *method) +static void * +test_migrate_precopy_tcp_multifd_start_common(QTestState *from, + QTestState *to, + const char *method) { - MigrateStart args =3D {}; - QTestState *from, *to; QDict *rsp; - g_autofree char *uri =3D NULL; - - if (test_migrate_start(&from, &to, "defer", &args)) { - return; - } - - /* - * We want to pick a speed slow enough that the test completes - * quickly, but that it doesn't complete precopy even on a slow - * machine, so also set the downtime. - */ - /* 1 ms should make it not converge*/ - migrate_set_parameter_int(from, "downtime-limit", 1); - /* 1GB/s */ - migrate_set_parameter_int(from, "max-bandwidth", 1000000000); =20 migrate_set_parameter_int(from, "multifd-channels", 16); migrate_set_parameter_int(to, "multifd-channels", 16); @@ -1775,41 +1761,58 @@ static void test_multifd_tcp(const char *method) " 'arguments': { 'uri': 'tcp:127.0.0.1:0' }}"); qobject_unref(rsp); =20 - /* Wait for the first serial output from the source */ - wait_for_serial("src_serial"); - - uri =3D migrate_get_socket_address(to, "socket-address"); - - migrate_qmp(from, uri, "{}"); - - wait_for_migration_pass(from); + return NULL; +} =20 - migrate_set_parameter_int(from, "downtime-limit", CONVERGE_DOWNTIME); +static void * +test_migrate_precopy_tcp_multifd_start(QTestState *from, + QTestState *to) +{ + return test_migrate_precopy_tcp_multifd_start_common(from, to, "none"); +} =20 - if (!got_stop) { - qtest_qmp_eventwait(from, "STOP"); - } - qtest_qmp_eventwait(to, "RESUME"); +static void * +test_migrate_precopy_tcp_multifd_zlib_start(QTestState *from, + QTestState *to) +{ + return test_migrate_precopy_tcp_multifd_start_common(from, to, "zlib"); +} =20 - wait_for_serial("dest_serial"); - wait_for_migration_complete(from); - test_migrate_end(from, to, true); +#ifdef CONFIG_ZSTD +static void * +test_migrate_precopy_tcp_multifd_zstd_start(QTestState *from, + QTestState *to) +{ + return test_migrate_precopy_tcp_multifd_start_common(from, to, "zstd"); } +#endif /* CONFIG_ZSTD */ =20 static void test_multifd_tcp_none(void) { - test_multifd_tcp("none"); + MigrateCommon args =3D { + .listen_uri =3D "defer", + .start_hook =3D test_migrate_precopy_tcp_multifd_start, + }; + test_precopy_common(&args); } =20 static void test_multifd_tcp_zlib(void) { - test_multifd_tcp("zlib"); + MigrateCommon args =3D { + .listen_uri =3D "defer", + .start_hook =3D test_migrate_precopy_tcp_multifd_zlib_start, + }; + test_precopy_common(&args); } =20 #ifdef CONFIG_ZSTD static void test_multifd_tcp_zstd(void) { - test_multifd_tcp("zstd"); + MigrateCommon args =3D { + .listen_uri =3D "defer", + .start_hook =3D test_migrate_precopy_tcp_multifd_zstd_start, + }; + test_precopy_common(&args); } #endif =20 --=20 2.36.1 From nobody Mon Apr 29 04:30:41 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1652718979; cv=none; d=zohomail.com; s=zohoarc; b=eMfAAfqTpgmaZci3sPvYX31/rzwMzPbfa7Dccz2Mi1e7SAvcYNCPsQdrqaiOY8iJ4PwB7j2UeSemKZGagidxBvzF58Adyj54DgdXcSFPcOS1X4+04ATKxr6XuZmEXK8zsWjg4aM6S2A4sfN3vmlZPRmR8AF7yDMg+boaxyqUCJ8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1652718979; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=87uk/BEncWcd8P/6afTzU4H6wTvyqWQlGlPNLEcY77M=; b=ffDMkJEEInDZxk9pYWorggEdiAWV7jJJJpuIloqoP4MKRWByIideyanDWon5P3iLzC53XvTNucdfAv21Xc+T+6PX3DrFR3ExqAc5La/H0TI061/xJT7qUxggXmFdc9tM1NtxDKuISK3P4Y7XhYHkk10mkdKDxz1rFaSZlNBXJz4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1652718979740884.0368956100239; Mon, 16 May 2022 09:36:19 -0700 (PDT) Received: from localhost ([::1]:34534 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nqdhi-0006Jh-LJ for importer@patchew.org; Mon, 16 May 2022 12:36:18 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35630) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqcnm-0002Fg-PO for qemu-devel@nongnu.org; Mon, 16 May 2022 11:38:30 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:27828) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqcnl-0008Ed-1v for qemu-devel@nongnu.org; Mon, 16 May 2022 11:38:30 -0400 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-445-S-R3g0mmOEGJyB6GRVm4Fw-1; Mon, 16 May 2022 11:38:26 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id A7DC629ABA1A for ; Mon, 16 May 2022 15:38:26 +0000 (UTC) Received: from dgilbert-t580.localhost (unknown [10.39.194.215]) by smtp.corp.redhat.com (Postfix) with ESMTP id C0EB415156FC; Mon, 16 May 2022 15:38:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1652715508; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=87uk/BEncWcd8P/6afTzU4H6wTvyqWQlGlPNLEcY77M=; b=hkcd5ZFyb43GEEDRcSgeZpDH/rZcTLvuw0xlmp1U8ZhJUy6J8fpBbrhSJGQ4IOtkFu0pdW lPwQwb+s+vKqRWI/48x0Ii5pSA2RBDLazqiCxpRPlkPXES7qJEg1YyowpQcJlTTHdZWmlu AxOMjVJGM2HJUbA51MwdSsBpCkruAUg= X-MC-Unique: S-R3g0mmOEGJyB6GRVm4Fw-1 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, quintela@redhat.com, peterx@redhat.com, leobras@redhat.com, berrange@redhat.com Subject: [PULL 07/17] tests: add multifd migration tests of TLS with PSK credentials Date: Mon, 16 May 2022 16:38:02 +0100 Message-Id: <20220516153812.127155-8-dgilbert@redhat.com> In-Reply-To: <20220516153812.127155-1-dgilbert@redhat.com> References: <20220516153812.127155-1-dgilbert@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.85 on 10.11.54.7 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=dgilbert@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1652718980743100001 From: Daniel P. Berrang=C3=A9 This validates that we correctly handle multifd migration success and failure scenarios when using TLS with pre shared keys. Signed-off-by: Daniel P. Berrang=C3=A9 Message-Id: <20220426160048.812266-8-berrange@redhat.com> Reviewed-by: Eric Blake Signed-off-by: Dr. David Alan Gilbert --- tests/qtest/migration-test.c | 60 +++++++++++++++++++++++++++++++++--- 1 file changed, 56 insertions(+), 4 deletions(-) diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c index f551c8d030..133665b500 100644 --- a/tests/qtest/migration-test.c +++ b/tests/qtest/migration-test.c @@ -1816,6 +1816,48 @@ static void test_multifd_tcp_zstd(void) } #endif =20 +#ifdef CONFIG_GNUTLS +static void * +test_migrate_multifd_tcp_tls_psk_start_match(QTestState *from, + QTestState *to) +{ + test_migrate_precopy_tcp_multifd_start_common(from, to, "none"); + return test_migrate_tls_psk_start_match(from, to); +} + +static void * +test_migrate_multifd_tcp_tls_psk_start_mismatch(QTestState *from, + QTestState *to) +{ + test_migrate_precopy_tcp_multifd_start_common(from, to, "none"); + return test_migrate_tls_psk_start_mismatch(from, to); +} + +static void test_multifd_tcp_tls_psk_match(void) +{ + MigrateCommon args =3D { + .listen_uri =3D "defer", + .start_hook =3D test_migrate_multifd_tcp_tls_psk_start_match, + .finish_hook =3D test_migrate_tls_psk_finish, + }; + test_precopy_common(&args); +} + +static void test_multifd_tcp_tls_psk_mismatch(void) +{ + MigrateCommon args =3D { + .start =3D { + .hide_stderr =3D true, + }, + .listen_uri =3D "defer", + .start_hook =3D test_migrate_multifd_tcp_tls_psk_start_mismatch, + .finish_hook =3D test_migrate_tls_psk_finish, + .result =3D MIG_TEST_FAIL, + }; + test_precopy_common(&args); +} +#endif /* CONFIG_GNUTLS */ + /* * This test does: * source target @@ -2026,12 +2068,22 @@ int main(int argc, char **argv) test_validate_uuid_dst_not_set); =20 qtest_add_func("/migration/auto_converge", test_migrate_auto_converge); - qtest_add_func("/migration/multifd/tcp/none", test_multifd_tcp_none); - qtest_add_func("/migration/multifd/tcp/cancel", test_multifd_tcp_cance= l); - qtest_add_func("/migration/multifd/tcp/zlib", test_multifd_tcp_zlib); + qtest_add_func("/migration/multifd/tcp/plain/none", + test_multifd_tcp_none); + qtest_add_func("/migration/multifd/tcp/plain/cancel", + test_multifd_tcp_cancel); + qtest_add_func("/migration/multifd/tcp/plain/zlib", + test_multifd_tcp_zlib); #ifdef CONFIG_ZSTD - qtest_add_func("/migration/multifd/tcp/zstd", test_multifd_tcp_zstd); + qtest_add_func("/migration/multifd/tcp/plain/zstd", + test_multifd_tcp_zstd); #endif +#ifdef CONFIG_GNUTLS + qtest_add_func("/migration/multifd/tcp/tls/psk/match", + test_multifd_tcp_tls_psk_match); + qtest_add_func("/migration/multifd/tcp/tls/psk/mismatch", + test_multifd_tcp_tls_psk_mismatch); +#endif /* CONFIG_GNUTLS */ =20 if (kvm_dirty_ring_supported()) { qtest_add_func("/migration/dirty_ring", --=20 2.36.1 From nobody Mon Apr 29 04:30:41 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1652720135; cv=none; d=zohomail.com; s=zohoarc; b=Yh0LaW8LnLsoSQy6/sT54ldbjJERx+g052vvNguEpgRUWpnxUzCQ1i/KpA7yItVSMrLdKufhdamNifsfD6G0cG+lAKXq9mH8oipia/76Tg/ZyzFiu50453xxMUyzVarOvTSvHm8/1MPBKgMB4npnznYyN0WUX+PwYwclXAYfUxc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1652720135; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=jiSiViv4bhHap7il9wtagSRAkjU5U1qvMOOHE8o9/jI=; b=OPPr5UWyTEghi59YzgS1loQ/fOy2mhDEpxG7649jhQGf5idKVb4CkmsQ2Xo63Vb+6V4SJW0e16ObYyH4dd8tS6Na2vbxQeZWUIobE4GpqRJPIwrzHy99dUv0pJXJyDJmsqNXOb6WV7FukW3ZdWsTfPLBQFg4mmA5Dav1CADKC5s= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1652720135576677.0274654238924; Mon, 16 May 2022 09:55:35 -0700 (PDT) Received: from localhost ([::1]:44620 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nqe0M-0007gi-CO for importer@patchew.org; Mon, 16 May 2022 12:55:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35660) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqcno-0002L7-Vp for qemu-devel@nongnu.org; Mon, 16 May 2022 11:38:33 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:51999) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqcnn-0008Ex-5G for qemu-devel@nongnu.org; Mon, 16 May 2022 11:38:32 -0400 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-508-WjWHapfeMpmC3DS6qs3yQg-1; Mon, 16 May 2022 11:38:28 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C9EAE1C05EAA for ; Mon, 16 May 2022 15:38:27 +0000 (UTC) Received: from dgilbert-t580.localhost (unknown [10.39.194.215]) by smtp.corp.redhat.com (Postfix) with ESMTP id E385F15215E9; Mon, 16 May 2022 15:38:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1652715510; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jiSiViv4bhHap7il9wtagSRAkjU5U1qvMOOHE8o9/jI=; b=XN7o+zQfnlGcKVRMhrSHlCDBTRFzgNrdOuCwbyHgx+o3/KMgalJxbTnboI4ZRQE1cF/VS2 qpwNU82NbJJQH0kWvWUEawxig1/Ty0QSSlAnXVBB8vikKJ2AJ2tGiIPpF6OoFcxbf1BqMt uYT5igQyG0+oW/HJCkrff0cEF7Cz7WA= X-MC-Unique: WjWHapfeMpmC3DS6qs3yQg-1 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, quintela@redhat.com, peterx@redhat.com, leobras@redhat.com, berrange@redhat.com Subject: [PULL 08/17] tests: add multifd migration tests of TLS with x509 credentials Date: Mon, 16 May 2022 16:38:03 +0100 Message-Id: <20220516153812.127155-9-dgilbert@redhat.com> In-Reply-To: <20220516153812.127155-1-dgilbert@redhat.com> References: <20220516153812.127155-1-dgilbert@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.85 on 10.11.54.7 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=dgilbert@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1652720136962100001 From: Daniel P. Berrang=C3=A9 This validates that we correctly handle multifd migration success and failure scenarios when using TLS with x509 certificates. There are quite a few different scenarios that matter in relation to hostname validation, but we skip a couple as we can assume that the non-multifd coverage applies to some extent. Signed-off-by: Daniel P. Berrang=C3=A9 Message-Id: <20220426160048.812266-9-berrange@redhat.com> Reviewed-by: Eric Blake Signed-off-by: Dr. David Alan Gilbert --- tests/qtest/migration-test.c | 127 +++++++++++++++++++++++++++++++++++ 1 file changed, 127 insertions(+) diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c index 133665b500..efc6ec1614 100644 --- a/tests/qtest/migration-test.c +++ b/tests/qtest/migration-test.c @@ -1833,6 +1833,48 @@ test_migrate_multifd_tcp_tls_psk_start_mismatch(QTes= tState *from, return test_migrate_tls_psk_start_mismatch(from, to); } =20 +#ifdef CONFIG_TASN1 +static void * +test_migrate_multifd_tls_x509_start_default_host(QTestState *from, + QTestState *to) +{ + test_migrate_precopy_tcp_multifd_start_common(from, to, "none"); + return test_migrate_tls_x509_start_default_host(from, to); +} + +static void * +test_migrate_multifd_tls_x509_start_override_host(QTestState *from, + QTestState *to) +{ + test_migrate_precopy_tcp_multifd_start_common(from, to, "none"); + return test_migrate_tls_x509_start_override_host(from, to); +} + +static void * +test_migrate_multifd_tls_x509_start_mismatch_host(QTestState *from, + QTestState *to) +{ + test_migrate_precopy_tcp_multifd_start_common(from, to, "none"); + return test_migrate_tls_x509_start_mismatch_host(from, to); +} + +static void * +test_migrate_multifd_tls_x509_start_allow_anon_client(QTestState *from, + QTestState *to) +{ + test_migrate_precopy_tcp_multifd_start_common(from, to, "none"); + return test_migrate_tls_x509_start_allow_anon_client(from, to); +} + +static void * +test_migrate_multifd_tls_x509_start_reject_anon_client(QTestState *from, + QTestState *to) +{ + test_migrate_precopy_tcp_multifd_start_common(from, to, "none"); + return test_migrate_tls_x509_start_reject_anon_client(from, to); +} +#endif /* CONFIG_TASN1 */ + static void test_multifd_tcp_tls_psk_match(void) { MigrateCommon args =3D { @@ -1856,6 +1898,79 @@ static void test_multifd_tcp_tls_psk_mismatch(void) }; test_precopy_common(&args); } + +#ifdef CONFIG_TASN1 +static void test_multifd_tcp_tls_x509_default_host(void) +{ + MigrateCommon args =3D { + .listen_uri =3D "defer", + .start_hook =3D test_migrate_multifd_tls_x509_start_default_host, + .finish_hook =3D test_migrate_tls_x509_finish, + }; + test_precopy_common(&args); +} + +static void test_multifd_tcp_tls_x509_override_host(void) +{ + MigrateCommon args =3D { + .listen_uri =3D "defer", + .start_hook =3D test_migrate_multifd_tls_x509_start_override_host, + .finish_hook =3D test_migrate_tls_x509_finish, + }; + test_precopy_common(&args); +} + +static void test_multifd_tcp_tls_x509_mismatch_host(void) +{ + /* + * This has different behaviour to the non-multifd case. + * + * In non-multifd case when client aborts due to mismatched + * cert host, the server has already started trying to load + * migration state, and so it exits with I/O failure. + * + * In multifd case when client aborts due to mismatched + * cert host, the server is still waiting for the other + * multifd connections to arrive so hasn't started trying + * to load migration state, and thus just aborts the migration + * without exiting. + */ + MigrateCommon args =3D { + .start =3D { + .hide_stderr =3D true, + }, + .listen_uri =3D "defer", + .start_hook =3D test_migrate_multifd_tls_x509_start_mismatch_host, + .finish_hook =3D test_migrate_tls_x509_finish, + .result =3D MIG_TEST_FAIL, + }; + test_precopy_common(&args); +} + +static void test_multifd_tcp_tls_x509_allow_anon_client(void) +{ + MigrateCommon args =3D { + .listen_uri =3D "defer", + .start_hook =3D test_migrate_multifd_tls_x509_start_allow_anon_cli= ent, + .finish_hook =3D test_migrate_tls_x509_finish, + }; + test_precopy_common(&args); +} + +static void test_multifd_tcp_tls_x509_reject_anon_client(void) +{ + MigrateCommon args =3D { + .start =3D { + .hide_stderr =3D true, + }, + .listen_uri =3D "defer", + .start_hook =3D test_migrate_multifd_tls_x509_start_reject_anon_cl= ient, + .finish_hook =3D test_migrate_tls_x509_finish, + .result =3D MIG_TEST_FAIL, + }; + test_precopy_common(&args); +} +#endif /* CONFIG_TASN1 */ #endif /* CONFIG_GNUTLS */ =20 /* @@ -2083,6 +2198,18 @@ int main(int argc, char **argv) test_multifd_tcp_tls_psk_match); qtest_add_func("/migration/multifd/tcp/tls/psk/mismatch", test_multifd_tcp_tls_psk_mismatch); +#ifdef CONFIG_TASN1 + qtest_add_func("/migration/multifd/tcp/tls/x509/default-host", + test_multifd_tcp_tls_x509_default_host); + qtest_add_func("/migration/multifd/tcp/tls/x509/override-host", + test_multifd_tcp_tls_x509_override_host); + qtest_add_func("/migration/multifd/tcp/tls/x509/mismatch-host", + test_multifd_tcp_tls_x509_mismatch_host); + qtest_add_func("/migration/multifd/tcp/tls/x509/allow-anon-client", + test_multifd_tcp_tls_x509_allow_anon_client); + qtest_add_func("/migration/multifd/tcp/tls/x509/reject-anon-client", + test_multifd_tcp_tls_x509_reject_anon_client); +#endif /* CONFIG_TASN1 */ #endif /* CONFIG_GNUTLS */ =20 if (kvm_dirty_ring_supported()) { --=20 2.36.1 From nobody Mon Apr 29 04:30:41 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1652719079; cv=none; d=zohomail.com; s=zohoarc; b=GujylchmfDuQL7jf9GSWtPwpHpwI3oHitC784vNlxNHTDA+FN5OJRkxybOgwDP4k4Zlbbfkp3HF3Ybs6cml33ZVaHpnXOnxlLVs6fjkUUqgEOC21BIv5e6FZ+stzv415Y37LQO+JiDdOjbiD0k1X7vWUwUQF3q8MJnexsx5Ev4U= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1652719079; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=yXXkZWKN8LrPxOaOLf+mp2FVQYecb/UvK2VWosQ8Q/U=; b=ba07CAFXcyitcfvoEbElFKacqjkQ8a86mwinS6YQW2iQGBryMlvo0P2lGInqDRWDAtYzvuk0o7sE6/fGkuxB3zaaRjyX2kTcwUWxNdlFt8jCvg0p+0lNX8pYZqvw32FOTSb08ZbBj45gVqsTUpSeHjwcgmBPSCaDOQKkWEUhPOY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1652719079446384.87130924989583; Mon, 16 May 2022 09:37:59 -0700 (PDT) Received: from localhost ([::1]:42628 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nqdjK-0003Xf-El for importer@patchew.org; Mon, 16 May 2022 12:37:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35664) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqcnp-0002MD-AN for qemu-devel@nongnu.org; Mon, 16 May 2022 11:38:33 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:31066) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqcnn-0008Ez-Ca for qemu-devel@nongnu.org; Mon, 16 May 2022 11:38:33 -0400 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-85-MmIOikY8N82o-pv7xYeIyg-1; Mon, 16 May 2022 11:38:29 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id EDA20395AFE5 for ; Mon, 16 May 2022 15:38:28 +0000 (UTC) Received: from dgilbert-t580.localhost (unknown [10.39.194.215]) by smtp.corp.redhat.com (Postfix) with ESMTP id 10F0015215E9; Mon, 16 May 2022 15:38:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1652715510; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yXXkZWKN8LrPxOaOLf+mp2FVQYecb/UvK2VWosQ8Q/U=; b=ZG2EcZT2GiHMP7psJlAfCkemoArp5qptF6uV0vcIYh/mpqM5MVdXSlDwJqS1vW0DPuNVtf aDai+OUB6UT2WpbS17aiT2pV7ES+g2VruEENZ7cCumXSu5gmpzWvtXmxKwKEC75GfkDlgF VkrMlMZWRhVoHlSvk/Z3Ox6QULBvTkY= X-MC-Unique: MmIOikY8N82o-pv7xYeIyg-1 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, quintela@redhat.com, peterx@redhat.com, leobras@redhat.com, berrange@redhat.com Subject: [PULL 09/17] tests: ensure migration status isn't reported as failed Date: Mon, 16 May 2022 16:38:04 +0100 Message-Id: <20220516153812.127155-10-dgilbert@redhat.com> In-Reply-To: <20220516153812.127155-1-dgilbert@redhat.com> References: <20220516153812.127155-1-dgilbert@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.85 on 10.11.54.7 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=dgilbert@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1652719080029100001 From: Daniel P. Berrang=C3=A9 Various methods in the migration test call 'query_migrate' to fetch the current status and then access a particular field. Almost all of these cases expect the migration to be in a non-failed state. In the case of 'wait_for_migration_pass' in particular, if the status is 'failed' then it will get into an infinite loop. By validating that the status is not 'failed' the test suite will assert rather than hang when getting into an unexpected state. Reviewed-by: Peter Xu Signed-off-by: Daniel P. Berrang=C3=A9 Message-Id: <20220426160048.812266-10-berrange@redhat.com> Signed-off-by: Dr. David Alan Gilbert --- tests/qtest/migration-helpers.c | 13 +++++++++++++ tests/qtest/migration-helpers.h | 1 + tests/qtest/migration-test.c | 6 +++--- 3 files changed, 17 insertions(+), 3 deletions(-) diff --git a/tests/qtest/migration-helpers.c b/tests/qtest/migration-helper= s.c index 4ee26014b7..a6aa59e4e6 100644 --- a/tests/qtest/migration-helpers.c +++ b/tests/qtest/migration-helpers.c @@ -107,6 +107,19 @@ QDict *migrate_query(QTestState *who) return wait_command(who, "{ 'execute': 'query-migrate' }"); } =20 +QDict *migrate_query_not_failed(QTestState *who) +{ + const char *status; + QDict *rsp =3D migrate_query(who); + status =3D qdict_get_str(rsp, "status"); + if (g_str_equal(status, "failed")) { + g_printerr("query-migrate shows failed migration: %s\n", + qdict_get_str(rsp, "error-desc")); + } + g_assert(!g_str_equal(status, "failed")); + return rsp; +} + /* * Note: caller is responsible to free the returned object via * g_free() after use diff --git a/tests/qtest/migration-helpers.h b/tests/qtest/migration-helper= s.h index 2731399aaa..78587c2b82 100644 --- a/tests/qtest/migration-helpers.h +++ b/tests/qtest/migration-helpers.h @@ -27,6 +27,7 @@ G_GNUC_PRINTF(3, 4) void migrate_qmp(QTestState *who, const char *uri, const char *fmt, ...); =20 QDict *migrate_query(QTestState *who); +QDict *migrate_query_not_failed(QTestState *who); =20 void wait_for_migration_status(QTestState *who, const char *goal, const char **ungoals); diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c index efc6ec1614..d33e8060f9 100644 --- a/tests/qtest/migration-test.c +++ b/tests/qtest/migration-test.c @@ -181,7 +181,7 @@ static int64_t read_ram_property_int(QTestState *who, c= onst char *property) QDict *rsp_return, *rsp_ram; int64_t result; =20 - rsp_return =3D migrate_query(who); + rsp_return =3D migrate_query_not_failed(who); if (!qdict_haskey(rsp_return, "ram")) { /* Still in setup */ result =3D 0; @@ -198,7 +198,7 @@ static int64_t read_migrate_property_int(QTestState *wh= o, const char *property) QDict *rsp_return; int64_t result; =20 - rsp_return =3D migrate_query(who); + rsp_return =3D migrate_query_not_failed(who); result =3D qdict_get_try_int(rsp_return, property, 0); qobject_unref(rsp_return); return result; @@ -213,7 +213,7 @@ static void read_blocktime(QTestState *who) { QDict *rsp_return; =20 - rsp_return =3D migrate_query(who); + rsp_return =3D migrate_query_not_failed(who); g_assert(qdict_haskey(rsp_return, "postcopy-blocktime")); qobject_unref(rsp_return); } --=20 2.36.1 From nobody Mon Apr 29 04:30:41 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1652720320; cv=none; d=zohomail.com; s=zohoarc; b=Pz/Mzzdv2A/4tSCHxUuGQDogVfpVbpIaug2R2D62aYvPu69tT9UuUO/xBgBunervx7Yf+A9yDJTT6SockNdLtHeV6CnoyD6B1YCKdefCSoHtHOeO2QWo3b8Xxpr1cIIRHyA8AgOVjs7MAqzpPVAw9B2aSewm4o58ovMEkIZd3+s= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1652720320; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=LSTZvulY/dEa/ZlC+jWoPHDZaCAA1l6KJkwIq3W4PuE=; b=Q2h//zSItzXusuHqUBfIjPu7y99odIBaBzYT50QQM0DQuKKx05qUIOfwIOKArSyYzK+B+vYu+IefuJ7E3PCghX2jMDzU3eBXe8L8/foRIg412xCcLDky7juPPQ3aDeK4BsstTe+Fh3Kf90nglw/wDahMSuWJ4npgVF8N3mWfajA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1652720320705223.7077973066215; Mon, 16 May 2022 09:58:40 -0700 (PDT) Received: from localhost ([::1]:53772 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nqe3L-0005Bo-Mu for importer@patchew.org; Mon, 16 May 2022 12:58:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35686) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqcnq-0002QB-Ef for qemu-devel@nongnu.org; Mon, 16 May 2022 11:38:34 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:37702) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqcno-0008F6-FG for qemu-devel@nongnu.org; Mon, 16 May 2022 11:38:34 -0400 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-383-nU66B_LWPXaj35Z7kdrU0w-1; Mon, 16 May 2022 11:38:30 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 1E8833C0D1A8 for ; Mon, 16 May 2022 15:38:30 +0000 (UTC) Received: from dgilbert-t580.localhost (unknown [10.39.194.215]) by smtp.corp.redhat.com (Postfix) with ESMTP id 36B2D15215E9; Mon, 16 May 2022 15:38:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1652715511; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LSTZvulY/dEa/ZlC+jWoPHDZaCAA1l6KJkwIq3W4PuE=; b=V645jbF/Yze/3BVvhEFRUGAXEEe7naoQWqEgdGA5cWdtV22Wf74t5JhRrMeeQ5bxowUFhd LakE2NjT+uVB0lGcKHDxh1l72diU3G+PBATZCfnOoTxhl/74fnwzU7oy1+8Gr6It0fRBnF jsIldOSVYigygX8QMqa4Fegvtt1LmtA= X-MC-Unique: nU66B_LWPXaj35Z7kdrU0w-1 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, quintela@redhat.com, peterx@redhat.com, leobras@redhat.com, berrange@redhat.com Subject: [PULL 10/17] meson.build: Fix docker-test-build@alpine when including linux/errqueue.h Date: Mon, 16 May 2022 16:38:05 +0100 Message-Id: <20220516153812.127155-11-dgilbert@redhat.com> In-Reply-To: <20220516153812.127155-1-dgilbert@redhat.com> References: <20220516153812.127155-1-dgilbert@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.85 on 10.11.54.7 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=dgilbert@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1652720322526100001 Content-Type: text/plain; charset="utf-8" From: Leonardo Bras A build error happens in alpine CI when linux/errqueue.h is included in io/channel-socket.c, due to redefining of 'struct __kernel_timespec': =3D=3D=3D ninja: job failed: [...] In file included from /usr/include/linux/errqueue.h:6, from ../io/channel-socket.c:29: /usr/include/linux/time_types.h:7:8: error: redefinition of 'struct __kerne= l_timespec' 7 | struct __kernel_timespec { | ^~~~~~~~~~~~~~~~~ In file included from /usr/include/liburing.h:19, from /builds/user/qemu/include/block/aio.h:18, from /builds/user/qemu/include/io/channel.h:26, from /builds/user/qemu/include/io/channel-socket.h:24, from ../io/channel-socket.c:24: /usr/include/liburing/compat.h:9:8: note: originally defined here 9 | struct __kernel_timespec { | ^~~~~~~~~~~~~~~~~ ninja: subcommand failed =3D=3D=3D As above error message suggests, 'struct __kernel_timespec' was already defined by liburing/compat.h. Fix alpine CI by adding test to disable liburing in configure step if a redefinition happens between linux/errqueue.h and liburing/compat.h. [dgilbert: This has been fixed in Alpine issue 13813 and liburing] Signed-off-by: Leonardo Bras Message-Id: <20220513062836.965425-2-leobras@redhat.com> Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Dr. David Alan Gilbert --- meson.build | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/meson.build b/meson.build index 93aa31a9e4..53a4728250 100644 --- a/meson.build +++ b/meson.build @@ -515,12 +515,23 @@ if not get_option('linux_aio').auto() or have_block required: get_option('linux_aio'), kwargs: static_kwargs) endif + +linux_io_uring_test =3D ''' + #include + #include + + int main(void) { return 0; }''' + linux_io_uring =3D not_found if not get_option('linux_io_uring').auto() or have_block linux_io_uring =3D dependency('liburing', version: '>=3D0.3', required: get_option('linux_io_uring'), method: 'pkg-config', kwargs: static_kwargs) + if not cc.links(linux_io_uring_test) + linux_io_uring =3D not_found + endif endif + libnfs =3D not_found if not get_option('libnfs').auto() or have_block libnfs =3D dependency('libnfs', version: '>=3D1.9.3', --=20 2.36.1 From nobody Mon Apr 29 04:30:41 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1652720498; cv=none; d=zohomail.com; s=zohoarc; b=EoLUE1+6XM7Ir5QNQMsMxHCR9K5gMbZYucO0rSuum9BW5HKP2YOXpSLc9y7j8pw6XP2cHyxwMt8ezqC9OoTpny8HgitRloj5FiwdZFqzjFH+Y857ZDlt1nCzY6Xlp8u+BDY88ZQHOGW8GSr3AObs9SdzpWQ/2qlq+2cuYy1DCx8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1652720498; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=CSMlChm4/VB0Gprmt2u8+fKwMxsiiyRvfMV8AW+Q4BE=; b=IEKKwOmkyfYJpR+GbK8eig9vSrN4j19Z/mkYlZPzOh9VyspwNePvC35aLadymnUtVJvb7Lnwlh0Rfs64NsiR+5MxI0uMUyWC13R7+oFmMEW8umeveYva9ggsa4T441APXCFmBGgmkJE2VQW5uq1fh5TDbXDTBiQiJr0dIcNBp2A= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1652720498494100.42292364249431; Mon, 16 May 2022 10:01:38 -0700 (PDT) Received: from localhost ([::1]:34750 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nqe6D-0002eH-4D for importer@patchew.org; Mon, 16 May 2022 13:01:37 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35702) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqcnr-0002T2-RA for qemu-devel@nongnu.org; Mon, 16 May 2022 11:38:35 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:21110) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqcnp-0008FK-Fu for qemu-devel@nongnu.org; Mon, 16 May 2022 11:38:35 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-526-4JwYHrW9O8OMy7nyH1ecUA-1; Mon, 16 May 2022 11:38:31 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 42E5D811E83 for ; Mon, 16 May 2022 15:38:31 +0000 (UTC) Received: from dgilbert-t580.localhost (unknown [10.39.194.215]) by smtp.corp.redhat.com (Postfix) with ESMTP id 59FCC15215E9; Mon, 16 May 2022 15:38:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1652715512; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=CSMlChm4/VB0Gprmt2u8+fKwMxsiiyRvfMV8AW+Q4BE=; b=caW4MRNvXp2GF5LrDrUjHBs/XDn8howk2/v5zUqlezhQ5TUek97+jXBOScO0LxHsYuS3pK N/qtpAWB7wrKRdiwLiqybdyVhMUu/QZ8OM2QIbl+azrzeOgPs9WfVm6rQdG81Uuq4evFXv H3P1zWRaO7hxTSEfKFtX0gmnUTu4U/c= X-MC-Unique: 4JwYHrW9O8OMy7nyH1ecUA-1 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, quintela@redhat.com, peterx@redhat.com, leobras@redhat.com, berrange@redhat.com Subject: [PULL 11/17] QIOChannel: Add flags on io_writev and introduce io_flush callback Date: Mon, 16 May 2022 16:38:06 +0100 Message-Id: <20220516153812.127155-12-dgilbert@redhat.com> In-Reply-To: <20220516153812.127155-1-dgilbert@redhat.com> References: <20220516153812.127155-1-dgilbert@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.85 on 10.11.54.7 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=dgilbert@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1652720500412100001 From: Leonardo Bras Add flags to io_writev and introduce io_flush as optional callback to QIOChannelClass, allowing the implementation of zero copy writes by subclasses. How to use them: - Write data using qio_channel_writev*(...,QIO_CHANNEL_WRITE_FLAG_ZERO_COPY= ), - Wait write completion with qio_channel_flush(). Notes: As some zero copy write implementations work asynchronously, it's recommended to keep the write buffer untouched until the return of qio_channel_flush(), to avoid the risk of sending an updated buffer instead of the buffer state during write. As io_flush callback is optional, if a subclass does not implement it, then: - io_flush will return 0 without changing anything. Also, some functions like qio_channel_writev_full_all() were adapted to receive a flag parameter. That allows shared code between zero copy and non-zero copy writev, and also an easier implementation on new flags. Signed-off-by: Leonardo Bras Reviewed-by: Daniel P. Berrang=C3=A9 Reviewed-by: Peter Xu Reviewed-by: Juan Quintela Message-Id: <20220513062836.965425-3-leobras@redhat.com> Signed-off-by: Dr. David Alan Gilbert --- chardev/char-io.c | 2 +- hw/remote/mpqemu-link.c | 2 +- include/io/channel.h | 38 +++++++++++++++++++++- io/channel-buffer.c | 1 + io/channel-command.c | 1 + io/channel-file.c | 1 + io/channel-socket.c | 2 ++ io/channel-tls.c | 1 + io/channel-websock.c | 1 + io/channel.c | 49 +++++++++++++++++++++++------ migration/rdma.c | 1 + scsi/pr-manager-helper.c | 2 +- tests/unit/test-io-channel-socket.c | 1 + 13 files changed, 88 insertions(+), 14 deletions(-) diff --git a/chardev/char-io.c b/chardev/char-io.c index 8ced184160..4451128cba 100644 --- a/chardev/char-io.c +++ b/chardev/char-io.c @@ -122,7 +122,7 @@ int io_channel_send_full(QIOChannel *ioc, =20 ret =3D qio_channel_writev_full( ioc, &iov, 1, - fds, nfds, NULL); + fds, nfds, 0, NULL); if (ret =3D=3D QIO_CHANNEL_ERR_BLOCK) { if (offset) { return offset; diff --git a/hw/remote/mpqemu-link.c b/hw/remote/mpqemu-link.c index 2a4aa651ca..9bd98e8219 100644 --- a/hw/remote/mpqemu-link.c +++ b/hw/remote/mpqemu-link.c @@ -68,7 +68,7 @@ bool mpqemu_msg_send(MPQemuMsg *msg, QIOChannel *ioc, Err= or **errp) } =20 if (!qio_channel_writev_full_all(ioc, send, G_N_ELEMENTS(send), - fds, nfds, errp)) { + fds, nfds, 0, errp)) { ret =3D true; } else { trace_mpqemu_send_io_error(msg->cmd, msg->size, nfds); diff --git a/include/io/channel.h b/include/io/channel.h index 88988979f8..c680ee7480 100644 --- a/include/io/channel.h +++ b/include/io/channel.h @@ -32,12 +32,15 @@ OBJECT_DECLARE_TYPE(QIOChannel, QIOChannelClass, =20 #define QIO_CHANNEL_ERR_BLOCK -2 =20 +#define QIO_CHANNEL_WRITE_FLAG_ZERO_COPY 0x1 + typedef enum QIOChannelFeature QIOChannelFeature; =20 enum QIOChannelFeature { QIO_CHANNEL_FEATURE_FD_PASS, QIO_CHANNEL_FEATURE_SHUTDOWN, QIO_CHANNEL_FEATURE_LISTEN, + QIO_CHANNEL_FEATURE_WRITE_ZERO_COPY, }; =20 =20 @@ -104,6 +107,7 @@ struct QIOChannelClass { size_t niov, int *fds, size_t nfds, + int flags, Error **errp); ssize_t (*io_readv)(QIOChannel *ioc, const struct iovec *iov, @@ -136,6 +140,8 @@ struct QIOChannelClass { IOHandler *io_read, IOHandler *io_write, void *opaque); + int (*io_flush)(QIOChannel *ioc, + Error **errp); }; =20 /* General I/O handling functions */ @@ -228,6 +234,7 @@ ssize_t qio_channel_readv_full(QIOChannel *ioc, * @niov: the length of the @iov array * @fds: an array of file handles to send * @nfds: number of file handles in @fds + * @flags: write flags (QIO_CHANNEL_WRITE_FLAG_*) * @errp: pointer to a NULL-initialized error object * * Write data to the IO channel, reading it from the @@ -260,6 +267,7 @@ ssize_t qio_channel_writev_full(QIOChannel *ioc, size_t niov, int *fds, size_t nfds, + int flags, Error **errp); =20 /** @@ -837,6 +845,7 @@ int qio_channel_readv_full_all(QIOChannel *ioc, * @niov: the length of the @iov array * @fds: an array of file handles to send * @nfds: number of file handles in @fds + * @flags: write flags (QIO_CHANNEL_WRITE_FLAG_*) * @errp: pointer to a NULL-initialized error object * * @@ -846,6 +855,14 @@ int qio_channel_readv_full_all(QIOChannel *ioc, * to be written, yielding from the current coroutine * if required. * + * If QIO_CHANNEL_WRITE_FLAG_ZERO_COPY is passed in flags, + * instead of waiting for all requested data to be written, + * this function will wait until it's all queued for writing. + * In this case, if the buffer gets changed between queueing and + * sending, the updated buffer will be sent. If this is not a + * desired behavior, it's suggested to call qio_channel_flush() + * before reusing the buffer. + * * Returns: 0 if all bytes were written, or -1 on error */ =20 @@ -853,6 +870,25 @@ int qio_channel_writev_full_all(QIOChannel *ioc, const struct iovec *iov, size_t niov, int *fds, size_t nfds, - Error **errp); + int flags, Error **errp); + +/** + * qio_channel_flush: + * @ioc: the channel object + * @errp: pointer to a NULL-initialized error object + * + * Will block until every packet queued with + * qio_channel_writev_full() + QIO_CHANNEL_WRITE_FLAG_ZERO_COPY + * is sent, or return in case of any error. + * + * If not implemented, acts as a no-op, and returns 0. + * + * Returns -1 if any error is found, + * 1 if every send failed to use zero copy. + * 0 otherwise. + */ + +int qio_channel_flush(QIOChannel *ioc, + Error **errp); =20 #endif /* QIO_CHANNEL_H */ diff --git a/io/channel-buffer.c b/io/channel-buffer.c index baa4e2b089..bf52011be2 100644 --- a/io/channel-buffer.c +++ b/io/channel-buffer.c @@ -81,6 +81,7 @@ static ssize_t qio_channel_buffer_writev(QIOChannel *ioc, size_t niov, int *fds, size_t nfds, + int flags, Error **errp) { QIOChannelBuffer *bioc =3D QIO_CHANNEL_BUFFER(ioc); diff --git a/io/channel-command.c b/io/channel-command.c index 4a1f969aaa..9f2f4a1793 100644 --- a/io/channel-command.c +++ b/io/channel-command.c @@ -276,6 +276,7 @@ static ssize_t qio_channel_command_writev(QIOChannel *i= oc, size_t niov, int *fds, size_t nfds, + int flags, Error **errp) { QIOChannelCommand *cioc =3D QIO_CHANNEL_COMMAND(ioc); diff --git a/io/channel-file.c b/io/channel-file.c index d146ace7db..b67687c2aa 100644 --- a/io/channel-file.c +++ b/io/channel-file.c @@ -114,6 +114,7 @@ static ssize_t qio_channel_file_writev(QIOChannel *ioc, size_t niov, int *fds, size_t nfds, + int flags, Error **errp) { QIOChannelFile *fioc =3D QIO_CHANNEL_FILE(ioc); diff --git a/io/channel-socket.c b/io/channel-socket.c index e531d7bd2a..05c425abb8 100644 --- a/io/channel-socket.c +++ b/io/channel-socket.c @@ -524,6 +524,7 @@ static ssize_t qio_channel_socket_writev(QIOChannel *io= c, size_t niov, int *fds, size_t nfds, + int flags, Error **errp) { QIOChannelSocket *sioc =3D QIO_CHANNEL_SOCKET(ioc); @@ -619,6 +620,7 @@ static ssize_t qio_channel_socket_writev(QIOChannel *io= c, size_t niov, int *fds, size_t nfds, + int flags, Error **errp) { QIOChannelSocket *sioc =3D QIO_CHANNEL_SOCKET(ioc); diff --git a/io/channel-tls.c b/io/channel-tls.c index 2ae1b92fc0..4ce890a538 100644 --- a/io/channel-tls.c +++ b/io/channel-tls.c @@ -301,6 +301,7 @@ static ssize_t qio_channel_tls_writev(QIOChannel *ioc, size_t niov, int *fds, size_t nfds, + int flags, Error **errp) { QIOChannelTLS *tioc =3D QIO_CHANNEL_TLS(ioc); diff --git a/io/channel-websock.c b/io/channel-websock.c index 55145a6a8c..9619906ac3 100644 --- a/io/channel-websock.c +++ b/io/channel-websock.c @@ -1127,6 +1127,7 @@ static ssize_t qio_channel_websock_writev(QIOChannel = *ioc, size_t niov, int *fds, size_t nfds, + int flags, Error **errp) { QIOChannelWebsock *wioc =3D QIO_CHANNEL_WEBSOCK(ioc); diff --git a/io/channel.c b/io/channel.c index e8b019dc36..0640941ac5 100644 --- a/io/channel.c +++ b/io/channel.c @@ -72,18 +72,32 @@ ssize_t qio_channel_writev_full(QIOChannel *ioc, size_t niov, int *fds, size_t nfds, + int flags, Error **errp) { QIOChannelClass *klass =3D QIO_CHANNEL_GET_CLASS(ioc); =20 - if ((fds || nfds) && - !qio_channel_has_feature(ioc, QIO_CHANNEL_FEATURE_FD_PASS)) { + if (fds || nfds) { + if (!qio_channel_has_feature(ioc, QIO_CHANNEL_FEATURE_FD_PASS)) { + error_setg_errno(errp, EINVAL, + "Channel does not support file descriptor pas= sing"); + return -1; + } + if (flags & QIO_CHANNEL_WRITE_FLAG_ZERO_COPY) { + error_setg_errno(errp, EINVAL, + "Zero Copy does not support file descriptor p= assing"); + return -1; + } + } + + if ((flags & QIO_CHANNEL_WRITE_FLAG_ZERO_COPY) && + !qio_channel_has_feature(ioc, QIO_CHANNEL_FEATURE_WRITE_ZERO_COPY)= ) { error_setg_errno(errp, EINVAL, - "Channel does not support file descriptor passing= "); + "Requested Zero Copy feature is not available"); return -1; } =20 - return klass->io_writev(ioc, iov, niov, fds, nfds, errp); + return klass->io_writev(ioc, iov, niov, fds, nfds, flags, errp); } =20 =20 @@ -217,14 +231,14 @@ int qio_channel_writev_all(QIOChannel *ioc, size_t niov, Error **errp) { - return qio_channel_writev_full_all(ioc, iov, niov, NULL, 0, errp); + return qio_channel_writev_full_all(ioc, iov, niov, NULL, 0, 0, errp); } =20 int qio_channel_writev_full_all(QIOChannel *ioc, const struct iovec *iov, size_t niov, int *fds, size_t nfds, - Error **errp) + int flags, Error **errp) { int ret =3D -1; struct iovec *local_iov =3D g_new(struct iovec, niov); @@ -237,8 +251,10 @@ int qio_channel_writev_full_all(QIOChannel *ioc, =20 while (nlocal_iov > 0) { ssize_t len; - len =3D qio_channel_writev_full(ioc, local_iov, nlocal_iov, fds, n= fds, - errp); + + len =3D qio_channel_writev_full(ioc, local_iov, nlocal_iov, fds, + nfds, flags, errp); + if (len =3D=3D QIO_CHANNEL_ERR_BLOCK) { if (qemu_in_coroutine()) { qio_channel_yield(ioc, G_IO_OUT); @@ -277,7 +293,7 @@ ssize_t qio_channel_writev(QIOChannel *ioc, size_t niov, Error **errp) { - return qio_channel_writev_full(ioc, iov, niov, NULL, 0, errp); + return qio_channel_writev_full(ioc, iov, niov, NULL, 0, 0, errp); } =20 =20 @@ -297,7 +313,7 @@ ssize_t qio_channel_write(QIOChannel *ioc, Error **errp) { struct iovec iov =3D { .iov_base =3D (char *)buf, .iov_len =3D buflen = }; - return qio_channel_writev_full(ioc, &iov, 1, NULL, 0, errp); + return qio_channel_writev_full(ioc, &iov, 1, NULL, 0, 0, errp); } =20 =20 @@ -473,6 +489,19 @@ off_t qio_channel_io_seek(QIOChannel *ioc, return klass->io_seek(ioc, offset, whence, errp); } =20 +int qio_channel_flush(QIOChannel *ioc, + Error **errp) +{ + QIOChannelClass *klass =3D QIO_CHANNEL_GET_CLASS(ioc); + + if (!klass->io_flush || + !qio_channel_has_feature(ioc, QIO_CHANNEL_FEATURE_WRITE_ZERO_COPY)= ) { + return 0; + } + + return klass->io_flush(ioc, errp); +} + =20 static void qio_channel_restart_read(void *opaque) { diff --git a/migration/rdma.c b/migration/rdma.c index ef1e65ec36..672d1958a9 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -2840,6 +2840,7 @@ static ssize_t qio_channel_rdma_writev(QIOChannel *io= c, size_t niov, int *fds, size_t nfds, + int flags, Error **errp) { QIOChannelRDMA *rioc =3D QIO_CHANNEL_RDMA(ioc); diff --git a/scsi/pr-manager-helper.c b/scsi/pr-manager-helper.c index 451c7631b7..3be52a98d5 100644 --- a/scsi/pr-manager-helper.c +++ b/scsi/pr-manager-helper.c @@ -77,7 +77,7 @@ static int pr_manager_helper_write(PRManagerHelper *pr_mg= r, iov.iov_base =3D (void *)buf; iov.iov_len =3D sz; n_written =3D qio_channel_writev_full(QIO_CHANNEL(pr_mgr->ioc), &i= ov, 1, - nfds ? &fd : NULL, nfds, errp); + nfds ? &fd : NULL, nfds, 0, er= rp); =20 if (n_written <=3D 0) { assert(n_written !=3D QIO_CHANNEL_ERR_BLOCK); diff --git a/tests/unit/test-io-channel-socket.c b/tests/unit/test-io-chann= el-socket.c index c49eec1f03..6713886d02 100644 --- a/tests/unit/test-io-channel-socket.c +++ b/tests/unit/test-io-channel-socket.c @@ -444,6 +444,7 @@ static void test_io_channel_unix_fd_pass(void) G_N_ELEMENTS(iosend), fdsend, G_N_ELEMENTS(fdsend), + 0, &error_abort); =20 qio_channel_readv_full(dst, --=20 2.36.1 From nobody Mon Apr 29 04:30:41 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1652716115; cv=none; d=zohomail.com; s=zohoarc; b=YTmR4vZvHqK/7o46JbYhAsfgLQcAfz6gJ01Fhc053Iuj7EdEenWof57ZtL79RWotvn5K6MOSt6BXHI456A47thar8BGyG81Ikh5fhDmtspL32gOw3ToK225mIMk+tTUpIjG21xRJi5vWXGK/R6TyspeKXG9wKqP0IDyTeV6+HNs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1652716115; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=c/MmgXjYJiRYgTnMYgo9XXB38y7lXIL3whWjA6dsIFM=; b=O0i3T5hbMq9pmI05pQ2m5oMpab/epnivbOshnwF2N+TLMkYUjszMntxkppFqIb0NP+Gfbs27HLQw7opy7VUT7LEQFIC87NVuhEt9o60ZznbIDFvEbY830fSa0kPe0zkv+Cx4P9M8H1Ziu+PuJMgoiz8+6+nQqRaLHPk2iuEaN0w= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 165271611589441.737627641341305; Mon, 16 May 2022 08:48:35 -0700 (PDT) Received: from localhost ([::1]:33300 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nqcxW-0002P6-BQ for importer@patchew.org; Mon, 16 May 2022 11:48:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35754) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqcnu-0002Yp-Gt for qemu-devel@nongnu.org; Mon, 16 May 2022 11:38:38 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:25576) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqcns-0008Fx-Ld for qemu-devel@nongnu.org; Mon, 16 May 2022 11:38:38 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-588-56MzTj6QNu-5MYPeWf7xsQ-1; Mon, 16 May 2022 11:38:32 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 62D5B811E7A for ; Mon, 16 May 2022 15:38:32 +0000 (UTC) Received: from dgilbert-t580.localhost (unknown [10.39.194.215]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7E32115215E9; Mon, 16 May 2022 15:38:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1652715516; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=c/MmgXjYJiRYgTnMYgo9XXB38y7lXIL3whWjA6dsIFM=; b=hTQgOUZqUF3v6mTilph1dVPUPL5BGOIgnqQpemhxpdquKVALz/Du2vzVQvFsEPMmLq1Mcj VxVVxB5943UYO+HEzlNy7RTGfRExmUaIV3wfukfKcfj04HWTMQP7MwwQ50Km88BjEFHSM5 acbAXALXWWNEHT86yopvtjmEUjZ/JcI= X-MC-Unique: 56MzTj6QNu-5MYPeWf7xsQ-1 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, quintela@redhat.com, peterx@redhat.com, leobras@redhat.com, berrange@redhat.com Subject: [PULL 12/17] QIOChannelSocket: Implement io_writev zero copy flag & io_flush for CONFIG_LINUX Date: Mon, 16 May 2022 16:38:07 +0100 Message-Id: <20220516153812.127155-13-dgilbert@redhat.com> In-Reply-To: <20220516153812.127155-1-dgilbert@redhat.com> References: <20220516153812.127155-1-dgilbert@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.85 on 10.11.54.7 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=dgilbert@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1652716116567100001 From: Leonardo Bras For CONFIG_LINUX, implement the new zero copy flag and the optional callback io_flush on QIOChannelSocket, but enables it only when MSG_ZEROCOPY feature is available in the host kernel, which is checked on qio_channel_socket_connect_sync() qio_channel_socket_flush() was implemented by counting how many times sendmsg(...,MSG_ZEROCOPY) was successfully called, and then reading the socket's error queue, in order to find how many of them finished sending. Flush will loop until those counters are the same, or until some error occu= rs. Notes on using writev() with QIO_CHANNEL_WRITE_FLAG_ZERO_COPY: 1: Buffer - As MSG_ZEROCOPY tells the kernel to use the same user buffer to avoid cop= ying, some caution is necessary to avoid overwriting any buffer before it's sent. If something like this happen, a newer version of the buffer may be sent in= stead. - If this is a problem, it's recommended to call qio_channel_flush() before= freeing or re-using the buffer. 2: Locked memory - When using MSG_ZERCOCOPY, the buffer memory will be locked after queued, = and unlocked after it's sent. - Depending on the size of each buffer, and how often it's sent, it may req= uire a larger amount of locked memory than usually available to non-root user. - If the required amount of locked memory is not available, writev_zero_copy will return an error, which can abort an operation like migration, - Because of this, when an user code wants to add zero copy as a feature, it requires a mechanism to disable it, so it can still be accessible to less privileged users. Signed-off-by: Leonardo Bras Reviewed-by: Peter Xu Reviewed-by: Daniel P. Berrang=C3=A9 Reviewed-by: Juan Quintela Message-Id: <20220513062836.965425-4-leobras@redhat.com> Signed-off-by: Dr. David Alan Gilbert --- include/io/channel-socket.h | 2 + io/channel-socket.c | 116 ++++++++++++++++++++++++++++++++++-- 2 files changed, 114 insertions(+), 4 deletions(-) diff --git a/include/io/channel-socket.h b/include/io/channel-socket.h index e747e63514..513c428fe4 100644 --- a/include/io/channel-socket.h +++ b/include/io/channel-socket.h @@ -47,6 +47,8 @@ struct QIOChannelSocket { socklen_t localAddrLen; struct sockaddr_storage remoteAddr; socklen_t remoteAddrLen; + ssize_t zero_copy_queued; + ssize_t zero_copy_sent; }; =20 =20 diff --git a/io/channel-socket.c b/io/channel-socket.c index 05c425abb8..dc9c165de1 100644 --- a/io/channel-socket.c +++ b/io/channel-socket.c @@ -25,6 +25,14 @@ #include "io/channel-watch.h" #include "trace.h" #include "qapi/clone-visitor.h" +#ifdef CONFIG_LINUX +#include +#include + +#if (defined(MSG_ZEROCOPY) && defined(SO_ZEROCOPY)) +#define QEMU_MSG_ZEROCOPY +#endif +#endif =20 #define SOCKET_MAX_FDS 16 =20 @@ -54,6 +62,8 @@ qio_channel_socket_new(void) =20 sioc =3D QIO_CHANNEL_SOCKET(object_new(TYPE_QIO_CHANNEL_SOCKET)); sioc->fd =3D -1; + sioc->zero_copy_queued =3D 0; + sioc->zero_copy_sent =3D 0; =20 ioc =3D QIO_CHANNEL(sioc); qio_channel_set_feature(ioc, QIO_CHANNEL_FEATURE_SHUTDOWN); @@ -153,6 +163,16 @@ int qio_channel_socket_connect_sync(QIOChannelSocket *= ioc, return -1; } =20 +#ifdef QEMU_MSG_ZEROCOPY + int ret, v =3D 1; + ret =3D setsockopt(fd, SOL_SOCKET, SO_ZEROCOPY, &v, sizeof(v)); + if (ret =3D=3D 0) { + /* Zero copy available on host */ + qio_channel_set_feature(QIO_CHANNEL(ioc), + QIO_CHANNEL_FEATURE_WRITE_ZERO_COPY); + } +#endif + return 0; } =20 @@ -533,6 +553,7 @@ static ssize_t qio_channel_socket_writev(QIOChannel *io= c, char control[CMSG_SPACE(sizeof(int) * SOCKET_MAX_FDS)]; size_t fdsize =3D sizeof(int) * nfds; struct cmsghdr *cmsg; + int sflags =3D 0; =20 memset(control, 0, CMSG_SPACE(sizeof(int) * SOCKET_MAX_FDS)); =20 @@ -557,15 +578,31 @@ static ssize_t qio_channel_socket_writev(QIOChannel *= ioc, memcpy(CMSG_DATA(cmsg), fds, fdsize); } =20 +#ifdef QEMU_MSG_ZEROCOPY + if (flags & QIO_CHANNEL_WRITE_FLAG_ZERO_COPY) { + sflags =3D MSG_ZEROCOPY; + } +#endif + retry: - ret =3D sendmsg(sioc->fd, &msg, 0); + ret =3D sendmsg(sioc->fd, &msg, sflags); if (ret <=3D 0) { - if (errno =3D=3D EAGAIN) { + switch (errno) { + case EAGAIN: return QIO_CHANNEL_ERR_BLOCK; - } - if (errno =3D=3D EINTR) { + case EINTR: goto retry; +#ifdef QEMU_MSG_ZEROCOPY + case ENOBUFS: + if (sflags & MSG_ZEROCOPY) { + error_setg_errno(errp, errno, + "Process can't lock enough memory for usi= ng MSG_ZEROCOPY"); + return -1; + } + break; +#endif } + error_setg_errno(errp, errno, "Unable to write to socket"); return -1; @@ -659,6 +696,74 @@ static ssize_t qio_channel_socket_writev(QIOChannel *i= oc, } #endif /* WIN32 */ =20 + +#ifdef QEMU_MSG_ZEROCOPY +static int qio_channel_socket_flush(QIOChannel *ioc, + Error **errp) +{ + QIOChannelSocket *sioc =3D QIO_CHANNEL_SOCKET(ioc); + struct msghdr msg =3D {}; + struct sock_extended_err *serr; + struct cmsghdr *cm; + char control[CMSG_SPACE(sizeof(*serr))]; + int received; + int ret =3D 1; + + msg.msg_control =3D control; + msg.msg_controllen =3D sizeof(control); + memset(control, 0, sizeof(control)); + + while (sioc->zero_copy_sent < sioc->zero_copy_queued) { + received =3D recvmsg(sioc->fd, &msg, MSG_ERRQUEUE); + if (received < 0) { + switch (errno) { + case EAGAIN: + /* Nothing on errqueue, wait until something is available = */ + qio_channel_wait(ioc, G_IO_ERR); + continue; + case EINTR: + continue; + default: + error_setg_errno(errp, errno, + "Unable to read errqueue"); + return -1; + } + } + + cm =3D CMSG_FIRSTHDR(&msg); + if (cm->cmsg_level !=3D SOL_IP && + cm->cmsg_type !=3D IP_RECVERR) { + error_setg_errno(errp, EPROTOTYPE, + "Wrong cmsg in errqueue"); + return -1; + } + + serr =3D (void *) CMSG_DATA(cm); + if (serr->ee_errno !=3D SO_EE_ORIGIN_NONE) { + error_setg_errno(errp, serr->ee_errno, + "Error on socket"); + return -1; + } + if (serr->ee_origin !=3D SO_EE_ORIGIN_ZEROCOPY) { + error_setg_errno(errp, serr->ee_origin, + "Error not from zero copy"); + return -1; + } + + /* No errors, count successfully finished sendmsg()*/ + sioc->zero_copy_sent +=3D serr->ee_data - serr->ee_info + 1; + + /* If any sendmsg() succeeded using zero copy, return 0 at the end= */ + if (serr->ee_code !=3D SO_EE_CODE_ZEROCOPY_COPIED) { + ret =3D 0; + } + } + + return ret; +} + +#endif /* QEMU_MSG_ZEROCOPY */ + static int qio_channel_socket_set_blocking(QIOChannel *ioc, bool enabled, @@ -789,6 +894,9 @@ static void qio_channel_socket_class_init(ObjectClass *= klass, ioc_klass->io_set_delay =3D qio_channel_socket_set_delay; ioc_klass->io_create_watch =3D qio_channel_socket_create_watch; ioc_klass->io_set_aio_fd_handler =3D qio_channel_socket_set_aio_fd_han= dler; +#ifdef QEMU_MSG_ZEROCOPY + ioc_klass->io_flush =3D qio_channel_socket_flush; +#endif } =20 static const TypeInfo qio_channel_socket_info =3D { --=20 2.36.1 From nobody Mon Apr 29 04:30:41 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1652720826; cv=none; d=zohomail.com; s=zohoarc; b=R95NMGYxcgHYwv5J/0cRze+YYMa6l5pXP8l5GZhRBe6h8rNKNoDk86lBWasQQnBRxy7f1fk52dskgwCPBxvZ11NTpbTGTepv91c5xi6CI7yjG+jJPTI9gwyOC6QiZKBs7hFZIwMyG2Nt5AK9qTj85VKK2+78EcdTiBN14vlGLLM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1652720826; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=fZUwmxBvnHlE/ys/hErjimpJ49gMUieat4miajW+lYo=; b=g+5qUqEeZPBru9p/zqWiLqOsKRkkORT/MoudhQWaUSxR7oOQdlp3h/xY5defmx0G11TE/Xn5yxyNeIY3Hn0jwnFqEbH3zT8xgqNf/lPSPxx1MXMNe+R4FOsdECEFZsFjPdhO2d16CWXuREsc4IyHtO9eQoUadFK937GJiIYpUKo= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1652720826521687.4568322603191; Mon, 16 May 2022 10:07:06 -0700 (PDT) Received: from localhost ([::1]:43554 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nqeBV-0000PK-3e for importer@patchew.org; Mon, 16 May 2022 13:07:05 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35740) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqcnt-0002X7-MX for qemu-devel@nongnu.org; Mon, 16 May 2022 11:38:37 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:29208) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqcnr-0008Fn-OV for qemu-devel@nongnu.org; Mon, 16 May 2022 11:38:37 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-660-fl_L5mbWP5yhFlfoUsZ2sw-1; Mon, 16 May 2022 11:38:33 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 8ABC2811E75 for ; Mon, 16 May 2022 15:38:33 +0000 (UTC) Received: from dgilbert-t580.localhost (unknown [10.39.194.215]) by smtp.corp.redhat.com (Postfix) with ESMTP id A08B715215E9; Mon, 16 May 2022 15:38:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1652715515; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fZUwmxBvnHlE/ys/hErjimpJ49gMUieat4miajW+lYo=; b=D2bp8RtfyPb3tdFR+pq+tJhyO/e4ta30ZmtVAuENNux/mIIERL0l2cUWtRdGUhDyARb1zW lPrAItp26p3qx1LoGC9Qn4k5fg9BGEP+x2kJDS1OscVs/kvAOf2g5+N24tLR0eYBzHWyCg P5j2b3c5GORA3TobOtwEjlS/kMHWZdg= X-MC-Unique: fl_L5mbWP5yhFlfoUsZ2sw-1 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, quintela@redhat.com, peterx@redhat.com, leobras@redhat.com, berrange@redhat.com Subject: [PULL 13/17] migration: Add zero-copy-send parameter for QMP/HMP for Linux Date: Mon, 16 May 2022 16:38:08 +0100 Message-Id: <20220516153812.127155-14-dgilbert@redhat.com> In-Reply-To: <20220516153812.127155-1-dgilbert@redhat.com> References: <20220516153812.127155-1-dgilbert@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.85 on 10.11.54.7 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=dgilbert@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1652720827512100001 From: Leonardo Bras Add property that allows zero-copy migration of memory pages on the sending side, and also includes a helper function migrate_use_zero_copy_send() to check if it's enabled. No code is introduced to actually do the migration, but it allow future implementations to enable/disable this feature. On non-Linux builds this parameter is compiled-out. Signed-off-by: Leonardo Bras Reviewed-by: Peter Xu Reviewed-by: Daniel P. Berrang=C3=A9 Reviewed-by: Juan Quintela Acked-by: Markus Armbruster Message-Id: <20220513062836.965425-5-leobras@redhat.com> Signed-off-by: Dr. David Alan Gilbert --- migration/migration.c | 32 ++++++++++++++++++++++++++++++++ migration/migration.h | 5 +++++ migration/socket.c | 11 +++++++++-- monitor/hmp-cmds.c | 6 ++++++ qapi/migration.json | 24 ++++++++++++++++++++++++ 5 files changed, 76 insertions(+), 2 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 5a31b23bd6..3e91f4b5e2 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -910,6 +910,10 @@ MigrationParameters *qmp_query_migrate_parameters(Erro= r **errp) params->multifd_zlib_level =3D s->parameters.multifd_zlib_level; params->has_multifd_zstd_level =3D true; params->multifd_zstd_level =3D s->parameters.multifd_zstd_level; +#ifdef CONFIG_LINUX + params->has_zero_copy_send =3D true; + params->zero_copy_send =3D s->parameters.zero_copy_send; +#endif params->has_xbzrle_cache_size =3D true; params->xbzrle_cache_size =3D s->parameters.xbzrle_cache_size; params->has_max_postcopy_bandwidth =3D true; @@ -1567,6 +1571,11 @@ static void migrate_params_test_apply(MigrateSetPara= meters *params, if (params->has_multifd_compression) { dest->multifd_compression =3D params->multifd_compression; } +#ifdef CONFIG_LINUX + if (params->has_zero_copy_send) { + dest->zero_copy_send =3D params->zero_copy_send; + } +#endif if (params->has_xbzrle_cache_size) { dest->xbzrle_cache_size =3D params->xbzrle_cache_size; } @@ -1679,6 +1688,11 @@ static void migrate_params_apply(MigrateSetParameter= s *params, Error **errp) if (params->has_multifd_compression) { s->parameters.multifd_compression =3D params->multifd_compression; } +#ifdef CONFIG_LINUX + if (params->has_zero_copy_send) { + s->parameters.zero_copy_send =3D params->zero_copy_send; + } +#endif if (params->has_xbzrle_cache_size) { s->parameters.xbzrle_cache_size =3D params->xbzrle_cache_size; xbzrle_cache_resize(params->xbzrle_cache_size, errp); @@ -2563,6 +2577,17 @@ int migrate_multifd_zstd_level(void) return s->parameters.multifd_zstd_level; } =20 +#ifdef CONFIG_LINUX +bool migrate_use_zero_copy_send(void) +{ + MigrationState *s; + + s =3D migrate_get_current(); + + return s->parameters.zero_copy_send; +} +#endif + int migrate_use_xbzrle(void) { MigrationState *s; @@ -4206,6 +4231,10 @@ static Property migration_properties[] =3D { DEFINE_PROP_UINT8("multifd-zstd-level", MigrationState, parameters.multifd_zstd_level, DEFAULT_MIGRATE_MULTIFD_ZSTD_LEVEL), +#ifdef CONFIG_LINUX + DEFINE_PROP_BOOL("zero_copy_send", MigrationState, + parameters.zero_copy_send, false), +#endif DEFINE_PROP_SIZE("xbzrle-cache-size", MigrationState, parameters.xbzrle_cache_size, DEFAULT_MIGRATE_XBZRLE_CACHE_SIZE), @@ -4303,6 +4332,9 @@ static void migration_instance_init(Object *obj) params->has_multifd_compression =3D true; params->has_multifd_zlib_level =3D true; params->has_multifd_zstd_level =3D true; +#ifdef CONFIG_LINUX + params->has_zero_copy_send =3D true; +#endif params->has_xbzrle_cache_size =3D true; params->has_max_postcopy_bandwidth =3D true; params->has_max_cpu_throttle =3D true; diff --git a/migration/migration.h b/migration/migration.h index a863032b71..e8f2941a55 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -375,6 +375,11 @@ MultiFDCompression migrate_multifd_compression(void); int migrate_multifd_zlib_level(void); int migrate_multifd_zstd_level(void); =20 +#ifdef CONFIG_LINUX +bool migrate_use_zero_copy_send(void); +#else +#define migrate_use_zero_copy_send() (false) +#endif int migrate_use_xbzrle(void); uint64_t migrate_xbzrle_cache_size(void); bool migrate_colo_enabled(void); diff --git a/migration/socket.c b/migration/socket.c index 05705a32d8..3754d8f72c 100644 --- a/migration/socket.c +++ b/migration/socket.c @@ -74,9 +74,16 @@ static void socket_outgoing_migration(QIOTask *task, =20 if (qio_task_propagate_error(task, &err)) { trace_migration_socket_outgoing_error(error_get_pretty(err)); - } else { - trace_migration_socket_outgoing_connected(data->hostname); + goto out; } + + trace_migration_socket_outgoing_connected(data->hostname); + + if (migrate_use_zero_copy_send()) { + error_setg(&err, "Zero copy send not available in migration"); + } + +out: migration_channel_connect(data->s, sioc, data->hostname, err); object_unref(OBJECT(sioc)); } diff --git a/monitor/hmp-cmds.c b/monitor/hmp-cmds.c index 93061a11af..622c783c32 100644 --- a/monitor/hmp-cmds.c +++ b/monitor/hmp-cmds.c @@ -1309,6 +1309,12 @@ void hmp_migrate_set_parameter(Monitor *mon, const Q= Dict *qdict) p->has_multifd_zstd_level =3D true; visit_type_uint8(v, param, &p->multifd_zstd_level, &err); break; +#ifdef CONFIG_LINUX + case MIGRATION_PARAMETER_ZERO_COPY_SEND: + p->has_zero_copy_send =3D true; + visit_type_bool(v, param, &p->zero_copy_send, &err); + break; +#endif case MIGRATION_PARAMETER_XBZRLE_CACHE_SIZE: p->has_xbzrle_cache_size =3D true; if (!visit_type_size(v, param, &cache_size, &err)) { diff --git a/qapi/migration.json b/qapi/migration.json index 409eb086a2..2222f44250 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -741,6 +741,13 @@ # will consume more CPU. # Defaults to 1. (Since 5.0) # +# @zero-copy-send: Controls behavior on sending memory pages on migration. +# When true, enables a zero-copy mechanism for sending +# memory pages, if host supports it. +# Requires that QEMU be permitted to use locked memory +# for guest RAM pages. +# Defaults to false. (Since 7.1) +# # @block-bitmap-mapping: Maps block nodes and bitmaps on them to # aliases for the purpose of dirty bitmap migration= . Such # aliases may for example be the corresponding name= s on the @@ -780,6 +787,7 @@ 'xbzrle-cache-size', 'max-postcopy-bandwidth', 'max-cpu-throttle', 'multifd-compression', 'multifd-zlib-level' ,'multifd-zstd-level', + { 'name': 'zero-copy-send', 'if' : 'CONFIG_LINUX'}, 'block-bitmap-mapping' ] } =20 ## @@ -906,6 +914,13 @@ # will consume more CPU. # Defaults to 1. (Since 5.0) # +# @zero-copy-send: Controls behavior on sending memory pages on migration. +# When true, enables a zero-copy mechanism for sending +# memory pages, if host supports it. +# Requires that QEMU be permitted to use locked memory +# for guest RAM pages. +# Defaults to false. (Since 7.1) +# # @block-bitmap-mapping: Maps block nodes and bitmaps on them to # aliases for the purpose of dirty bitmap migration= . Such # aliases may for example be the corresponding name= s on the @@ -960,6 +975,7 @@ '*multifd-compression': 'MultiFDCompression', '*multifd-zlib-level': 'uint8', '*multifd-zstd-level': 'uint8', + '*zero-copy-send': { 'type': 'bool', 'if': 'CONFIG_LINUX' }, '*block-bitmap-mapping': [ 'BitmapMigrationNodeAlias' ] } } =20 ## @@ -1106,6 +1122,13 @@ # will consume more CPU. # Defaults to 1. (Since 5.0) # +# @zero-copy-send: Controls behavior on sending memory pages on migration. +# When true, enables a zero-copy mechanism for sending +# memory pages, if host supports it. +# Requires that QEMU be permitted to use locked memory +# for guest RAM pages. +# Defaults to false. (Since 7.1) +# # @block-bitmap-mapping: Maps block nodes and bitmaps on them to # aliases for the purpose of dirty bitmap migration= . Such # aliases may for example be the corresponding name= s on the @@ -1158,6 +1181,7 @@ '*multifd-compression': 'MultiFDCompression', '*multifd-zlib-level': 'uint8', '*multifd-zstd-level': 'uint8', + '*zero-copy-send': { 'type': 'bool', 'if': 'CONFIG_LINUX' }, '*block-bitmap-mapping': [ 'BitmapMigrationNodeAlias' ] } } =20 ## --=20 2.36.1 From nobody Mon Apr 29 04:30:41 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1652721092; cv=none; d=zohomail.com; s=zohoarc; b=ALjqY4oDoy7TKipq2SuNMsam9XPg+Dhm+5kE9kDsclm5E7FEGmQCMnZuMlfjV0xxcNU6huEFhnYpljz2q1LTu5xgYK7Jj5dI82d3MMRbbZBUrUtQXJbc5SlHmCczzVkH5KPRGIoZXHQ1eMQ01lVktBr1UDK8NtQfho1kqwcH31U= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1652721092; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=BSXYhmLsxmjQvh+hxf/hj8hhkAuQ7N548iXyDzZnUWw=; b=eg896zYb3AEyPRZN+0RHIzJHpAL3/iFjtr8//81QolN/nkNUxAny1ISewKvXdyxiMFpjPkEbMiYny6Pw1ux6ZBz9qcrDSewjuSTjpgmThfaTegOZtZKvXo5a5e48Sj1EHZHAHTQiOmzGvjTNZzhfbJlEYIuM95rSJDSbRzLkq5c= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1652721092098267.74166469614045; Mon, 16 May 2022 10:11:32 -0700 (PDT) Received: from localhost ([::1]:52372 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nqeFn-0006en-2h for importer@patchew.org; Mon, 16 May 2022 13:11:31 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35762) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqcnu-0002Zh-RC for qemu-devel@nongnu.org; Mon, 16 May 2022 11:38:38 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:33797) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqcnt-0008G2-3X for qemu-devel@nongnu.org; Mon, 16 May 2022 11:38:38 -0400 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-67-qDxonCVYMNu7BVh8QYECvQ-1; Mon, 16 May 2022 11:38:35 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id DF1C43C0D18A for ; Mon, 16 May 2022 15:38:34 +0000 (UTC) Received: from dgilbert-t580.localhost (unknown [10.39.194.215]) by smtp.corp.redhat.com (Postfix) with ESMTP id C4D1415215E9; Mon, 16 May 2022 15:38:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1652715516; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=BSXYhmLsxmjQvh+hxf/hj8hhkAuQ7N548iXyDzZnUWw=; b=gsFwSciiI0ra15q3Vk1uHXInDVg/QB2gWBj7hfjopFnvswBeISLPTtET5bwNN51Yb5m+0H bKRw55iDyZUBhawOuARlLA0ivnCXMC2k56wU6HRnpq3KRELqxTh1Fl+EFaqHOUlXKj24C3 MX11lbsTFQfmxs8xy+hg1w5cegM9lg8= X-MC-Unique: qDxonCVYMNu7BVh8QYECvQ-1 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, quintela@redhat.com, peterx@redhat.com, leobras@redhat.com, berrange@redhat.com Subject: [PULL 14/17] migration: Add migrate_use_tls() helper Date: Mon, 16 May 2022 16:38:09 +0100 Message-Id: <20220516153812.127155-15-dgilbert@redhat.com> In-Reply-To: <20220516153812.127155-1-dgilbert@redhat.com> References: <20220516153812.127155-1-dgilbert@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.85 on 10.11.54.7 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=dgilbert@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1652721093271100001 From: Leonardo Bras A lot of places check parameters.tls_creds in order to evaluate if TLS is in use, and sometimes call migrate_get_current() just for that test. Add new helper function migrate_use_tls() in order to simplify testing for TLS usage. Signed-off-by: Leonardo Bras Reviewed-by: Juan Quintela Reviewed-by: Peter Xu Reviewed-by: Daniel P. Berrang=C3=A9 Message-Id: <20220513062836.965425-6-leobras@redhat.com> Signed-off-by: Dr. David Alan Gilbert --- migration/channel.c | 3 +-- migration/migration.c | 9 +++++++++ migration/migration.h | 1 + migration/multifd.c | 5 +---- 4 files changed, 12 insertions(+), 6 deletions(-) diff --git a/migration/channel.c b/migration/channel.c index c6a8dcf1d7..a162d00fea 100644 --- a/migration/channel.c +++ b/migration/channel.c @@ -38,8 +38,7 @@ void migration_channel_process_incoming(QIOChannel *ioc) trace_migration_set_incoming_channel( ioc, object_get_typename(OBJECT(ioc))); =20 - if (s->parameters.tls_creds && - *s->parameters.tls_creds && + if (migrate_use_tls() && !object_dynamic_cast(OBJECT(ioc), TYPE_QIO_CHANNEL_TLS)) { migration_tls_channel_process_incoming(s, ioc, &local_err); diff --git a/migration/migration.c b/migration/migration.c index 3e91f4b5e2..4b6df2eb5e 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -2588,6 +2588,15 @@ bool migrate_use_zero_copy_send(void) } #endif =20 +int migrate_use_tls(void) +{ + MigrationState *s; + + s =3D migrate_get_current(); + + return s->parameters.tls_creds && *s->parameters.tls_creds; +} + int migrate_use_xbzrle(void) { MigrationState *s; diff --git a/migration/migration.h b/migration/migration.h index e8f2941a55..485d58b95f 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -380,6 +380,7 @@ bool migrate_use_zero_copy_send(void); #else #define migrate_use_zero_copy_send() (false) #endif +int migrate_use_tls(void); int migrate_use_xbzrle(void); uint64_t migrate_xbzrle_cache_size(void); bool migrate_colo_enabled(void); diff --git a/migration/multifd.c b/migration/multifd.c index 9ea4f581e2..2a8c8570c3 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -782,15 +782,12 @@ static bool multifd_channel_connect(MultiFDSendParams= *p, QIOChannel *ioc, Error *error) { - MigrationState *s =3D migrate_get_current(); - trace_multifd_set_outgoing_channel( ioc, object_get_typename(OBJECT(ioc)), migrate_get_current()->hostname, error); =20 if (!error) { - if (s->parameters.tls_creds && - *s->parameters.tls_creds && + if (migrate_use_tls() && !object_dynamic_cast(OBJECT(ioc), TYPE_QIO_CHANNEL_TLS)) { multifd_tls_channel_connect(p, ioc, &error); --=20 2.36.1 From nobody Mon Apr 29 04:30:41 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1652718495; cv=none; d=zohomail.com; s=zohoarc; b=TvYsIBH/Fd0+xL6yx1cJBMJILeRNgwGDtcIsjxjVqo1BGzOZkFWIgYvzrZVTTXXcbAtosn+IIfqv09bXjTh37WETT6o8EBaagV5S+u9Zj48l+HnokSfa06DWy9M1QNxrzk9XIlysRkWtcJid/SIbmk/8y5DZD2k+pw8VklbUY5c= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1652718495; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=15j9pU0tJ13K7sS8jEpVQWgMXdlXyxE6n+Y7TJESQys=; b=MSW4BXSyX41Z8e5+7vF9g0hqss4n6vdjCtY8USQrQMBxrSNYlYurPA5yOdbGy8Jxijgtn1MDgIvh4T+vtK16eLpcCXTK6DVP6nsqQ7xa9kXoRAaywX4OZXMS/pGWVW7egpQUrLGGHR6VnUJwAkwSGcT3Tw5gsdiQ6wqple/72M4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1652718495277801.033013852092; Mon, 16 May 2022 09:28:15 -0700 (PDT) Received: from localhost ([::1]:42820 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nqdZu-0000kB-6H for importer@patchew.org; Mon, 16 May 2022 12:28:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35784) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqcnv-0002bD-TN for qemu-devel@nongnu.org; Mon, 16 May 2022 11:38:40 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:34376) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqcnu-0008GI-7o for qemu-devel@nongnu.org; Mon, 16 May 2022 11:38:39 -0400 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-283-Zxhwwub0P3qa1qN_ZCoiNQ-1; Mon, 16 May 2022 11:38:36 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 11F443C0D18B for ; Mon, 16 May 2022 15:38:36 +0000 (UTC) Received: from dgilbert-t580.localhost (unknown [10.39.194.215]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2763C15215EA; Mon, 16 May 2022 15:38:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1652715517; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=15j9pU0tJ13K7sS8jEpVQWgMXdlXyxE6n+Y7TJESQys=; b=T9WDGEtOsq6X8prhMDkeJCeg0m40dMpdn1kRAao5VhfUcf8ghAp2O+CYAGrRPsIQVICmng FVlHb1J3FBHIgUWfxzku1K7OUnidSnLEMk26nC6zw2r9f3Hp12G6a46BfI0tDwYuhNVVgY 9egBF43ZbD4n6rpeiFRaiJyyUKdwUtg= X-MC-Unique: Zxhwwub0P3qa1qN_ZCoiNQ-1 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, quintela@redhat.com, peterx@redhat.com, leobras@redhat.com, berrange@redhat.com Subject: [PULL 15/17] multifd: multifd_send_sync_main now returns negative on error Date: Mon, 16 May 2022 16:38:10 +0100 Message-Id: <20220516153812.127155-16-dgilbert@redhat.com> In-Reply-To: <20220516153812.127155-1-dgilbert@redhat.com> References: <20220516153812.127155-1-dgilbert@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.85 on 10.11.54.7 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=dgilbert@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1652718497548100001 From: Leonardo Bras Even though multifd_send_sync_main() currently emits error_reports, it's callers don't really check it before continuing. Change multifd_send_sync_main() to return -1 on error and 0 on success. Also change all it's callers to make use of this change and possibly fail earlier. (This change is important to next patch on multifd zero copy implementation, to make it sure an error in zero-copy flush does not go unnoticed. Signed-off-by: Leonardo Bras Reviewed-by: Daniel P. Berrang=C3=A9 Reviewed-by: Peter Xu Message-Id: <20220513062836.965425-7-leobras@redhat.com> Signed-off-by: Dr. David Alan Gilbert --- migration/multifd.c | 10 ++++++---- migration/multifd.h | 2 +- migration/ram.c | 29 ++++++++++++++++++++++------- 3 files changed, 29 insertions(+), 12 deletions(-) diff --git a/migration/multifd.c b/migration/multifd.c index 2a8c8570c3..15fb668e64 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -566,17 +566,17 @@ void multifd_save_cleanup(void) multifd_send_state =3D NULL; } =20 -void multifd_send_sync_main(QEMUFile *f) +int multifd_send_sync_main(QEMUFile *f) { int i; =20 if (!migrate_use_multifd()) { - return; + return 0; } if (multifd_send_state->pages->num) { if (multifd_send_pages(f) < 0) { error_report("%s: multifd_send_pages fail", __func__); - return; + return -1; } } for (i =3D 0; i < migrate_multifd_channels(); i++) { @@ -589,7 +589,7 @@ void multifd_send_sync_main(QEMUFile *f) if (p->quit) { error_report("%s: channel %d has already quit", __func__, i); qemu_mutex_unlock(&p->mutex); - return; + return -1; } =20 p->packet_num =3D multifd_send_state->packet_num++; @@ -608,6 +608,8 @@ void multifd_send_sync_main(QEMUFile *f) qemu_sem_wait(&p->sem_sync); } trace_multifd_send_sync_main(multifd_send_state->packet_num); + + return 0; } =20 static void *multifd_send_thread(void *opaque) diff --git a/migration/multifd.h b/migration/multifd.h index 7d0effcb03..bcf5992945 100644 --- a/migration/multifd.h +++ b/migration/multifd.h @@ -20,7 +20,7 @@ int multifd_load_cleanup(Error **errp); bool multifd_recv_all_channels_created(void); bool multifd_recv_new_channel(QIOChannel *ioc, Error **errp); void multifd_recv_sync_main(void); -void multifd_send_sync_main(QEMUFile *f); +int multifd_send_sync_main(QEMUFile *f); int multifd_queue_page(QEMUFile *f, RAMBlock *block, ram_addr_t offset); =20 /* Multifd Compression flags */ diff --git a/migration/ram.c b/migration/ram.c index a2489a2699..5f5e37f64d 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -2909,6 +2909,7 @@ static int ram_save_setup(QEMUFile *f, void *opaque) { RAMState **rsp =3D opaque; RAMBlock *block; + int ret; =20 if (compress_threads_save_setup()) { return -1; @@ -2943,7 +2944,11 @@ static int ram_save_setup(QEMUFile *f, void *opaque) ram_control_before_iterate(f, RAM_CONTROL_SETUP); ram_control_after_iterate(f, RAM_CONTROL_SETUP); =20 - multifd_send_sync_main(f); + ret =3D multifd_send_sync_main(f); + if (ret < 0) { + return ret; + } + qemu_put_be64(f, RAM_SAVE_FLAG_EOS); qemu_fflush(f); =20 @@ -3052,7 +3057,11 @@ static int ram_save_iterate(QEMUFile *f, void *opaqu= e) out: if (ret >=3D 0 && migration_is_setup_or_active(migrate_get_current()->state)) { - multifd_send_sync_main(rs->f); + ret =3D multifd_send_sync_main(rs->f); + if (ret < 0) { + return ret; + } + qemu_put_be64(f, RAM_SAVE_FLAG_EOS); qemu_fflush(f); ram_transferred_add(8); @@ -3112,13 +3121,19 @@ static int ram_save_complete(QEMUFile *f, void *opa= que) ram_control_after_iterate(f, RAM_CONTROL_FINISH); } =20 - if (ret >=3D 0) { - multifd_send_sync_main(rs->f); - qemu_put_be64(f, RAM_SAVE_FLAG_EOS); - qemu_fflush(f); + if (ret < 0) { + return ret; } =20 - return ret; + ret =3D multifd_send_sync_main(rs->f); + if (ret < 0) { + return ret; + } + + qemu_put_be64(f, RAM_SAVE_FLAG_EOS); + qemu_fflush(f); + + return 0; } =20 static void ram_save_pending(QEMUFile *f, void *opaque, uint64_t max_size, --=20 2.36.1 From nobody Mon Apr 29 04:30:41 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1652719235; cv=none; d=zohomail.com; s=zohoarc; b=PPA1X4tIUn8MBp0ooSXGBsr5o0qzQEz2lKJ8G7cyZcvJ0ho4UZdpa/0pkYmTyCAZaZwYY/f8PmZcolKqWDadfxVd4oV3de+zV31bLo3eTHr/TcxTZM9wOKO1lGPe1/9HjIEVceczJa5pAIp4nXeHwvtDuupRMPMfD/RDlY38Erk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1652719235; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=u8a9dX1TUn1G6l02f7/hh8fMb3yp/47BCORsBB8Jq/w=; b=U67RFxRtkU/KgFvQ87KXZdJmkly1NPty4/xvn5LTBq5+Mucc03JgFYPSGVoKk9oUwpYSxk+th5dLx1UVnf1fhpRIMyd3bXsD0zpSlZjMCyhStqHU8JMgd4mQGI43dUtznUCLFsqeJsRkPqcM0byMAlqGuW7FWUR/4BbId3QmX0s= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1652719235023506.7988300718757; Mon, 16 May 2022 09:40:35 -0700 (PDT) Received: from localhost ([::1]:48690 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nqdlq-0007ZP-0n for importer@patchew.org; Mon, 16 May 2022 12:40:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35798) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqcnw-0002cb-RP for qemu-devel@nongnu.org; Mon, 16 May 2022 11:38:40 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:26353) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqcnv-0008GY-BB for qemu-devel@nongnu.org; Mon, 16 May 2022 11:38:40 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-550--pCIqjItN6mIlXwFHmwHsA-1; Mon, 16 May 2022 11:38:37 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 3502B18E0048 for ; Mon, 16 May 2022 15:38:37 +0000 (UTC) Received: from dgilbert-t580.localhost (unknown [10.39.194.215]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4D87415215E9; Mon, 16 May 2022 15:38:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1652715518; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=u8a9dX1TUn1G6l02f7/hh8fMb3yp/47BCORsBB8Jq/w=; b=ePaDYzAy1gfOtAAJAPeKA68OsEejU3pV5MmV5PbmaUIE+jXk2HohJjrMJB/lrOqBhF0PYz Uv071Ido1/WlOEGCNsIenhc6abuN46P1wmSF/7nd3cpwrzXiDjJTwmR4E3KwWM1ugAYw4O 2rl3bF7c0IQYD4vPDeYmzhga1vBKYcA= X-MC-Unique: -pCIqjItN6mIlXwFHmwHsA-1 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, quintela@redhat.com, peterx@redhat.com, leobras@redhat.com, berrange@redhat.com Subject: [PULL 16/17] multifd: Send header packet without flags if zero-copy-send is enabled Date: Mon, 16 May 2022 16:38:11 +0100 Message-Id: <20220516153812.127155-17-dgilbert@redhat.com> In-Reply-To: <20220516153812.127155-1-dgilbert@redhat.com> References: <20220516153812.127155-1-dgilbert@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.85 on 10.11.54.7 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=dgilbert@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1652719235738100001 From: Leonardo Bras Since d48c3a0445 ("multifd: Use a single writev on the send side"), sending the header packet and the memory pages happens in the same writev, which can potentially make the migration faster. Using channel-socket as example, this works well with the default copying mechanism of sendmsg(), but with zero-copy-send=3Dtrue, it will cause the migration to often break. This happens because the header packet buffer gets reused quite often, and there is a high chance that by the time the MSG_ZEROCOPY mechanism get to send the buffer, it has already changed, sending the wrong data and causing the migration to abort. It means that, as it is, the buffer for the header packet is not suitable for sending with MSG_ZEROCOPY. In order to enable zero copy for multifd, send the header packet on an individual write(), without any flags, and the remanining pages with a writev(), as it was happening before. This only changes how a migration with zero-copy-send=3Dtrue works, not changing any current behavior for migrations with zero-copy-send=3Dfalse. Signed-off-by: Leonardo Bras Reviewed-by: Peter Xu Reviewed-by: Daniel P. Berrang=C3=A9 Message-Id: <20220513062836.965425-8-leobras@redhat.com> Signed-off-by: Dr. David Alan Gilbert --- migration/multifd.c | 22 +++++++++++++++++++--- 1 file changed, 19 insertions(+), 3 deletions(-) diff --git a/migration/multifd.c b/migration/multifd.c index 15fb668e64..2541cd2322 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -617,6 +617,7 @@ static void *multifd_send_thread(void *opaque) MultiFDSendParams *p =3D opaque; Error *local_err =3D NULL; int ret =3D 0; + bool use_zero_copy_send =3D migrate_use_zero_copy_send(); =20 trace_multifd_send_thread_start(p->id); rcu_register_thread(); @@ -639,9 +640,14 @@ static void *multifd_send_thread(void *opaque) if (p->pending_job) { uint64_t packet_num =3D p->packet_num; uint32_t flags =3D p->flags; - p->iovs_num =3D 1; p->normal_num =3D 0; =20 + if (use_zero_copy_send) { + p->iovs_num =3D 0; + } else { + p->iovs_num =3D 1; + } + for (int i =3D 0; i < p->pages->num; i++) { p->normal[p->normal_num] =3D p->pages->offset[i]; p->normal_num++; @@ -665,8 +671,18 @@ static void *multifd_send_thread(void *opaque) trace_multifd_send(p->id, packet_num, p->normal_num, flags, p->next_packet_size); =20 - p->iov[0].iov_len =3D p->packet_len; - p->iov[0].iov_base =3D p->packet; + if (use_zero_copy_send) { + /* Send header first, without zerocopy */ + ret =3D qio_channel_write_all(p->c, (void *)p->packet, + p->packet_len, &local_err); + if (ret !=3D 0) { + break; + } + } else { + /* Send header using the same writev call */ + p->iov[0].iov_len =3D p->packet_len; + p->iov[0].iov_base =3D p->packet; + } =20 ret =3D qio_channel_writev_all(p->c, p->iov, p->iovs_num, &local_err); --=20 2.36.1 From nobody Mon Apr 29 04:30:41 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1652719537; cv=none; d=zohomail.com; s=zohoarc; b=PSkJwQf2/zHomdpUhdwjG73axML1QJLmjtXa6+mDnl/6Qa/696C7MJwio+9yscZ9hK97DPGYIjdfQ/AI5ZSo6EMvrn8TTSDxn1LjPbsrhjGHjTfiboc9fD7v4SqB8h7ycseCsGFAJC/u9+cYQn5iEfgyzWokknhH76J5R9xXVLQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1652719537; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=tIPMnL8MH2d7LdaVzn9UDMy9r0k0h38rx7Qbe8Laa4U=; b=UjoEyyL+ItLMOBthmpgAQMDMOx5PcpFBkY4bDqqeFPiRKFrX9GUtBjH6OS2C5BMlMHrIQFFw+d5lIPOoE4wqThZ4L12DEL1qhlpeSTmpWvx4TlwflR1/XgH5yN+ZMlDimjCp2wn3+mQmWf7tfWy2zsuBq30PPQzW/M7Ggz7fQp4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1652719537585128.47258648264005; Mon, 16 May 2022 09:45:37 -0700 (PDT) Received: from localhost ([::1]:56662 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nqdqi-0004nZ-B2 for importer@patchew.org; Mon, 16 May 2022 12:45:36 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35808) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqcny-0002iN-Fk for qemu-devel@nongnu.org; Mon, 16 May 2022 11:38:42 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:28487) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqcnw-0008Gj-Lz for qemu-devel@nongnu.org; Mon, 16 May 2022 11:38:42 -0400 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-115-3cQb95J8Nv-YbrxwsfBWig-1; Mon, 16 May 2022 11:38:38 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 5DF6929ABA13 for ; Mon, 16 May 2022 15:38:38 +0000 (UTC) Received: from dgilbert-t580.localhost (unknown [10.39.194.215]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7169D15215E9; Mon, 16 May 2022 15:38:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1652715520; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tIPMnL8MH2d7LdaVzn9UDMy9r0k0h38rx7Qbe8Laa4U=; b=cNGSnIHG2kOrZZh6d4RhPoaYtQdML4wgACwpOqLy71MTB9b0vhrpWwC5U+qYrB/w/fvYCw zGiXYxpgQsQuhrFIXCgCVC/bIZHt9a3I4W/ErN/sng1NAUBatDIQoy/D5R02Jt8KUAbKLH 4PxxfG1XU+xWdCzc/tcnuBCODXvzg4g= X-MC-Unique: 3cQb95J8Nv-YbrxwsfBWig-1 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, quintela@redhat.com, peterx@redhat.com, leobras@redhat.com, berrange@redhat.com Subject: [PULL 17/17] multifd: Implement zero copy write in multifd migration (multifd-zero-copy) Date: Mon, 16 May 2022 16:38:12 +0100 Message-Id: <20220516153812.127155-18-dgilbert@redhat.com> In-Reply-To: <20220516153812.127155-1-dgilbert@redhat.com> References: <20220516153812.127155-1-dgilbert@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.85 on 10.11.54.7 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=dgilbert@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1652719538738100001 From: Leonardo Bras Implement zero copy send on nocomp_send_write(), by making use of QIOChannel writev + flags & flush interface. Change multifd_send_sync_main() so flush_zero_copy() can be called after each iteration in order to make sure all dirty pages are sent before a new iteration is started. It will also flush at the beginning and at the end of migration. Also make it return -1 if flush_zero_copy() fails, in order to cancel the migration process, and avoid resuming the guest in the target host without receiving all current RAM. This will work fine on RAM migration because the RAM pages are not usually = freed, and there is no problem on changing the pages content between writev_zero_c= opy() and the actual sending of the buffer, because this change will dirty the page a= nd cause it to be re-sent on a next iteration anyway. A lot of locked memory may be needed in order to use multifd migration with zero-copy enabled, so disabling the feature should be necessary for low-privileged users trying to perform multifd migrations. Signed-off-by: Leonardo Bras Reviewed-by: Peter Xu Reviewed-by: Daniel P. Berrang=C3=A9 Message-Id: <20220513062836.965425-9-leobras@redhat.com> Signed-off-by: Dr. David Alan Gilbert --- migration/migration.c | 11 ++++++++++- migration/multifd.c | 37 +++++++++++++++++++++++++++++++++++-- migration/multifd.h | 2 ++ migration/socket.c | 5 +++-- 4 files changed, 50 insertions(+), 5 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 4b6df2eb5e..31739b2af9 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -1497,7 +1497,16 @@ static bool migrate_params_check(MigrationParameters= *params, Error **errp) error_prepend(errp, "Invalid mapping given for block-bitmap-mappin= g: "); return false; } - +#ifdef CONFIG_LINUX + if (params->zero_copy_send && + (!migrate_use_multifd() || + params->multifd_compression !=3D MULTIFD_COMPRESSION_NONE || + (params->tls_creds && *params->tls_creds))) { + error_setg(errp, + "Zero copy only available for non-compressed non-TLS mu= ltifd migration"); + return false; + } +#endif return true; } =20 diff --git a/migration/multifd.c b/migration/multifd.c index 2541cd2322..9282ab6aa4 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -569,6 +569,7 @@ void multifd_save_cleanup(void) int multifd_send_sync_main(QEMUFile *f) { int i; + bool flush_zero_copy; =20 if (!migrate_use_multifd()) { return 0; @@ -579,6 +580,20 @@ int multifd_send_sync_main(QEMUFile *f) return -1; } } + + /* + * When using zero-copy, it's necessary to flush the pages before any = of + * the pages can be sent again, so we'll make sure the new version of = the + * pages will always arrive _later_ than the old pages. + * + * Currently we achieve this by flushing the zero-page requested writes + * per ram iteration, but in the future we could potentially optimize = it + * to be less frequent, e.g. only after we finished one whole scanning= of + * all the dirty bitmaps. + */ + + flush_zero_copy =3D migrate_use_zero_copy_send(); + for (i =3D 0; i < migrate_multifd_channels(); i++) { MultiFDSendParams *p =3D &multifd_send_state->params[i]; =20 @@ -600,6 +615,17 @@ int multifd_send_sync_main(QEMUFile *f) ram_counters.transferred +=3D p->packet_len; qemu_mutex_unlock(&p->mutex); qemu_sem_post(&p->sem); + + if (flush_zero_copy && p->c) { + int ret; + Error *err =3D NULL; + + ret =3D qio_channel_flush(p->c, &err); + if (ret < 0) { + error_report_err(err); + return -1; + } + } } for (i =3D 0; i < migrate_multifd_channels(); i++) { MultiFDSendParams *p =3D &multifd_send_state->params[i]; @@ -684,8 +710,8 @@ static void *multifd_send_thread(void *opaque) p->iov[0].iov_base =3D p->packet; } =20 - ret =3D qio_channel_writev_all(p->c, p->iov, p->iovs_num, - &local_err); + ret =3D qio_channel_writev_full_all(p->c, p->iov, p->iovs_num,= NULL, + 0, p->write_flags, &local_er= r); if (ret !=3D 0) { break; } @@ -913,6 +939,13 @@ int multifd_save_setup(Error **errp) /* We need one extra place for the packet header */ p->iov =3D g_new0(struct iovec, page_count + 1); p->normal =3D g_new0(ram_addr_t, page_count); + + if (migrate_use_zero_copy_send()) { + p->write_flags =3D QIO_CHANNEL_WRITE_FLAG_ZERO_COPY; + } else { + p->write_flags =3D 0; + } + socket_send_channel_create(multifd_new_send_channel_async, p); } =20 diff --git a/migration/multifd.h b/migration/multifd.h index bcf5992945..4d8d89e5e5 100644 --- a/migration/multifd.h +++ b/migration/multifd.h @@ -92,6 +92,8 @@ typedef struct { uint32_t packet_len; /* pointer to the packet */ MultiFDPacket_t *packet; + /* multifd flags for sending ram */ + int write_flags; /* multifd flags for each packet */ uint32_t flags; /* size of the next packet that contains pages */ diff --git a/migration/socket.c b/migration/socket.c index 3754d8f72c..4fd5e85f50 100644 --- a/migration/socket.c +++ b/migration/socket.c @@ -79,8 +79,9 @@ static void socket_outgoing_migration(QIOTask *task, =20 trace_migration_socket_outgoing_connected(data->hostname); =20 - if (migrate_use_zero_copy_send()) { - error_setg(&err, "Zero copy send not available in migration"); + if (migrate_use_zero_copy_send() && + !qio_channel_has_feature(sioc, QIO_CHANNEL_FEATURE_WRITE_ZERO_COPY= )) { + error_setg(&err, "Zero copy send feature not detected in host kern= el"); } =20 out: --=20 2.36.1