From nobody Sat Feb 7 07:38:02 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1599146994; cv=none; d=zohomail.com; s=zohoarc; b=JgVKsGWRf1dRdjvBWp1Rg2C/a7ZtNQcGoJbL4pibsWllS2hNv3Om1xcPlqdz7nhvN64FpysQtGKIEHPnD4jEDiORyuWmPVsegmOcyKBARbkb82k0xgtjkIC5bvQ06kIRKY+j0ILONxfIQuwcpNRVLKlswbyK9a4fDZjo7Gyj42w= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1599146994; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=q9eafikgUXeUM2nyemK+4rrXIL5OR+qKMW3xwigy0Do=; b=bAsr3/ksub02g8ZU2ojdFJKmvQY1AW5Q+ErraxcX12IEWGRQSlVo5aDDeKxKrYst7go5JdE1wCvjX2z31SH8lJCgn/Sj9MfWELBNpKcFe3/4Nll3FW7A6tLMsT9U9bCpFRt9M4HfXGvTc23xcXC2EjyVCQb9tPMXXFGEJPOGxrY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1599146993908696.2579094539535; Thu, 3 Sep 2020 08:29:53 -0700 (PDT) Received: from localhost ([::1]:41232 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kDrBR-0004Gj-4G for importer@patchew.org; Thu, 03 Sep 2020 11:29:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50772) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kDr8Z-00006J-Pw for qemu-devel@nongnu.org; Thu, 03 Sep 2020 11:26:55 -0400 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:53165 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kDr8X-0000DD-Rw for qemu-devel@nongnu.org; Thu, 03 Sep 2020 11:26:55 -0400 Received: from mail-qk1-f198.google.com (mail-qk1-f198.google.com [209.85.222.198]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-135-R7K1qk2nPm25vKxMv7kwdw-1; Thu, 03 Sep 2020 11:26:51 -0400 Received: by mail-qk1-f198.google.com with SMTP id 139so1769226qkl.11 for ; Thu, 03 Sep 2020 08:26:51 -0700 (PDT) Received: from xz-x1.redhat.com (bras-vprn-toroon474qw-lp130-11-70-53-122-15.dsl.bell.ca. [70.53.122.15]) by smtp.gmail.com with ESMTPSA id l38sm2319889qtl.58.2020.09.03.08.26.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 03 Sep 2020 08:26:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1599146812; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=q9eafikgUXeUM2nyemK+4rrXIL5OR+qKMW3xwigy0Do=; b=brND23z47sYRAVHqO/QqTegf4scpbizoAwiELPL6EX6qMeLl93Vpy10sDqad/L5P+TstF4 rGjZh3HsxYb+nB2pe2/JngxXAvbNyXm9rZOO25HcgMhbNwv3cTa8ijWMHRx3+ud3tXCfWE fQZ1qRqqsTX0K8eYy1/7n/2LQLjaw/g= X-MC-Unique: R7K1qk2nPm25vKxMv7kwdw-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=q9eafikgUXeUM2nyemK+4rrXIL5OR+qKMW3xwigy0Do=; b=cfyVjAKCcF4qjVuDMfM805Viz9GcMOXxPmxAyET5WLZVzfGqiBNY1bX0taTnmQTtYI Tib3pq2j/4401Zt+RFKCjmyku5fKfyp62Gmk5Iri4pA80Tqp0HuQGMzPheUA1q7FkAq5 zPaLbTJ9rWdQ3a31yiiJMJ99VIpyj3n1JtaOiaXR81N9BEUg/GQxsebnuyKlnIb2GPyj WY01e3YV/mjmRGWHeeFg8RnObmn5kc9WMjBlmmlYScbn1CyfFL0yBQF5QBWDWgZEUijP 52IZqBHU0fhEBRMv0j7ExjEwYl0sg3N7GZRbr8VIpMiXYLjy+zVyczuJL3bQe5UQzC/8 DYGg== X-Gm-Message-State: AOAM5320J/704eoq0WHyPOD1St7d9yV4AIPdmV4EIe2JGZALnmYEb2ox j09Q7hzvnhGBggqJwDkVO2pBuCuyPZ66f+nFFpjPdbTf15G9M+G/U3SmNB/B768rmbk4ARS3EP2 1Wz14vqhHIdQgFE8= X-Received: by 2002:ac8:474d:: with SMTP id k13mr4050440qtp.20.1599146810641; Thu, 03 Sep 2020 08:26:50 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwa02dLLYxCKWZewAX/vzPnpK9Hkt5gxeRJ1DAD5hd+LVSyV5gy6uyRg5nukDwORqq44rDZ8w== X-Received: by 2002:ac8:474d:: with SMTP id k13mr4050409qtp.20.1599146810383; Thu, 03 Sep 2020 08:26:50 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Subject: [PATCH 1/5] migration: Rework migrate_send_rp_req_pages() function Date: Thu, 3 Sep 2020 11:26:42 -0400 Message-Id: <20200903152646.93336-2-peterx@redhat.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200903152646.93336-1-peterx@redhat.com> References: <20200903152646.93336-1-peterx@redhat.com> MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=peterx@redhat.com X-Mimecast-Spam-Score: 0.002 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=207.211.31.120; envelope-from=peterx@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/09/03 00:24:51 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Xiaohui Li , "Dr . David Alan Gilbert" , peterx@redhat.com, Juan Quintela Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" We duplicated the logic of maintaining the last_rb variable at both callers= of this function. Pass *rb pointer into the function so that we can avoid duplicating the logic. Also, when we have the rb pointer, it's also easier= to remove the original 2nd & 4th parameters, because both of them (name of the ramblock when needed, or the page size) can be fetched from the ramblock pointer too. Signed-off-by: Peter Xu Reviewed-by: Dr. David Alan Gilbert --- migration/migration.c | 26 ++++++++++++++++++-------- migration/migration.h | 4 ++-- migration/postcopy-ram.c | 24 ++---------------------- 3 files changed, 22 insertions(+), 32 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 58a5452471..6761e3f233 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -311,25 +311,35 @@ error: return ret; } =20 -/* Request a range of pages from the source VM at the given - * start address. - * rbname: Name of the RAMBlock to request the page in, if NULL it's the= same - * as the last request (a name must have been given previously) +/* Request one page from the source VM at the given start address. + * rb: the RAMBlock to request the page in * Start: Address offset within the RB * Len: Length in bytes required - must be a multiple of pagesize */ -int migrate_send_rp_req_pages(MigrationIncomingState *mis, const char *rbn= ame, - ram_addr_t start, size_t len) +int migrate_send_rp_req_pages(MigrationIncomingState *mis, RAMBlock *rb, + ram_addr_t start) { uint8_t bufc[12 + 1 + 255]; /* start (8), len (4), rbname up to 256 */ size_t msglen =3D 12; /* start + len */ + size_t len =3D qemu_ram_pagesize(rb); enum mig_rp_message_type msg_type; + const char *rbname; + int rbname_len; =20 *(uint64_t *)bufc =3D cpu_to_be64((uint64_t)start); *(uint32_t *)(bufc + 8) =3D cpu_to_be32((uint32_t)len); =20 - if (rbname) { - int rbname_len =3D strlen(rbname); + /* + * We maintain the last ramblock that we requested for page. Note tha= t we + * don't need locking because this function will only be called within= the + * postcopy ram fault thread. + */ + if (rb !=3D mis->last_rb) { + mis->last_rb =3D rb; + + rbname =3D qemu_ram_get_idstr(rb); + rbname_len =3D strlen(rbname); + assert(rbname_len < 256); =20 bufc[msglen++] =3D rbname_len; diff --git a/migration/migration.h b/migration/migration.h index ae497bd45a..ca8dc4c773 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -328,8 +328,8 @@ void migrate_send_rp_shut(MigrationIncomingState *mis, uint32_t value); void migrate_send_rp_pong(MigrationIncomingState *mis, uint32_t value); -int migrate_send_rp_req_pages(MigrationIncomingState *mis, const char* rbn= ame, - ram_addr_t start, size_t len); +int migrate_send_rp_req_pages(MigrationIncomingState *mis, RAMBlock *rb, + ram_addr_t start); void migrate_send_rp_recv_bitmap(MigrationIncomingState *mis, char *block_name); void migrate_send_rp_resume_ack(MigrationIncomingState *mis, uint32_t valu= e); diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index 1bb22f2b6c..11a70441a6 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -684,14 +684,7 @@ int postcopy_request_shared_page(struct PostCopyFD *pc= fd, RAMBlock *rb, qemu_ram_get_idstr(rb), rb_offset); return postcopy_wake_shared(pcfd, client_addr, rb); } - if (rb !=3D mis->last_rb) { - mis->last_rb =3D rb; - migrate_send_rp_req_pages(mis, qemu_ram_get_idstr(rb), - aligned_rbo, pagesize); - } else { - /* Save some space */ - migrate_send_rp_req_pages(mis, NULL, aligned_rbo, pagesize); - } + migrate_send_rp_req_pages(mis, rb, aligned_rbo); return 0; } =20 @@ -986,20 +979,7 @@ retry: * Send the request to the source - we want to request one * of our host page sizes (which is >=3D TPS) */ - if (rb !=3D mis->last_rb) { - mis->last_rb =3D rb; - ret =3D migrate_send_rp_req_pages(mis, - qemu_ram_get_idstr(rb), - rb_offset, - qemu_ram_pagesize(rb)); - } else { - /* Save some space */ - ret =3D migrate_send_rp_req_pages(mis, - NULL, - rb_offset, - qemu_ram_pagesize(rb)); - } - + ret =3D migrate_send_rp_req_pages(mis, rb, rb_offset); if (ret) { /* May be network failure, try to wait for recovery */ if (ret =3D=3D -EIO && postcopy_pause_fault_thread(mis)) { --=20 2.26.2 From nobody Sat Feb 7 07:38:02 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1599147141; cv=none; d=zohomail.com; s=zohoarc; b=Plk87r113J9gZjqQf5jNLBuoKzeR7LFJJudcOuj3bwFJW6cfUy8kEyAFEA2y/L9TH9b9JrFs8aeShbTCkLvTG8myrHnYinrt5N+aYjb/zxEfTI90Eb08DK05xLcxbfxDVl0OLev1F2bA5QbTzZlAg4hJLgbDNVDqvLKNJT8Y+as= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1599147141; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=DAmUmQ2oPw8sOeJlLTMAYNZr3L9wUUBWGxTofVgM/fc=; b=D/yLYzhRVeW+UX3gv0dxeTuRCLXZK6RPqYfC8bm/i/4qtzOPTjeuctRqdICuvfbGnfcqV9kFY+ancCpYJHixrNVao2FCmN6XNgtNYb6/OHZx3M+wht2TTz0l0rDxkIrL5QDUlJione3m6iJm2LJoElchoBlDgLhnRRytbyyRtes= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1599147141546936.943082119358; Thu, 3 Sep 2020 08:32:21 -0700 (PDT) Received: from localhost ([::1]:47664 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kDrDo-00072E-1k for importer@patchew.org; Thu, 03 Sep 2020 11:32:20 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50798) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kDr8b-00009Q-5E for qemu-devel@nongnu.org; Thu, 03 Sep 2020 11:26:57 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:41179 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kDr8Z-0000DO-99 for qemu-devel@nongnu.org; Thu, 03 Sep 2020 11:26:56 -0400 Received: from mail-qt1-f198.google.com (mail-qt1-f198.google.com [209.85.160.198]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-478-MWWE9awJNxWsXs4qZJcB-g-1; Thu, 03 Sep 2020 11:26:52 -0400 Received: by mail-qt1-f198.google.com with SMTP id o14so331948qtq.0 for ; Thu, 03 Sep 2020 08:26:52 -0700 (PDT) Received: from xz-x1.redhat.com (bras-vprn-toroon474qw-lp130-11-70-53-122-15.dsl.bell.ca. [70.53.122.15]) by smtp.gmail.com with ESMTPSA id l38sm2319889qtl.58.2020.09.03.08.26.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 03 Sep 2020 08:26:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1599146814; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DAmUmQ2oPw8sOeJlLTMAYNZr3L9wUUBWGxTofVgM/fc=; b=RCGmH7Je/RbngJnccyEaDC63BTnc5ePItgydqsUeM5scbjtORysYu+Tb/VoZNrWLlUSa8y 5J3SwSfkt9j6ce+YYv6tdBqfgL9dcIWfmHbku6hPzNRvlA3Pi5BhAsPrtfEsrOM5xZHyOq nxr5oStFYVnN/fgB8e8xqBN2db9IVoo= X-MC-Unique: MWWE9awJNxWsXs4qZJcB-g-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=DAmUmQ2oPw8sOeJlLTMAYNZr3L9wUUBWGxTofVgM/fc=; b=MKaNksgtxHwyY8OH2ho9rMDH9fLHqoWLkm/mMyMsoWOqgCqXZCweVBl2BQjynrfxke tGDjzxsAMzXVrBOYzmuLckpRCK6hvIRIemdjDDzxmm6NXGLoKQp4uMZlYMWiJsHFI+d8 2HJNA/pTgkyBXVuye5HVpPFHHQAPOI8rfKNWqiYitseJtoLJjVI+hiRd3OUU83yaHexn dalSlwA3oDdP0zo4d5MjCOaE9qjet8HZTEsH1wjQNi4M5Yl8q+LVbyfP+Jz1+frHjlYk doSSAF9bNH0R10V6h7bmd9X/Y+HjXGjhQMWmaBb/5v9540npAr0frWAKmKizovrwxDK/ uuag== X-Gm-Message-State: AOAM53277tc0GnKEXeoUd12uwpNWK33AYahFh9ZzxV9/EQ2r7Y6CjIZR 6LKJu0pUZ0eSPPKKgRxy1E9H8x8AvxK4CPKTQlzdHS4cQfEh+WTg5B1LymiBMqNoQwUoJ4GXsFD afK2jvtVdQGYoEn4= X-Received: by 2002:a37:654b:: with SMTP id z72mr3485892qkb.365.1599146812087; Thu, 03 Sep 2020 08:26:52 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxhxnmxNHEzcd+Y0bZIUNW0PwSMhPvUejDGjAMadilAkSkNQXTPL/pBO3elKGG8hCC7YBwtKw== X-Received: by 2002:a37:654b:: with SMTP id z72mr3485872qkb.365.1599146811895; Thu, 03 Sep 2020 08:26:51 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Subject: [PATCH 2/5] migration: Introduce migrate_send_rp_message_req_pages() Date: Thu, 3 Sep 2020 11:26:43 -0400 Message-Id: <20200903152646.93336-3-peterx@redhat.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200903152646.93336-1-peterx@redhat.com> References: <20200903152646.93336-1-peterx@redhat.com> MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=peterx@redhat.com X-Mimecast-Spam-Score: 0.002 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.139.110.120; envelope-from=peterx@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/09/03 01:58:20 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Xiaohui Li , "Dr . David Alan Gilbert" , peterx@redhat.com, Juan Quintela Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" This is another layer wrapper for sending a page request to the source VM, Signed-off-by: Peter Xu Reviewed-by: Dr. David Alan Gilbert --- migration/migration.c | 10 ++++++++-- migration/migration.h | 2 ++ 2 files changed, 10 insertions(+), 2 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 6761e3f233..6b43ffddbd 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -316,8 +316,8 @@ error: * Start: Address offset within the RB * Len: Length in bytes required - must be a multiple of pagesize */ -int migrate_send_rp_req_pages(MigrationIncomingState *mis, RAMBlock *rb, - ram_addr_t start) +int migrate_send_rp_message_req_pages(MigrationIncomingState *mis, + RAMBlock *rb, ram_addr_t start) { uint8_t bufc[12 + 1 + 255]; /* start (8), len (4), rbname up to 256 */ size_t msglen =3D 12; /* start + len */ @@ -353,6 +353,12 @@ int migrate_send_rp_req_pages(MigrationIncomingState *= mis, RAMBlock *rb, return migrate_send_rp_message(mis, msg_type, msglen, bufc); } =20 +int migrate_send_rp_req_pages(MigrationIncomingState *mis, + RAMBlock *rb, ram_addr_t start) +{ + return migrate_send_rp_message_req_pages(mis, rb, start); +} + static bool migration_colo_enabled; bool migration_incoming_colo_enabled(void) { diff --git a/migration/migration.h b/migration/migration.h index ca8dc4c773..f552725305 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -330,6 +330,8 @@ void migrate_send_rp_pong(MigrationIncomingState *mis, uint32_t value); int migrate_send_rp_req_pages(MigrationIncomingState *mis, RAMBlock *rb, ram_addr_t start); +int migrate_send_rp_message_req_pages(MigrationIncomingState *mis, + RAMBlock *rb, ram_addr_t start); void migrate_send_rp_recv_bitmap(MigrationIncomingState *mis, char *block_name); void migrate_send_rp_resume_ack(MigrationIncomingState *mis, uint32_t valu= e); --=20 2.26.2 From nobody Sat Feb 7 07:38:02 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1599146996; cv=none; d=zohomail.com; s=zohoarc; b=Rv7fy5clM8e0grzlol8yPKeqEPdRECWv435aCfDPL6CSgM0FyIc+XqNczcCt3ef+uOjuvnQaBW91ijgOWaIRJ2X3D2qL89l915/Tsv9CflQ3DFH3mVkS6L/Nhu17tYneidJFavniI19brCAHwVGvkA5ppZzAK/5c1PJQxXUBMBg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1599146996; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=yNN9HMxErQBgUM+loGmpJTxuhPYQQVhkbsp50pLpF+c=; b=U/jfTQzmuPYGR82QLZPEORwQtwFXgIJM0kKMFKQTEUMGUQwOAN3fbFYAqrl3HOoctbrUG1hqt5/JOcYtbsFHSbghlsM3NpiJQYS7fGUmhuHsQmUTYaplLJ56dBatfREVtAwaqVmxGfJOkXubkJgwdmdWRCE2c6fSG5KWfxLSoc8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1599146996911350.94814923969966; Thu, 3 Sep 2020 08:29:56 -0700 (PDT) Received: from localhost ([::1]:41540 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kDrBU-0004OD-1T for importer@patchew.org; Thu, 03 Sep 2020 11:29:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50844) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kDr8e-0000IX-87 for qemu-devel@nongnu.org; Thu, 03 Sep 2020 11:27:00 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:58885) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kDr8c-0000E6-FB for qemu-devel@nongnu.org; Thu, 03 Sep 2020 11:26:59 -0400 Received: from mail-qt1-f199.google.com (mail-qt1-f199.google.com [209.85.160.199]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-525-SA9GxHsdNJ2BjR_p84eRYg-1; Thu, 03 Sep 2020 11:26:55 -0400 Received: by mail-qt1-f199.google.com with SMTP id j35so2350750qtk.14 for ; Thu, 03 Sep 2020 08:26:55 -0700 (PDT) Received: from xz-x1.redhat.com (bras-vprn-toroon474qw-lp130-11-70-53-122-15.dsl.bell.ca. [70.53.122.15]) by smtp.gmail.com with ESMTPSA id l38sm2319889qtl.58.2020.09.03.08.26.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 03 Sep 2020 08:26:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1599146817; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yNN9HMxErQBgUM+loGmpJTxuhPYQQVhkbsp50pLpF+c=; b=RJSOZMOirC+FFmRFAsu7Z7Alfq/Z2+XKK9+IS7XodMvyWgxZdQMu5VS9FVkEUQUVb5jUA8 Knz9fla0PwX9J+5fmPsFAn2fmxJd106Mz93X0Yyrpf/HGUnCJncMp+LY+K46PooRlRgdnu aSjKv9ZCC7mnjetc42nrcS8ksHGg/YY= X-MC-Unique: SA9GxHsdNJ2BjR_p84eRYg-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=yNN9HMxErQBgUM+loGmpJTxuhPYQQVhkbsp50pLpF+c=; b=F1UhqyLgotfvZTUvcE9+l6Ee9X4msTS/ts/1VmeK8DUqXVLqQ1+C1ylht5yzjYML72 ukyY5m3DZ/RNBjad/9w2bRydOygtLbaARVUF+v1HJaNvmiHGmuo9nwiao1Hz2uxxNUNC WZ+6+aKssDS6Jz+QyOGnX1QjS19G1aySPR6VoLbncyblv3ytwDapwK7yLZ1PChWeb+yL RQzDDxBWPeRRoGwU6ple3xyvChtQrebs5uE/qp0tIHnlEYVmkLpQ4PSxagCaeI3byP8W ZIjI+Pyk7U9NsfWAd/utprND16IpU3oP7nbJzvlewYycK7YpSETLkuRIwEwXHOpuQqo2 9OTA== X-Gm-Message-State: AOAM5300O3nQFDdR9ks6fcNMaSD4zpT0NcH/uzjg13gq1yfb3rJDP1R2 WDN3Py+g55b6SPGg1ds9vbELGfLeb+/ujPUcmUTAS7CBhLUJnKLLKUWu61S9Nz0wPdQZnV11JwR Xi//Yfh8hgJpMI14= X-Received: by 2002:a37:68c7:: with SMTP id d190mr3553533qkc.127.1599146813432; Thu, 03 Sep 2020 08:26:53 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzhbtQYN6UmM0Ksh7FFhjMatqeLXu86Q7riw5gzhopdLkqndQyT0hc81xQA1N+MNil12P3LoA== X-Received: by 2002:a37:68c7:: with SMTP id d190mr3553514qkc.127.1599146813196; Thu, 03 Sep 2020 08:26:53 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Subject: [PATCH 3/5] migration: Pass incoming state into qemu_ufd_copy_ioctl() Date: Thu, 3 Sep 2020 11:26:44 -0400 Message-Id: <20200903152646.93336-4-peterx@redhat.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200903152646.93336-1-peterx@redhat.com> References: <20200903152646.93336-1-peterx@redhat.com> MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=peterx@redhat.com X-Mimecast-Spam-Score: 0.003 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/09/03 01:47:17 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Xiaohui Li , "Dr . David Alan Gilbert" , peterx@redhat.com, Juan Quintela Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" It'll be used in follow up patches to access more fields out of it. Meanwh= ile fetch the userfaultfd inside the function. Signed-off-by: Peter Xu Reviewed-by: Dr. David Alan Gilbert --- migration/postcopy-ram.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index 11a70441a6..d333c3fd0e 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -1128,10 +1128,12 @@ int postcopy_ram_incoming_setup(MigrationIncomingSt= ate *mis) return 0; } =20 -static int qemu_ufd_copy_ioctl(int userfault_fd, void *host_addr, +static int qemu_ufd_copy_ioctl(MigrationIncomingState *mis, void *host_add= r, void *from_addr, uint64_t pagesize, RAMBloc= k *rb) { + int userfault_fd =3D mis->userfault_fd; int ret; + if (from_addr) { struct uffdio_copy copy_struct; copy_struct.dst =3D (uint64_t)(uintptr_t)host_addr; @@ -1185,7 +1187,7 @@ int postcopy_place_page(MigrationIncomingState *mis, = void *host, void *from, * which would be slightly cheaper, but we'd have to be careful * of the order of updating our page state. */ - if (qemu_ufd_copy_ioctl(mis->userfault_fd, host, from, pagesize, rb)) { + if (qemu_ufd_copy_ioctl(mis, host, from, pagesize, rb)) { int e =3D errno; error_report("%s: %s copy host: %p from: %p (size: %zd)", __func__, strerror(e), host, from, pagesize); @@ -1212,7 +1214,7 @@ int postcopy_place_page_zero(MigrationIncomingState *= mis, void *host, * but it's not available for everything (e.g. hugetlbpages) */ if (qemu_ram_is_uf_zeroable(rb)) { - if (qemu_ufd_copy_ioctl(mis->userfault_fd, host, NULL, pagesize, r= b)) { + if (qemu_ufd_copy_ioctl(mis, host, NULL, pagesize, rb)) { int e =3D errno; error_report("%s: %s zero host: %p", __func__, strerror(e), host); --=20 2.26.2 From nobody Sat Feb 7 07:38:02 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1599147220; cv=none; d=zohomail.com; s=zohoarc; b=jmYYkFvLg5XXm2Zn3GIyKVr+TqEqfX+VL5pPn2MTHush10hlOX1HPlcDj1xGwRGSn5VrYIVWQXz5jNQ3VS+EN5UXhZPvuIxTIouYqt4fVk5PctRi4YcUB0a9+bo8aVAqUbbFPJXqQQgS1jEBBBqxvcAgikJLQKf4J8LKF4qaoAY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1599147220; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=7qNwKw3bP7jnzU4s7QDMSOC0XIYxDM97KbjcVzP6q/Q=; b=aZkpise1qGHEGrgdeNtz6daAgYUTnbxkEnJ7zP2ojB+Uy2tgRmHf0cMSjmqWx890bwAQehQx6X0Sw6XlGurDJn1V1AfMKhueV+gPTBY4SCNh/MRNgR3oq8G01mhTb1yk4VtgrPWGBbr7F+Id6+SagLZTQmcZOgzH1rWPf6MGz9g= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1599147220567724.5938543801025; Thu, 3 Sep 2020 08:33:40 -0700 (PDT) Received: from localhost ([::1]:53104 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kDrF5-00011o-M7 for importer@patchew.org; Thu, 03 Sep 2020 11:33:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50884) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kDr8g-0000NP-6E for qemu-devel@nongnu.org; Thu, 03 Sep 2020 11:27:02 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:35182 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kDr8d-0000EC-1C for qemu-devel@nongnu.org; Thu, 03 Sep 2020 11:27:01 -0400 Received: from mail-qk1-f197.google.com (mail-qk1-f197.google.com [209.85.222.197]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-554-Uiy9quutPPGFeVfTglBkqA-1; Thu, 03 Sep 2020 11:26:56 -0400 Received: by mail-qk1-f197.google.com with SMTP id s141so1770471qka.13 for ; Thu, 03 Sep 2020 08:26:56 -0700 (PDT) Received: from xz-x1.redhat.com (bras-vprn-toroon474qw-lp130-11-70-53-122-15.dsl.bell.ca. [70.53.122.15]) by smtp.gmail.com with ESMTPSA id l38sm2319889qtl.58.2020.09.03.08.26.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 03 Sep 2020 08:26:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1599146817; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=7qNwKw3bP7jnzU4s7QDMSOC0XIYxDM97KbjcVzP6q/Q=; b=SN2MujVRtJN3cNRkGDkwHWkisHVriywgRPhcQFS4S1A9q6B5hb09bmqRAfphJycqwVNEm3 I07/yE0HcU5+oITTN+xX+nnd7Cz8NXbaz6k6b+B9y9mNHmi3k1nTobDPLFIlz1cFu+m88C JRdBTU9/7kpIPWkQeebW1lJ0e3NVsa0= X-MC-Unique: Uiy9quutPPGFeVfTglBkqA-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=7qNwKw3bP7jnzU4s7QDMSOC0XIYxDM97KbjcVzP6q/Q=; b=hTJ2XpT10WUATPQ0oKfF09jQCILA8imacj1e+xHwP6QsZHOgz+xM+lvbbZVRlU9yiw ARlv9lIPuz+Y+R081GAGBm9cZvLzZBO2pQ6TezdX9sq5gZ0uxpxrDXixszcG3vfBtsPL bj60R8ai4WBJ85CXR1DiVs226Iz3ZlR1y9iNTaellRjkl3VLMu5bZawYGW78T62PXdmd aFEu1SdKmokiSRlvnyAngD165MbNMfoQFHP2LbIfP8nJpYtdnQMA/Ob79bAR2a/EbRh/ cDc+hFu5ECWnTuukmu69NCF47WLmuVXgvwg99XRigmJvFNyl0hG32l6VcH0Qpu3/fhFO DHVg== X-Gm-Message-State: AOAM531uOzXgbFLz0HnbqvD7YBiti2r556lnq+8e7+Q22/hbQPH00zn+ oU5f+gyA41c4S0WZj9HuvpeElHUoS4KcYoejXS3R0QGPtxjxLvhGDEMDlQDIomwDR/8yGtS3Y7i sG5JaoANvyokqTaQ= X-Received: by 2002:a0c:e8c9:: with SMTP id m9mr3544002qvo.51.1599146815462; Thu, 03 Sep 2020 08:26:55 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwWbXH6HXo7H+gxm2bIni2HTO1WNh0nnxCO+55B6eeyoEQx3e4CsvMaXYz4hM329OdRchSDjQ== X-Received: by 2002:a0c:e8c9:: with SMTP id m9mr3543969qvo.51.1599146815114; Thu, 03 Sep 2020 08:26:55 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Subject: [PATCH 4/5] migration: Maintain postcopy faulted addresses Date: Thu, 3 Sep 2020 11:26:45 -0400 Message-Id: <20200903152646.93336-5-peterx@redhat.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200903152646.93336-1-peterx@redhat.com> References: <20200903152646.93336-1-peterx@redhat.com> MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=peterx@redhat.com X-Mimecast-Spam-Score: 0.001 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.139.110.120; envelope-from=peterx@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/09/03 01:58:20 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Xiaohui Li , "Dr . David Alan Gilbert" , peterx@redhat.com, Juan Quintela Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" Maintain a list of faulted addresses on the destination host for which we're waiting on. This is implemented using a GTree rather than a real list to m= ake sure even there're plenty of vCPUs/threads that are faulting, the lookup wi= ll still be fast with O(log(N)) (because we'll do that after placing each page= ). It should bring a slight overhead, but ideally that shouldn't be a big prob= lem simply because in most cases the requested page list will be short. Actually we did similar things for postcopy blocktime measurements. This p= atch didn't use that simply because: (1) blocktime measurement is towards vcpu threads only, but here we need = to record all faulted addresses, including main thread and external thread (like, DPDK via vhost-user). (2) blocktime measurement will require UFFD_FEATURE_THREAD_ID, but here we don't want to add that extra dependency on the kernel version since n= ot necessary. E.g., we don't need to know which thread faulted on which page, we also don't care about multiple threads faulting on the same page. But we only care about what addresses are faulted so waiting f= or a page copying from src. (3) blocktime measurement is not enabled by default. However we need thi= s by default especially for postcopy recover. Another thing to mention is that this patch introduced a new mutex to seria= lize the receivedmap and the page_requested tree, however that serialization does not cover other procedures like UFFDIO_COPY. Signed-off-by: Peter Xu --- migration/migration.c | 41 +++++++++++++++++++++++++++++++++++++++- migration/migration.h | 19 ++++++++++++++++++- migration/postcopy-ram.c | 18 +++++++++++++++--- migration/trace-events | 2 ++ 4 files changed, 75 insertions(+), 5 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 6b43ffddbd..e943d96c1b 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -143,6 +143,13 @@ static int migration_maybe_pause(MigrationState *s, int new_state); static void migrate_fd_cancel(MigrationState *s); =20 +static gint page_request_addr_cmp(gconstpointer ap, gconstpointer bp) +{ + uint64_t a =3D (uint64_t) ap, b =3D (uint64_t) bp; + + return (a > b) - (a < b); +} + void migration_object_init(void) { MachineState *ms =3D MACHINE(qdev_get_machine()); @@ -165,6 +172,8 @@ void migration_object_init(void) qemu_event_init(¤t_incoming->main_thread_load_event, false); qemu_sem_init(¤t_incoming->postcopy_pause_sem_dst, 0); qemu_sem_init(¤t_incoming->postcopy_pause_sem_fault, 0); + qemu_mutex_init(¤t_incoming->page_request_mutex); + current_incoming->page_requested =3D g_tree_new(page_request_addr_cmp); =20 if (!migration_object_check(current_migration, &err)) { error_report_err(err); @@ -238,6 +247,11 @@ void migration_incoming_state_destroy(void) mis->postcopy_remote_fds =3D NULL; } =20 + if (mis->page_requested) { + g_tree_destroy(mis->page_requested); + mis->page_requested =3D NULL; + } + qemu_event_reset(&mis->main_thread_load_event); =20 if (mis->socket_address_list) { @@ -354,8 +368,33 @@ int migrate_send_rp_message_req_pages(MigrationIncomin= gState *mis, } =20 int migrate_send_rp_req_pages(MigrationIncomingState *mis, - RAMBlock *rb, ram_addr_t start) + RAMBlock *rb, ram_addr_t start, uint64_t had= dr) { + uint64_t aligned =3D haddr & (-qemu_target_page_size()); + bool received; + + qemu_mutex_lock(&mis->page_request_mutex); + received =3D ramblock_recv_bitmap_test_byte_offset(rb, start); + if (!received && !g_tree_lookup(mis->page_requested, (gpointer) aligne= d)) { + /* + * The page has not been received, and it's not yet in the page re= quest + * list. Queue it. Set the value of element to 1, so that things= like + * g_tree_lookup() will return TRUE (1) when found. + */ + g_tree_insert(mis->page_requested, (gpointer) aligned, (gpointer) = 1); + mis->page_requested_count++; + trace_postcopy_page_req_add(aligned, mis->page_requested_count); + } + qemu_mutex_unlock(&mis->page_request_mutex); + + /* + * If the page is there, skip sending the message. We don't even need= the + * lock because as long as the page arrived, it'll be there forever. + */ + if (received) { + return 0; + } + return migrate_send_rp_message_req_pages(mis, rb, start); } =20 diff --git a/migration/migration.h b/migration/migration.h index f552725305..81311dc154 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -103,6 +103,23 @@ struct MigrationIncomingState { =20 /* List of listening socket addresses */ SocketAddressList *socket_address_list; + + /* A tree of pages that we requested to the source VM */ + GTree *page_requested; + /* For debugging purpose only, but would be nice to keep */ + int page_requested_count; + /* + * The mutex helps to maintain the requested pages that we sent to the + * source, IOW, to guarantee coherent between the page_requests tree a= nd + * the per-ramblock receivedmap. Note! This does not guarantee consis= tency + * of the real page copy procedures (using UFFDIO_[ZERO]COPY). E.g., = even + * if one bit in receivedmap is cleared, UFFDIO_COPY could have happen= ed + * for that page already. This is intended so that the mutex won't + * serialize and blocked by slow operations like UFFDIO_* ioctls. How= ever + * this should be enough to make sure the page_requested tree always + * contains valid information. + */ + QemuMutex page_request_mutex; }; =20 MigrationIncomingState *migration_incoming_get_current(void); @@ -329,7 +346,7 @@ void migrate_send_rp_shut(MigrationIncomingState *mis, void migrate_send_rp_pong(MigrationIncomingState *mis, uint32_t value); int migrate_send_rp_req_pages(MigrationIncomingState *mis, RAMBlock *rb, - ram_addr_t start); + ram_addr_t start, uint64_t haddr); int migrate_send_rp_message_req_pages(MigrationIncomingState *mis, RAMBlock *rb, ram_addr_t start); void migrate_send_rp_recv_bitmap(MigrationIncomingState *mis, diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index d333c3fd0e..a30627e838 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -684,7 +684,7 @@ int postcopy_request_shared_page(struct PostCopyFD *pcf= d, RAMBlock *rb, qemu_ram_get_idstr(rb), rb_offset); return postcopy_wake_shared(pcfd, client_addr, rb); } - migrate_send_rp_req_pages(mis, rb, aligned_rbo); + migrate_send_rp_req_pages(mis, rb, aligned_rbo, client_addr); return 0; } =20 @@ -979,7 +979,8 @@ retry: * Send the request to the source - we want to request one * of our host page sizes (which is >=3D TPS) */ - ret =3D migrate_send_rp_req_pages(mis, rb, rb_offset); + ret =3D migrate_send_rp_req_pages(mis, rb, rb_offset, + msg.arg.pagefault.address); if (ret) { /* May be network failure, try to wait for recovery */ if (ret =3D=3D -EIO && postcopy_pause_fault_thread(mis)) { @@ -1149,10 +1150,21 @@ static int qemu_ufd_copy_ioctl(MigrationIncomingSta= te *mis, void *host_addr, ret =3D ioctl(userfault_fd, UFFDIO_ZEROPAGE, &zero_struct); } if (!ret) { + qemu_mutex_lock(&mis->page_request_mutex); ramblock_recv_bitmap_set_range(rb, host_addr, pagesize / qemu_target_page_size()); + /* + * If this page resolves a page fault for a previous recorded faul= ted + * address, take a special note to maintain the requested page lis= t. + */ + if (g_tree_lookup(mis->page_requested, (gconstpointer)host_addr)) { + g_tree_remove(mis->page_requested, (gconstpointer)host_addr); + mis->page_requested_count--; + trace_postcopy_page_req_del((uint64_t)host_addr, + mis->page_requested_count); + } + qemu_mutex_unlock(&mis->page_request_mutex); mark_postcopy_blocktime_end((uintptr_t)host_addr); - } return ret; } diff --git a/migration/trace-events b/migration/trace-events index 4ab0a503d2..b89ce02cb0 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -157,6 +157,7 @@ postcopy_pause_return_path(void) "" postcopy_pause_return_path_continued(void) "" postcopy_pause_continued(void) "" postcopy_start_set_run(void) "" +postcopy_page_req_add(uint64_t addr, int count) "new page req 0x%lx total = %d" source_return_path_thread_bad_end(void) "" source_return_path_thread_end(void) "" source_return_path_thread_entry(void) "" @@ -267,6 +268,7 @@ postcopy_ram_incoming_cleanup_blocktime(uint64_t total)= "total blocktime %" PRIu postcopy_request_shared_page(const char *sharer, const char *rb, uint64_t = rb_offset) "for %s in %s offset 0x%"PRIx64 postcopy_request_shared_page_present(const char *sharer, const char *rb, u= int64_t rb_offset) "%s already %s offset 0x%"PRIx64 postcopy_wake_shared(uint64_t client_addr, const char *rb) "at 0x%"PRIx64"= in %s" +postcopy_page_req_del(uint64_t addr, int count) "resolved page req 0x%lx t= otal %d" =20 get_mem_fault_cpu_index(int cpu, uint32_t pid) "cpu: %d, pid: %u" =20 --=20 2.26.2 From nobody Sat Feb 7 07:38:02 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1599147314; cv=none; d=zohomail.com; s=zohoarc; b=dcwseIXS1JnBFxS4ss0LO35LaMfDpwp360CSV5zYUTMh0oIbfNPEcKLl3watifj9pSogS1RlEHyEqPZjJWDPM97i3r0gP/44P+JKBlQR1tDi0FJCTacDp6La58MmjuHQS0kjS7uwydZat7cGAZ0A+UiWILzTuxCVTowx+H1UZZA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1599147314; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=SN+zpDe63b9OXABJW135pgj4q08uRgy/TMocpHMnT88=; b=CYf0aVnB7brOV3fYBeZR4JrEADawHg3SPfLbPDYDDd7CS9XExSjPkMc3HDR4wn+kb9tvQryLoxIIAHhgDvwJeF2MrDcifIbkWoUAzFDC+Yr0eIlQmfDduGMS28tzE08q4jD9BpBAzyuHAHyy4u7YDDAlBjDFr8yNHGyAhxC7lNE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1599147314985396.4855179989753; Thu, 3 Sep 2020 08:35:14 -0700 (PDT) Received: from localhost ([::1]:56846 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kDrGb-0002dJ-71 for importer@patchew.org; Thu, 03 Sep 2020 11:35:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50892) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kDr8h-0000QU-5k for qemu-devel@nongnu.org; Thu, 03 Sep 2020 11:27:03 -0400 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:46181) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kDr8f-0000Fl-2J for qemu-devel@nongnu.org; Thu, 03 Sep 2020 11:27:02 -0400 Received: from mail-qk1-f197.google.com (mail-qk1-f197.google.com [209.85.222.197]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-168-nndpTprDPEWybfrL-1_LBQ-1; Thu, 03 Sep 2020 11:26:58 -0400 Received: by mail-qk1-f197.google.com with SMTP id g6so1742683qko.21 for ; Thu, 03 Sep 2020 08:26:57 -0700 (PDT) Received: from xz-x1.redhat.com (bras-vprn-toroon474qw-lp130-11-70-53-122-15.dsl.bell.ca. [70.53.122.15]) by smtp.gmail.com with ESMTPSA id l38sm2319889qtl.58.2020.09.03.08.26.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 03 Sep 2020 08:26:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1599146820; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=SN+zpDe63b9OXABJW135pgj4q08uRgy/TMocpHMnT88=; b=DzsB7784ZeG1akKNy10wM27wTNVSB2vHQuiAzU7IfGyPZycl95dK+h3gjjdXfMCagut1Gl 0QgvqF8bT/8ztcMPledpDVwykO0+87n53S2n1y/JxHz7xUyygDuhbGwAj4HRpGfv5uCc3z T6NQPVFnfRCQFYsL/olg44AR7vU+YmM= X-MC-Unique: nndpTprDPEWybfrL-1_LBQ-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=SN+zpDe63b9OXABJW135pgj4q08uRgy/TMocpHMnT88=; b=UzM8aa1dQX1ZYrQNqyi25p0qZUVbGmKYPoES+cLWpBg9NvSfg2CKQPJU29LLj/3MMr 6zw2facEVAs6PwW4nuoYLq068wCIJsbH5QJ7D4+NdZQUXY6aUho6QrvonEgNOKzhHH2O 1p7L7NgSXJzqAooK+1p920Xt7AFM0BvqUAxTzr64XIKXQOFsaixWGa35mUUlpfXySe93 9w3U/PTBHM6COyJ43NB/ldbktj/Zq9KguzAWrVBu3ODQbwhpnayasZ5U0RmzmnvRc8D7 HOZFg3wEJ/ZJLMI4lC66CzwPSTXvxDnTNEH3gjbrFjpYwTPh5hS4FWJEhRGmPrlOGCjp 4qsw== X-Gm-Message-State: AOAM533WrJ8SUrloNUZmTL0cxKsBxvAMY83yG5QlyFVoGp4xH3nWQjh9 q8JoXOUhIIyBDOvCm9rzXvjR8jmtVqH/GRmtIGPCi93S/PIW8T3Tgt1LDAqHEWMoTUBXexcrpSA 1BYIHd+wlDYQeMI8= X-Received: by 2002:a0c:de0f:: with SMTP id t15mr3449302qvk.90.1599146817095; Thu, 03 Sep 2020 08:26:57 -0700 (PDT) X-Google-Smtp-Source: ABdhPJx5VWYzIE3WiP4K3dcvtkqv9LeAY2SZSUaV46AuCecyFjoS83lup39Dwj7+QPW8Xw1mFlzGaA== X-Received: by 2002:a0c:de0f:: with SMTP id t15mr3449277qvk.90.1599146816767; Thu, 03 Sep 2020 08:26:56 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Subject: [PATCH 5/5] migration: Sync requested pages after postcopy recovery Date: Thu, 3 Sep 2020 11:26:46 -0400 Message-Id: <20200903152646.93336-6-peterx@redhat.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200903152646.93336-1-peterx@redhat.com> References: <20200903152646.93336-1-peterx@redhat.com> MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=peterx@redhat.com X-Mimecast-Spam-Score: 0.003 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=63.128.21.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/09/03 04:23:54 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Xiaohui Li , "Dr . David Alan Gilbert" , peterx@redhat.com, Juan Quintela Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" We synchronize the requested pages right after a postcopy recovery happens. This helps to synchronize the prioritized pages on source so that the fault= ed threads can be served faster. Reported-by: Xiaohui Li Signed-off-by: Peter Xu Reviewed-by: Dr. David Alan Gilbert --- migration/savevm.c | 56 ++++++++++++++++++++++++++++++++++++++++++ migration/trace-events | 1 + 2 files changed, 57 insertions(+) diff --git a/migration/savevm.c b/migration/savevm.c index 304d98ff78..f998dd230d 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -2011,6 +2011,48 @@ static int loadvm_postcopy_handle_run(MigrationIncom= ingState *mis) return LOADVM_QUIT; } =20 +/* We must be with page_request_mutex held */ +static gboolean postcopy_sync_page_req(gpointer key, gpointer value, + gpointer data) +{ + MigrationIncomingState *mis =3D data; + void *host_addr =3D (void *) key; + ram_addr_t rb_offset; + RAMBlock *rb; + int ret; + + rb =3D qemu_ram_block_from_host(host_addr, true, &rb_offset); + if (!rb) { + /* + * This should _never_ happen. However be nice for a migrating VM= to + * not crash/assert. Post an error (note: intended to not use *_o= nce + * because we do want to see all the illegal addresses; and this c= an + * never be triggered by the guest so we're safe) and move on next. + */ + error_report("%s: illegal host addr %p", __func__, host_addr); + /* Try the next entry */ + return FALSE; + } + + ret =3D migrate_send_rp_message_req_pages(mis, rb, rb_offset); + if (ret) { + /* Refer to above comment - just try our best to continue */ + error_report("%s: send rp message failed for addr %p", + __func__, host_addr); + } + + trace_postcopy_page_req_sync((uint64_t)host_addr); + + return FALSE; +} + +static void migrate_send_rp_req_pages_pending(MigrationIncomingState *mis) +{ + qemu_mutex_lock(&mis->page_request_mutex); + g_tree_foreach(mis->page_requested, postcopy_sync_page_req, mis); + qemu_mutex_unlock(&mis->page_request_mutex); +} + static int loadvm_postcopy_handle_resume(MigrationIncomingState *mis) { if (mis->state !=3D MIGRATION_STATUS_POSTCOPY_RECOVER) { @@ -2033,6 +2075,20 @@ static int loadvm_postcopy_handle_resume(MigrationIn= comingState *mis) /* Tell source that "we are ready" */ migrate_send_rp_resume_ack(mis, MIGRATION_RESUME_ACK_VALUE); =20 + /* + * After a postcopy recovery, the source should have lost the postcopy + * queue, or potentially the requested pages could have been lost duri= ng + * the network down phase. Let's re-sync with the source VM by re-sen= ding + * all the pending pages that we eagerly need, so these threads won't = get + * blocked too long due to the recovery. + * + * Without this procedure, the faulted destination VM threads (waiting= for + * page requests right before the postcopy is interrupted) can keep ha= nging + * until the pages are sent by the source during the background copyin= g of + * pages, or another thread faulted on the same address accidentally. + */ + migrate_send_rp_req_pages_pending(mis); + return 0; } =20 diff --git a/migration/trace-events b/migration/trace-events index b89ce02cb0..54a6dd2761 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -49,6 +49,7 @@ vmstate_save(const char *idstr, const char *vmsd_name) "%= s, %s" vmstate_load(const char *idstr, const char *vmsd_name) "%s, %s" postcopy_pause_incoming(void) "" postcopy_pause_incoming_continued(void) "" +postcopy_page_req_sync(uint64_t host_addr) "sync page req 0x%"PRIx64 =20 # vmstate.c vmstate_load_field_error(const char *field, int ret) "field \"%s\" load fa= iled, ret =3D %d" --=20 2.26.2