From nobody Sun May 19 13:34:33 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1601661330; cv=none; d=zohomail.com; s=zohoarc; b=n3fuJ2ly/IJXwxOGvA7fVxlXDlzZynWx73h0cpmDC1BLpz2Xxk3sEqOtx9FRl53ppMwMNv+0d8VYV903R7Vi95vOdZ+YMy8l++jsuqMDALHBN2T/+zOShqVQBDxFzK22pgJqWDmzkGqrTAvWtY8W4+fQECYDWTi98FeL936OPkQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1601661330; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=n4IR530G0jn3KaGrAMU+kHWU9guPG9Q80wHEffp3OJw=; b=MRmjEfkX8PvlMLee7VUws3XDyYbgA2VKSgrVeFYLfFM/0mcB1sjA2ZrG/YwIvIf63ft/ja4QA+IKMPtp+uTldxq3kp2f3PBmsO36SPdbJVbGx5G9Y40V+mp1eID3r+vSpq8cc7be4CMbpEEFsMym+uKmxyKGiMMyIULmvLV3qjE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1601661330849853.1244120214692; Fri, 2 Oct 2020 10:55:30 -0700 (PDT) Received: from localhost ([::1]:43626 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kOPHB-0002VC-Fe for importer@patchew.org; Fri, 02 Oct 2020 13:55:25 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48080) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kOPFZ-0001I5-0s for qemu-devel@nongnu.org; Fri, 02 Oct 2020 13:53:45 -0400 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:55109) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kOPFX-00056r-AQ for qemu-devel@nongnu.org; Fri, 02 Oct 2020 13:53:44 -0400 Received: from mail-qt1-f198.google.com (mail-qt1-f198.google.com [209.85.160.198]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-346-iHJH8sICNOuIXoPy1FEVUA-1; Fri, 02 Oct 2020 13:53:40 -0400 Received: by mail-qt1-f198.google.com with SMTP id y20so143019qta.6 for ; Fri, 02 Oct 2020 10:53:40 -0700 (PDT) Received: from xz-x1.redhat.com (toroon474qw-lp130-09-184-147-14-204.dsl.bell.ca. [184.147.14.204]) by smtp.gmail.com with ESMTPSA id a3sm1562229qtp.63.2020.10.02.10.53.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 02 Oct 2020 10:53:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1601661222; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=n4IR530G0jn3KaGrAMU+kHWU9guPG9Q80wHEffp3OJw=; b=CpriClChmq0QWDaCZV2w70Vxcg57nFNXim7hp+uzFT6lTTNv9YUWCnfutLTaTsV6oTwPwR l4ZRylWT61SLvqnms4jKsjwf0OggvxAG4TcEzn59dTrFyVgcTxBFo6M8e3dO/bGAB6LqI8 V4bdHtumOPMM3aAlbLmnbE81PHVBCcA= X-MC-Unique: iHJH8sICNOuIXoPy1FEVUA-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=n4IR530G0jn3KaGrAMU+kHWU9guPG9Q80wHEffp3OJw=; b=mJhQgRyLcxqxHL1Pm7AZ9DGM4fheu+SDYWOghrR+7P9ReQT91tXOWqID7/cu2Ops0+ RibKkauuc3Vnv/DWJ9Lm9D+/3Pip+8MwdbDaDsZ/4MVCOjFhMuSh4b5RBrhJDyG4PqDf hwgCS5tr2wZC8IYBdv4Rbghy6ie7zCYgwb+FtsA6Cnsct910kuRlPqVBQblevejQnWYx dlXL5HD1u5p7BRdk6cQu4FB+xRYDlp8bOW8GO7SkZBhDu3w7ZXwEczeh3c2zErAclWHS YazdapSw3oADLkAGf1k9rkCZ6Cy+OBVioIyLcYdM1XlESgbowRoAfWkjkCIzO/lVAeRX QwKQ== X-Gm-Message-State: AOAM530obCJ6V5+MSwRXehxhkufSk28ne0q1kemd6Rn0lpf7WWvoOIyl siRAOv/unQYrpY1VwwJ45eiJYVDcEOuz7L1+4RRAkh0V3sG68T6sALnrLttYjo/kgK4zxbvAR9N eggf9eLlPIZlHbwE= X-Received: by 2002:aed:278a:: with SMTP id a10mr3579714qtd.261.1601661220211; Fri, 02 Oct 2020 10:53:40 -0700 (PDT) X-Google-Smtp-Source: ABdhPJy7DN7dwDxIW3w9Rmt7xM7ZdW13boK2yoAnuKZENMhrjpAOuqT4kZ81tLEmcajYW2Dy1gMpVA== X-Received: by 2002:aed:278a:: with SMTP id a10mr3579691qtd.261.1601661219987; Fri, 02 Oct 2020 10:53:39 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Subject: [PATCH v4 1/4] migration: Pass incoming state into qemu_ufd_copy_ioctl() Date: Fri, 2 Oct 2020 13:53:33 -0400 Message-Id: <20201002175336.30858-2-peterx@redhat.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20201002175336.30858-1-peterx@redhat.com> References: <20201002175336.30858-1-peterx@redhat.com> MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=peterx@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=63.128.21.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/01 23:37:29 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "Dr . David Alan Gilbert" , peterx@redhat.com, Juan Quintela Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" It'll be used in follow up patches to access more fields out of it. Meanwh= ile fetch the userfaultfd inside the function. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu --- migration/postcopy-ram.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index 0a2f88a87d..722034dc01 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -1128,10 +1128,12 @@ int postcopy_ram_incoming_setup(MigrationIncomingSt= ate *mis) return 0; } =20 -static int qemu_ufd_copy_ioctl(int userfault_fd, void *host_addr, +static int qemu_ufd_copy_ioctl(MigrationIncomingState *mis, void *host_add= r, void *from_addr, uint64_t pagesize, RAMBloc= k *rb) { + int userfault_fd =3D mis->userfault_fd; int ret; + if (from_addr) { struct uffdio_copy copy_struct; copy_struct.dst =3D (uint64_t)(uintptr_t)host_addr; @@ -1185,7 +1187,7 @@ int postcopy_place_page(MigrationIncomingState *mis, = void *host, void *from, * which would be slightly cheaper, but we'd have to be careful * of the order of updating our page state. */ - if (qemu_ufd_copy_ioctl(mis->userfault_fd, host, from, pagesize, rb)) { + if (qemu_ufd_copy_ioctl(mis, host, from, pagesize, rb)) { int e =3D errno; error_report("%s: %s copy host: %p from: %p (size: %zd)", __func__, strerror(e), host, from, pagesize); @@ -1212,7 +1214,7 @@ int postcopy_place_page_zero(MigrationIncomingState *= mis, void *host, * but it's not available for everything (e.g. hugetlbpages) */ if (qemu_ram_is_uf_zeroable(rb)) { - if (qemu_ufd_copy_ioctl(mis->userfault_fd, host, NULL, pagesize, r= b)) { + if (qemu_ufd_copy_ioctl(mis, host, NULL, pagesize, rb)) { int e =3D errno; error_report("%s: %s zero host: %p", __func__, strerror(e), host); --=20 2.26.2 From nobody Sun May 19 13:34:33 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1601661331; cv=none; d=zohomail.com; s=zohoarc; b=gf0DsAM7BImBfUG3R2URAbI7Vy2I/H9WwxXXvSZpY+p2U/hTNK5P5u+Eirf7ram/BN6U1k6I7dbw6goyuq2oHXDtpfGDS/jmdsNrQuztw7TVqLYf+1gK6dshDLwMORQ4Wq06PclTesFBsiYvezgxvTbudoPfPPNfJx2KoG4vwPg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1601661331; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=hSeDqMHpB+FHPSAPyKgXEXUo5azyTlLZe7OlYAX6W4M=; b=LSsHycxFIv8DXhbcc0GzzfoRlaHu7FHojgvTJFk8IY/gxB9m1Fx25AOYLK5zogHtLCFV9ASnFbWWTdmuvEUpO9VQ1FS8Nk6KtiNnSK/QXTkfsbsh122uglj+0yRD/DLsGBjs3WGBhRi9CYQUi37ol/i21cF70EZMdHacHyUatDU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1601661331079728.0126828259253; Fri, 2 Oct 2020 10:55:31 -0700 (PDT) Received: from localhost ([::1]:43884 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kOPHD-0002bX-Sd for importer@patchew.org; Fri, 02 Oct 2020 13:55:27 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48106) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kOPFa-0001IW-RN for qemu-devel@nongnu.org; Fri, 02 Oct 2020 13:53:46 -0400 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:57882) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kOPFY-000575-8K for qemu-devel@nongnu.org; Fri, 02 Oct 2020 13:53:46 -0400 Received: from mail-qt1-f199.google.com (mail-qt1-f199.google.com [209.85.160.199]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-138-OkiZA_ivN6eqHlGdn-OsGw-1; Fri, 02 Oct 2020 13:53:42 -0400 Received: by mail-qt1-f199.google.com with SMTP id f5so1568982qtk.11 for ; Fri, 02 Oct 2020 10:53:42 -0700 (PDT) Received: from xz-x1.redhat.com (toroon474qw-lp130-09-184-147-14-204.dsl.bell.ca. [184.147.14.204]) by smtp.gmail.com with ESMTPSA id a3sm1562229qtp.63.2020.10.02.10.53.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 02 Oct 2020 10:53:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1601661223; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hSeDqMHpB+FHPSAPyKgXEXUo5azyTlLZe7OlYAX6W4M=; b=OT+U63VvLxpHCAUuCF4jO/xBZcs39ckU53PNZjEnfVlJmWDassrpWQRNPcN26vEGiEksP/ whYvaD15MSwSddm9bypieAD7zIjrppI6AKH3zO2jGnFJ2+KgGNzlOv5ZD3/I3NKGXrYX8b GHnbHJ5VscaENM+KNNn9GjaidDdPnb4= X-MC-Unique: OkiZA_ivN6eqHlGdn-OsGw-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=hSeDqMHpB+FHPSAPyKgXEXUo5azyTlLZe7OlYAX6W4M=; b=fKTjHd8khOLmzTHxLMkzRDgIlD+4yhbXIC4PAMGLdNv2nwprmsh0KMkAmFshcBDL+p K1qah0n5h9mAVQzSaKhBMWIi4FJtb1bmB4WCh9qfXlyXkCueWJlaqdMNXmhbrFmBctRO XGFukcfbqDqAkoOFUiRSYQkG0bp/OAX8uFl7v17Hjup2ueX6k+X0zhE1Sufd2I608MrU hS7IFe6yS8avTtfj2wtY2YB/wA1Riv8kraO334O70aZJN4WbguFEwYrxyr2XqzNJFKxv 8FdI5KdfMMkNbQxIULqh706YbDXRbxmql5VN311IMtRUMl9qzOaXblZdc93uPovEQNWI /reg== X-Gm-Message-State: AOAM533IP8IVNrHp5ZzpjkeoII7/k0F1bSx/4b6S8L9KOTsFfdPMQjF1 8V1vmdCjywDziqiWe8rTqC5Qofqb/vvrAaIJPjWwkWtUR3KRblzd9Zp18dxyslWfxg8zA9RAqHK HtO30/VYo7jOS13I= X-Received: by 2002:ac8:4658:: with SMTP id f24mr3476265qto.158.1601661221582; Fri, 02 Oct 2020 10:53:41 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxfx/otY6zgjxIY+jEr+ON6ECTGV4tt+ojztdSnRtEZ6F4anDX7tIqREb5Zc56uzJdzUjhCMQ== X-Received: by 2002:ac8:4658:: with SMTP id f24mr3476248qto.158.1601661221383; Fri, 02 Oct 2020 10:53:41 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Subject: [PATCH v4 2/4] migration: Introduce migrate_send_rp_message_req_pages() Date: Fri, 2 Oct 2020 13:53:34 -0400 Message-Id: <20201002175336.30858-3-peterx@redhat.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20201002175336.30858-1-peterx@redhat.com> References: <20201002175336.30858-1-peterx@redhat.com> MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=peterx@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=63.128.21.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/01 23:37:29 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "Dr . David Alan Gilbert" , peterx@redhat.com, Juan Quintela Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" This is another layer wrapper for sending a page request to the source VM. = The new migrate_send_rp_message_req_pages() will be used elsewhere in coming patches. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu --- migration/migration.c | 10 ++++++++-- migration/migration.h | 2 ++ 2 files changed, 10 insertions(+), 2 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index aca7fdcd0b..b2dac6b39c 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -316,8 +316,8 @@ error: * Start: Address offset within the RB * Len: Length in bytes required - must be a multiple of pagesize */ -int migrate_send_rp_req_pages(MigrationIncomingState *mis, RAMBlock *rb, - ram_addr_t start) +int migrate_send_rp_message_req_pages(MigrationIncomingState *mis, + RAMBlock *rb, ram_addr_t start) { uint8_t bufc[12 + 1 + 255]; /* start (8), len (4), rbname up to 256 */ size_t msglen =3D 12; /* start + len */ @@ -353,6 +353,12 @@ int migrate_send_rp_req_pages(MigrationIncomingState *= mis, RAMBlock *rb, return migrate_send_rp_message(mis, msg_type, msglen, bufc); } =20 +int migrate_send_rp_req_pages(MigrationIncomingState *mis, + RAMBlock *rb, ram_addr_t start) +{ + return migrate_send_rp_message_req_pages(mis, rb, start); +} + static bool migration_colo_enabled; bool migration_incoming_colo_enabled(void) { diff --git a/migration/migration.h b/migration/migration.h index deb411aaad..e853ccf8b1 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -333,6 +333,8 @@ void migrate_send_rp_pong(MigrationIncomingState *mis, uint32_t value); int migrate_send_rp_req_pages(MigrationIncomingState *mis, RAMBlock *rb, ram_addr_t start); +int migrate_send_rp_message_req_pages(MigrationIncomingState *mis, + RAMBlock *rb, ram_addr_t start); void migrate_send_rp_recv_bitmap(MigrationIncomingState *mis, char *block_name); void migrate_send_rp_resume_ack(MigrationIncomingState *mis, uint32_t valu= e); --=20 2.26.2 From nobody Sun May 19 13:34:33 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1601661521; cv=none; d=zohomail.com; s=zohoarc; b=G4Dx9VlTsi9C0ShYBGFjD6I9f9SkU7kkwr3ZM6kZrAYrAywtWpZeMLG4Xd9//lgQ5/h/h7wTH7Pa7VmgG2P95cKCvssiuSl0hjEx3eqsyLqgSYYS/+Wa95Lhp/0VRe1rk7xSVMYg//Yo3AwzQl+/IxHRWupnQi9jK90AD0Y209k= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1601661521; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=0Ie6DlgGu7L1VKQQrv8O11rbSk9IFIx5+vdovmVLcaw=; b=AVsQXFRfUQUeL/iEJVOkejKL9Cp2blNoX2j79iJUfLTR614nb8VYAoFiIFzRZMgf5pwkf+sCRLr0cLc/iEiaUwobyKminRFAYJA8LalE+Y3BIXObPkuPDgYm0cx2NqjvM6OYACilkJ0bKl9AzGitqJNq4fpBTcc72yjauomgbmg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1601661521396618.8960375077723; Fri, 2 Oct 2020 10:58:41 -0700 (PDT) Received: from localhost ([::1]:54656 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kOPKJ-00074e-C0 for importer@patchew.org; Fri, 02 Oct 2020 13:58:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48124) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kOPFc-0001Lv-U1 for qemu-devel@nongnu.org; Fri, 02 Oct 2020 13:53:48 -0400 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:41437) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kOPFa-00057R-Mg for qemu-devel@nongnu.org; Fri, 02 Oct 2020 13:53:48 -0400 Received: from mail-qv1-f72.google.com (mail-qv1-f72.google.com [209.85.219.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-326-7kehWfHbNfyhKLK1OxqWZQ-1; Fri, 02 Oct 2020 13:53:44 -0400 Received: by mail-qv1-f72.google.com with SMTP id i17so1287104qvj.22 for ; Fri, 02 Oct 2020 10:53:44 -0700 (PDT) Received: from xz-x1.redhat.com (toroon474qw-lp130-09-184-147-14-204.dsl.bell.ca. [184.147.14.204]) by smtp.gmail.com with ESMTPSA id a3sm1562229qtp.63.2020.10.02.10.53.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 02 Oct 2020 10:53:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1601661225; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0Ie6DlgGu7L1VKQQrv8O11rbSk9IFIx5+vdovmVLcaw=; b=A0BtXOb+q0/pjH1OtRZvBiFiqBmZR2Nof3E30VawEoFuq6ArJCci2FSMEoyfPlMfNDEEqj SwG8ySWVfikeLBsCkXQu9GmZ8N4qKYNFZhdLBBlqeHUcvh1QTT/x6g+op5N4sUnDqDYMUT hleu3iFYWuLIkeaB7O/h3OmPN1n7mWU= X-MC-Unique: 7kehWfHbNfyhKLK1OxqWZQ-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=0Ie6DlgGu7L1VKQQrv8O11rbSk9IFIx5+vdovmVLcaw=; b=OCgBELp6KuAMPqufPjlTNecpkujbuiK7rJ92Xm8sQr9kXHCUvTl/W34U6ARUmNeq/C 2TFtwl3+aIGov/88ptGSh5sRXTan6tByGvuW0urKfO1W4ajzIqtOsb22rVVFUwpOGBBI q+QC4QApgTUq1j8tUHf6thuKy+58FVM9avqHQy1tr7Jv4Hw052E/jU8XZhfKeeql98/E 6rD8NeGUWo2UJsDC+dW9pCwraPK4xoQfGu0iSPnViMShnRtbxYp9GIZb3cR/H0di8QfY 8Avv1noZL+aA2Y0D2x2VfynbvbYJ6EuI1Jz/IRfwKqMnhYt3+suRsZs5udDpqL3ZY0vR SyGQ== X-Gm-Message-State: AOAM531F36MGErosG1K1rKhRdt/nSsOZXYghgLpYMRKxI7yTr8EUoJeT VgdK+Nw78gKaeN0DKqdsuKQu2Ceqm2ZQZEB0qZUyXg4sjXwSXp01w2NwiKYq4xeb0hQA1Ijte13 yJWJtbVTBLlnPDWk= X-Received: by 2002:a05:620a:919:: with SMTP id v25mr3089432qkv.461.1601661223074; Fri, 02 Oct 2020 10:53:43 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxCIHnP5VrQ2FBWTya2FgKLbqueJZdoHdWRkvn5hZZo8ySOaNfKcLKardfR+tjKD9NKD0aWYA== X-Received: by 2002:a05:620a:919:: with SMTP id v25mr3089408qkv.461.1601661222732; Fri, 02 Oct 2020 10:53:42 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Subject: [PATCH v4 3/4] migration: Maintain postcopy faulted addresses Date: Fri, 2 Oct 2020 13:53:35 -0400 Message-Id: <20201002175336.30858-4-peterx@redhat.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20201002175336.30858-1-peterx@redhat.com> References: <20201002175336.30858-1-peterx@redhat.com> MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=peterx@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=63.128.21.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/01 23:37:29 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "Dr . David Alan Gilbert" , peterx@redhat.com, Juan Quintela Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" Maintain a list of faulted addresses on the destination host for which we're waiting on. This is implemented using a GTree rather than a real list to m= ake sure even there're plenty of vCPUs/threads that are faulting, the lookup wi= ll still be fast with O(log(N)) (because we'll do that after placing each page= ). It should bring a slight overhead, but ideally that shouldn't be a big prob= lem simply because in most cases the requested page list will be short. Actually we did similar things for postcopy blocktime measurements. This p= atch didn't use that simply because: (1) blocktime measurement is towards vcpu threads only, but here we need = to record all faulted addresses, including main thread and external thread (like, DPDK via vhost-user). (2) blocktime measurement will require UFFD_FEATURE_THREAD_ID, but here we don't want to add that extra dependency on the kernel version since n= ot necessary. E.g., we don't need to know which thread faulted on which page, we also don't care about multiple threads faulting on the same page. But we only care about what addresses are faulted so waiting f= or a page copying from src. (3) blocktime measurement is not enabled by default. However we need thi= s by default especially for postcopy recover. Another thing to mention is that this patch introduced a new mutex to seria= lize the receivedmap and the page_requested tree, however that serialization does not cover other procedures like UFFDIO_COPY. Signed-off-by: Peter Xu Reviewed-by: Dr. David Alan Gilbert --- migration/migration.c | 41 +++++++++++++++++++++++++++++++++++++++- migration/migration.h | 19 ++++++++++++++++++- migration/postcopy-ram.c | 17 ++++++++++++++--- migration/trace-events | 2 ++ 4 files changed, 74 insertions(+), 5 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index b2dac6b39c..e7d179bffc 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -143,6 +143,13 @@ static int migration_maybe_pause(MigrationState *s, int new_state); static void migrate_fd_cancel(MigrationState *s); =20 +static gint page_request_addr_cmp(gconstpointer ap, gconstpointer bp) +{ + unsigned long a =3D (unsigned long) ap, b =3D (unsigned long) bp; + + return (a > b) - (a < b); +} + void migration_object_init(void) { MachineState *ms =3D MACHINE(qdev_get_machine()); @@ -165,6 +172,8 @@ void migration_object_init(void) qemu_event_init(¤t_incoming->main_thread_load_event, false); qemu_sem_init(¤t_incoming->postcopy_pause_sem_dst, 0); qemu_sem_init(¤t_incoming->postcopy_pause_sem_fault, 0); + qemu_mutex_init(¤t_incoming->page_request_mutex); + current_incoming->page_requested =3D g_tree_new(page_request_addr_cmp); =20 if (!migration_object_check(current_migration, &err)) { error_report_err(err); @@ -240,6 +249,11 @@ void migration_incoming_state_destroy(void) =20 qemu_event_reset(&mis->main_thread_load_event); =20 + if (mis->page_requested) { + g_tree_destroy(mis->page_requested); + mis->page_requested =3D NULL; + } + if (mis->socket_address_list) { qapi_free_SocketAddressList(mis->socket_address_list); mis->socket_address_list =3D NULL; @@ -354,8 +368,33 @@ int migrate_send_rp_message_req_pages(MigrationIncomin= gState *mis, } =20 int migrate_send_rp_req_pages(MigrationIncomingState *mis, - RAMBlock *rb, ram_addr_t start) + RAMBlock *rb, ram_addr_t start, uint64_t had= dr) { + void *aligned =3D (void *)(uintptr_t)(haddr & (-qemu_target_page_size(= ))); + bool received; + + WITH_QEMU_LOCK_GUARD(&mis->page_request_mutex) { + received =3D ramblock_recv_bitmap_test_byte_offset(rb, start); + if (!received && !g_tree_lookup(mis->page_requested, aligned)) { + /* + * The page has not been received, and it's not yet in the page + * request list. Queue it. Set the value of element to 1, so= that + * things like g_tree_lookup() will return TRUE (1) when found. + */ + g_tree_insert(mis->page_requested, aligned, (gpointer)1); + mis->page_requested_count++; + trace_postcopy_page_req_add(aligned, mis->page_requested_count= ); + } + } + + /* + * If the page is there, skip sending the message. We don't even need= the + * lock because as long as the page arrived, it'll be there forever. + */ + if (received) { + return 0; + } + return migrate_send_rp_message_req_pages(mis, rb, start); } =20 diff --git a/migration/migration.h b/migration/migration.h index e853ccf8b1..8d2d1ce839 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -104,6 +104,23 @@ struct MigrationIncomingState { =20 /* List of listening socket addresses */ SocketAddressList *socket_address_list; + + /* A tree of pages that we requested to the source VM */ + GTree *page_requested; + /* For debugging purpose only, but would be nice to keep */ + int page_requested_count; + /* + * The mutex helps to maintain the requested pages that we sent to the + * source, IOW, to guarantee coherent between the page_requests tree a= nd + * the per-ramblock receivedmap. Note! This does not guarantee consis= tency + * of the real page copy procedures (using UFFDIO_[ZERO]COPY). E.g., = even + * if one bit in receivedmap is cleared, UFFDIO_COPY could have happen= ed + * for that page already. This is intended so that the mutex won't + * serialize and blocked by slow operations like UFFDIO_* ioctls. How= ever + * this should be enough to make sure the page_requested tree always + * contains valid information. + */ + QemuMutex page_request_mutex; }; =20 MigrationIncomingState *migration_incoming_get_current(void); @@ -332,7 +349,7 @@ void migrate_send_rp_shut(MigrationIncomingState *mis, void migrate_send_rp_pong(MigrationIncomingState *mis, uint32_t value); int migrate_send_rp_req_pages(MigrationIncomingState *mis, RAMBlock *rb, - ram_addr_t start); + ram_addr_t start, uint64_t haddr); int migrate_send_rp_message_req_pages(MigrationIncomingState *mis, RAMBlock *rb, ram_addr_t start); void migrate_send_rp_recv_bitmap(MigrationIncomingState *mis, diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index 722034dc01..ca1daf0024 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -684,7 +684,7 @@ int postcopy_request_shared_page(struct PostCopyFD *pcf= d, RAMBlock *rb, qemu_ram_get_idstr(rb), rb_offset); return postcopy_wake_shared(pcfd, client_addr, rb); } - migrate_send_rp_req_pages(mis, rb, aligned_rbo); + migrate_send_rp_req_pages(mis, rb, aligned_rbo, client_addr); return 0; } =20 @@ -979,7 +979,8 @@ retry: * Send the request to the source - we want to request one * of our host page sizes (which is >=3D TPS) */ - ret =3D migrate_send_rp_req_pages(mis, rb, rb_offset); + ret =3D migrate_send_rp_req_pages(mis, rb, rb_offset, + msg.arg.pagefault.address); if (ret) { /* May be network failure, try to wait for recovery */ if (ret =3D=3D -EIO && postcopy_pause_fault_thread(mis)) { @@ -1149,10 +1150,20 @@ static int qemu_ufd_copy_ioctl(MigrationIncomingSta= te *mis, void *host_addr, ret =3D ioctl(userfault_fd, UFFDIO_ZEROPAGE, &zero_struct); } if (!ret) { + qemu_mutex_lock(&mis->page_request_mutex); ramblock_recv_bitmap_set_range(rb, host_addr, pagesize / qemu_target_page_size()); + /* + * If this page resolves a page fault for a previous recorded faul= ted + * address, take a special note to maintain the requested page lis= t. + */ + if (g_tree_lookup(mis->page_requested, host_addr)) { + g_tree_remove(mis->page_requested, host_addr); + mis->page_requested_count--; + trace_postcopy_page_req_del(host_addr, mis->page_requested_cou= nt); + } + qemu_mutex_unlock(&mis->page_request_mutex); mark_postcopy_blocktime_end((uintptr_t)host_addr); - } return ret; } diff --git a/migration/trace-events b/migration/trace-events index 338f38b3dd..e4d5eb94ca 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -162,6 +162,7 @@ postcopy_pause_return_path(void) "" postcopy_pause_return_path_continued(void) "" postcopy_pause_continued(void) "" postcopy_start_set_run(void) "" +postcopy_page_req_add(void *addr, int count) "new page req %p total %d" source_return_path_thread_bad_end(void) "" source_return_path_thread_end(void) "" source_return_path_thread_entry(void) "" @@ -272,6 +273,7 @@ postcopy_ram_incoming_cleanup_blocktime(uint64_t total)= "total blocktime %" PRIu postcopy_request_shared_page(const char *sharer, const char *rb, uint64_t = rb_offset) "for %s in %s offset 0x%"PRIx64 postcopy_request_shared_page_present(const char *sharer, const char *rb, u= int64_t rb_offset) "%s already %s offset 0x%"PRIx64 postcopy_wake_shared(uint64_t client_addr, const char *rb) "at 0x%"PRIx64"= in %s" +postcopy_page_req_del(void *addr, int count) "resolved page req %p total %= d" =20 get_mem_fault_cpu_index(int cpu, uint32_t pid) "cpu: %d, pid: %u" =20 --=20 2.26.2 From nobody Sun May 19 13:34:33 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1601661400; cv=none; d=zohomail.com; s=zohoarc; b=GO3CSKyJpMeJr0DWOWgyCaWKIgBaFBFXefBA61Pk9dAWqK3j5wazFbibTX+Ij4emH/aF2G9kfmNpo2wEt5LHfDqLNwevIX0Pxnecf44Bzi5ndzRQWnRh4rMM5MB3Xc8oKxyskMTNt5i5UNXWcpJ2Imww+9OEOmEk+qebM1E81yo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1601661400; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=Vx+Hot1wSXytFGZn9Y3x89aH6Pp0nwmA/pTo/g6zg/M=; b=ek+I3Ne3oYehMKTuOGUVVdCtRqTmT2Xqg+ab6svTn9p8ib9GHCT4dEyL9SGqXBXGqItbLnPbfKzQud+zqoTtc/EmLP4dlXpYfbCFTN+aEO5v0PV2L4E3WhFGpcGEwIVrBu2tPLWCthF3qbvBATN21UWFcfSHbcehTMfx+KaNWC0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1601661400179512.6735877883605; Fri, 2 Oct 2020 10:56:40 -0700 (PDT) Received: from localhost ([::1]:47460 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kOPIM-00047O-SH for importer@patchew.org; Fri, 02 Oct 2020 13:56:38 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48368) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kOPGp-0002Mz-5z for qemu-devel@nongnu.org; Fri, 02 Oct 2020 13:55:04 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:55206) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kOPGl-0005El-SQ for qemu-devel@nongnu.org; Fri, 02 Oct 2020 13:55:02 -0400 Received: from mail-qt1-f197.google.com (mail-qt1-f197.google.com [209.85.160.197]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-217-LwwSpMGHMv60mEQA7wJP6g-1; Fri, 02 Oct 2020 13:53:44 -0400 Received: by mail-qt1-f197.google.com with SMTP id 60so1550892qtf.21 for ; Fri, 02 Oct 2020 10:53:44 -0700 (PDT) Received: from xz-x1.redhat.com (toroon474qw-lp130-09-184-147-14-204.dsl.bell.ca. [184.147.14.204]) by smtp.gmail.com with ESMTPSA id a3sm1562229qtp.63.2020.10.02.10.53.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 02 Oct 2020 10:53:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1601661299; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Vx+Hot1wSXytFGZn9Y3x89aH6Pp0nwmA/pTo/g6zg/M=; b=ReOcH29ysagrJrLRROC7huBbWaeZTmIKPqe3Hjs3xOz+rCIOuTHkz2pYet1+IvyGsQjo5V /pVucsOg6N8HmboCgDvj3ZpsHybx7hAYBNmrCBc6+M4n3U8758pWPBSgTnP51krTyNyZqI YfebhWzpCmi9YMKTP4MY8bWsAa9gOOg= X-MC-Unique: LwwSpMGHMv60mEQA7wJP6g-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Vx+Hot1wSXytFGZn9Y3x89aH6Pp0nwmA/pTo/g6zg/M=; b=UVVKnFBjonFilePhTBCJ8bpNfEQswJcdOeKrKSpBERpss36av2UiYZXebwyMGhUZQ8 SOggbUnph4jsIXmm2sErq0KT7GpCOAMG/6dg9zipnT7fDd2/SNX8uaaBw5XFFQNY1Z7y 2N1BSwTbCYnYuuPeaEqbMJP5XVTWq1GCs7wCo2ZUysLJLZToP09fCsbZGKZ2EKGcVIw/ TE4yfrK7P5njBTHS5nhlWdn2SNsrpk2mRIjxWuLtGyJ6+5MQWwjTtEhhxh8G2KBsNdAb n7Xj/v384CB/ER/uH19yD14f0ZVFpLv2iKRqDLx1t46v7Ku/nyfnojqGAVs0tixkAGTe Uo0g== X-Gm-Message-State: AOAM530242Jrapbek7eMUTlSDfQoyqwYSX5zHT3NCcIXeJpaPcOYm9ZW UdgerKBXTWTj/kAxoczNVZccbRO7vYVTJvkzda+9z0BJRG9BMBvvGOttRr2RcKDTdCprhjeu/Nk oF6jjcq15SrLZfVo= X-Received: by 2002:ad4:52e3:: with SMTP id p3mr3072689qvu.42.1601661224157; Fri, 02 Oct 2020 10:53:44 -0700 (PDT) X-Google-Smtp-Source: ABdhPJy68Blgtf9zKN5k5AKJUK220BH4aZWwCjaOC0kiM0FUXYTB/HW8Sop5hbRsjmYWw16udz49wg== X-Received: by 2002:ad4:52e3:: with SMTP id p3mr3072670qvu.42.1601661223911; Fri, 02 Oct 2020 10:53:43 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Subject: [PATCH v4 4/4] migration: Sync requested pages after postcopy recovery Date: Fri, 2 Oct 2020 13:53:36 -0400 Message-Id: <20201002175336.30858-5-peterx@redhat.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20201002175336.30858-1-peterx@redhat.com> References: <20201002175336.30858-1-peterx@redhat.com> MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=peterx@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/02 01:13:31 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Xiaohui Li , "Dr . David Alan Gilbert" , peterx@redhat.com, Juan Quintela Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" We synchronize the requested pages right after a postcopy recovery happens. This helps to synchronize the prioritized pages on source so that the fault= ed threads can be served faster. Reported-by: Xiaohui Li Signed-off-by: Peter Xu Reviewed-by: Dr. David Alan Gilbert --- migration/savevm.c | 57 ++++++++++++++++++++++++++++++++++++++++++ migration/trace-events | 1 + 2 files changed, 58 insertions(+) diff --git a/migration/savevm.c b/migration/savevm.c index 34e4b71052..1dc021ee53 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -2011,6 +2011,49 @@ static int loadvm_postcopy_handle_run(MigrationIncom= ingState *mis) return LOADVM_QUIT; } =20 +/* We must be with page_request_mutex held */ +static gboolean postcopy_sync_page_req(gpointer key, gpointer value, + gpointer data) +{ + MigrationIncomingState *mis =3D data; + void *host_addr =3D (void *) key; + ram_addr_t rb_offset; + RAMBlock *rb; + int ret; + + rb =3D qemu_ram_block_from_host(host_addr, true, &rb_offset); + if (!rb) { + /* + * This should _never_ happen. However be nice for a migrating VM= to + * not crash/assert. Post an error (note: intended to not use *_o= nce + * because we do want to see all the illegal addresses; and this c= an + * never be triggered by the guest so we're safe) and move on next. + */ + error_report("%s: illegal host addr %p", __func__, host_addr); + /* Try the next entry */ + return FALSE; + } + + ret =3D migrate_send_rp_message_req_pages(mis, rb, rb_offset); + if (ret) { + /* Please refer to above comment. */ + error_report("%s: send rp message failed for addr %p", + __func__, host_addr); + return FALSE; + } + + trace_postcopy_page_req_sync(host_addr); + + return FALSE; +} + +static void migrate_send_rp_req_pages_pending(MigrationIncomingState *mis) +{ + WITH_QEMU_LOCK_GUARD(&mis->page_request_mutex) { + g_tree_foreach(mis->page_requested, postcopy_sync_page_req, mis); + } +} + static int loadvm_postcopy_handle_resume(MigrationIncomingState *mis) { if (mis->state !=3D MIGRATION_STATUS_POSTCOPY_RECOVER) { @@ -2033,6 +2076,20 @@ static int loadvm_postcopy_handle_resume(MigrationIn= comingState *mis) /* Tell source that "we are ready" */ migrate_send_rp_resume_ack(mis, MIGRATION_RESUME_ACK_VALUE); =20 + /* + * After a postcopy recovery, the source should have lost the postcopy + * queue, or potentially the requested pages could have been lost duri= ng + * the network down phase. Let's re-sync with the source VM by re-sen= ding + * all the pending pages that we eagerly need, so these threads won't = get + * blocked too long due to the recovery. + * + * Without this procedure, the faulted destination VM threads (waiting= for + * page requests right before the postcopy is interrupted) can keep ha= nging + * until the pages are sent by the source during the background copyin= g of + * pages, or another thread faulted on the same address accidentally. + */ + migrate_send_rp_req_pages_pending(mis); + return 0; } =20 diff --git a/migration/trace-events b/migration/trace-events index e4d5eb94ca..0fbfd2da60 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -49,6 +49,7 @@ vmstate_save(const char *idstr, const char *vmsd_name) "%= s, %s" vmstate_load(const char *idstr, const char *vmsd_name) "%s, %s" postcopy_pause_incoming(void) "" postcopy_pause_incoming_continued(void) "" +postcopy_page_req_sync(void *host_addr) "sync page req %p" =20 # vmstate.c vmstate_load_field_error(const char *field, int ret) "field \"%s\" load fa= iled, ret =3D %d" --=20 2.26.2