From nobody Sat May 18 02:22:07 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1644514079; cv=none; d=zohomail.com; s=zohoarc; b=FQTMeyL6SLJwJRYAT3OIoNHMMGwA/xdUfKvir+9YKhfr25Y85K19zOK9zd7ZUtcBPEwevT3R0sQY/C6zAmrE7S26GR2bWZxexhm2qhSvMbpBTLfzasSRJU8wve7ojivRdBcGGGnZA0fdV4QrDwwr1s55Z+mBKy7CTjKrceDBVrw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1644514079; h=Content-Type:Content-Transfer-Encoding:Date:From:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:To; bh=Px77cD0FejDu6XuNndK0FPvrCK+eDUGXBcpEO5pgJPI=; b=fDyZvsM/to/pe9Tb5ZcA3nZjWR7+X+CFPOETJlhVDjHqZA8DwyIshi2x55XuwnlaFVFp0lypL8nQF2P2S+Sq8QI1GcMp2WB1SqpMmTKjBt8E7/1lZeQSjX5E49It8QpeMqIwPfgCnSopblarLZ4lcTbSFfjwAl4vU25LnGyW+gs= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1644514078991396.53352576616953; Thu, 10 Feb 2022 09:27:58 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.269941.464091 (Exim 4.92) (envelope-from ) id 1nIDE2-0005ez-Cr; Thu, 10 Feb 2022 17:27:22 +0000 Received: by outflank-mailman (output) from mailman id 269941.464091; Thu, 10 Feb 2022 17:27:22 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1nIDE2-0005es-9p; Thu, 10 Feb 2022 17:27:22 +0000 Received: by outflank-mailman (input) for mailman id 269941; Thu, 10 Feb 2022 17:27:20 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1nIDE0-0005em-GW for xen-devel@lists.xenproject.org; Thu, 10 Feb 2022 17:27:20 +0000 Received: from mail-qt1-x82c.google.com (mail-qt1-x82c.google.com [2607:f8b0:4864:20::82c]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id b1e030d6-8a96-11ec-8f75-fffcc8bd4f1a; Thu, 10 Feb 2022 18:27:18 +0100 (CET) Received: by mail-qt1-x82c.google.com with SMTP id k25so6064689qtp.4 for ; Thu, 10 Feb 2022 09:27:18 -0800 (PST) Received: from development (c-73-166-253-254.hsd1.tx.comcast.net. [73.166.253.254]) by smtp.gmail.com with ESMTPSA id de43sm9781703qkb.4.2022.02.10.09.27.16 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Feb 2022 09:27:16 -0800 (PST) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: b1e030d6-8a96-11ec-8f75-fffcc8bd4f1a DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:subject:from:to:date:user-agent:mime-version :content-transfer-encoding; bh=Px77cD0FejDu6XuNndK0FPvrCK+eDUGXBcpEO5pgJPI=; b=DlrBbAIeZ0CUD+nbjiaqZ6oYHEee3N47hOmnqKvGv1PkCECOfVqo6KoMAhluN24VbK Fb5scT5xHBDhffaBfpKF16Pgx/7bJuqPfjfF/ehhPsBxZhepelOpTcOFDqYT3dPycT6q exUXOLtiLfg3rA8CmGZIWMZjiXwvmq2YnZs93UmupZfdV3MQBamgzGYdJ7Md3Ihcj7gJ Rpm0prpXjiQXas8iRQ5U5LYSzRb68iFR6QtMKfobyLGK5AXHreMLLhBQD9bKbb/D0TMV SLqpQPvWGL7B9kjRtCGMqCdO5RPTiwnO1WqA+jXqniYJPo1bB7tW+9NduDPil0YexZBN 3f8g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:subject:from:to:date:user-agent :mime-version:content-transfer-encoding; bh=Px77cD0FejDu6XuNndK0FPvrCK+eDUGXBcpEO5pgJPI=; b=mL6GmtK3zcmfhGt/6pmVH6sBGD1/lqIpzuJWHhjIM84u+0lI1qnYCz7QjGdRqP932/ iUyoNd2uxhqPpz9oAKeLlJPryswafU0DQAKSQTx7PVEonaZfmi99FLWKT5AAO/ljdGKs 5qbvx/AZC4JJ3OJefV913MgYZXYZcCOFweQ03/QnKdJhl2brr1Rl3TY0rjyUjtNajkzW cy2AlqpOaDRLKbi84CcP+XKpgAEz2EX7Q7O8PytcPgwmoG4DWEMcGIeSZ6dB0Kij0vAO HwZ6e39BomEWDCl1JEpj1MmAg656kF9PkrpRLwOYCcz1m6HxCKAIz9vqNuOfeR8FqKS9 Vadw== X-Gm-Message-State: AOAM531t1eJWElXbeypBhWP+vp5Z244/fREYDWS+5G8ZSbyvdW5BH6yN cMzEK3Ol4vYEidgv+X3vh04znUf754gMcQ== X-Google-Smtp-Source: ABdhPJzcHqnBlz+AlHEqOuJWO+6b4zYfnmMaNfsuC9MoBxFlNJwGBuKKjsksUSa00VKuK6Vq3/1gDg== X-Received: by 2002:a05:622a:349:: with SMTP id r9mr5683472qtw.37.1644514036893; Thu, 10 Feb 2022 09:27:16 -0800 (PST) Message-ID: <949b4776e23e4607776685a7e2705b9e77f5b717.camel@gmail.com> Subject: [RFC] Avoid dom0/HVM performance penalty from MSR access tightening From: Alex Olson To: xen-devel@lists.xenproject.org Date: Thu, 10 Feb 2022 11:27:15 -0600 User-Agent: Evolution 3.36.5-0ubuntu1 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1644514083629100001 Content-Type: text/plain; charset="utf-8" I'm seeing strange performance issues under Xen on a Supermicro server with= a Xeon D-1541 CPU caused by an MSR-related commit. Commit 322ec7c89f6640ee2a99d1040b6f786cf04872cf 'x86/pv: disallow access to= unknown MSRs' surprisingly introduces a severe performance penality where dom0 has about = 1/8th the normal CPU performance. Even even when 'xenpm' is used to select the performance governor and operate the CPU at maximum frequency, actual CPU performance is still 1/2 of normal (as well as using "cpufreq=3Dxen,perform= ance"). The patch below fixes it but I don't fully understand why. Basically, when *reads* of MSR_IA32_THERM_CONTROL are blocked, dom0 and guests (pinned to other CPUs) see the performance issues. For benchmarking purposes, I built a small C program that runs a "for loop"=20 4Billion iterations and timed its execution. In dom0, the performance issues also cause HVM guest startup time to go from 9-10 seconds to almost 80 seconds. I assumed Xen was managing CPU frequency and thus blocking related MSR access by dom0 (or any other domain). However, clearly something else is happening and I don't understand why. I initially attempted to copy the same logic as the write MSR case. This was effective at fixing the dom0 performance issue, but still left other domains running at 1/2 speed. Hence, the change below has no access control. If anyone has any insight as to what is really happening, I would be all ea= rs as I am unsure if the change below is a proper solution. Thanks -Alex --- --- xen/arch/x86/pv/emul-priv-op.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/xen/arch/x86/pv/emul-priv-op.c b/xen/arch/x86/pv/emul-priv-op.c index 7f4279a051..f254479bda 100644 --- a/xen/arch/x86/pv/emul-priv-op.c +++ b/xen/arch/x86/pv/emul-priv-op.c @@ -970,6 +970,18 @@ static int read_msr(unsigned int reg, uint64_t *val, *val =3D 0; return X86EMUL_OKAY; =20 + /* being unable to read MSR_IA32_THERM_CONTROL seems to significantly = affect + * dom0 and thus HVM guest startup performance, as well as PVH VMs. + */ + case MSR_IA32_THERM_CONTROL: + case MSR_IA32_ENERGY_PERF_BIAS: + *val =3D 0; + if ( boot_cpu_data.x86_vendor !=3D X86_VENDOR_INTEL ) + break; + if ( rdmsr_safe(reg, *val) =3D=3D 0) + return X86EMUL_OKAY; + break; + case MSR_P6_PERFCTR(0) ... MSR_P6_PERFCTR(7): case MSR_P6_EVNTSEL(0) ... MSR_P6_EVNTSEL(3): case MSR_CORE_PERF_FIXED_CTR0 ... MSR_CORE_PERF_FIXED_CTR2: --=20 2.30.2