From nobody Fri May 17 09:01:31 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass header.i=@wdc.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=wdc.com); dmarc=fail(p=quarantine dis=none) header.from=wdc.com ARC-Seal: i=2; a=rsa-sha256; t=1710495450; cv=pass; d=zohomail.com; s=zohoarc; b=aridPiJE58yvsOYvWIJOJturiPtxiBzi8r7EMnz34TqPkzpvY8ULcQyeIVJ1aXqhUCd5fu6FLSjKsmcFNwIND+QnxeXdmsY785erW7iVS8KrsRFZvA4Zvr2Emg+ytqdA0smUKDSyrcMiwkGRVjRuBQlcTujAD9tR9WgEOJgmNV0= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1710495450; h=Content-ID:Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=/gV9w7K+UxXbMJgFMjMQxOAsPyVcPJGHbxV8Wp1W8XY=; b=Szoahwsr6Cjk9dS2cJva8vlWEE4jQ71Yst33+ValPihlccjf53WZbiVtdCxKMZLw3dgKABZIeLEszDcN/mm4Hendr8Vqu+mse00GryQ2K45llQXdWpbT9XXCpx42GQPnxpItD9mJ6mTX8OFf5OBOcIROo3cg9OGa7QuCp0DTres= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass header.i=@wdc.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=wdc.com); dmarc=fail header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1710495450116474.1603618518234; Fri, 15 Mar 2024 02:37:30 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rl3zK-0001mX-G9; Fri, 15 Mar 2024 05:36:30 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rl3zI-0001m4-KL for qemu-devel@nongnu.org; Fri, 15 Mar 2024 05:36:28 -0400 Received: from esa3.hgst.iphmx.com ([216.71.153.141]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rl3zG-000532-9F for qemu-devel@nongnu.org; Fri, 15 Mar 2024 05:36:28 -0400 Received: from mail-dm6nam12lp2169.outbound.protection.outlook.com (HELO NAM12-DM6-obe.outbound.protection.outlook.com) ([104.47.59.169]) by ob1.hgst.iphmx.com with ESMTP; 15 Mar 2024 17:36:20 +0800 Received: from BYAPR04MB5431.namprd04.prod.outlook.com (2603:10b6:a03:ce::16) by PH0PR04MB7526.namprd04.prod.outlook.com (2603:10b6:510:58::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7386.22; Fri, 15 Mar 2024 09:36:17 +0000 Received: from BYAPR04MB5431.namprd04.prod.outlook.com ([fe80::2ab2:43a3:658b:b8c9]) by BYAPR04MB5431.namprd04.prod.outlook.com ([fe80::2ab2:43a3:658b:b8c9%7]) with mapi id 15.20.7386.021; Fri, 15 Mar 2024 09:36:16 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1710495386; x=1742031386; h=from:to:cc:subject:date:message-id:content-id: content-transfer-encoding:mime-version; bh=/gV9w7K+UxXbMJgFMjMQxOAsPyVcPJGHbxV8Wp1W8XY=; b=QOwdlt9JcUXdwbM2YxeO7khnShV7jtgfKB18MZyHJK0cZlWY1PKI/jXT 3lMrSstC69a/3bXttAjz8EKcLifFRYoBhVmzvBZnT7e8NMzkF3sgA6OpX cT8USQc9rJC+tyUi4dLVovEe1eSlZ78q0w/qD3gqcqwugneQeXAHX+cWL ytMtOHHktDcnGuoc8R7P5xgQO5DChLMMSTPEBZDndw6eWg1Fy7lMupWoF Tskdwxug4JncMJB9JGZ2Zs3s1fRz1WarzpcFmMJpoal3ubKFbTnNlbDyf KYHwLOwAgolghGBZqdUsjcE/lB41BM9KpCiN0nX2MtkF8UlmSqiZNRYAU A==; X-CSE-ConnectionGUID: L5mkrpWsT1q1pG7MPsrRwQ== X-CSE-MsgGUID: wAJpL+3HSvep4hNJHvcYOw== X-IronPort-AV: E=Sophos;i="6.07,128,1708358400"; d="scan'208";a="11511678" ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=LsLYBwFvcdZOpTRMi31nxB5wBmQ8l8Fm3zaJ+MHXn10hD9e+c045c5aUB0QEpkbJGrMdGLqi88IH9v7RTnxYqTZP93sRApWOTy4PKj3y6KcWe4qNCNad5EQV86wI1xOyKTmuAAx+nOFsDicaxAsfhUbWTnsToQ7mjJ9NsGt2yYm6KBuMJ9zs4SLN1RRHhGTX5I3G5yb3UE19GikO4ERPotV1sCAkVwUfsGKhXAOLBJKXZa36mVrDuAYzInkdNmxKIRSvqHEjL8/CjBwD98uPKHlUrin/aC9tbnNjAXD2dWwxqMMLrUhgsTGgJwB+vHQLsKqVIKsxvyeWkP1Q6LZVWw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=/gV9w7K+UxXbMJgFMjMQxOAsPyVcPJGHbxV8Wp1W8XY=; b=GbxULhgnaehHr4z7Mbf19LgtWPW3VRggD1TQxBczE8n7DPCH4yPTg7fcFjtDDk6gdlFfQtrjykx3sCgb9BYVfp4GkXzJrIB2EZwcYkkhA6hnWuys7ILignsflutumOzIqPLD79WC6US95+8HYUZR6oo0y6PUfg0t+RyO5PlxvzMgPtsxIK37CoVgJJVsB+rWtLN/LVx5KLVvWST6FNnxbJUi1os9Ioi7rPWyaZr0g+ppGJqRDdmezqNHpgcgw8uuihHYrKpo3HNijhkis3sFkQKB7vI0rwuLOpLRNVsWQOd4OKkVZ9pYl+diVkiUoW+GJOPF3jc+EhlLWZTuFkblCg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=wdc.com; dmarc=pass action=none header.from=wdc.com; dkim=pass header.d=wdc.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sharedspace.onmicrosoft.com; s=selector2-sharedspace-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=/gV9w7K+UxXbMJgFMjMQxOAsPyVcPJGHbxV8Wp1W8XY=; b=LBeaxjkNNaQvUUitBsYufuVEhA2GBVzyZKqzhtSMO0HvgYBND3QdG/ncAss3v1FcYmCzd/r8yhBSwE5vv335pJDj8b+s6eXp4HvMNkqKU1sjxjFjd4l8ESkGwSHoDAJp0/63HF68u2khlDLHbUoExTk9HdGtXiDhjk1A6Y2uUaU= From: =?utf-8?B?SsO4cmdlbiBIYW5zZW4=?= To: "qemu-devel@nongnu.org" , "peter.maydell@linaro.org" , "richard.henderson@linaro.org" , Jonathan Cameron CC: "linux-cxl@vger.kernel.org" Subject: Another CXL/MMIO tcg tlb corner case Thread-Topic: Another CXL/MMIO tcg tlb corner case Thread-Index: AQHadrw6KncL+X/w106el4Z9yvqctA== Date: Fri, 15 Mar 2024 09:36:16 +0000 Message-ID: <33748BB7-E617-4661-BDE3-5D29780FC9FF@wdc.com> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=wdc.com; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: BYAPR04MB5431:EE_|PH0PR04MB7526:EE_ x-ms-office365-filtering-correlation-id: 23242598-8316-48d1-0193-08dc44d35cfc wdcipoutbound: EOP-TRUE x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: HMYNOezN/4++ezSt/tsyJRPU/HQ6Soknqyzh4g4/rMNYYDdYHM3mN8wHg6JrmzJstjAjTP9LthkJJusxOPJUkaDTDVOgBXbk/EVh6fgpbkX+y3uDMjI+waZ6ibHZGoTj5jgQiZeunOEnDRgSSDVlB4RRdueJFjritsQxLMDTQqlvVUfhICwUjOPwZPp9fMLv54BTXMOBrTKNCLd8Pt2G7+8Goy3WBHSY6pIHSzNYwhCqcSoPsnRPrm79a1l4P2HPzt4X4WtmSrNiOU9z5i34jfyBFOsr+6J2YNqCi+2L0/xYjeshbGwBHNmpMkRFJilfhaufI/+aWAXW7y6kgYn/QyiGQ4ymbJqizg34IAb2phqyaRr1cqxzZOJVYlXpStAdK9KdTfxsjogtfm4hF6kma8QorRignHeVPA3ZDS21usN1ty37cwqhzcyhgJYsqIjqqUgi4Nfi6u1UoIND/1NXVWYOEXSDVqLe/SpWknfz5nBjTEiH8EhMRVcXUCLIrEQR40ML4ZF67pFPqTt1T+lHIZZrgEEaHB3Gp9LxY75HkxpCTbGwIIgMNMq7+EKce3oOou2i05OJC6ZtK/oRksqx2jnM3sxgL0+z5sCEm+ejx/gQ5xuJafWy2ZXJ17vqADpY1ftUYvSsGn1e9ryclERQUHWCXd1qAjEy8ftHxoTWSPk= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BYAPR04MB5431.namprd04.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230031)(366007)(1800799015)(376005)(38070700009); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?utf-8?B?VzZJbXgxUVE4OGdMZkpUbk0vMklaOC94VVo5VlBVTWlNaHNjQ2RTY0h0ZlM0?= =?utf-8?B?NlovemgwNWxiN2tVa1pxWjMvVWl3Tmg1RDVDQkJxZVAvZ05mVFVzL1BtbG9U?= =?utf-8?B?T3JXaXh1OGZvb3ZCL0RCenF4UFJvKzU0N2xuaWQ2NmJZdUVtWVlUZUFoWFNx?= =?utf-8?B?ZjcxNENqMzlXSm5odzhNNk1tTVdGditUb2Y3M2pxWU9LNGoyRFFuU1pLMFBT?= =?utf-8?B?TUFjMjF6Ry96NG9zUWVHVUdLc3lOSkVuTC96bURmK3lySUZlU1NreTZ3dlRk?= =?utf-8?B?V0dpNGk0dDJJTURmODFHZlJXZHFCK2YyRlpwdFNkVFVJbHU0Mk1yYnh2aTN0?= =?utf-8?B?VGswR1ZVMG5SZmwvWkNSOTk1clJ1NEZEVHNNU3dJMWoxcG9ZT3B6R1B3TnVQ?= =?utf-8?B?cjAxb2pXLzJQQXpGc0hPSWVqYzZNcXNhSTVCU1prMnhlV2hJeFY0aTZ0RkpY?= =?utf-8?B?a2tNK3FGN2xFb05jeHpGZ1luQm80c043bTBkY2pPc1JmU0VRRWdpcDJUb1kz?= =?utf-8?B?NUdaYlk4WWpUS01Kc0sxYlEyUG8vSzAwejFpWUp6SWFuSUgxckpCdTF6bGRr?= =?utf-8?B?UEZpN3ZoM1d3T3N0eEhEVjZDaENaSjljbnlpeG4vaWpBREFuM3FORUN1MWtP?= =?utf-8?B?QWcveW1EZTc5NkdVUWFQNDF5aHdGbW1qdWkyQnpZZ0duVlhXS3pSWWZFZW8z?= =?utf-8?B?MlF0SVFHOFU3bWFFS1cxc0dWQk44c2tkK3N5ZFM4U05icEk1cDR5SXRNNGVK?= =?utf-8?B?OFc1ZFdvZDBNazNGaEgwL3Jyb3Y5U0FkK1dPczNKTFdBUWVIUEtGMy83VGRZ?= =?utf-8?B?QnFPZXhXVXJ2QUhGa3pGb1JmY09KVUwvNEZRQm5RR2NRTVhHcGp2eGxCNFZN?= =?utf-8?B?UkNkSE16MnJYdXlLR1BqUndxclFWRHlUdEl6ZmVHQUQzTFhFUllpeGw3ajB1?= =?utf-8?B?Vkt5WW1udVY2Tkp6d0NjNmlCcmVoeDNkTkdVS2dvcHo3NUQ4M3VGd1RGT0F2?= =?utf-8?B?QUE4Z0F3Q0oxelZ0VURRZXV5RStsT3hqSFdLc3BnbjJMVjdHcVNLWStnVlA1?= =?utf-8?B?WVMxUXJFdUg0bER6bXJmVkY3d014Tm8vTER6MVFIQ0wvVWV3NVZNeHI4ZEpx?= =?utf-8?B?U3RSNXMwQ1RNM3p2NmttR0lHaFEvclZQb1hLOVdGNWRQS2toZ09qcE05Z29p?= =?utf-8?B?NU1kRXZtUm9MdDV0K1BWeG1Rd243SGxwMEl4ZVBmdzM3ZGFtNm5SZ0JOeXJO?= =?utf-8?B?YVNDemNkWGI4eGU0L2hsYUdLb3UyTFZoUHVLQi9DbHJ6LzlwNzZPR3BxSktI?= =?utf-8?B?OTVVRCtsemo4Nld5VXNEQk9vbk9lT0NHN2FndmtYeUt2YklxbmgxUU1rUHFh?= =?utf-8?B?S1pUaUJJMGdCbTdSUEd6S0xwemFoUFhrZTJvNGVjODJ4UjRxMmx5Q0N6aU9I?= =?utf-8?B?OWNQUXBDM0s0QmF6a0oyejVsMUV6VzhWOWZUZklid0hWYWxnamVsYWV5UnJF?= =?utf-8?B?MkVtWFkzUGhlbjJtK1dTYkRTNDJDVGdWV2t4SEkyWDJ0SkgyYVN5NlBsN2ZV?= =?utf-8?B?dm9kQTUrb2xlWVR1bHk2YTF2N2Jqb2tZcXBnRmpGMzNjZHhuZjU1WnJJalB1?= =?utf-8?B?cXpJTE5lSjNIRG5RWm01eGdhaUhyVXc1SUVqR2c3WEZXMDd6RmRuQUo1Z1Nk?= =?utf-8?B?QnJBYlZrL3ovQnRJdkpveWRCdVVaUGVZNVplNTJjN0ZZT0lRUXRiUzNhb0JM?= =?utf-8?B?UFdGTVZhemE2MHdPVkZJb21XekY4REZhK0dDYldiRnlRZHFSY2NuL3FFWmRp?= =?utf-8?B?NjhHTWhiQUE4ZGViSURPMlJlWkQvK21Edk5yQm1UdStmdElqTkhYOGNOTmZt?= =?utf-8?B?d0pZZ0hXRnJWcmRpamJCTXgvUFpQcC8yamRjMzRBNmlTMmlFU3RMK3dReHB6?= =?utf-8?B?SlREQklxQ1JrYjB5K1JWVHZ1WnNqSWNIZ0swT1pMYmtYK3M4RG5KUWFJZTRz?= =?utf-8?B?RHEvdzlTaEo5WlluaUtTbHB4c3hFemsvUWZDMXNSSnZ3RjhnWWpRdW5yWmxW?= =?utf-8?B?bU14d2gzUXM5UHZUbWx6dFh2YkdRM2sxMnFVdEkrMGdsRmJWeEdwY1JMeVdw?= =?utf-8?B?b3hiclZiNkpJQ0EzQkI1STZVa0N4cHlKTXd1QWFCSWRrcENVcVVvcFhJcU0x?= =?utf-8?B?VEE9PQ==?= Content-Type: text/plain; charset="utf-8" Content-ID: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: X0Tf4KZfcm3MWRGwKYEt0hL6BLajXL7kXe4qDnqJCcy1VwjkgMz0zb7s3yKiVLtVMawQc9byt5wq75uzJBRa45tA5IO9kd/vQX7SX3t2XaIN20JnwGmU60GbClxDfL1T4K8G/SyFjlLjuISfrZpTU+XPMzwVzLu1WsgghPoRvOwEv+YACh/TvnLD7tcAIsXZmufgwEAg2f+i17vh7ybW0ZVRvM87xDl/qqxdgLYOHXOkt+PKsz9NsoZFCqqoc3sek5ktoAnH6mjpLOtHso324AASQtvcP6o7oBjpjphWWnZDgZH98CqVz8Q2nYm0jwSbPpDbpxlEyE8V091/JSq7Y5sPEqMaqunoZeU9h4U+Chn/XalNOkxE5pe2vja8msco+ayHlSLJwKfbGq6ku5oFxkwkXS+jmOZhInd9igT/7Kltf7H8aN3Zpu0s7H6v9K5aS+BkkwwGEH/E0B/cLdfKk2ZgzG979pMRE/+HUXwSZqnC5mF/BPggua5BWJBqMOV3l0YwoM0juNVnJ8/uaTtNp7ltrnIGpk/ubjTQckrWAnAtaCHpczDJk/V3Ujhnu/T89QY1NI9AdWtkYhY0OGlleiKBWmVZO4lwiSqlO6seKx/6jNH6VVDdMHJ9UPHAoGGM X-OriginatorOrg: wdc.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: BYAPR04MB5431.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 23242598-8316-48d1-0193-08dc44d35cfc X-MS-Exchange-CrossTenant-originalarrivaltime: 15 Mar 2024 09:36:16.7945 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: b61c8803-16f3-4c35-9b17-6f65f441df86 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: dszx7PcktgoGGkI14ihiICVH/dvLN1AOObZt5eGpLgrT0fjBpbf6GmQPzZORyS6FklMrlBRv9StzWEvYrRZOzw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR04MB7526 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.71.153.141; envelope-from=prvs=797307635=Jorgen.Hansen@wdc.com; helo=esa3.hgst.iphmx.com X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @sharedspace.onmicrosoft.com) X-ZM-MESSAGEID: 1710495450847100001 Hi, While doing some testing using numactl-based interleaving of application me= mory across regular memory and CXL-based memory using QEMU with tcg, I ran into = an issue similar to what we saw a while back - link to old issue: https://lore.kernel.org/qemu-devel/CAFEAcA_a_AyQ=3DEpz3_+CheAT8Crsk9mOu894w= bNW_FywamkZiw@mail.gmail.com/#t. When running: numactl --interleave 0,1 ./cachebench =E2=80=A6 I hit the following: numactl --interleave 0,1 ./cachebench --json_test_config ../test_configs/hi= t_ratio/graph_cache_follower_assocs/config.json qemu: fatal: cpu_io_recompile: could not find TB for pc=3D0x7fffc3926dd4 RAX=3D00007f65df55ba18 RBX=3D00007f65df55ba60 RCX=3D00007f65df221620 RDX=3D= 0000000000000000 RSI=3D00000000011c0260 RDI=3D00007f65df55ba60 RBP=3D00007ffdb4b4b280 RSP=3D= 00007ffdb4b4b1d0 R8 =3D00000000011c02c0 R9 =3D00007f65debf6b20 R10=3D00000000011bf5d0 R11=3D= 00007f65deb7d300 R12=3D00007ffdb4b4b260 R13=3D00007ffdb4b4b200 R14=3D00007ffdb4b4b220 R15=3D= 00000000011bf5a0 RIP=3D00007f65df18affc RFL=3D00000246 [---Z-P-] CPL=3D3 II=3D0 A20=3D1 SMM= =3D0 HLT=3D0 ES =3D0000 0000000000000000 00000000 00000000 CS =3D0033 0000000000000000 ffffffff 00affb00 DPL=3D3 CS64 [-RA] SS =3D002b 0000000000000000 ffffffff 00cff300 DPL=3D3 DS [-WA] DS =3D0000 0000000000000000 00000000 00000000 FS =3D0000 00007f65de2f64c0 00000000 00000000 GS =3D0000 0000000000000000 00000000 00000000 LDT=3D0000 0000000000000000 00000000 00008200 DPL=3D0 LDT TR =3D0040 fffffe6c37990000 00004087 00008900 DPL=3D0 TSS64-avl GDT=3D fffffe6c3798e000 0000007f IDT=3D fffffe0000000000 00000fff CR0=3D80050033 CR2=3D00007f65df1b3eb0 CR3=3D0000000152a1e000 CR4=3D00350ef0 DR0=3D0000000000000000 DR1=3D0000000000000000 DR2=3D0000000000000000 DR3=3D= 0000000000000000 DR6=3D00000000ffff0ff0 DR7=3D0000000000000400 CCS=3D0000000000000000 CCD=3D0000000000000001 CCO=3DCLR EFER=3D0000000000000d01 FCW=3D037f FSW=3D0000 [ST=3D0] FTW=3D00 MXCSR=3D00001f80 FPR0=3D0000000000000000 0000 FPR1=3D0000000000000000 0000 FPR2=3D0000000000000000 0000 FPR3=3D0000000000000000 0000 FPR4=3D0000000000000000 0000 FPR5=3D0000000000000000 0000 FPR6=3D0000000000000000 0000 FPR7=3D0000000000000000 0000 YMM00=3D0000000000000000 0000000000000000 00007f65df2233e0 00007f65df221620 YMM01=3D0000000000000000 0000000000000000 0000000000000000 43e0000000000000 YMM02=3D0000000000000000 0000000000000000 0000000000000000 0000000000000000 YMM03=3D0000000000000000 0000000000000000 0000000000000000 0000000000000000 YMM04=3D0000000000000000 0000000000000000 0000000000000000 3ff0000000000000 YMM05=3D0000000000000000 0000000000000000 0000000000000000 00007f65df2233e0 YMM06=3D0000000000000000 0000000000000000 0000000000000000 00007f65df2233b0 YMM07=3D0000000000000000 0000000000000000 62694c6568636143 2f65636170736b72 YMM08=3D0000000000000000 0000000000000000 6d622070656d7320 327876612031696d YMM09=3D0000000000000000 0000000000000000 0000000000000004 0000000000000004 YMM10=3D0000000000000000 0000000000000000 0000000000000002 0000000000000002 YMM11=3D0000000000000000 0000000000000000 0000000000000010 0000000000000010 YMM12=3D0000000000000000 0000000000000000 0000000000ff00fb 0000000000fe00fa YMM13=3D0000000000000000 0000000000000000 0000000000000000 00ff00fd00fb00f9 YMM14=3D0000000000000000 0000000000000000 0000000000000000 0000000000000000 YMM15=3D0000000000000000 0000000000000000 0000000000000000 0000000000000000 The backtrace is (using Jonathans cxl-2024-03-05 branch): (gdb) bt #0 __pthread_kill_implementation (no_tid=3D0, signo=3D6, threadid=3D140737= 297516096) at ./nptl/pthread_kill.c:44 #1 __pthread_kill_internal (signo=3D6, threadid=3D140737297516096) at ./np= tl/pthread_kill.c:78 #2 __GI___pthread_kill (threadid=3D140737297516096, signo=3Dsigno@entry=3D= 6) at ./nptl/pthread_kill.c:89 #3 0x00007ffff7642476 in __GI_raise (sig=3Dsig@entry=3D6) at ../sysdeps/po= six/raise.c:26 #4 0x00007ffff76287f3 in __GI_abort () at ./stdlib/abort.c:79 #5 0x0000555555c5a9df in cpu_abort (cpu=3Dcpu@entry=3D0x5555578c19c0, fmt= =3Dfmt@entry=3D0x55555605d100 "cpu_io_recompile: could not find TB for pc= =3D%p") at ../cpu-target.c:371 #6 0x0000555555caa065 in cpu_io_recompile (cpu=3Dcpu@entry=3D0x5555578c19c= 0, retaddr=3D140736474541524) at ../accel/tcg/translate-all.c:610 #7 0x0000555555cacee7 in io_prepare (retaddr=3D140736474541524, addr=3D140= 075515361944, attrs=3D..., xlat=3D, cpu=3D0x5555578c19c0, ou= t_offset=3D) at ../accel/tcg/cputlb.c:1336 #8 do_st_mmio_leN (cpu=3D0x5555578c19c0, full=3D0x7ffd1a1554d0, val_le=3D1= 40075515361816, addr=3D140075515361944, size=3D8, mmu_idx=3D3, ra=3D1407364= 74541524) at ../accel/tcg/cputlb.c:2591 #9 0x0000555555cb179d in do_st_8 (ra=3D, memop=3D, mmu_idx=3D, val=3D140075515361816, p=3D, cpu=3D) at ../accel/tcg/cputlb.c:2784 #10 do_st8_mmu (cpu=3D0x5555578c19c0, addr=3D39050, val=3D140075515361816, = oi=3D6, ra=3D140736474541524) at ../accel/tcg/cputlb.c:2862 #11 0x00007fffc3926e15 in code_gen_buffer () #12 0x0000555555ca0e5b in cpu_tb_exec (cpu=3Dcpu@entry=3D0x5555578c19c0, it= b=3Ditb@entry=3D0x7fffc3926cc0 , tb_exit=3Dtb_ex= it@entry=3D0x7ffff49ff6d8) at ../accel/tcg/cpu-exec.c:449 #13 0x0000555555ca13ac in cpu_loop_exec_tb (tb_exit=3D0x7ffff49ff6d8, last_= tb=3D, pc=3D, tb=3D0x7fffc3926cc0 , cpu=3D0x5555578c19c0) at ../accel/tcg/cpu-exec.c:904 #14 cpu_exec_loop (cpu=3Dcpu@entry=3D0x5555578c19c0, sc=3Dsc@entry=3D0x7fff= f49ff770) at ../accel/tcg/cpu-exec.c:1019 #15 0x0000555555ca1bb1 in cpu_exec_setjmp (cpu=3Dcpu@entry=3D0x5555578c19c0= , sc=3Dsc@entry=3D0x7ffff49ff770) at ../accel/tcg/cpu-exec.c:1036 #16 0x0000555555ca2388 in cpu_exec (cpu=3Dcpu@entry=3D0x5555578c19c0) at ..= /accel/tcg/cpu-exec.c:1062 #17 0x0000555555cc65c4 in tcg_cpu_exec (cpu=3Dcpu@entry=3D0x5555578c19c0) a= t ../accel/tcg/tcg-accel-ops.c:76 #18 0x0000555555cc671f in mttcg_cpu_thread_fn (arg=3Darg@entry=3D0x5555578c= 19c0) at ../accel/tcg/tcg-accel-ops-mttcg.c:95 #19 0x0000555555e61261 in qemu_thread_start (args=3D) at ../= util/qemu-thread-posix.c:541 #20 0x00007ffff7694ac3 in start_thread (arg=3D) at ./nptl/pt= hread_create.c:442 #21 0x00007ffff7726850 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/cl= one3.S:81 Looking at the tb being executed, it looks like it is a single instruction = tb, so with my _very_ limited understanding of tcg, it shouldn=E2=80=99t be nec= essary to do the IO recompile: (gdb) up 13 #13 0x0000555555ca13ac in cpu_loop_exec_tb (tb_exit=3D0x7ffff49ff6d8, last_= tb=3D, pc=3D, tb=3D0x7fffc3926cc0 , cpu=3D0x5555578c19c0) at ../accel/tcg/cpu-exec.c:904 904 tb =3D cpu_tb_exec(cpu, tb, tb_exit); (gdb) print *tb $1 =3D {pc =3D 0, cs_base =3D 0, flags =3D 415285939, cflags =3D 4278321152= , size =3D 7, icount =3D 1, tc =3D {ptr =3D 0x7fffc3926d80 , size =3D 176}, page_next =3D {0, 0}, page_addr =3D {1844674407= 3709551615, 18446744073709551615}, jmp_lock =3D {value =3D 0}, jmp_reset_offset =3D= {65535, 65535}, jmp_insn_offset =3D {65535, 65535}, jmp_target_addr =3D {0= , 0}, jmp_list_head =3D 140736474540928, jmp_list_next =3D {0, 0}, jmp_dest= =3D {0, 0}} If the application is run entirely out of MMIO memory, things work fine (the previous patches related to this is in Jonathans branch), so one thought is= that it is related to having the code on a mix of regular and CXL memory. Since = we previously had issues with code crossing page boundaries where only the sec= ond page is MMIO, I tried out the following change to the fix introduced for th= at issue thinking that reverting to the slow path in the middle of the transla= tion might not correctly update can_do_io: diff --git a/accel/tcg/translator.c b/accel/tcg/translator.c index 38c34009a5..db6ea360e0 100644 --- a/accel/tcg/translator.c +++ b/accel/tcg/translator.c @@ -258,6 +258,7 @@ static void *translator_access(CPUArchState *env, Disas= ContextBase *db, if (unlikely(new_page1 =3D=3D -1)) { tb_unlock_pages(tb); tb_set_page_addr0(tb, -1); + set_can_do_io(db, true); return NULL; } With that change, things work fine. Not saying that this is a valid fix for= the issue, but just trying to provide as much information as possible :) Thanks, Jorgen