From nobody Thu Apr 3 11:36:53 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1741166629; cv=none; d=zohomail.com; s=zohoarc; b=aLtrgEvwYt3pge/LArOlUaHZegPmDzbbEQhJf3U6bRDXFTpVKusYbAlW7CeffHzkUDGiTwgqtyqJco7RJ70/fdH0BvoT6JwCmmaRimYz7pFdUw6hFxzgsTpzdLaRA4icNt0QeTFDGrTZEjRuTGGZ23s4DLLDnQo4X3oglJIm+fU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1741166629; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Reply-To:Reply-To:References:Sender:Subject:Subject:To:To:Message-Id; bh=8PEr0IX7E0U+5BY2b4GPGJG1YEJPWHpNkreZ74IrVW0=; b=feSUlXqsYslel3W1w+zXcvWF7zX+DUbgNd6DJOHL9zr6vBr/WSQyb2nvMwgjUR7w5sLCBRa7lwgzZjYGSXDo402JFH8oB8Kr2bPDAenkBVLqoRQRtz461mZoFvQv8XVqFrCrvQrRE9OHlZXEiVsI6s7wUhSoP6bbs8UGJPFo9S0= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1741166629529898.1136528876459; Wed, 5 Mar 2025 01:23:49 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tpkyV-0006Eb-A1; Wed, 05 Mar 2025 04:23:35 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tpkyN-00069y-Ia for qemu-devel@nongnu.org; Wed, 05 Mar 2025 04:23:29 -0500 Received: from frasgout.his.huawei.com ([185.176.79.56]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tpkyK-0000jQ-VM for qemu-devel@nongnu.org; Wed, 05 Mar 2025 04:23:27 -0500 Received: from mail.maildlp.com (unknown [172.18.186.231]) by frasgout.his.huawei.com (SkyGuard) with ESMTP id 4Z76SN40RPz67D7Q; Wed, 5 Mar 2025 17:19:16 +0800 (CST) Received: from frapeml500008.china.huawei.com (unknown [7.182.85.71]) by mail.maildlp.com (Postfix) with ESMTPS id 656C7140A70; Wed, 5 Mar 2025 17:23:22 +0800 (CST) Received: from SecurePC-101-06.china.huawei.com (10.122.19.247) by frapeml500008.china.huawei.com (7.182.85.71) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.39; Wed, 5 Mar 2025 10:23:21 +0100 To: , CC: , , Yuquan Wang , Arpit Kumar , Sweta Kumari , Vinayak Holikatti , Davidlohr Bueso , Ajay Joshi Subject: [PATCH qemu 1/8] hw/cxl: Support aborting background commands Date: Wed, 5 Mar 2025 09:22:41 +0000 Message-ID: <20250305092249.191812-2-Jonathan.Cameron@huawei.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250305092249.191812-1-Jonathan.Cameron@huawei.com> References: <20250305092249.191812-1-Jonathan.Cameron@huawei.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.122.19.247] X-ClientProxiedBy: lhrpeml100001.china.huawei.com (7.191.160.183) To frapeml500008.china.huawei.com (7.182.85.71) Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=185.176.79.56; envelope-from=jonathan.cameron@huawei.com; helo=frasgout.his.huawei.com X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H2=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-to: Jonathan Cameron From: Jonathan Cameron via Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZM-MESSAGEID: 1741166631614019000 Content-Type: text/plain; charset="utf-8" From: Davidlohr Bueso As of 3.1 spec, background commands can be canceled with a new abort command. Implement the support, which is advertised in the CEL. No ad-hoc context undoing is necessary as all the command logic of the running bg command is done upon completion. Arbitrarily, the on-going background cmd will not be aborted if already at least 85% done; A mutex is introduced to stabilize mbox request cancel command vs the timer callback being fired scenarios (as well as reading the mbox registers). While some operations under critical regions may be unnecessary (irq notifying, cmd callbacks), this is not a path where performance is important, so simplicity is preferred. Tested-by: Ajay Joshi Reviewed-by: Ajay Joshi Signed-off-by: Davidlohr Bueso Signed-off-by: Jonathan Cameron --- Modified with changes Davidlohr posted on list. --- include/hw/cxl/cxl_device.h | 4 ++ include/hw/cxl/cxl_mailbox.h | 1 + hw/cxl/cxl-device-utils.c | 14 ++++--- hw/cxl/cxl-mailbox-utils.c | 72 ++++++++++++++++++++++++++++++++---- hw/mem/cxl_type3.c | 8 +++- 5 files changed, 86 insertions(+), 13 deletions(-) diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h index 3a0ee7e8e7..d21695507f 100644 --- a/include/hw/cxl/cxl_device.h +++ b/include/hw/cxl/cxl_device.h @@ -176,10 +176,12 @@ typedef struct CXLCCI { uint16_t opcode; uint16_t complete_pct; uint16_t ret_code; /* Current value of retcode */ + bool aborted; uint64_t starttime; /* set by each bg cmd, cleared by the bg_timer when complete */ uint64_t runtime; QEMUTimer *timer; + QemuMutex lock; /* serializes mbox abort vs timer cb */ } bg; =20 /* firmware update */ @@ -201,6 +203,7 @@ typedef struct CXLCCI { DeviceState *d; /* Pointer to the device hosting the protocol conversion */ DeviceState *intf; + bool initialized; } CXLCCI; =20 typedef struct cxl_device_state { @@ -316,6 +319,7 @@ void cxl_initialize_mailbox_t3(CXLCCI *cci, DeviceState= *d, size_t payload_max); void cxl_initialize_mailbox_swcci(CXLCCI *cci, DeviceState *intf, DeviceState *d, size_t payload_max); void cxl_init_cci(CXLCCI *cci, size_t payload_max); +void cxl_destroy_cci(CXLCCI *cci); void cxl_add_cci_commands(CXLCCI *cci, const struct cxl_cmd (*cxl_cmd_set)= [256], size_t payload_max); int cxl_process_cci_message(CXLCCI *cci, uint8_t set, uint8_t cmd, diff --git a/include/hw/cxl/cxl_mailbox.h b/include/hw/cxl/cxl_mailbox.h index beb048052e..9008402d1c 100644 --- a/include/hw/cxl/cxl_mailbox.h +++ b/include/hw/cxl/cxl_mailbox.h @@ -14,5 +14,6 @@ #define CXL_MBOX_IMMEDIATE_LOG_CHANGE (1 << 4) #define CXL_MBOX_SECURITY_STATE_CHANGE (1 << 5) #define CXL_MBOX_BACKGROUND_OPERATION (1 << 6) +#define CXL_MBOX_BACKGROUND_OPERATION_ABORT (1 << 7) =20 #endif diff --git a/hw/cxl/cxl-device-utils.c b/hw/cxl/cxl-device-utils.c index 52ad1e4c3f..e150d74457 100644 --- a/hw/cxl/cxl-device-utils.c +++ b/hw/cxl/cxl-device-utils.c @@ -95,11 +95,15 @@ static uint64_t mailbox_reg_read(void *opaque, hwaddr o= ffset, unsigned size) } if (offset =3D=3D A_CXL_DEV_MAILBOX_STS) { uint64_t status_reg =3D cxl_dstate->mbox_reg_state64[offset / = size]; - if (cci->bg.complete_pct) { - status_reg =3D FIELD_DP64(status_reg, CXL_DEV_MAILBOX_STS,= BG_OP, - 0); - cxl_dstate->mbox_reg_state64[offset / size] =3D status_reg; - } + int bgop; + + qemu_mutex_lock(&cci->bg.lock); + bgop =3D !(cci->bg.complete_pct =3D=3D 100 || cci->bg.aborted); + + status_reg =3D FIELD_DP64(status_reg, CXL_DEV_MAILBOX_STS, BG_= OP, + bgop); + cxl_dstate->mbox_reg_state64[offset / size] =3D status_reg; + qemu_mutex_unlock(&cci->bg.lock); } return cxl_dstate->mbox_reg_state64[offset / size]; default: diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c index 516c01d840..4401f446d9 100644 --- a/hw/cxl/cxl-mailbox-utils.c +++ b/hw/cxl/cxl-mailbox-utils.c @@ -56,6 +56,7 @@ enum { INFOSTAT =3D 0x00, #define IS_IDENTIFY 0x1 #define BACKGROUND_OPERATION_STATUS 0x2 + #define BACKGROUND_OPERATION_ABORT 0x5 EVENTS =3D 0x01, #define GET_RECORDS 0x0 #define CLEAR_RECORDS 0x1 @@ -636,6 +637,41 @@ static CXLRetCode cmd_infostat_bg_op_sts(const struct = cxl_cmd *cmd, return CXL_MBOX_SUCCESS; } =20 +/* + * CXL r3.1 Section 8.2.9.1.5: + * Request Abort Background Operation (Opcode 0005h) + */ +static CXLRetCode cmd_infostat_bg_op_abort(const struct cxl_cmd *cmd, + uint8_t *payload_in, + size_t len_in, + uint8_t *payload_out, + size_t *len_out, + CXLCCI *cci) +{ + int bg_set =3D cci->bg.opcode >> 8; + int bg_cmd =3D cci->bg.opcode & 0xff; + const struct cxl_cmd *bg_c =3D &cci->cxl_cmd_set[bg_set][bg_cmd]; + + if (!(bg_c->effect & CXL_MBOX_BACKGROUND_OPERATION_ABORT)) { + return CXL_MBOX_REQUEST_ABORT_NOTSUP; + } + + qemu_mutex_lock(&cci->bg.lock); + if (cci->bg.runtime) { + /* operation is near complete, let it finish */ + if (cci->bg.complete_pct < 85) { + timer_del(cci->bg.timer); + cci->bg.ret_code =3D CXL_MBOX_ABORTED; + cci->bg.starttime =3D 0; + cci->bg.runtime =3D 0; + cci->bg.aborted =3D true; + } + } + qemu_mutex_unlock(&cci->bg.lock); + + return CXL_MBOX_SUCCESS; +} + #define CXL_FW_SLOTS 2 #define CXL_FW_SIZE 0x02000000 /* 32 mb */ =20 @@ -2715,6 +2751,8 @@ static CXLRetCode cmd_dcd_release_dyn_cap(const struc= t cxl_cmd *cmd, } =20 static const struct cxl_cmd cxl_cmd_set[256][256] =3D { + [INFOSTAT][BACKGROUND_OPERATION_ABORT] =3D { "BACKGROUND_OPERATION_ABO= RT", + cmd_infostat_bg_op_abort, 0, 0 }, [EVENTS][GET_RECORDS] =3D { "EVENTS_GET_RECORDS", cmd_events_get_records, 1, 0 }, [EVENTS][CLEAR_RECORDS] =3D { "EVENTS_CLEAR_RECORDS", @@ -2727,9 +2765,11 @@ static const struct cxl_cmd cxl_cmd_set[256][256] = =3D { [FIRMWARE_UPDATE][GET_INFO] =3D { "FIRMWARE_UPDATE_GET_INFO", cmd_firmware_update_get_info, 0, 0 }, [FIRMWARE_UPDATE][TRANSFER] =3D { "FIRMWARE_UPDATE_TRANSFER", - cmd_firmware_update_transfer, ~0, CXL_MBOX_BACKGROUND_OPERATION }, + cmd_firmware_update_transfer, ~0, + CXL_MBOX_BACKGROUND_OPERATION | CXL_MBOX_BACKGROUND_OPERATION_ABOR= T }, [FIRMWARE_UPDATE][ACTIVATE] =3D { "FIRMWARE_UPDATE_ACTIVATE", - cmd_firmware_update_activate, 2, CXL_MBOX_BACKGROUND_OPERATION }, + cmd_firmware_update_activate, 2, + CXL_MBOX_BACKGROUND_OPERATION | CXL_MBOX_BACKGROUND_OPERATION_ABOR= T }, [TIMESTAMP][GET] =3D { "TIMESTAMP_GET", cmd_timestamp_get, 0, 0 }, [TIMESTAMP][SET] =3D { "TIMESTAMP_SET", cmd_timestamp_set, 8, CXL_MBOX_IMMEDIATE_POLICY_CHANGE }, @@ -2758,7 +2798,8 @@ static const struct cxl_cmd cxl_cmd_set[256][256] =3D= { [SANITIZE][OVERWRITE] =3D { "SANITIZE_OVERWRITE", cmd_sanitize_overwri= te, 0, (CXL_MBOX_IMMEDIATE_DATA_CHANGE | CXL_MBOX_SECURITY_STATE_CHANGE | - CXL_MBOX_BACKGROUND_OPERATION)}, + CXL_MBOX_BACKGROUND_OPERATION | + CXL_MBOX_BACKGROUND_OPERATION_ABORT)}, [PERSISTENT_MEM][GET_SECURITY_STATE] =3D { "GET_SECURITY_STATE", cmd_get_security_state, 0, 0 }, [MEDIA_AND_POISON][GET_POISON_LIST] =3D { "MEDIA_AND_POISON_GET_POISON= _LIST", @@ -2771,7 +2812,8 @@ static const struct cxl_cmd cxl_cmd_set[256][256] =3D= { "MEDIA_AND_POISON_GET_SCAN_MEDIA_CAPABILITIES", cmd_media_get_scan_media_capabilities, 16, 0 }, [MEDIA_AND_POISON][SCAN_MEDIA] =3D { "MEDIA_AND_POISON_SCAN_MEDIA", - cmd_media_scan_media, 17, CXL_MBOX_BACKGROUND_OPERATION }, + cmd_media_scan_media, 17, + (CXL_MBOX_BACKGROUND_OPERATION | CXL_MBOX_BACKGROUND_OPERATION_ABO= RT)}, [MEDIA_AND_POISON][GET_SCAN_MEDIA_RESULTS] =3D { "MEDIA_AND_POISON_GET_SCAN_MEDIA_RESULTS", cmd_media_get_scan_media_results, 0, 0 }, @@ -2795,6 +2837,8 @@ static const struct cxl_cmd cxl_cmd_set_sw[256][256] = =3D { [INFOSTAT][IS_IDENTIFY] =3D { "IDENTIFY", cmd_infostat_identify, 0, 0 = }, [INFOSTAT][BACKGROUND_OPERATION_STATUS] =3D { "BACKGROUND_OPERATION_ST= ATUS", cmd_infostat_bg_op_sts, 0, 0 }, + [INFOSTAT][BACKGROUND_OPERATION_ABORT] =3D { "BACKGROUND_OPERATION_ABO= RT", + cmd_infostat_bg_op_abort, 0, 0 }, [TIMESTAMP][GET] =3D { "TIMESTAMP_GET", cmd_timestamp_get, 0, 0 }, [TIMESTAMP][SET] =3D { "TIMESTAMP_SET", cmd_timestamp_set, 8, CXL_MBOX_IMMEDIATE_POLICY_CHANGE }, @@ -2881,6 +2925,7 @@ int cxl_process_cci_message(CXLCCI *cci, uint8_t set,= uint8_t cmd, cci->bg.opcode =3D (set << 8) | cmd; =20 cci->bg.complete_pct =3D 0; + cci->bg.aborted =3D false; cci->bg.ret_code =3D 0; =20 now =3D qemu_clock_get_ms(QEMU_CLOCK_VIRTUAL); @@ -2894,10 +2939,12 @@ int cxl_process_cci_message(CXLCCI *cci, uint8_t se= t, uint8_t cmd, static void bg_timercb(void *opaque) { CXLCCI *cci =3D opaque; - uint64_t now =3D qemu_clock_get_ms(QEMU_CLOCK_VIRTUAL); - uint64_t total_time =3D cci->bg.starttime + cci->bg.runtime; + uint64_t now, total_time; + + qemu_mutex_lock(&cci->bg.lock); =20 - assert(cci->bg.runtime > 0); + now =3D qemu_clock_get_ms(QEMU_CLOCK_VIRTUAL); + total_time =3D cci->bg.starttime + cci->bg.runtime; =20 if (now >=3D total_time) { /* we are done */ uint16_t ret =3D CXL_MBOX_SUCCESS; @@ -2950,6 +2997,8 @@ static void bg_timercb(void *opaque) msi_notify(pdev, cxl_dstate->mbox_msi_n); } } + + qemu_mutex_unlock(&cci->bg.lock); } =20 static void cxl_rebuild_cel(CXLCCI *cci) @@ -2978,12 +3027,21 @@ void cxl_init_cci(CXLCCI *cci, size_t payload_max) cci->bg.complete_pct =3D 0; cci->bg.starttime =3D 0; cci->bg.runtime =3D 0; + cci->bg.aborted =3D false; cci->bg.timer =3D timer_new_ms(QEMU_CLOCK_VIRTUAL, bg_timercb, cci); + qemu_mutex_init(&cci->bg.lock); =20 memset(&cci->fw, 0, sizeof(cci->fw)); cci->fw.active_slot =3D 1; cci->fw.slot[cci->fw.active_slot - 1] =3D true; + cci->initialized =3D true; +} + +void cxl_destroy_cci(CXLCCI *cci) +{ + qemu_mutex_destroy(&cci->bg.lock); + cci->initialized =3D false; } =20 static void cxl_copy_cci_commands(CXLCCI *cci, const struct cxl_cmd (*cxl_= cmds)[256]) diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c index 6fffa21ead..be670ae3f3 100644 --- a/hw/mem/cxl_type3.c +++ b/hw/mem/cxl_type3.c @@ -970,6 +970,7 @@ static void ct3_exit(PCIDevice *pci_dev) cxl_doe_cdat_release(cxl_cstate); msix_uninit_exclusive_bar(pci_dev); g_free(regs->special_ops); + cxl_destroy_cci(&ct3d->cci); if (ct3d->dc.host_dc) { cxl_destroy_dc_regions(ct3d); address_space_destroy(&ct3d->dc.host_dc_as); @@ -1225,12 +1226,17 @@ static void ct3d_reset(DeviceState *dev) * Bring up an endpoint to target with MCTP over VDM. * This device is emulating an MLD with single LD for now. */ + if (ct3d->vdm_fm_owned_ld_mctp_cci.initialized) { + cxl_destroy_cci(&ct3d->vdm_fm_owned_ld_mctp_cci); + } cxl_initialize_t3_fm_owned_ld_mctpcci(&ct3d->vdm_fm_owned_ld_mctp_cci, DEVICE(ct3d), DEVICE(ct3d), 512); /* Max payload made up */ + if (ct3d->ld0_cci.initialized) { + cxl_destroy_cci(&ct3d->ld0_cci); + } cxl_initialize_t3_ld_cci(&ct3d->ld0_cci, DEVICE(ct3d), DEVICE(ct3d), 512); /* Max payload made up */ - } =20 static const Property ct3_props[] =3D { --=20 2.43.0