From nobody Sun May 5 14:44:00 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=linux.ibm.com ARC-Seal: i=1; a=rsa-sha256; t=1606377418; cv=none; d=zohomail.com; s=zohoarc; b=fWdfIVIPolsE+h2DAViDwHsmJSndI2rRwybCiV4pwOxZw3hsIb9AA+bC7AXwM/ueUCkyGGAjxbrHyxTPQ6PLNxYFRJFI9rLhdTLdawTU0+CyVodRU+XWJ/PPVSHsu8hOblFAlzKKLTqYkZTt1QxpnvCOmkqae7HJPza8hf1CDQc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1606377418; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=vmynOO0OC8i6baSfxu+2Y4bdfbLG58xZnKltpDHGDf0=; b=lIGF53rNqiPNAsGlfJ9dz3zsF7Ce2AlDqjjpYWapr5ObhkK8Zc30P3Hb2qcaQuSa8t4xqCntPeITi4QmgtQ3vIIMeNVhXPYkUwCAG8PNHLlDBz3Ytf8nbIWPasQ632KBfDHauRhAFfBNTcj1BgZ96l3uXQ9IIbpXnTf7TkQ9TYw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1606377418552761.8058181611342; Wed, 25 Nov 2020 23:56:58 -0800 (PST) Received: from localhost ([::1]:34770 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kiC9B-0001dT-8l for importer@patchew.org; Thu, 26 Nov 2020 02:56:57 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:58256) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kiC7X-0000TT-0l; Thu, 26 Nov 2020 02:55:15 -0500 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:7758) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kiC7T-0005Ui-36; Thu, 26 Nov 2020 02:55:14 -0500 Received: from pps.filterd (m0098421.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 0AQ71JM5147890; Thu, 26 Nov 2020 02:55:05 -0500 Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 3523k1ebhr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 26 Nov 2020 02:55:05 -0500 Received: from m0098421.ppops.net (m0098421.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.36/8.16.0.36) with SMTP id 0AQ7ChY0192222; Thu, 26 Nov 2020 02:55:05 -0500 Received: from ppma04fra.de.ibm.com (6a.4a.5195.ip4.static.sl-reverse.com [149.81.74.106]) by mx0a-001b2d01.pphosted.com with ESMTP id 3523k1ebgx-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 26 Nov 2020 02:55:05 -0500 Received: from pps.filterd (ppma04fra.de.ibm.com [127.0.0.1]) by ppma04fra.de.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 0AQ7mpui018630; Thu, 26 Nov 2020 07:55:03 GMT Received: from b06cxnps4076.portsmouth.uk.ibm.com (d06relay13.portsmouth.uk.ibm.com [9.149.109.198]) by ppma04fra.de.ibm.com with ESMTP id 34xth8asgq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 26 Nov 2020 07:55:03 +0000 Received: from d06av25.portsmouth.uk.ibm.com (d06av25.portsmouth.uk.ibm.com [9.149.105.61]) by b06cxnps4076.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 0AQ7t1ih8651386 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 26 Nov 2020 07:55:01 GMT Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E9D3A11C05B; Thu, 26 Nov 2020 07:55:00 +0000 (GMT) Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id BE12111C050; Thu, 26 Nov 2020 07:54:59 +0000 (GMT) Received: from lep8c.aus.stglabs.ibm.com (unknown [9.40.192.207]) by d06av25.portsmouth.uk.ibm.com (Postfix) with ESMTP; Thu, 26 Nov 2020 07:54:59 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=pp1; bh=vmynOO0OC8i6baSfxu+2Y4bdfbLG58xZnKltpDHGDf0=; b=Y8vNha1f8uBll12MyA8P3H/GRLnHdWFennkkvJ47hjE9aDowVpfRbr7SrGNqrFgsOXP+ bxaeR4yrGsvCdwF6o1CuULeACmcebZ1avFaSYwRNCYFdIRloCltYCmhtjDY+15RRhyMg QCHejjRVO6cF8xh1x3+IfiBNbEMmGVY+U6NB9rHC1WR/OXOfm5PUsxgxMrgxBfBsFn+8 YIn71Ptif010c/8h52+CL2niO/Zey16DVELNWZ6gmFBak1SXnqVFVIoEWGjdet3nTOKg y3C5vec4E9gRMN/rmVuTQt+jztN/RLBdfeN2X/XTSMVivr+A5TRCit1MBdpnO0JvYVbc pw== Subject: [RFC PATCH 1/2] spapr: drc: Add support for async hcalls at the drc level From: Shivaprasad G Bhat To: xiaoguangrong.eric@gmail.com, mst@redhat.com, imammedo@redhat.com, david@gibson.dropbear.id.au, qemu-devel@nongnu.org, qemu-ppc@nongnu.org Date: Thu, 26 Nov 2020 01:54:55 -0600 Message-ID: <160637728389.1977658.13576068142432992889.stgit@lep8c.aus.stglabs.ibm.com> In-Reply-To: <160637722874.1977658.13777818163262816915.stgit@lep8c.aus.stglabs.ibm.com> References: <160637722874.1977658.13777818163262816915.stgit@lep8c.aus.stglabs.ibm.com> User-Agent: StGit/0.19 MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.312, 18.0.737 definitions=2020-11-26_01:2020-11-26, 2020-11-26 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 adultscore=0 phishscore=0 impostorscore=0 mlxlogscore=999 bulkscore=0 spamscore=0 suspectscore=2 malwarescore=0 mlxscore=0 priorityscore=1501 lowpriorityscore=0 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2011260037 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.158.5; envelope-from=sbhat@linux.ibm.com; helo=mx0b-001b2d01.pphosted.com X-Spam_score_int: -19 X-Spam_score: -2.0 X-Spam_bar: -- X-Spam_report: (-2.0 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: shivaprasadbhat@gmail.com, bharata@linux.vnet.ibm.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @ibm.com) The patch adds support for async hcalls at the DRC level for the spapr devices. To be used by spapr-scm devices in the patch/es to follow. Signed-off-by: Shivaprasad G Bhat --- hw/ppc/spapr_drc.c | 146 ++++++++++++++++++++++++++++++++++++++++= ++++ include/hw/ppc/spapr_drc.h | 25 ++++++++ 2 files changed, 171 insertions(+) diff --git a/hw/ppc/spapr_drc.c b/hw/ppc/spapr_drc.c index 77718cde1f..2cecccf701 100644 --- a/hw/ppc/spapr_drc.c +++ b/hw/ppc/spapr_drc.c @@ -15,6 +15,7 @@ #include "qapi/qmp/qnull.h" #include "cpu.h" #include "qemu/cutils.h" +#include "qemu/guest-random.h" #include "hw/ppc/spapr_drc.h" #include "qom/object.h" #include "migration/vmstate.h" @@ -421,6 +422,145 @@ void spapr_drc_detach(SpaprDrc *drc) spapr_drc_release(drc); } =20 + +/* + * @drc : device DRC targetting which the async hcalls to be made. + * + * All subsequent requests to run/query the status should use the + * unique token returned here. + */ +uint64_t spapr_drc_get_new_async_hcall_token(SpaprDrc *drc) +{ + Error *err =3D NULL; + uint64_t token; + SpaprDrcDeviceAsyncHCallState *tmp, *next, *state; + + state =3D g_malloc0(sizeof(*state)); + state->pending =3D true; + + qemu_mutex_lock(&drc->async_hcall_states_lock); +retry: + if (qemu_guest_getrandom(&token, sizeof(token), &err) < 0) { + error_report_err(err); + g_free(state); + return 0; + } + + if (!token) /* Token should be non-zero */ + goto retry; + + if (!QLIST_EMPTY(&drc->async_hcall_states)) { + QLIST_FOREACH_SAFE(tmp, &drc->async_hcall_states, node, next) { + if (tmp->continue_token =3D=3D token) { + /* If the token already in use, get a new one */ + goto retry; + } + } + } + + state->continue_token =3D token; + QLIST_INSERT_HEAD(&drc->async_hcall_states, state, node); + + qemu_mutex_unlock(&drc->async_hcall_states_lock); + + return state->continue_token; +} + +static void *spapr_drc_async_hcall_runner(void *opaque) +{ + int response =3D -1; + SpaprDrcDeviceAsyncHCallState *state =3D opaque; + + /* + * state is freed only after this thread finishes(after pthread_join()= ), + * don't worry about it becoming NULL. + */ + + response =3D state->func(state->data); + + state->hcall_ret =3D response; + state->pending =3D 0; + + return NULL; +} + +/* + * @drc : device DRC targetting which the async hcalls to be made. + * token : The continue token to be used for tracking as recived from + * spapr_drc_get_new_async_hcall_token + * @func() : the worker function which needs to be executed asynchronously + * @data : data to be passed to the asynchronous function. Worker is suppo= sed + * to free/cleanup the data that is passed here + */ +void spapr_drc_run_async_hcall(SpaprDrc *drc, uint64_t token, + SpaprDrcAsyncHcallWorkerFunc *func, void *d= ata) +{ + SpaprDrcDeviceAsyncHCallState *state, *next; + + qemu_mutex_lock(&drc->async_hcall_states_lock); + QLIST_FOREACH_SAFE(state, &drc->async_hcall_states, node, next) { + if (state->continue_token =3D=3D token) { + state->func =3D func; + state->data =3D data; + qemu_thread_create(&state->thread, "sPAPR Async HCALL", + spapr_drc_async_hcall_runner, state, + QEMU_THREAD_JOINABLE); + break; + } + } + qemu_mutex_unlock(&drc->async_hcall_states_lock); +} + +/* + * spapr_drc_finish_async_hcalls + * Waits for all pending async requests to complete + * thier execution and free the states + */ +static void spapr_drc_finish_async_hcalls(SpaprDrc *drc) +{ + SpaprDrcDeviceAsyncHCallState *state, *next; + + if (QLIST_EMPTY(&drc->async_hcall_states)) { + return; + } + + QLIST_FOREACH_SAFE(state, &drc->async_hcall_states, node, next) { + qemu_thread_join(&state->thread); + QLIST_REMOVE(state, node); + g_free(state); + } +} + +/* + * spapr_drc_get_async_hcall_status + * Fetches the status of the hcall worker and returns H_BUSY + * if the worker is still running. + */ +int spapr_drc_get_async_hcall_status(SpaprDrc *drc, uint64_t token) +{ + int ret =3D H_PARAMETER; + SpaprDrcDeviceAsyncHCallState *state, *node; + + qemu_mutex_lock(&drc->async_hcall_states_lock); + QLIST_FOREACH_SAFE(state, &drc->async_hcall_states, node, node) { + if (state->continue_token =3D=3D token) { + if (state->pending) { + ret =3D H_BUSY; + break; + } else { + ret =3D state->hcall_ret; + qemu_thread_join(&state->thread); + QLIST_REMOVE(state, node); + g_free(state); + break; + } + } + } + qemu_mutex_unlock(&drc->async_hcall_states_lock); + + return ret; +} + void spapr_drc_reset(SpaprDrc *drc) { SpaprDrcClass *drck =3D SPAPR_DR_CONNECTOR_GET_CLASS(drc); @@ -448,6 +588,7 @@ void spapr_drc_reset(SpaprDrc *drc) drc->ccs_offset =3D -1; drc->ccs_depth =3D -1; } + spapr_drc_finish_async_hcalls(drc); } =20 static bool spapr_drc_unplug_requested_needed(void *opaque) @@ -558,6 +699,7 @@ SpaprDrc *spapr_dr_connector_new(Object *owner, const c= har *type, drc->owner =3D owner; prop_name =3D g_strdup_printf("dr-connector[%"PRIu32"]", spapr_drc_index(drc)); + object_property_add_child(owner, prop_name, OBJECT(drc)); object_unref(OBJECT(drc)); qdev_realize(DEVICE(drc), NULL, NULL); @@ -577,6 +719,10 @@ static void spapr_dr_connector_instance_init(Object *o= bj) object_property_add(obj, "fdt", "struct", prop_get_fdt, NULL, NULL, NULL); drc->state =3D drck->empty_state; + + qemu_mutex_init(&drc->async_hcall_states_lock); + QLIST_INIT(&drc->async_hcall_states); + } =20 static void spapr_dr_connector_class_init(ObjectClass *k, void *data) diff --git a/include/hw/ppc/spapr_drc.h b/include/hw/ppc/spapr_drc.h index 165b281496..77f6e4386c 100644 --- a/include/hw/ppc/spapr_drc.h +++ b/include/hw/ppc/spapr_drc.h @@ -18,6 +18,7 @@ #include "sysemu/runstate.h" #include "hw/qdev-core.h" #include "qapi/error.h" +#include "block/thread-pool.h" =20 #define TYPE_SPAPR_DR_CONNECTOR "spapr-dr-connector" #define SPAPR_DR_CONNECTOR_GET_CLASS(obj) \ @@ -168,6 +169,21 @@ typedef enum { SPAPR_DRC_STATE_PHYSICAL_CONFIGURED =3D 8, } SpaprDrcState; =20 +typedef struct SpaprDrc SpaprDrc; + +typedef int SpaprDrcAsyncHcallWorkerFunc(void *opaque); +typedef struct SpaprDrcDeviceAsyncHCallState { + uint64_t continue_token; + bool pending; + + int hcall_ret; + SpaprDrcAsyncHcallWorkerFunc *func; + void *data; + + QemuThread thread; + + QLIST_ENTRY(SpaprDrcDeviceAsyncHCallState) node; +} SpaprDrcDeviceAsyncHCallState; typedef struct SpaprDrc { /*< private >*/ DeviceState parent; @@ -182,6 +198,10 @@ typedef struct SpaprDrc { int ccs_offset; int ccs_depth; =20 + /* async hcall states */ + QemuMutex async_hcall_states_lock; + QLIST_HEAD(, SpaprDrcDeviceAsyncHCallState) async_hcall_states; + /* device pointer, via link property */ DeviceState *dev; bool unplug_requested; @@ -241,6 +261,11 @@ void spapr_drc_detach(SpaprDrc *drc); /* Returns true if a hot plug/unplug request is pending */ bool spapr_drc_transient(SpaprDrc *drc); =20 +uint64_t spapr_drc_get_new_async_hcall_token(SpaprDrc *drc); +void spapr_drc_run_async_hcall(SpaprDrc *drc, uint64_t token, + SpaprDrcAsyncHcallWorkerFunc, void *data); +int spapr_drc_get_async_hcall_status(SpaprDrc *drc, uint64_t token); + static inline bool spapr_drc_unplug_requested(SpaprDrc *drc) { return drc->unplug_requested; From nobody Sun May 5 14:44:00 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=linux.ibm.com ARC-Seal: i=1; a=rsa-sha256; t=1606377506; cv=none; d=zohomail.com; s=zohoarc; b=DRZcioMJEmmJoXzzcIIG22lEOrfJXpvAC8hH3m42cBkt3s6ZtJLP/Wtz7EmSVVn7FI8bvqUQ24cHXBYsHwl402nlY2dIEqeoo2qkI5BqCctTvoE8VvU7owS+Iq0x0zoKVD6239Cd5xW5YNEFawLH720wjtQ+dQ+VQ4gVLuFeUWM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1606377506; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=q4qOxUv82Vlz1okT0uQaPo+5wm5xH7ihGdQ6JZq/NVw=; b=lGeEfCLwyW53jgpy1X8RztbmwyF02HVaM5cSC6gYf6+T7DbUiLzgohs8dkAtb3JzI4EwrEQw+X88SX1NdAU/yi/A1U7m2mjmPmLQuPxnA10uyddtXIXDU93i5/TjPIo8YGv2nviil/FhRxb7p6Eo2IsySUrech0KtWJntP5hGXI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 160637750648596.55309737185564; Wed, 25 Nov 2020 23:58:26 -0800 (PST) Received: from localhost ([::1]:37740 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kiCAb-0002ua-DE for importer@patchew.org; Thu, 26 Nov 2020 02:58:25 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:58354) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kiC7m-0000lw-Nl; Thu, 26 Nov 2020 02:55:30 -0500 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:20378) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kiC7k-0005l8-Ey; Thu, 26 Nov 2020 02:55:30 -0500 Received: from pps.filterd (m0098410.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 0AQ734lv118747; Thu, 26 Nov 2020 02:55:24 -0500 Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 3522mrfksj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 26 Nov 2020 02:55:24 -0500 Received: from m0098410.ppops.net (m0098410.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.36/8.16.0.36) with SMTP id 0AQ7W1Qx067462; Thu, 26 Nov 2020 02:55:24 -0500 Received: from ppma04ams.nl.ibm.com (63.31.33a9.ip4.static.sl-reverse.com [169.51.49.99]) by mx0a-001b2d01.pphosted.com with ESMTP id 3522mrfkqq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 26 Nov 2020 02:55:24 -0500 Received: from pps.filterd (ppma04ams.nl.ibm.com [127.0.0.1]) by ppma04ams.nl.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 0AQ7lTpB022807; Thu, 26 Nov 2020 07:55:21 GMT Received: from b06cxnps4074.portsmouth.uk.ibm.com (d06relay11.portsmouth.uk.ibm.com [9.149.109.196]) by ppma04ams.nl.ibm.com with ESMTP id 3518j8hen8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 26 Nov 2020 07:55:21 +0000 Received: from d06av21.portsmouth.uk.ibm.com (d06av21.portsmouth.uk.ibm.com [9.149.105.232]) by b06cxnps4074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 0AQ7tJEw8848060 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 26 Nov 2020 07:55:19 GMT Received: from d06av21.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 9016A52051; Thu, 26 Nov 2020 07:55:19 +0000 (GMT) Received: from lep8c.aus.stglabs.ibm.com (unknown [9.40.192.207]) by d06av21.portsmouth.uk.ibm.com (Postfix) with ESMTP id 6F6FC52050; Thu, 26 Nov 2020 07:55:18 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=pp1; bh=q4qOxUv82Vlz1okT0uQaPo+5wm5xH7ihGdQ6JZq/NVw=; b=GGLDezuvYTD3pO+4aHYZZeQ8vkAFdaLpEmQ1tb0D6iBJLNppZ4XPm3RnXoB7jHgbRaha gSH+O2snDveGNTaAqQoDxp+Tppv4Wa8ITnC7tpQPvBUhRYl2coweChceCSkioORP5XVa Kiz4fqUDeFmLCW1Qe/96wMG2DhRVzLN03fLdI/0B1MRStC83777BJf2/loG3un/aqcNF kRJkOFGbYiJekwroauTqCONLJR+q634tNWNqDmloGdGCR0zmN2AzvRoz54gG4qKza0FY pXWXZZ4g2dULD4lrqAabCQgRbDf5BMdc90OGm1wu7RHBuRdt7cR3Raq4JOV7FJtsWRa0 Uw== Subject: [RFC PATCH 2/2] spapr: nvdimm: Implement async flush hcalls From: Shivaprasad G Bhat To: xiaoguangrong.eric@gmail.com, mst@redhat.com, imammedo@redhat.com, david@gibson.dropbear.id.au, qemu-devel@nongnu.org, qemu-ppc@nongnu.org Date: Thu, 26 Nov 2020 01:55:17 -0600 Message-ID: <160637730609.1977658.17051667828648659623.stgit@lep8c.aus.stglabs.ibm.com> In-Reply-To: <160637722874.1977658.13777818163262816915.stgit@lep8c.aus.stglabs.ibm.com> References: <160637722874.1977658.13777818163262816915.stgit@lep8c.aus.stglabs.ibm.com> User-Agent: StGit/0.19 MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.312, 18.0.737 definitions=2020-11-26_01:2020-11-26, 2020-11-26 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 spamscore=0 priorityscore=1501 mlxlogscore=999 clxscore=1015 phishscore=0 lowpriorityscore=0 bulkscore=0 suspectscore=2 mlxscore=0 adultscore=0 impostorscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2011260037 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.156.1; envelope-from=sbhat@linux.ibm.com; helo=mx0a-001b2d01.pphosted.com X-Spam_score_int: -19 X-Spam_score: -2.0 X-Spam_bar: -- X-Spam_report: (-2.0 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: shivaprasadbhat@gmail.com, bharata@linux.vnet.ibm.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @ibm.com) When the persistent memory beacked by a file, a cpu cache flush instruction is not sufficient to ensure the stores are correctly flushed to the media. The patch implements the async hcalls for flush operation on demand from the guest kernel. The device option sync-dax is by default off and enables explicit asynchron= ous flush requests from guest. It can be disabled by setting syn-dax=3Don. Signed-off-by: Shivaprasad G Bhat --- hw/mem/nvdimm.c | 1 + hw/ppc/spapr_nvdimm.c | 79 +++++++++++++++++++++++++++++++++++++++++++= ++++ include/hw/mem/nvdimm.h | 10 ++++++ include/hw/ppc/spapr.h | 3 +- 4 files changed, 92 insertions(+), 1 deletion(-) diff --git a/hw/mem/nvdimm.c b/hw/mem/nvdimm.c index 03c2201b56..37a4db0135 100644 --- a/hw/mem/nvdimm.c +++ b/hw/mem/nvdimm.c @@ -220,6 +220,7 @@ static void nvdimm_write_label_data(NVDIMMDevice *nvdim= m, const void *buf, =20 static Property nvdimm_properties[] =3D { DEFINE_PROP_BOOL(NVDIMM_UNARMED_PROP, NVDIMMDevice, unarmed, false), + DEFINE_PROP_BOOL(NVDIMM_SYNC_DAX_PROP, NVDIMMDevice, sync_dax, false), DEFINE_PROP_END_OF_LIST(), }; =20 diff --git a/hw/ppc/spapr_nvdimm.c b/hw/ppc/spapr_nvdimm.c index a833a63b5e..557e36aa98 100644 --- a/hw/ppc/spapr_nvdimm.c +++ b/hw/ppc/spapr_nvdimm.c @@ -22,6 +22,7 @@ * THE SOFTWARE. */ #include "qemu/osdep.h" +#include "qemu/cutils.h" #include "qapi/error.h" #include "hw/ppc/spapr_drc.h" #include "hw/ppc/spapr_nvdimm.h" @@ -155,6 +156,11 @@ static int spapr_dt_nvdimm(SpaprMachineState *spapr, v= oid *fdt, "operating-system"))); _FDT(fdt_setprop(fdt, child_offset, "ibm,cache-flush-required", NULL, = 0)); =20 + if (!nvdimm->sync_dax) { + _FDT(fdt_setprop(fdt, child_offset, "ibm,async-flush-required", + NULL, 0)); + } + return child_offset; } =20 @@ -370,6 +376,78 @@ static target_ulong h_scm_bind_mem(PowerPCCPU *cpu, Sp= aprMachineState *spapr, return H_SUCCESS; } =20 +typedef struct SCMAsyncFlushData { + int fd; + uint64_t token; +} SCMAsyncFlushData; + +static int flush_worker_cb(void *opaque) +{ + int ret =3D H_SUCCESS; + SCMAsyncFlushData *req_data =3D opaque; + + /* flush raw backing image */ + if (qemu_fdatasync(req_data->fd) < 0) { + error_report("papr_scm: Could not sync nvdimm to backend file: %s", + strerror(errno)); + ret =3D H_HARDWARE; + } + + g_free(req_data); + + return ret; +} + +static target_ulong h_scm_async_flush(PowerPCCPU *cpu, SpaprMachineState *= spapr, + target_ulong opcode, target_ulong *a= rgs) +{ + int ret; + uint32_t drc_index =3D args[0]; + uint64_t continue_token =3D args[1]; + SpaprDrc *drc =3D spapr_drc_by_index(drc_index); + PCDIMMDevice *dimm; + HostMemoryBackend *backend =3D NULL; + SCMAsyncFlushData *req_data =3D NULL; + + if (!drc || !drc->dev || + spapr_drc_type(drc) !=3D SPAPR_DR_CONNECTOR_TYPE_PMEM) { + return H_PARAMETER; + } + + if (continue_token !=3D 0) { + ret =3D spapr_drc_get_async_hcall_status(drc, continue_token); + if (ret =3D=3D H_BUSY) { + args[0] =3D continue_token; + return H_LONG_BUSY_ORDER_1_SEC; + } + + return ret; + } + + dimm =3D PC_DIMM(drc->dev); + backend =3D MEMORY_BACKEND(dimm->hostmem); + + req_data =3D g_malloc0(sizeof(SCMAsyncFlushData)); + req_data->fd =3D memory_region_get_fd(&backend->mr); + + continue_token =3D spapr_drc_get_new_async_hcall_token(drc); + if (!continue_token) { + g_free(req_data); + return H_P2; + } + req_data->token =3D continue_token; + + spapr_drc_run_async_hcall(drc, continue_token, &flush_worker_cb, req_d= ata); + + ret =3D spapr_drc_get_async_hcall_status(drc, continue_token); + if (ret =3D=3D H_BUSY) { + args[0] =3D req_data->token; + return ret; + } + + return ret; +} + static target_ulong h_scm_unbind_mem(PowerPCCPU *cpu, SpaprMachineState *s= papr, target_ulong opcode, target_ulong *ar= gs) { @@ -486,6 +564,7 @@ static void spapr_scm_register_types(void) spapr_register_hypercall(H_SCM_BIND_MEM, h_scm_bind_mem); spapr_register_hypercall(H_SCM_UNBIND_MEM, h_scm_unbind_mem); spapr_register_hypercall(H_SCM_UNBIND_ALL, h_scm_unbind_all); + spapr_register_hypercall(H_SCM_ASYNC_FLUSH, h_scm_async_flush); } =20 type_init(spapr_scm_register_types) diff --git a/include/hw/mem/nvdimm.h b/include/hw/mem/nvdimm.h index c699842dd0..9e8795766e 100644 --- a/include/hw/mem/nvdimm.h +++ b/include/hw/mem/nvdimm.h @@ -51,6 +51,7 @@ OBJECT_DECLARE_TYPE(NVDIMMDevice, NVDIMMClass, NVDIMM) #define NVDIMM_LABEL_SIZE_PROP "label-size" #define NVDIMM_UUID_PROP "uuid" #define NVDIMM_UNARMED_PROP "unarmed" +#define NVDIMM_SYNC_DAX_PROP "sync-dax" =20 struct NVDIMMDevice { /* private */ @@ -85,6 +86,15 @@ struct NVDIMMDevice { */ bool unarmed; =20 + /* + * On PPC64, + * The 'off' value results in the async-flush-required property set + * in the device tree for pseries machines. When 'off', the guest + * initiates explicity flush requests to the backend device ensuring + * write persistence. + */ + bool sync_dax; + /* * The PPC64 - spapr requires each nvdimm device have a uuid. */ diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h index 2e89e36cfb..6d7110b7dc 100644 --- a/include/hw/ppc/spapr.h +++ b/include/hw/ppc/spapr.h @@ -535,8 +535,9 @@ struct SpaprMachineState { #define H_SCM_BIND_MEM 0x3EC #define H_SCM_UNBIND_MEM 0x3F0 #define H_SCM_UNBIND_ALL 0x3FC +#define H_SCM_ASYNC_FLUSH 0x4A0 =20 -#define MAX_HCALL_OPCODE H_SCM_UNBIND_ALL +#define MAX_HCALL_OPCODE H_SCM_ASYNC_FLUSH =20 /* The hcalls above are standardized in PAPR and implemented by pHyp * as well.