From nobody Mon May 6 22:16:24 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=quarantine dis=none) header.from=suse.com ARC-Seal: i=1; a=rsa-sha256; t=1606201305; cv=none; d=zohomail.com; s=zohoarc; b=esE7nbK5QFzsUBa/kis/esZg4W2fHErl6fNdZKEy5oslzvbofWctJ28gQbd6FySJ/N3CxHBI8SWTI+/KcBHy2bGb5R4crcf7xzLGctb9JoV3J5CfS42wp4XFVW1YDg/jvcXtm9WMCfbbsqVQSgobBwdQ9l5hvSXxZsosnkHCjUE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1606201305; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=QEucFGeONQJdQQwxfd3osX/7AAW0Pbr2/fwfaONhMq0=; b=lY0AlGHF2yCmLFgj83IG5IWNSIbN/jBAYZZKWpjVTpHPK8H3DlNIc0SgTGDvPZwLX5BPCKVvapEculKASegNwEj7gVtoRCR/YRPlQkyrzAxZLheRZ8msiZdHyOcB6JOixHgUncYbp+36j+vgHKOhu8mbUPU83S53fcc9vcXfaVE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=quarantine dis=none) header.from= Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1606201305261181.0500902496674; Mon, 23 Nov 2020 23:01:45 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.35379.66908 (Exim 4.92) (envelope-from ) id 1khSKI-0001qZ-LL; Tue, 24 Nov 2020 07:01:22 +0000 Received: by outflank-mailman (output) from mailman id 35379.66908; Tue, 24 Nov 2020 07:01:22 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1khSKI-0001qS-Hj; Tue, 24 Nov 2020 07:01:22 +0000 Received: by outflank-mailman (input) for mailman id 35379; Tue, 24 Nov 2020 07:01:22 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1khSKI-0001kq-4R for xen-devel@lists.xenproject.org; Tue, 24 Nov 2020 07:01:22 +0000 Received: from mx2.suse.de (unknown [195.135.220.15]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id d691081d-7325-4f16-9a49-8c4ac1eeac55; Tue, 24 Nov 2020 07:01:10 +0000 (UTC) Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id 60C81ADAA; Tue, 24 Nov 2020 07:01:09 +0000 (UTC) Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1khSKI-0001kq-4R for xen-devel@lists.xenproject.org; Tue, 24 Nov 2020 07:01:22 +0000 Received: from mx2.suse.de (unknown [195.135.220.15]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id d691081d-7325-4f16-9a49-8c4ac1eeac55; Tue, 24 Nov 2020 07:01:10 +0000 (UTC) Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id 60C81ADAA; Tue, 24 Nov 2020 07:01:09 +0000 (UTC) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: d691081d-7325-4f16-9a49-8c4ac1eeac55 X-Virus-Scanned: by amavisd-new at test-mx.suse.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1606201269; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QEucFGeONQJdQQwxfd3osX/7AAW0Pbr2/fwfaONhMq0=; b=dUiEWDaF3N/1coDsYe6Jr8SuU+TPBtmwdsNxCgNq+fnPQaBki8NY0UTBQjD2XRacgs+JPD 8xWQqvdH26vDPVUS0G1RaeAidD4K7C+eOOFTyyThhOuUvyTUnh94MXG5DbWPSRgL8MpFeG E7Mv5c+1/No5wxDNGTAjWizgL0G1wbs= From: Juergen Gross To: xen-devel@lists.xenproject.org Cc: Juergen Gross , Andrew Cooper , George Dunlap , Ian Jackson , Jan Beulich , Julien Grall , Stefano Stabellini , Wei Liu , Julien Grall Subject: [PATCH v7 1/3] xen/events: access last_priority and last_vcpu_id together Date: Tue, 24 Nov 2020 08:01:04 +0100 Message-Id: <20201124070106.26854-2-jgross@suse.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20201124070106.26854-1-jgross@suse.com> References: <20201124070106.26854-1-jgross@suse.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @suse.com) Content-Type: text/plain; charset="utf-8" The queue for a fifo event is depending on the vcpu_id and the priority of the event. When sending an event it might happen the event needs to change queues and the old queue needs to be kept for keeping the links between queue elements intact. For this purpose the event channel contains last_priority and last_vcpu_id values elements for being able to identify the old queue. In order to avoid races always access last_priority and last_vcpu_id with a single atomic operation avoiding any inconsistencies. Signed-off-by: Juergen Gross Reviewed-by: Julien Grall --- xen/common/event_fifo.c | 25 +++++++++++++++++++------ xen/include/xen/sched.h | 3 +-- 2 files changed, 20 insertions(+), 8 deletions(-) diff --git a/xen/common/event_fifo.c b/xen/common/event_fifo.c index c6e58d2a1a..79090c04ca 100644 --- a/xen/common/event_fifo.c +++ b/xen/common/event_fifo.c @@ -42,6 +42,14 @@ struct evtchn_fifo_domain { unsigned int num_evtchns; }; =20 +union evtchn_fifo_lastq { + uint32_t raw; + struct { + uint8_t last_priority; + uint16_t last_vcpu_id; + }; +}; + static inline event_word_t *evtchn_fifo_word_from_port(const struct domain= *d, unsigned int port) { @@ -86,16 +94,18 @@ static struct evtchn_fifo_queue *lock_old_queue(const s= truct domain *d, struct vcpu *v; struct evtchn_fifo_queue *q, *old_q; unsigned int try; + union evtchn_fifo_lastq lastq; =20 for ( try =3D 0; try < 3; try++ ) { - v =3D d->vcpu[evtchn->last_vcpu_id]; - old_q =3D &v->evtchn_fifo->queue[evtchn->last_priority]; + lastq.raw =3D read_atomic(&evtchn->fifo_lastq); + v =3D d->vcpu[lastq.last_vcpu_id]; + old_q =3D &v->evtchn_fifo->queue[lastq.last_priority]; =20 spin_lock_irqsave(&old_q->lock, *flags); =20 - v =3D d->vcpu[evtchn->last_vcpu_id]; - q =3D &v->evtchn_fifo->queue[evtchn->last_priority]; + v =3D d->vcpu[lastq.last_vcpu_id]; + q =3D &v->evtchn_fifo->queue[lastq.last_priority]; =20 if ( old_q =3D=3D q ) return old_q; @@ -246,8 +256,11 @@ static void evtchn_fifo_set_pending(struct vcpu *v, st= ruct evtchn *evtchn) /* Moved to a different queue? */ if ( old_q !=3D q ) { - evtchn->last_vcpu_id =3D v->vcpu_id; - evtchn->last_priority =3D q->priority; + union evtchn_fifo_lastq lastq =3D { }; + + lastq.last_vcpu_id =3D v->vcpu_id; + lastq.last_priority =3D q->priority; + write_atomic(&evtchn->fifo_lastq, lastq.raw); =20 spin_unlock_irqrestore(&old_q->lock, flags); spin_lock_irqsave(&q->lock, flags); diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h index 7251b3ae3e..a345cc01f8 100644 --- a/xen/include/xen/sched.h +++ b/xen/include/xen/sched.h @@ -117,8 +117,7 @@ struct evtchn #ifndef NDEBUG u8 old_state; /* State when taking lock in write mode. */ #endif - u8 last_priority; - u16 last_vcpu_id; + u32 fifo_lastq; /* Data for fifo events identifying last queue. */ #ifdef CONFIG_XSM union { #ifdef XSM_NEED_GENERIC_EVTCHN_SSID --=20 2.26.2 From nobody Mon May 6 22:16:24 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=quarantine dis=none) header.from=suse.com ARC-Seal: i=1; a=rsa-sha256; t=1606201307; cv=none; d=zohomail.com; s=zohoarc; b=CJ/nHiWwOdtXMiQDd/V+oTzk4uWTmhJVKxOGR/rQWPz2UaoEY+7CqY3+m4IUad1VXOBkccI8FUopymRNE+kk8WebVAx7Q0s1Cz8l4kJ/tofyTmFnzQd9+0n+Yl6k6DjX2OlXL7G4f2UtXeQ0maAfoYzYAwnNN8mivn10QCus56Y= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1606201307; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=uY+rObtaesBiYwhFR54/hfk+UTURWjLhSDOKmBjo9Hk=; b=YYh27kteUQ7c1HENtdoSBTsndrmmP+5LgJJxLOkc+gSmyrWJEwlnIS5Wd5IeNfHnL/Kqjbk8SkuhDX+vdL1GZlb1gtrVN8Qv/ZtmWJkgNNOQcl7TGocnl76YsEuXZL4UEiLdD3CLB/ohQTTPXk3hTO3eqlerflSw+zE+sXDvdr0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=quarantine dis=none) header.from= Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1606201307205381.984064554464; Mon, 23 Nov 2020 23:01:47 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.35378.66897 (Exim 4.92) (envelope-from ) id 1khSKE-0001nR-CI; Tue, 24 Nov 2020 07:01:18 +0000 Received: by outflank-mailman (output) from mailman id 35378.66897; Tue, 24 Nov 2020 07:01:18 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1khSKE-0001nH-7p; Tue, 24 Nov 2020 07:01:18 +0000 Received: by outflank-mailman (input) for mailman id 35378; Tue, 24 Nov 2020 07:01:17 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1khSKD-0001kq-3z for xen-devel@lists.xenproject.org; Tue, 24 Nov 2020 07:01:17 +0000 Received: from mx2.suse.de (unknown [195.135.220.15]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 38370360-fb20-49fc-88ce-cf4fc5afc72c; Tue, 24 Nov 2020 07:01:10 +0000 (UTC) Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id A013DADCD; Tue, 24 Nov 2020 07:01:09 +0000 (UTC) Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1khSKD-0001kq-3z for xen-devel@lists.xenproject.org; Tue, 24 Nov 2020 07:01:17 +0000 Received: from mx2.suse.de (unknown [195.135.220.15]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 38370360-fb20-49fc-88ce-cf4fc5afc72c; Tue, 24 Nov 2020 07:01:10 +0000 (UTC) Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id A013DADCD; Tue, 24 Nov 2020 07:01:09 +0000 (UTC) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 38370360-fb20-49fc-88ce-cf4fc5afc72c X-Virus-Scanned: by amavisd-new at test-mx.suse.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1606201269; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=uY+rObtaesBiYwhFR54/hfk+UTURWjLhSDOKmBjo9Hk=; b=DuiB6acuqrH4C2absqcxEFRbDrXB2fDjBgSxIwxTz1tlybaHV+QEpsJU02wI1BLe46mAw0 QMHW2k4JCUWGGaNTw0+FqZKNpJPpydJ3mutCADpZN9FTGdUIzLDMcOLuaegMoB+IhE4m7h YMuNtsNn46pIrv/Xlh+AWDys/+6aZfQ= From: Juergen Gross To: xen-devel@lists.xenproject.org Cc: Juergen Gross , Andrew Cooper , George Dunlap , Ian Jackson , Jan Beulich , Julien Grall , Stefano Stabellini , Wei Liu Subject: [PATCH v7 2/3] xen/events: modify struct evtchn layout Date: Tue, 24 Nov 2020 08:01:05 +0100 Message-Id: <20201124070106.26854-3-jgross@suse.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20201124070106.26854-1-jgross@suse.com> References: <20201124070106.26854-1-jgross@suse.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @suse.com) Content-Type: text/plain; charset="utf-8" In order to avoid latent races when updating an event channel put xen_consumer and pending fields in different bytes. At the same time move some other fields around to have less implicit paddings and to keep related fields more closely together. Signed-off-by: Juergen Gross --- xen/include/xen/sched.h | 22 ++++++++++++---------- 1 file changed, 12 insertions(+), 10 deletions(-) diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h index a345cc01f8..e6d09aa055 100644 --- a/xen/include/xen/sched.h +++ b/xen/include/xen/sched.h @@ -80,8 +80,7 @@ extern domid_t hardware_domid; #define EVTCHNS_PER_GROUP (BUCKETS_PER_GROUP * EVTCHNS_PER_BUCKET) #define NR_EVTCHN_GROUPS DIV_ROUND_UP(MAX_NR_EVTCHNS, EVTCHNS_PER_GROUP) =20 -#define XEN_CONSUMER_BITS 3 -#define NR_XEN_CONSUMERS ((1 << XEN_CONSUMER_BITS) - 1) +#define NR_XEN_CONSUMERS 8 =20 struct evtchn { @@ -94,9 +93,10 @@ struct evtchn #define ECS_VIRQ 5 /* Channel is bound to a virtual IRQ line. = */ #define ECS_IPI 6 /* Channel is bound to a virtual IPI line. = */ u8 state; /* ECS_* */ - u8 xen_consumer:XEN_CONSUMER_BITS; /* Consumer in Xen if nonzero */ - u8 pending:1; - u16 notify_vcpu_id; /* VCPU for local delivery notification */ +#ifndef NDEBUG + u8 old_state; /* State when taking lock in write mode. */ +#endif + u8 xen_consumer; /* Consumer in Xen if nonzero */ u32 port; union { struct { @@ -113,11 +113,13 @@ struct evtchn } pirq; /* state =3D=3D ECS_PIRQ */ u16 virq; /* state =3D=3D ECS_VIRQ */ } u; - u8 priority; -#ifndef NDEBUG - u8 old_state; /* State when taking lock in write mode. */ -#endif - u32 fifo_lastq; /* Data for fifo events identifying last queue. */ + + /* FIFO event channels only. */ + u8 pending; + u8 priority; + u16 notify_vcpu_id; /* VCPU for local delivery notification */ + u32 fifo_lastq; /* Data for identifying last queue. */ + #ifdef CONFIG_XSM union { #ifdef XSM_NEED_GENERIC_EVTCHN_SSID --=20 2.26.2 From nobody Mon May 6 22:16:24 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=quarantine dis=none) header.from=suse.com ARC-Seal: i=1; a=rsa-sha256; t=1606201305; cv=none; d=zohomail.com; s=zohoarc; b=E7CrX67QFl1MuZKrmz6EwTqe4GCMCPO92yola+ClyW34rwtnjPXfc87C45kmM6rhTBGN8R04ZbTAVtOsaAEV7hyDdJaRDyk8ojmRjmeGgcz9LtIItJyoNbIGc6JcaAcf6l2T1uX7y90vmcTGPR9ND7X8TdDU25BUrJIwpkPDMMQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1606201305; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=Yz6J7wdCjuva4czegeT7IAWoCsKh1vZ7NlNUOs0HkPI=; b=NNh9x/h89Bh4UOodgCsEcWq97iPTdnZHdzARsqTfnPKKIPszOB8l9MoA8C6zOxDE9uAVr3Y5TbPnxfT9Y+RcrgHrop26Lj1Mtp/CGv/HcRsh0eLIj6MIO/BZnwdTmettOxHJ7viyKGESo+EwnLXgMP6njTE0mD0d1beiF091hEQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=quarantine dis=none) header.from= Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 16062013053701019.3551561240986; Mon, 23 Nov 2020 23:01:45 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.35377.66885 (Exim 4.92) (envelope-from ) id 1khSKB-0001lz-3J; Tue, 24 Nov 2020 07:01:15 +0000 Received: by outflank-mailman (output) from mailman id 35377.66885; Tue, 24 Nov 2020 07:01:15 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1khSKA-0001ls-Vw; Tue, 24 Nov 2020 07:01:14 +0000 Received: by outflank-mailman (input) for mailman id 35377; Tue, 24 Nov 2020 07:01:13 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1khSK9-0001kv-BO for xen-devel@lists.xenproject.org; Tue, 24 Nov 2020 07:01:13 +0000 Received: from mx2.suse.de (unknown [195.135.220.15]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 3f16b6f2-66de-4860-8df4-aa00258c40a5; Tue, 24 Nov 2020 07:01:10 +0000 (UTC) Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id E99DCADD5; Tue, 24 Nov 2020 07:01:09 +0000 (UTC) Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1khSK9-0001kv-BO for xen-devel@lists.xenproject.org; Tue, 24 Nov 2020 07:01:13 +0000 Received: from mx2.suse.de (unknown [195.135.220.15]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 3f16b6f2-66de-4860-8df4-aa00258c40a5; Tue, 24 Nov 2020 07:01:10 +0000 (UTC) Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id E99DCADD5; Tue, 24 Nov 2020 07:01:09 +0000 (UTC) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 3f16b6f2-66de-4860-8df4-aa00258c40a5 X-Virus-Scanned: by amavisd-new at test-mx.suse.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1606201270; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Yz6J7wdCjuva4czegeT7IAWoCsKh1vZ7NlNUOs0HkPI=; b=Ri1obx6a7eZp4WW1J7KUmJbRyO6puXP6b/7u2DmSYjNGeiGGfx3+oXKKljh8zElqbgaWp1 OVWRG7/iRhdGbZ9kytMy5IbkDc1fVMtpwgnSf43rYs+nmbRXvRefHBag1Ktys+AuPZFHNY P+pFkpiB0GwOeK/fYwk42cme8KNBbrs= From: Juergen Gross To: xen-devel@lists.xenproject.org Cc: Juergen Gross , Andrew Cooper , George Dunlap , Ian Jackson , Jan Beulich , Julien Grall , Stefano Stabellini , Wei Liu Subject: [PATCH v7 3/3] xen/events: rework fifo queue locking Date: Tue, 24 Nov 2020 08:01:06 +0100 Message-Id: <20201124070106.26854-4-jgross@suse.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20201124070106.26854-1-jgross@suse.com> References: <20201124070106.26854-1-jgross@suse.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @suse.com) Content-Type: text/plain; charset="utf-8" Two cpus entering evtchn_fifo_set_pending() for the same event channel can race in case the first one gets interrupted after setting EVTCHN_FIFO_PENDING and when the other one manages to set EVTCHN_FIFO_LINKED before the first one is testing that bit. This can lead to evtchn_check_pollers() being called before the event is put properly into the queue, resulting eventually in the guest not seeing the event pending and thus blocking forever afterwards. Note that commit 5f2df45ead7c1195 ("xen/evtchn: rework per event channel lock") made the race just more obvious, while the fifo event channel implementation had this race from the beginning when an unmask operation was running in parallel with an event channel send operation. For avoiding this race the queue locking in evtchn_fifo_set_pending() needs to be reworked to cover the test of EVTCHN_FIFO_PENDING, EVTCHN_FIFO_MASKED and EVTCHN_FIFO_LINKED, too. Additionally when an event channel needs to change queues both queues need to be locked initially. Fixes: 5f2df45ead7c1195 ("xen/evtchn: rework per event channel lock") Fixes: 88910061ec615b2d ("evtchn: add FIFO-based event channel hypercalls a= nd port ops") Signed-off-by: Juergen Gross --- xen/common/event_fifo.c | 115 ++++++++++++++++++++-------------------- 1 file changed, 58 insertions(+), 57 deletions(-) diff --git a/xen/common/event_fifo.c b/xen/common/event_fifo.c index 79090c04ca..a57d459cc2 100644 --- a/xen/common/event_fifo.c +++ b/xen/common/event_fifo.c @@ -87,38 +87,6 @@ static void evtchn_fifo_init(struct domain *d, struct ev= tchn *evtchn) d->domain_id, evtchn->port); } =20 -static struct evtchn_fifo_queue *lock_old_queue(const struct domain *d, - struct evtchn *evtchn, - unsigned long *flags) -{ - struct vcpu *v; - struct evtchn_fifo_queue *q, *old_q; - unsigned int try; - union evtchn_fifo_lastq lastq; - - for ( try =3D 0; try < 3; try++ ) - { - lastq.raw =3D read_atomic(&evtchn->fifo_lastq); - v =3D d->vcpu[lastq.last_vcpu_id]; - old_q =3D &v->evtchn_fifo->queue[lastq.last_priority]; - - spin_lock_irqsave(&old_q->lock, *flags); - - v =3D d->vcpu[lastq.last_vcpu_id]; - q =3D &v->evtchn_fifo->queue[lastq.last_priority]; - - if ( old_q =3D=3D q ) - return old_q; - - spin_unlock_irqrestore(&old_q->lock, *flags); - } - - gprintk(XENLOG_WARNING, - "dom%d port %d lost event (too many queue changes)\n", - d->domain_id, evtchn->port); - return NULL; -} =20 - static int try_set_link(event_word_t *word, event_word_t *w, uint32_t link) { event_word_t new, old; @@ -190,6 +158,9 @@ static void evtchn_fifo_set_pending(struct vcpu *v, str= uct evtchn *evtchn) event_word_t *word; unsigned long flags; bool_t was_pending; + struct evtchn_fifo_queue *q, *old_q; + unsigned int try; + bool linked =3D true; =20 port =3D evtchn->port; word =3D evtchn_fifo_word_from_port(d, port); @@ -204,6 +175,48 @@ static void evtchn_fifo_set_pending(struct vcpu *v, st= ruct evtchn *evtchn) return; } =20 + for ( try =3D 0; ; try++ ) + { + union evtchn_fifo_lastq lastq; + struct vcpu *old_v; + + lastq.raw =3D read_atomic(&evtchn->fifo_lastq); + old_v =3D d->vcpu[lastq.last_vcpu_id]; + + q =3D &v->evtchn_fifo->queue[evtchn->priority]; + old_q =3D &old_v->evtchn_fifo->queue[lastq.last_priority]; + + if ( q <=3D old_q ) + { + spin_lock_irqsave(&q->lock, flags); + if ( q !=3D old_q ) + spin_lock(&old_q->lock); + } + else + { + spin_lock_irqsave(&old_q->lock, flags); + spin_lock(&q->lock); + } + + lastq.raw =3D read_atomic(&evtchn->fifo_lastq); + old_v =3D d->vcpu[lastq.last_vcpu_id]; + if ( q =3D=3D &v->evtchn_fifo->queue[evtchn->priority] && + old_q =3D=3D &old_v->evtchn_fifo->queue[lastq.last_priority] ) + break; + + if ( q !=3D old_q ) + spin_unlock(&old_q->lock); + spin_unlock_irqrestore(&q->lock, flags); + + if ( try =3D=3D 3 ) + { + gprintk(XENLOG_WARNING, + "dom%d port %d lost event (too many queue changes)\n", + d->domain_id, evtchn->port); + return; + } + } + was_pending =3D guest_test_and_set_bit(d, EVTCHN_FIFO_PENDING, word); =20 /* @@ -212,9 +225,7 @@ static void evtchn_fifo_set_pending(struct vcpu *v, str= uct evtchn *evtchn) if ( !guest_test_bit(d, EVTCHN_FIFO_MASKED, word) && !guest_test_bit(d, EVTCHN_FIFO_LINKED, word) ) { - struct evtchn_fifo_queue *q, *old_q; event_word_t *tail_word; - bool_t linked =3D 0; =20 /* * Control block not mapped. The guest must not unmask an @@ -228,22 +239,8 @@ static void evtchn_fifo_set_pending(struct vcpu *v, st= ruct evtchn *evtchn) goto done; } =20 - /* - * No locking around getting the queue. This may race with - * changing the priority but we are allowed to signal the - * event once on the old priority. - */ - q =3D &v->evtchn_fifo->queue[evtchn->priority]; - - old_q =3D lock_old_queue(d, evtchn, &flags); - if ( !old_q ) - goto done; - if ( guest_test_and_set_bit(d, EVTCHN_FIFO_LINKED, word) ) - { - spin_unlock_irqrestore(&old_q->lock, flags); goto done; - } =20 /* * If this event was a tail, the old queue is now empty and @@ -262,8 +259,8 @@ static void evtchn_fifo_set_pending(struct vcpu *v, str= uct evtchn *evtchn) lastq.last_priority =3D q->priority; write_atomic(&evtchn->fifo_lastq, lastq.raw); =20 - spin_unlock_irqrestore(&old_q->lock, flags); - spin_lock_irqsave(&q->lock, flags); + spin_unlock(&old_q->lock); + old_q =3D q; } =20 /* @@ -276,6 +273,7 @@ static void evtchn_fifo_set_pending(struct vcpu *v, str= uct evtchn *evtchn) * If the queue is empty (i.e., we haven't linked to the new * event), head must be updated. */ + linked =3D false; if ( q->tail ) { tail_word =3D evtchn_fifo_word_from_port(d, q->tail); @@ -284,15 +282,18 @@ static void evtchn_fifo_set_pending(struct vcpu *v, s= truct evtchn *evtchn) if ( !linked ) write_atomic(q->head, port); q->tail =3D port; - - spin_unlock_irqrestore(&q->lock, flags); - - if ( !linked - && !guest_test_and_set_bit(d, q->priority, - &v->evtchn_fifo->control_block->re= ady) ) - vcpu_mark_events_pending(v); } + done: + if ( q !=3D old_q ) + spin_unlock(&old_q->lock); + spin_unlock_irqrestore(&q->lock, flags); + + if ( !linked && + !guest_test_and_set_bit(d, q->priority, + &v->evtchn_fifo->control_block->ready) ) + vcpu_mark_events_pending(v); + if ( !was_pending ) evtchn_check_pollers(d, port); } --=20 2.26.2