From nobody Fri Dec 27 03:16:20 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass header.i=teddy.astie@vates.tech; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=quarantine dis=none) header.from=vates.tech ARC-Seal: i=1; a=rsa-sha256; t=1732284128; cv=none; d=zohomail.com; s=zohoarc; b=KemwaL+W6o8VCReG+Eh3I9QqN51kwHehSx3erfCBrMJRG24GuVVz2hcNKj3jyzDvJL26fIRnpFnEOlBMLO3eSaoHWc9v3zAuy+TjNRPeXw0cgNykQwtg6o6J3D+7oerfP/isIodhu31IFqFgnvnU27Yw0yHkFYOLdKTTCDatBcw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1732284128; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=jWuIepqC8yaWMkVG+vgwEXX3z5Z3/zOLNl3TS3kIncc=; b=IX+VSMTZFYlfLSWGnSDSmZeGBYKRQmGU/zGQPzjF8Re2IahAHn1cTckIFl3VwOFqUtkAjNenY4syg+w/2XfAxeB3bnUTlfYpKmC+ix3+r+hZRcbB7ZaWCXgVTZeW4l6A7xGccOUhOpr94ASdVg/QcKPccxJGgkgBTRqmQQbWVow= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=teddy.astie@vates.tech; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1732284128217833.4796764981533; Fri, 22 Nov 2024 06:02:08 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.841905.1257383 (Exim 4.92) (envelope-from ) id 1tEUE8-0005n7-NK; Fri, 22 Nov 2024 14:01:40 +0000 Received: by outflank-mailman (output) from mailman id 841905.1257383; Fri, 22 Nov 2024 14:01:40 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tEUE8-0005n0-JP; Fri, 22 Nov 2024 14:01:40 +0000 Received: by outflank-mailman (input) for mailman id 841905; Fri, 22 Nov 2024 14:01:40 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tEUE8-0005mu-4R for xen-devel@lists.xenproject.org; Fri, 22 Nov 2024 14:01:40 +0000 Received: from mail134-5.atl141.mandrillapp.com (mail134-5.atl141.mandrillapp.com [198.2.134.5]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 48bdf5b1-a8da-11ef-99a3-01e77a169b0f; Fri, 22 Nov 2024 15:01:35 +0100 (CET) Received: from pmta10.mandrill.prod.atl01.rsglab.com (localhost [127.0.0.1]) by mail134-5.atl141.mandrillapp.com (Mailchimp) with ESMTP id 4Xvxbd54mJzG0CJtY for ; Fri, 22 Nov 2024 14:01:33 +0000 (GMT) Received: from [37.26.189.201] by mandrillapp.com id 287a498457444c1f939bb637caa01303; Fri, 22 Nov 2024 14:01:33 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 48bdf5b1-a8da-11ef-99a3-01e77a169b0f X-Custom-Connection: eyJyZW1vdGVpcCI6IjE5OC4yLjEzNC41IiwiaGVsbyI6Im1haWwxMzQtNS5hdGwxNDEubWFuZHJpbGxhcHAuY29tIn0= X-Custom-Transaction: eyJpZCI6IjQ4YmRmNWIxLWE4ZGEtMTFlZi05OWEzLTAxZTc3YTE2OWIwZiIsInRzIjoxNzMyMjg0MDk1Ljg2MTI0NCwic2VuZGVyIjoiYm91bmNlLW1kXzMwNTA0OTYyLjY3NDA4ZWJkLnYxLTI4N2E0OTg0NTc0NDRjMWY5MzliYjYzN2NhYTAxMzAzQGJvdW5jZS52YXRlcy50ZWNoIiwicmVjaXBpZW50IjoieGVuLWRldmVsQGxpc3RzLnhlbnByb2plY3Qub3JnIn0= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mandrillapp.com; s=mte1; t=1732284093; x=1732544593; bh=jWuIepqC8yaWMkVG+vgwEXX3z5Z3/zOLNl3TS3kIncc=; h=From:Subject:To:Cc:Message-Id:Feedback-ID:Date:MIME-Version: Content-Type:Content-Transfer-Encoding:CC:Date:Subject:From; b=IUUXvgIDDsqeWZqJmIHD1gEibcp2TaTt4ZhgUKFk0tBwCDR9ee/BBS/2a+P9xGAJ7 1CeahGg5RG6OuArYq/e53l6vJBd3tKWwTrZCfx88zE++5u8Rvi65m/Ruc0Hypyks2Q ghmeyB7XuSC8VxsL4nweN0NaXbm4b8HBfGy28NwaR70tHE03oUqWqdvh5WqT2qNhmj hsQkGLXidsROeIsZRZvE8OMBkqB3CcaYRlQI/rLWbUxLZn+wgIoH2bXVxbKAZfyOUO cjWtNGgTxIu0twEbZRfHMhsYgeK9ShloRFprihXOpejO6pBN5pfKFbGcdJVNzoxbts XRCPqIrv02dKQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=vates.tech; s=mte1; t=1732284093; x=1732544593; i=teddy.astie@vates.tech; bh=jWuIepqC8yaWMkVG+vgwEXX3z5Z3/zOLNl3TS3kIncc=; h=From:Subject:To:Cc:Message-Id:Feedback-ID:Date:MIME-Version: Content-Type:Content-Transfer-Encoding:CC:Date:Subject:From; b=kb7atfSoLkhf+3B2nxXJjH6JXnkSXeVKt+narDrrQtuZjnexDG7ItWxwxISG6PFo2 TJaJOv0tZZlpITbhdLRV18IX1KZPL26eGep/SFjByy+8bwkeSdcXJCwJki6BRPrNLI ccKVBNW1fsjTR5xhMo+jk82GNnLbXjLscHYkyL9LWnmz5FnmYbOAoI0lNB6y32EFW1 bm3mJVX6MzFZezd3lKArgM04+OvSbgPk7QHGiVaeqyCAK3bJIjQjMukJliEBxRbuSd lLmjYCywsblkxecY8jqYxWbrVOHNSn8noUf6QaynZoltB3GwT6nseQu9Z2oxI/edNX v3GWMGbF/Vnyg== From: "Teddy Astie" Subject: =?utf-8?Q?[RFC=20XEN=20PATCH]=20docs/designs:=20Add=20a=20design=20document=20for=20'xen'=20Rust=20crate?= X-Mailer: git-send-email 2.45.2 X-Bm-Disclaimer: Yes X-Bm-Milter-Handled: 4ffbd6c1-ee69-4e1b-aabd-f977039bd3e2 X-Bm-Transport-Timestamp: 1732284092312 To: xen-devel@lists.xenproject.org Cc: "Teddy Astie" , "Andrew Cooper" , "Jan Beulich" , "Julien Grall" , "Stefano Stabellini" , "Alejandro Vallejo" , "Anthony PERARD" Message-Id: X-Native-Encoded: 1 X-Report-Abuse: =?UTF-8?Q?Please=20forward=20a=20copy=20of=20this=20message,=20including=20all=20headers,=20to=20abuse@mandrill.com.=20You=20can=20also=20report=20abuse=20here:=20https://mandrillapp.com/contact/abuse=3Fid=3D30504962.287a498457444c1f939bb637caa01303?= X-Mandrill-User: md_30504962 Feedback-ID: 30504962:30504962.20241122:md Date: Fri, 22 Nov 2024 14:01:33 +0000 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity teddy.astie@vates.tech) (identity @mandrillapp.com) X-ZM-MESSAGEID: 1732284130879116600 Content-Type: text/plain; charset="utf-8" Add a design document for the 'xen' rust crate. Signed-off-by: Teddy Astie --- Follows Alejandro works with https://lore.kernel.org/xen-devel/D5SQEZIL2SZV= .QR3X5MRVQJJP@cloud.com/T/#t There is also a WIP branch at https://gitlab.com/TSnake41/xen/-/tree/rust-r= fc --- docs/designs/xen-rust-crate.md | 199 +++++++++++++++++++++++++++++++++ 1 file changed, 199 insertions(+) create mode 100644 docs/designs/xen-rust-crate.md diff --git a/docs/designs/xen-rust-crate.md b/docs/designs/xen-rust-crate.md new file mode 100644 index 0000000000..5eab603d4d --- /dev/null +++ b/docs/designs/xen-rust-crate.md @@ -0,0 +1,199 @@ +# Rust 'xen' crate interface design + +See [1] for more context. + +This RFC proposes parts of the 'xen' crate interface that would directly o= r indirectly +(through internal wrappers) be used by users. +Those users could be a userland toolstack, a unikernel application (e.g XT= F, Unikraft), +some other freestanding environment (e.g OVMF, Linux, Redox OS), ... + +These users can have a very different execution environment, this crate ai= ms to provide +a uniform interface while allowing flexibility for exposing platform-speci= fic bits. + +## Design philosophy + +This crate should feel natural for a Rust developper, thus, any Rust devel= opper with some +understanding on the Xen hypercall operations should be able to use this c= rate. +Moreover, we want this crate to be maintainable and feel "idiomatic", and = not introduce +confusing behavior onto the user. Note that this crate will heavily use un= safe code. + +Some principles are proposed : + +Use or provide idiomatic abstractions when relevant (reuse standard traits= ). + +Examples: + Provide (optional) Future<...> abstractions for event channels + Provide a iterator-based (Stream ? [2]) abstraction for guest console. + +Don't restrict features to some execution environment, use modular abstrac= tions (e.g traits) +to allow the user to specify the missing bits himself / provide its own im= plementation. +Note that it doesn't prevent us from exposing the platform-specific bits o= nto the +types themselves (e.g UnixXenEventChannel can still expose its file descri= ptor). + +Example: + If we provide a event channel abstraction based on hypercall but it does= n't implement Future<...>, + the user can still implement its own type on top of the hypercall implem= entation, and + use its own async runtime (e.g based on interrupts) to implement Future<= ...> himself. + There could be 2 traits for varying needs : + EventChannel (base trait) and AsyncEventChannel (await-able EventChann= el) + + We can have both RawEventChannel based on XenHypercall that only impleme= nts EventChannel + and another type TokioEventChannel that provides both EventChannel and A= syncEventChannel + and integrates with tokio runtime. + +Safe wrappers must be "sound" and unsafe code shall not cause undefined be= havior. +- safe wrappers must not cause undefined behavior on their own +- unsafe code should not cause undefined behavior if properly used + +This is a bit tricky due to some Xen specificities, but we expect hypercal= l to be well +formed (we can add validation tools for that) including have its parameter= indirectly +respect the aliasing rules [3]. +Although, we assume that Xen is well-behaving regarding its ABI. +We don't define misuse of a hypercall that can harm the guest himself, but= we care +about not causing a undefined behavior (e.g by passing a buggy pointer) th= rough the +hypercall interface that can overwrite unrelated/arbitrary kernel memory. + +## Hypercall interface + +We introduce a XenHypercall trait that exposes a raw primitive for making = hypercalls. +This interface supposes nothing on the ABI used in Xen, and its the respon= sibility +of the user of such interface (often safe wrappers) that the hypercall mad= e is +meaningful. + +This interface is mostly to only be used by the crate developpers to build= safe +wrappers on top, or by advanced user for using non-exposed/WIP hypercall i= nterfaces +or bypassing the safe wrappers. + +We can implement this interface for freestanding platforms using direct na= tive hypercalls +(e.g using inline assembly) for freestanding platforms or in userland usin= g special devices +like privcmd/xencall on most Unix platforms. + +```rust +pub trait XenHypercall: Sized { + unsafe fn hypercall5(&self, cmd: usize, param: [usize; 5]) -> usize; + + unsafe fn hypercall4(&self, cmd: usize, param: [usize; 4]) -> usize; + unsafe fn hypercall3(&self, cmd: usize, param: [usize; 3]) -> usize; + unsafe fn hypercall2(&self, cmd: usize, param: [usize; 2]) -> usize; + unsafe fn hypercall1(&self, cmd: usize, param: usize) -> usize; + unsafe fn hypercall0(&self, cmd: usize) -> usize; + + /* ... */ +} +``` + +### Hypercall-safe buffers + +One difficulty is that in a freestanding environment, we need to use point= ers to +original data. But in a hosted environment, we need to make special buffer= s instead +for that. + +We introduce the Xen{Const/Mut}Buffer generic trait that wraps a reference= in a +"hypercall-safe" buffer that may or may not be a bounce buffer. + +```rust +/// Wrapper of a reference into a hypercall-safe buffer. +pub trait XenConstBuffer { + /// Get a hypercall-safe reference to underlying data. + fn as_hypercall_ptr(&self) -> *const T; +} + +/// Wrapper of a mutable reference into a mutable hypercall-safe buffer. +pub trait XenMutBuffer { + /// Get a hypercall-safe mutable reference to underlying data. + fn as_hypercall_ptr(&mut self) -> *mut T; + + /// Update original reference with new data. + unsafe fn update(&mut self); +} + +// The user will make those wrappers using dedicated functions in XenHyper= call trait. + +trait XenHypercall: Sized { + /* ... */ + type Error; + + fn make_const_object( + &self, + buffer: &T, + ) -> Result, Self::Error>; + + fn make_mut_buffer( + &self, + buffer: &mut T, + ) -> Result, Self::Error>; + + fn make_const_slice( + &self, + slice: &[T], + ) -> Result, Self::Error>; + + fn make_mut_slice( + &self, + slice: &mut [T], + ) -> Result, Self::Error>; +} +``` + +Example use: + +```rust +fn demo_hypercall(interface: &H, buffer: &mut [u8]) -> Re= sult<(), H::Error> { + let buffer_size =3D buffer.len(); + // make a hypercall-safe wrapper of `buffer` + let hyp_buffer =3D interface.make_mut_slice(buffer)?; + + let op =3D SomeHypercallStruct { + buffer: hyp_buffer.as_hypercall_ptr(), + buffer_size: buffer_size as _, + }; + // Do the same for hyp_op + let hyp_op =3D interface.make_const_object(&op)?; + + unsafe { + interface.hypercall1(SOME_CMD, hyp_op.as_hypercall_ptr().addr()); + // assume success + hyp_buffer.update(); // update buffer back + } + + Ok(()) +} +``` + +Note that freestanding case, we can use a thin zero-copy wrapper : +```rust +/// Constant xen buffer that passes the reference as-is. +pub(super) struct DirectConstXenBuffer<'a, T>(&'a T); + +impl XenConstBuffer for DirectConstXenBuffer<'_, T> { + fn as_hypercall_ptr(&self) -> *const T { + self.0 + } +} +// ... +``` + +TODO: +Do we need to clarify the lifetimes (e.g should trait indicate a lifetime = binding with +original data) ? Try to answer with RPITIT and Rust 2024 capture rules [4]. + +Try to unify make_const_object and make_const_slice (along with mut varian= t). `*const [T]` +is a bit more subtle to create and we cannot trivially cast a address into= a pointer and +need to use special functions for that (`core::ptr::slice_from_raw_parts` = ?). +But for that, we need to know that T is actually a slice before using this= function. + +## Event channels + +TODO + +[1] - Interfacing Rust with Xen - Alejandro Vallejo, XenServer BU, Cloud S= oftware Group +https://youtu.be/iFh4n2kbAwM + +[2] - The Stream Trait +https://rust-lang.github.io/async-book/05_streams/01_chapter.html + +[3] - Aliasing +https://doc.rust-lang.org/nomicon/aliasing.html + +[4] - https://blog.rust-lang.org/2023/12/21/async-fn-rpit-in-traits.html +https://rust-lang.github.io/rfcs/3498-lifetime-capture-rules-2024.html --=20 2.45.2 Teddy Astie | Vates XCP-ng Developer XCP-ng & Xen Orchestra - Vates solutions web: https://vates.tech