From nobody Thu Dec 18 00:34:59 2025 Received: from mail-wr1-f51.google.com (mail-wr1-f51.google.com [209.85.221.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B036138DEC for ; Thu, 24 Apr 2025 14:13:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504032; cv=none; b=Bh05Jo580AulINFLaqka3qY209sdgbf3HCdV+bKj/dTOu8G9ExfhcdB6kTpQhnoymP3kVSD5/6qA5yRPtosTDaxVmCDx3M531NWlKFCp22+/T6GXt0C+q5bVW9dhm5HMYV1qQwfCyjeks78EoGd/Sk3LWMudZtAfn8sEqNVa8cw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504032; c=relaxed/simple; bh=g59SsG0N84wb4f6piJJJcigRQ9wXVyh8p4R6BkHSxiU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=BwRZwnEiZo7e6o3G03szRKEFQXbNB86FvrlZfEWvBEghdUcTrhmpTKzQYmqE80wqhrxmYzRtC3zXb3vFuMcMY38ACMATbddnbJkeRKch2ou8ttkoREXZp5EbDY7iwk44p8idXuiL4JZBvSOqzGzcD/oF8piCreklFnnWaZ898IE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=d4ub933F; arc=none smtp.client-ip=209.85.221.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="d4ub933F" Received: by mail-wr1-f51.google.com with SMTP id ffacd0b85a97d-39c266c1389so744596f8f.1 for ; Thu, 24 Apr 2025 07:13:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504026; x=1746108826; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9pJfw58KOVJlcvsi2YwD0da6h5VnpsCyDZ/he/rc1kk=; b=d4ub933FvrTaDPxCrebBqFETW/r4aHP5Zgr0uloJ5fW2q1BgsSLYE9WfvREpVpK5Yz rFlJg17xycIpGpNDwGK1zzI7gb73tqT1kOVRMlGs7gMFeSid/hTjvGdK6tahBOKODrA9 MIxU7d7+HD7laNzNMfGSRvGavpy/f6Zy7eHGHmV0O8Ii55ik8j1NGSeVDdWQlkcSaI6z le/xGuUuVHwERSloTou3x5Ocpx3XILcqiE2eyLPMMyRuEnmmiAM+J8jK63LuX3uz/rpZ y4zVZGqKyx6CMRrPLhtw2EZmfxMNZSgCyjBgTPAQ6bb1QZRnylv5EaQbwNSG0ZYHJqTG uOng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504027; x=1746108827; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9pJfw58KOVJlcvsi2YwD0da6h5VnpsCyDZ/he/rc1kk=; b=sT4Ju63ptsy7o4404OVmre/eTvU6h21nbYBZDkBhkjXtCj1xuERG9KaZRh5mAe18mR lQc5BT02o8daFq/BVibWqaZwht3GFdievdC3FppUkIfmTQnNPyhpcqtufQuw89QuQvxG 1XG+d8++6iVJz9jpzH6gJWFqW8gyPHdxSmIevPC4HCxFs8n2JIn0qZ9b4HhOX03IUzBx MSm+Z4JzRlaC241IVgs2RV7rcO4FlA5a833KwVkTxitjMZaaTPGYOP2vu3DgukHnsauK PQZSW6qjbUfiGpgLxZBLMuM1nwNBVH/jRSmdE34gQPEWfI9aoiEC8Z/tdnJYf7Xa3AAF u5kg== X-Gm-Message-State: AOJu0YzY6udaFCC65YNK0JjNcTa0vZ+BdGBjOPijWmLEgawUNlgNsOUK 0eTK/qtr6ZnK6mhRNNHnywShbqENnhygv++4mYVid0xwIm/11CaAzEJxuphJGqKV8+pyIpKA7vf U+7o= X-Gm-Gg: ASbGncvu8LIQ5fyHK8kgwNGfqdPrkV4Vtnx1OiFNCJeBNJEfIW1fSmIfPpgNvGabsU8 4N8CRLbZ8+hbVLbMlE8nVIkF2YDv+knUGG+x8lCkcE4w9r7s4zhSsNf1D4whM0JRQUf+ODCpqyE 0sofG3GCkvwfYjx4OQAyTOo1Tl3Egca9UQ7Jcpq91Cvce2ATWi6R7dX1XEC6FmPOhNRPQ8GLhQI Db45PWIO8kqQajIPK04eNKZgTYfvDHJaM0Z3tWJcjacktXWkSvNqJzhNkFNjgzW+wvoDSbAQ0S9 S25AbCRebLwZ8mFhwxV7Cfcfzcb1Kb7nA0uYFn9KW7WYLkjZUf6CMeBDRqBnagtUg+SRDOqQECy T9cpPZtxlim9RazQW X-Google-Smtp-Source: AGHT+IGfRrIJRrexUH30juVxzQxuzQYHkS7hGTbMmJLV2Lp0+vmOpc/tkASkqUk2GbTXQepziYNH6A== X-Received: by 2002:a05:6000:2510:b0:391:304f:34e7 with SMTP id ffacd0b85a97d-3a06cfab9abmr2560733f8f.44.1745504026564; Thu, 24 Apr 2025 07:13:46 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.13.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:13:46 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt Subject: [RFC PATCH 01/34] KVM: Allow arch-specific vCPU allocation and freeing Date: Thu, 24 Apr 2025 15:13:08 +0100 Message-Id: <20250424141341.841734-2-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Gunyah KVM backend [1] requires custom vCPU allocation to associate architecture-specific state with each virtual CPU. The generic KVM core currently allocates vCPUs directly using the kvm_vcpu_cache slab, which does not allow architecture code to intervene in the allocation process. Introduce two weakly-defined functions, kvm_arch_vcpu_alloc() and kvm_arch_vcpu_free(), which default to using kmem_cache_zalloc() and kmem_cache_free() respectively. Architectures can override these functions to implement custom vCPU allocation behavior. Replace all direct allocations and frees of vCPUs in kvm_main.c with calls to these helper functions to allow arch-specific substitution. This change is required to support architectures such as Gunyah that must allocate architecture-private state along with the vCPU. [1] https://github.com/quic/gunyah-hypervisor Signed-off-by: Karim Manaouil --- include/linux/kvm_host.h | 2 ++ virt/kvm/kvm_main.c | 16 +++++++++++++--- 2 files changed, 15 insertions(+), 3 deletions(-) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 1dedc421b3e3..3461346b37e0 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1581,6 +1581,8 @@ int kvm_arch_vcpu_precreate(struct kvm *kvm, unsigned= int id); int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu); void kvm_arch_vcpu_postcreate(struct kvm_vcpu *vcpu); void kvm_arch_vcpu_destroy(struct kvm_vcpu *vcpu); +struct kvm_vcpu *kvm_arch_vcpu_alloc(void); +void kvm_arch_vcpu_free(struct kvm_vcpu *vcpu); =20 #ifdef CONFIG_HAVE_KVM_PM_NOTIFIER int kvm_arch_pm_notifier(struct kvm *kvm, unsigned long state); diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 69782df3617f..dbb7ed95523f 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -476,7 +476,7 @@ static void kvm_vcpu_destroy(struct kvm_vcpu *vcpu) put_pid(vcpu->pid); =20 free_page((unsigned long)vcpu->run); - kmem_cache_free(kvm_vcpu_cache, vcpu); + kvm_arch_vcpu_free(vcpu); } =20 void kvm_destroy_vcpus(struct kvm *kvm) @@ -4067,6 +4067,16 @@ static void kvm_create_vcpu_debugfs(struct kvm_vcpu = *vcpu) } #endif =20 +struct kvm_vcpu __attribute__((weak)) *kvm_arch_vcpu_alloc(void) +{ + return kmem_cache_zalloc(kvm_vcpu_cache, GFP_KERNEL_ACCOUNT); +} + +void __attribute__((weak)) kvm_arch_vcpu_free(struct kvm_vcpu *vcpu) +{ + return kmem_cache_free(kvm_vcpu_cache, vcpu); +} + /* * Creates some virtual cpus. Good luck creating more than one. */ @@ -4103,7 +4113,7 @@ static int kvm_vm_ioctl_create_vcpu(struct kvm *kvm, = unsigned long id) kvm->created_vcpus++; mutex_unlock(&kvm->lock); =20 - vcpu =3D kmem_cache_zalloc(kvm_vcpu_cache, GFP_KERNEL_ACCOUNT); + vcpu =3D kvm_arch_vcpu_alloc(); if (!vcpu) { r =3D -ENOMEM; goto vcpu_decrement; @@ -4182,7 +4192,7 @@ static int kvm_vm_ioctl_create_vcpu(struct kvm *kvm, = unsigned long id) vcpu_free_run_page: free_page((unsigned long)vcpu->run); vcpu_free: - kmem_cache_free(kvm_vcpu_cache, vcpu); + kvm_arch_vcpu_free(vcpu); vcpu_decrement: mutex_lock(&kvm->lock); kvm->created_vcpus--; --=20 2.39.5 From nobody Thu Dec 18 00:34:59 2025 Received: from mail-wr1-f43.google.com (mail-wr1-f43.google.com [209.85.221.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5CA3619EED2 for ; Thu, 24 Apr 2025 14:13:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.43 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504032; cv=none; b=r6hB37DQMneD21sKl0MvkMW8/J7ZV/s4YlIVSoapn4rEKEFHa4Y+7BS4qigTKK+oicOv6RYhrA7F/B/D6LwIShZoLizPpSsjA6M8a/YJEYr9fLMJE9Ly01vZGH+n5rJld/p/zxh4c/PYeCaD7dGdhp/svRje7ZyoIPOaWUgDetI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504032; c=relaxed/simple; bh=CvFb1okbnEK9wiiZwwdcGSqHE02tQmbzZOFs1CBcYGk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=MY+vNNK1PkEEUrjoi+2kXQ6tjcSgqo6kRYq32kA/4QmDQmqZpduzRtF7DxcrWBQCcFOzk7MUas4r8zifKxlyUJd5uLjZ6/UV5qhthZhKBZRFCPQyuZQD+1TJ5vpkZS54EDrASo4h3BMT8udTrjdu5+TrgXl5lEM6F5DraMC6jfI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=iLOwI++f; arc=none smtp.client-ip=209.85.221.43 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="iLOwI++f" Received: by mail-wr1-f43.google.com with SMTP id ffacd0b85a97d-3912fdddf8fso1551221f8f.1 for ; Thu, 24 Apr 2025 07:13:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504028; x=1746108828; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=hvXrACbOQaGlhym8epnVf2FRavXrOILN2RCY7mrPgRw=; b=iLOwI++fMXomw84Au6sHZmvf4XEzvfBtSvOEbGXBg1IqUMdLW0+v/JK3OSvPQBiK07 G7DAOlkh3Dn7nSPFwYfcmRBHN4JhU6Z7fOWSzh4GJgLLSKO1hmkm6NG5WZH0AfoEC9Bo NaVFDFNV39iAFQcXUNIXRLyqVzXYxsfBRnVLgKEH3URDsPi8jYpP5eiVPiklRMRJO+fD KUIEEE9ixPIFXkDrY/7Qt3c6/QbTS76DZoKYstIM7n24LZL98RND02LEMC+RNX+PpEON PTHZTJSA7cSemMaHjeV4uQBh4Xu/4+vnuIYtqe3zD61Dyl0L28stdByuBTQVTBs9JVfw TXHg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504028; x=1746108828; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hvXrACbOQaGlhym8epnVf2FRavXrOILN2RCY7mrPgRw=; b=R8/a5c937dnrXOnSxEI+auQAeB/rf/t9uEqRTg7yKkFhVSfg6JmaH0Wn9xTaIlerpy XWfNeVA4zjxQxF3WgtzzNAJRjyLinzFV86WwhUZSQQEQYoCREAaDweVJ5ymEUjmn41si KJDBNDLkfyxkZIc71oC6rEozayzxwNjau9c7ShtfkYoT7OM+ajexWg4/ABtTuI2rMjjk z3AL6SEgB0eto6c0ycKjlrJa76wcuWRDoLvJNsKT+bIvShGIAmhMSGn+hlV8s2A7tXqQ u8GG/VSa0KmOQG/UeCDk15ZtN1nFC1y/7/4hFWjP11Ws6IsIF9dNvDVEy7GV6ItO1cK+ ZdXA== X-Gm-Message-State: AOJu0YwHn44QLMjPNorUrx/z/BMzpNitTQ1jFX1LFwz8aBdhahoUlRxd 3tbuMDRIPzY/0dcgKtRWRBRi4zgpSYIuOAtTZXwvAxW1sn2qmVVmlRrQufwx6zb+YtFu3GF/45m cDlM= X-Gm-Gg: ASbGncsyxrZvCH2cGE2vw/Odfle1Ux+kYUByM9oUXCGxPGr1gAwLPtkiNP3rGXGKvpY csB3KoFql5NNhUXd4SX2GGh7PPuf10rU9d/MUlp8bNVERozKaUyIaT/qM1fv5JPKdZG66tR9D1/ d6eswyxNyecNhY4hOwlLibNociTTv904Zv7XtJMsXwLuf+VEOQfUqRfJ/BDY2WISKUjGc1ohkFO Vu1UmSZqhtZMd8ptow5zyPVb/1H64/XmQVTjAwNRjC29ntwgRmiY0Ad4jr9CwtZNwus3qU2iuP7 aAPXGDM/2DrlRKZITzYJVQ0kYj3AzfkzWeAVslxrT/emwwWt8YeCEF7HXfIrXPxPgFqOD3uYPGD ww58O5xIh41uQiq0Nrvm43TfADu8= X-Google-Smtp-Source: AGHT+IEBkZ1IBJCII/ywyTdMC0fLKprAp65skAqA7s+zC8i8AN1EGINZeECW9VcRyNwMhKCFTJTxcg== X-Received: by 2002:adf:fb4c:0:b0:391:1218:d5f4 with SMTP id ffacd0b85a97d-3a06d698d62mr2225510f8f.23.1745504027803; Thu, 24 Apr 2025 07:13:47 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.13.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:13:47 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt Subject: [RFC PATCH 02/34] KVM: irqfd: Add architecture hooks for irqfd allocation and initialization Date: Thu, 24 Apr 2025 15:13:09 +0100 Message-Id: <20250424141341.841734-3-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Some KVM backends, such as Gunyah, require irqfd structures to carry platform-specific state or setup logic. To support these use cases, introduce three weakly-defined functions: - kvm_arch_irqfd_alloc() - kvm_arch_irqfd_free() - kvm_arch_irqfd_init() These allow KVM backends to override irqfd allocation, teardown, and initialization logic. The default implementations simply allocate/finalize a standard `struct kvm_kernel_irqfd`, maintaining existing behaviour. This change is required by the Gunyah backend, which uses these hooks to associate irqfd objects with Gunyah-specific bell resource handles for IRQ injection via hypercalls. Signed-off-by: Karim Manaouil --- include/linux/kvm_irqfd.h | 4 ++++ virt/kvm/eventfd.c | 31 ++++++++++++++++++++++++++++--- 2 files changed, 32 insertions(+), 3 deletions(-) diff --git a/include/linux/kvm_irqfd.h b/include/linux/kvm_irqfd.h index 8ad43692e3bb..e8d21d443c58 100644 --- a/include/linux/kvm_irqfd.h +++ b/include/linux/kvm_irqfd.h @@ -61,4 +61,8 @@ struct kvm_kernel_irqfd { struct irq_bypass_producer *producer; }; =20 +struct kvm_kernel_irqfd *kvm_arch_irqfd_alloc(void); +void kvm_arch_irqfd_free(struct kvm_kernel_irqfd *irqfd); +int kvm_arch_irqfd_init(struct kvm_kernel_irqfd *irqfd); + #endif /* __LINUX_KVM_IRQFD_H */ diff --git a/virt/kvm/eventfd.c b/virt/kvm/eventfd.c index 11e5d1e3f12e..5f3776a1b960 100644 --- a/virt/kvm/eventfd.c +++ b/virt/kvm/eventfd.c @@ -32,6 +32,24 @@ =20 static struct workqueue_struct *irqfd_cleanup_wq; =20 +struct kvm_kernel_irqfd __attribute__((weak)) +*kvm_arch_irqfd_alloc(void) +{ + return kzalloc(sizeof(struct kvm_kernel_irqfd), GFP_KERNEL_ACCOUNT); +} + +void __attribute__((weak)) +kvm_arch_irqfd_free(struct kvm_kernel_irqfd *irqfd) +{ + kfree(irqfd); +} + +int __attribute__((weak)) +kvm_arch_irqfd_init(struct kvm_kernel_irqfd *irqfd) +{ + return 0; +} + bool __attribute__((weak)) kvm_arch_irqfd_allowed(struct kvm *kvm, struct kvm_irqfd *args) { @@ -153,7 +171,7 @@ irqfd_shutdown(struct work_struct *work) irq_bypass_unregister_consumer(&irqfd->consumer); #endif eventfd_ctx_put(irqfd->eventfd); - kfree(irqfd); + kvm_arch_irqfd_free(irqfd); } =20 =20 @@ -315,7 +333,7 @@ kvm_irqfd_assign(struct kvm *kvm, struct kvm_irqfd *arg= s) if (!kvm_arch_irqfd_allowed(kvm, args)) return -EINVAL; =20 - irqfd =3D kzalloc(sizeof(*irqfd), GFP_KERNEL_ACCOUNT); + irqfd =3D kvm_arch_irqfd_alloc(); if (!irqfd) return -ENOMEM; =20 @@ -396,6 +414,13 @@ kvm_irqfd_assign(struct kvm *kvm, struct kvm_irqfd *ar= gs) init_waitqueue_func_entry(&irqfd->wait, irqfd_wakeup); init_poll_funcptr(&irqfd->pt, irqfd_ptable_queue_proc); =20 + /* + * Give a chance to archiectures to finish initilization. + * E.g. Gunyah needs to register a resource ticket for this irq. + */ + if (kvm_arch_irqfd_init(irqfd)) + goto fail; + spin_lock_irq(&kvm->irqfds.lock); =20 ret =3D 0; @@ -452,7 +477,7 @@ kvm_irqfd_assign(struct kvm *kvm, struct kvm_irqfd *arg= s) eventfd_ctx_put(eventfd); =20 out: - kfree(irqfd); + kvm_arch_irqfd_free(irqfd); return ret; } =20 --=20 2.39.5 From nobody Thu Dec 18 00:34:59 2025 Received: from mail-wr1-f52.google.com (mail-wr1-f52.google.com [209.85.221.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 46A131B3939 for ; Thu, 24 Apr 2025 14:13:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.52 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504033; cv=none; b=cchJ1KYt0aYz/KU/YTrI+/WEoVeIfAdTTlDChKXxv5B1sIOeCVI7GmXtz2m6Pj6IAAdYj4sbBeVw3AYjly9PwRdQm4UdSnwGIrv0RSi9Cukr/kH62O034CCxWqU2QgMP6wrHcm1yfQ0wXGJu7yycH1xKUPmeIjs+7SfmShw5o9M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504033; c=relaxed/simple; bh=wzaNO/FV98NvOhVwcurDeWV9o2lzkiDEMgzO9vOQkfc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=CpBk0g0oplUdurWgbEkY/mkONqs1c3MMl8bIGjfvOtNFaA1QmwyRArN8bWtzz9VlsgW8BHY1BIRY+9APjdSOiRg3/sqwgV2J+kLZ0oXVSYqtE+GXHsgvfEhk6qNIQiqkLtxnUEboZ2aMerhAM+cZ9bgRObSHdIeI7CxSHLwoupY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=AJZId15e; arc=none smtp.client-ip=209.85.221.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="AJZId15e" Received: by mail-wr1-f52.google.com with SMTP id ffacd0b85a97d-39c0dfad22aso840340f8f.2 for ; Thu, 24 Apr 2025 07:13:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504029; x=1746108829; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9Hk5UBQTvuxPsQW0619Tu3I4qsqkftwTtmBE5Ybb85g=; b=AJZId15e7pdgeqWr+FMAv4lLCiYL8lS9X4n246zeYuB5IaR/DGRrpcovPDeJif1ult pCCP+lQCy1PBuRRsue4enbNhuItViUP50zrTPxmT0nYVJuKw/mn8cCZ5OQlhg6eDHW9M a05wR8Ubv3dLoceKbyhzZ/riusaGcx+r9eKtqhXZP/YlnDIR6V5OqRyMhOR7PXfzMpu9 bd39oniRh5fAZXnOSJ6Smcyyg/jnOY7Ep8YHqXr5Ac8Lg6Ki8jPdYS6b+DDfyDjPiHA+ KmVtSGfJ5KzOQuk2U26Ugg9Q5sqD/HC/WlFq2brgPR9T1soyDON1tKSxraJ1r6NZZ3nr p44Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504029; x=1746108829; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9Hk5UBQTvuxPsQW0619Tu3I4qsqkftwTtmBE5Ybb85g=; b=DaSRSYTj2VWKIR7S/KegRfvkW+YQeUO1YhtlEO83SJRLIfPjjRTs8+HPUi5oGwuU/L lBP3ubLpLYf321eEXa+oE67MEapdR9IybnsYyyYKiikaT4IcG1rN+hl3Ry8OUcCK7iQW Tf8l2DqaCs4EmfGd4IIwkiyQpiKN+XUwKU/XC/JJaizWqjzUe5fF34IfYqThMleK4471 iJIUIaXiB0cmPuqYGYS/9UhOBTjEt40P1uhCL7Km/R7yHpU3alHUCYo/4VBJjisLZ8hg 0nyS02/jgWq51O/pGJBqEPtKy+KhJgWmwDhlCL0/OkJ3YCABi5P1H+Nheb7t3iLenDs3 DS7Q== X-Gm-Message-State: AOJu0YyPFRBXyeNBYm3EwrJ+K0JtIIfDGvvtQrims9lSfpqeiJHO/A9H CfDcl9snSAYfA3Yf606pOGCaoTupokirct8qA7W/h/pPs8sr5Taq8jIyJ89nIVNH+WBplbTZ17W HSA0= X-Gm-Gg: ASbGnctj5EClpKDHkAlRtHsoiWrdHBd9weYJ4gpAtbWb4HyTpRkjNUF62tNknmmIgOG FZCsWa4r+b7pTMBvTch/wXtN0sA0//o6918DFON/el1AjV6GdLLPtWzVvGASofZaRCsuc/SR0mf Mnr5PhPYYnr7x4GiHmhPBCoql/RZoOIpjOdwGgJOaivTkFbi8rK+uaJ5ULyAtht5Qeatetrc0S7 l8OtjMa1S2AndEiEYdGUujYTkJ2RpUU0xY+xqrYasGcBW0yJxmRX+xcS6C0fuKFhReL3V0ekXG8 sRFZZc/HB9WBclvB306O42lbAykDpLgRiF4/bCuQ1aB7gHgngIFJCArZqVQv7ksij7H3qfhY6XD wag2StZAOFF6y/gxX/uzus+5a5t4= X-Google-Smtp-Source: AGHT+IEX65fw+AuwIbm8DfCqVPIwMLBPu7X4mEmOAObvmlxgMqJT5rSe8ZZGCXBt4CgJFkEQIJbeNw== X-Received: by 2002:a05:6000:1a85:b0:391:39fb:59c8 with SMTP id ffacd0b85a97d-3a06cf61418mr2487543f8f.25.1745504029007; Thu, 24 Apr 2025 07:13:49 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.13.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:13:48 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt Subject: [RFC PATCH 03/34] KVM: irqfd: Allow KVM backends to override IRQ injection via set_irq callback Date: Thu, 24 Apr 2025 15:13:10 +0100 Message-Id: <20250424141341.841734-4-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Some KVM backends, such as Gunyah, require custom mechanisms to inject interrupts into the guest. For example, Gunyah performs IRQ injection through a hypercall to the underlying hypervisor. To support such use case, this patch introduces a new optional callback field `set_irq` in `struct kvm_kernel_irqfd`. If this callback is set, irqfd injection will use the provided function instead of calling kvm_set_irq() directly. The default behavior is unchanged for existing users that do not override the `set_irq` field. Signed-off-by: Karim Manaouil --- include/linux/kvm_irqfd.h | 1 + virt/kvm/eventfd.c | 5 +++++ 2 files changed, 6 insertions(+) diff --git a/include/linux/kvm_irqfd.h b/include/linux/kvm_irqfd.h index e8d21d443c58..7d54bc12c4bf 100644 --- a/include/linux/kvm_irqfd.h +++ b/include/linux/kvm_irqfd.h @@ -46,6 +46,7 @@ struct kvm_kernel_irqfd { /* Used for level IRQ fast-path */ int gsi; struct work_struct inject; + int (*set_irq)(struct kvm_kernel_irqfd *); /* The resampler used by this irqfd (resampler-only) */ struct kvm_kernel_irqfd_resampler *resampler; /* Eventfd notified on resample (resampler-only) */ diff --git a/virt/kvm/eventfd.c b/virt/kvm/eventfd.c index 5f3776a1b960..d6702225e7f2 100644 --- a/virt/kvm/eventfd.c +++ b/virt/kvm/eventfd.c @@ -63,6 +63,11 @@ irqfd_inject(struct work_struct *work) container_of(work, struct kvm_kernel_irqfd, inject); struct kvm *kvm =3D irqfd->kvm; =20 + if (irqfd->set_irq) { + irqfd->set_irq(irqfd); + return; + } + if (!irqfd->resampler) { kvm_set_irq(kvm, KVM_USERSPACE_IRQ_SOURCE_ID, irqfd->gsi, 1, false); --=20 2.39.5 From nobody Thu Dec 18 00:34:59 2025 Received: from mail-wr1-f46.google.com (mail-wr1-f46.google.com [209.85.221.46]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 79AAF2036EC for ; Thu, 24 Apr 2025 14:13:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.46 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504034; cv=none; b=ebj88A630ReoiPzj4m+4U0f/vnvqwFCOqgPJFuouIQFXFQuUhIZmjwy1eYQ5HXeKhYCIwYaBHuSMO22hYaK7ud7EnDnxYiS9Y/QPnTJ2VYCQItViAlsQ+OF5wRW/wKYzuEr74QBkf7bcUZeP4x2krrdhv+wLJDgjnX2T+mgLJIs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504034; c=relaxed/simple; bh=yHQVbNXozVjVHt5pOWEf6CFk11nfYNM4wHopdbI5gbI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=r+lOvLYC7PSVNfBL2pSu3FXXwc6412t2X+396qiNGNaH96bXrlqPXwlQ6QmQl39xzFGIcLi1Rho7x97z6XqqlTDAX/nsgFnNSv1U/WQA8evdw/Zd6pBEN3mDIXlqyIOQyqTvD8tSBbRsH3wn7nF7exQ52z5Yzr4fq4QAhPdPRSY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=TMpYcwLk; arc=none smtp.client-ip=209.85.221.46 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="TMpYcwLk" Received: by mail-wr1-f46.google.com with SMTP id ffacd0b85a97d-39c0e0bc733so989121f8f.1 for ; Thu, 24 Apr 2025 07:13:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504030; x=1746108830; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=AlljNambtbwwvkzAI4LjAz2eo7Ti1KUJrgs8M2UB6Bg=; b=TMpYcwLkfLTj/ATVziMuksWaNsV3xs053pw6b7Tre2eq7ME2fqLNfMFtBGsCfJ0eLv YaiXKTnIikVyMFLz6c8RFczV2dxEuTPlqUn0VbE8hvXxjTAi2mOelNzXjdxPAmnkSCfd 3LifOZSqAm46lqRHFyEjsJsmzn/kOOCCpL0PCLxtWs3mOsRWSy+t7/E8RaWNKpD4LyWW QEM8MkS1ukD7WBR9ph7HvsZQWjM1KNmig+Vr6IRoe9cSARPdbP8NtrVV38KehoUVsO5h vE9r4xl6/rZ+05p1xaciTkPpWpp6a1HaUXeVGrmcWltf1wygu6Br9SLBLSbQuwtYg8jY 0m/w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504030; x=1746108830; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=AlljNambtbwwvkzAI4LjAz2eo7Ti1KUJrgs8M2UB6Bg=; b=UlTyR3yzgg99lAK7eOblPjZZ9vSiR8FgfF5Xc5YFfQSGDYZfNqSIih8YKZZLU2dTw0 MnvFXGMRW0niFEuWLDgYi3Z8kjs4SuyBz+QZ5anhSG4WhL/MRSH5I0Nzr09ewQgOtfTc E5zRpHP8wpo0X4BNnJHLDIzzQXcS15en57zT9d+/gpf7YsCABIxsNNfnc4/XI5E4J3Cm bcQV7pMmKdod3RIkvrlWUUL9Nft4uVSxgP8tMvBBvsUeDu0baKIyFx7EgF5RRHJT9oAe B14Y8+FHAOrY80HCQd+nKA74emdKeJyj+2Zgcb2JG/3K35sRoRLm972jUG5N2NRVboaQ ShCw== X-Gm-Message-State: AOJu0YxZ/XbY/P0a6r+JOGwj7Xsew2vXiR5omZthJQHcyh23NL3O1dBN omuGnsFr2rhLiehgc4ijvWn6uU/cShUuiB9QLmjSue5DYE3pspW9wbvdOAKDZbxu2AXYIQCikGX prA0= X-Gm-Gg: ASbGncsp/EPNfoIjHud8OVGkDWdsreKUZ6uIXi9B2Ac64EzEVOp0zAo1QsiZajmW+qi 3Pb1UvVSAhxIkgswMrj8HjHGInHGNvVJdncYbXcB3eRIQbZCl+tpBco+B5FIV4N+SbTsfdW06OI sdONfYf4LqvMM9jOAERFz1ZHUYuPWZ+xUvEq5iHhYcx9JZHeNnWH+PA3x4flC2iq51CQ0rTtNQ2 WNVQbQozFvkIhDW9tSn/LM4dBel2sz9tnkRU7HsDX4TJasDT0HHjjqwFZVPpmXkAKWVK4bmXl6i 2VRXX/vzD2Dpl4s9t1UPAO9/qb2i666tIorMEBaShi6kOI2bv5rdFTbq+2JU/TKnnB9wUi8OcPQ iTKfRP0RVj8RYO6yh X-Google-Smtp-Source: AGHT+IGjTSYHUF3tyGvvpZJy9ttFB+SjTk/OBEoax4K5qs9LZf6N8lXayXTA0lInN+EeJy++KbQTNQ== X-Received: by 2002:a05:6000:18a2:b0:391:47d8:de2d with SMTP id ffacd0b85a97d-3a06cf5ed4cmr2481988f8f.23.1745504030257; Thu, 24 Apr 2025 07:13:50 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.13.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:13:49 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt Subject: [RFC PATCH 04/34] KVM: Add weak stubs for irqchip-related functions for Gunyah builds Date: Thu, 24 Apr 2025 15:13:11 +0100 Message-Id: <20250424141341.841734-5-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The generic KVM core code (e.g., kvm_main.c and eventfd.c) calls into irqchip-specific helpers such as kvm_set_irq(), kvm_irq_map_gsi(), and kvm_irq_map_chip_pin(). These functions are defined in kvm_irqchip.c, which is not required or compiled when porting KVM to run on top of the Gunyah hypervisor. To allow building the KVM core code without linking errors in such configurations, provide weak stub implementations of these functions in eventfd.c. These stubs return appropriate default values (e.g., -ENXIO or -1) to indicate that the functionality is not available. This allows the KVM core to build successfully for platforms that do not use the in-kernel irqchip support, such as Gunyah. Signed-off-by: Karim Manaouil --- virt/kvm/eventfd.c | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/virt/kvm/eventfd.c b/virt/kvm/eventfd.c index d6702225e7f2..2a658d8277ed 100644 --- a/virt/kvm/eventfd.c +++ b/virt/kvm/eventfd.c @@ -56,6 +56,26 @@ kvm_arch_irqfd_allowed(struct kvm *kvm, struct kvm_irqfd= *args) return true; } =20 +int __attribute__((weak)) +kvm_set_irq(struct kvm *kvm, int irq_source_id, u32 irq, int level, + bool line_status) +{ + return -ENXIO; +} + +int __attribute__((weak)) +kvm_irq_map_gsi(struct kvm *kvm, + struct kvm_kernel_irq_routing_entry *entries, int gsi) +{ + return 0; +} + +int __attribute__((weak)) +kvm_irq_map_chip_pin(struct kvm *kvm, unsigned irqchip, unsigned pin) +{ + return -1; +} + static void irqfd_inject(struct work_struct *work) { --=20 2.39.5 From nobody Thu Dec 18 00:34:59 2025 Received: from mail-wm1-f53.google.com (mail-wm1-f53.google.com [209.85.128.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AD37C27B516 for ; Thu, 24 Apr 2025 14:13:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.53 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504036; cv=none; b=mel1V7Uu9D+3RokDigsEZ8kXmNMQawYcWIO6ccBJRCiTRHn6nSx/94q+8wjpE3xf6hWzmWfj5/3pHcF+/71NY5MHsBf6Z94rGJ5YjRybkBc2tCroZxvkrFTaPWocYWKlR53UwuGYO7rT2R0AwJAIRIrwbFXzi2AQTTNH5uvGMiQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504036; c=relaxed/simple; bh=62G1u7Mmcj0LoLg/GsZOxRpsdU6hf0LcgEiDACKmXjQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=EHCghYlf7G91jIgcRBH9M573boOmj4Oajs8yzEsw7zBgh/I09/S5+Lxa4A5v4FynJeZlAXY0LL86URz6y0GC5mEggR/Tni7tl0EraOzvLM/KW8eSfZGwAyFEvoGr5axrnAQ86Mkh2H8COYtSgq3Vn1Rxv4k9NMZ4kFWmDXdE05M= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=YyLEy0uC; arc=none smtp.client-ip=209.85.128.53 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="YyLEy0uC" Received: by mail-wm1-f53.google.com with SMTP id 5b1f17b1804b1-43cf680d351so13779685e9.0 for ; Thu, 24 Apr 2025 07:13:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504031; x=1746108831; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=BvSif0ZiFcbo45REElIUdI7xtRbISNuAr1Y97HbCqdI=; b=YyLEy0uC+aKHIkM4oaNNOOvTEZxl/5B4cXMJWSiA5CH0VKETdRs1XxMP9euG6CgE2e Z23rgG1SUAKMC/rdVHWjNbC+XWmyfueGcOirkOcsGMCQPLCujAKCkPge1SHWgctMolRg VsK7AbhITHdDJZ/mZTkqKU2opmMA5akZbn1I1JWbXeLTDgeuLCR+bqweC7o3r8fqG0JP rvxIZ6AW6JAFEQIIudhQnHCQgdgEg6zG4+kSRQJN8FbIPXzyGwglOHS25g10OTkEfunL lbyuyS6pqq7X1Q8tQ/K+fZGkXN0vkHcZtBXrR60SkIIevTg4zC9iJj8ScVY3BHkXRwUW mzag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504031; x=1746108831; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=BvSif0ZiFcbo45REElIUdI7xtRbISNuAr1Y97HbCqdI=; b=rKFdMnOYyALmK+KZO74UvLOKqsjftrfkvkRmoR6ZAs18Z7fix8QSlAH7cn2VlNHDLG zuE4JK0LNaPB2C4Eyq0pDNEwddRGADqH89iacIOQ1knNIOtBfkt9/hCcQhKv68z97DFC SAIV0IvXRU0Q4F5sqW/VqjOWis+eaODFzS0v1AGoFYvUWdBQCXeLAweaD/DZrQAXaDm7 X1y+hxQc0DgFdZEcMC8yUTbn11lZKJFx8mvHEfyupl1V1buJEKA2sO3rPtzy0DajJguI 8i9S0gv/9cK2BpNSJ3A9lGshxLGD1DYrQa2yT9GMCJ+dQo6etTuhtNTb1i/3t9ORGltW rfPQ== X-Gm-Message-State: AOJu0Yy1Rp1bGv7J2two1Qkztdn7iffU4R6Ed7ckLDEiaFinOUmc8jdl 2HGLxMIDz77Fz40lPWD2+D5YptGJQnTWsNO5942UMciBU7nqCWIfJuhsOiH7OmzeqvcJY28kcUf cfi0= X-Gm-Gg: ASbGnctJ1Pr0uxtgSX3k0EfZ50oXDung62cTb5jFcGypEz1jpwKmh6U6S68nqLqXPaR h9VJAWolaQtQvU68oLjBxbQB4XMIlQKQla9XNImRnvYeBJ2bojMNcqaXgR9gHRHJ3bBRvUnDODA QOYv7LQaa+2KGIeSeXAcmqt58LPjI5J8VUZpCu2PtdxSuXwSlEh2nZT9owhwkgIT4k6GgXGHwM8 BI9guOAvEOClKcTSaRjR7Q5z8KslsZwlhH06iRChMpemCPSqse3D8ETR4X6R1itequGNeMO/tO4 9G9aNNk7FnrkH0TjEzMAoX7f2cvOdNZwrLDbLxly4yWKa062pcsKJkndNR76wboz21LYFteHlDa jvBod0aSNHrNrkqNV X-Google-Smtp-Source: AGHT+IGs6TOHidyOdLjO/aiH4v937+mffmDnW0sidZAyNfcPHTexYT+C0CxkEqdPf2DwJcaHqEXYnw== X-Received: by 2002:adf:ebcc:0:b0:39c:1efb:eec9 with SMTP id ffacd0b85a97d-3a06d673778mr1956533f8f.13.1745504031472; Thu, 24 Apr 2025 07:13:51 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.13.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:13:51 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt Subject: [RFC PATCH 05/34] KVM: Add KVM_SET_DTB_ADDRESS ioctl to pass guest DTB address from userspace Date: Thu, 24 Apr 2025 15:13:12 +0100 Message-Id: <20250424141341.841734-6-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Some hypervisors, such as Gunyah, require access to the guest's device tree blob (DTB) in order to inspect and/or modify it before starting the guest. The userspace virtual machine monitor (e.g. QEMU) is responsible for loading the guest's DTB into memory at a guest physical address, but the hypervisor backend must be informed of that address and size. To support this use case, introduce a new ioctl: KVM_SET_DTB_ADDRESS. This allows userspace to provide the guest physical address and size of the DTB via a `struct kvm_dtb`, which is now stored in `struct kvm`. The ioctl allows platform-specific backends like Gunyah to retrieve the DTB location when configuring the VM. This patch also increments the KVM API version to 13 to reflect the addition of this new ioctl. Signed-off-by: Karim Manaouil --- include/linux/kvm_host.h | 1 + include/uapi/linux/kvm.h | 14 ++++++++++---- virt/kvm/kvm_main.c | 8 ++++++++ 3 files changed, 19 insertions(+), 4 deletions(-) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 3461346b37e0..4e98c7cad2bd 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -862,6 +862,7 @@ struct kvm { /* Protected by slots_locks (for writes) and RCU (for reads) */ struct xarray mem_attr_array; #endif + struct kvm_dtb dtb; char stats_id[KVM_STATS_NAME_SIZE]; }; =20 diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index c6988e2c68d5..8f8161cd61a7 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -14,7 +14,7 @@ #include #include =20 -#define KVM_API_VERSION 12 +#define KVM_API_VERSION 13 =20 /* * Backwards-compatible definitions. @@ -43,6 +43,11 @@ struct kvm_userspace_memory_region2 { __u64 pad2[14]; }; =20 +struct kvm_dtb { + __u64 guest_phys_addr; + __u64 size; +}; + /* * The bit 0 ~ bit 15 of kvm_userspace_memory_region::flags are visible for * userspace, other bits are reserved for kvm internal use which are defin= ed @@ -1190,11 +1195,12 @@ struct kvm_vfio_spapr_tce { #define KVM_SET_IDENTITY_MAP_ADDR _IOW(KVMIO, 0x48, __u64) #define KVM_SET_USER_MEMORY_REGION2 _IOW(KVMIO, 0x49, \ struct kvm_userspace_memory_region2) +#define KVM_SET_DTB_ADDRESS _IOW(KVMIO, 0x50, struct kvm_dtb) =20 /* enable ucontrol for s390 */ -#define KVM_S390_UCAS_MAP _IOW(KVMIO, 0x50, struct kvm_s390_ucas_ma= pping) -#define KVM_S390_UCAS_UNMAP _IOW(KVMIO, 0x51, struct kvm_s390_ucas_ma= pping) -#define KVM_S390_VCPU_FAULT _IOW(KVMIO, 0x52, unsigned long) +#define KVM_S390_UCAS_MAP _IOW(KVMIO, 0x55, struct kvm_s390_ucas_ma= pping) +#define KVM_S390_UCAS_UNMAP _IOW(KVMIO, 0x56, struct kvm_s390_ucas_ma= pping) +#define KVM_S390_VCPU_FAULT _IOW(KVMIO, 0x57, unsigned long) =20 /* Device model IOC */ #define KVM_CREATE_IRQCHIP _IO(KVMIO, 0x60) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index dbb7ed95523f..a984051e2470 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -5121,6 +5121,14 @@ static long kvm_vm_ioctl(struct file *filp, r =3D kvm_vm_ioctl_set_memory_region(kvm, &mem); break; } + case KVM_SET_DTB_ADDRESS: { + r =3D 0; + if (copy_from_user(&kvm->dtb, argp, sizeof(struct kvm_dtb))) { + r =3D -EFAULT; + goto out; + } + break; + } case KVM_GET_DIRTY_LOG: { struct kvm_dirty_log log; =20 --=20 2.39.5 From nobody Thu Dec 18 00:34:59 2025 Received: from mail-wr1-f41.google.com (mail-wr1-f41.google.com [209.85.221.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 51BC51B4227 for ; Thu, 24 Apr 2025 14:13:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.41 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504039; cv=none; b=ridwioemK8QSunoLGe4y1LozUyI983veebKno6UVPVtc5z011YHcqXoOxViKkdvGXisQ717z12590A0/rrIf7FpCyHaEc+R/2lqazjRRyylcl8FI3kfrbjNP6PY1VmR3fc8DwFyP8q/i7deZFUeNzgmAll08LHNxlFO2zLat2q4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504039; c=relaxed/simple; bh=mRPK+89hbVEkRoZvoKf3k11XmtiTEyAylGP5LEAOcB4=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=ORFXaF6nbHXgFLQ4Qdv46Ulkb8EjYPrykolCzqoc2tU4aeDI1E5WZAC1TIB7mrlDrOd3+/EPOe/+1kdFnq9DVXrKwArwxzrY9gNOp40HZYWmXguCj+cwRrDecbytezP4FwCqcwP+MPa2DIOnJTFTykdE62aPlI2lzZDsjmt3Yrc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=NHz03KRf; arc=none smtp.client-ip=209.85.221.41 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="NHz03KRf" Received: by mail-wr1-f41.google.com with SMTP id ffacd0b85a97d-39c1ef4ae3aso756837f8f.1 for ; Thu, 24 Apr 2025 07:13:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504033; x=1746108833; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=8jfERcd+/8hI2lWUkL6Z3PV6Xql42InuHmm1RJaBik0=; b=NHz03KRfKRU1ydrBTpF950pSDo/fHHj/qFQsZBAH0nAr4J9XrBebUanz4RG0Ixizpb zizFOX7Vu6tkDrj2qPw3CD+XTvSRf647mi6yu15W/sUo85Ma/WYJ03VBSbAyl3QveuSj y+rNVChsDvDU1Owq9C37MZKVnLhZcFMRtLzhu9lxwqU4fxn2gQzaTp0qmCCwCCdkyUob GKkWZ9icDjECa2TnEhe8KvnOI/31ftMaK1j05MmE06f1oYp23/LsxAb0xjpcv5N66sz+ xsWF4R4435th0RzZhvo28yWhWepgCqYIkW7Is/iTl37ZQJa/SEUSfIzKb2Tm1RJkuXbn BUjA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504033; x=1746108833; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8jfERcd+/8hI2lWUkL6Z3PV6Xql42InuHmm1RJaBik0=; b=JX1+3+tkd1nCbjyhWm2wUbI31Nvr8ZbZHpVcuUy6RAgWTOMquGxeh7htf4IUsW6a1U +YL0a+O+2XvlkMw4KsWECOyMow664kNBmBTJTb6+Zt0vqwkr4TuIfJR3JVaqDCxBTI5K YU+gbPehRLDHBTyNVf/qxJ8q05TQ1DmMgP6y0d6nPMjYjuecCNXoHfWBBygOH77c7BFE 2MO0N/I7tlomfQH1oSGCxDkKM+AIIS8dR6p1e37cqqMPVtuMYfQL81oewCLjzsRBFIqz sGUa9Fm35UbDbw3nqGGlzD0bDVvnIG1bN/A9T2O+OIo331LRyZTe/GcQDlUr78zyeWVB NlQw== X-Gm-Message-State: AOJu0YzSdDkc3tV7cwAVYg+k2BAO/YsNpUkoAoX+6qGguxP7wQxvO974 I/KbWqGxQlOuZVE3ROmLEpl9r1X/M8cJgE7ps1EKjte7tJWfROPRxMxgE9MRZBvhS182oggbNsv 9oCQ= X-Gm-Gg: ASbGncuO7EVErnJx8wm0mdR3+cRZ87DPGfOckafHUtCm5Xz6uM96nO60w02dZ70SNrI n4D+BKFfSpl6Kjx0ksrnT1ZCUyyalUaSrmUtIhDV4bnb2PMaC36hKB7vM5kgARCckCFrRjJ/SR4 DU98pinU+5+jQ2MeZhSzztlVXaVGSWDanp6jNWLjS8eeLiJflnuu5lBXNk797aIIknpTTXC0Hy4 L+IM5GOcppIQMaVDC+jB3vCLMCl6B3gNQ+flhQmaeGHuj5EVKNubFDUOPlj+uOu6NDUZ8zUPqwo a/AXn7K7ytSQg7o9sR7G9XOrn7jkBHv8ZRiV2BfzcCx9qn3FXnjhS0eTk81mBqq3tBShvOf9jiq xWtysTstPe8+zWo3A X-Google-Smtp-Source: AGHT+IGx0xNiXWRY0y1pEy2kIqf26zFRwFHT6eClNX/GVzcfRlYZlyiUPQIhI5t0dInRmkb4qoBxLw== X-Received: by 2002:adf:e2d0:0:b0:39c:1f02:44d8 with SMTP id ffacd0b85a97d-3a06d647344mr1972721f8f.4.1745504032796; Thu, 24 Apr 2025 07:13:52 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.13.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:13:52 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt Subject: [RFC PATCH 06/34] KVM: gunyah: Add initial Gunyah backend support Date: Thu, 24 Apr 2025 15:13:13 +0100 Message-Id: <20250424141341.841734-7-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" This patch introduces the initial skeleton for supporting the Gunyah hypervisor [1] as a KVM backend on arm64. The Gunyah backend implements a different KVM architecture backend under `arch/arm64/kvm/` alongside the existing support for AArch64's native virtualization. Please, note that the two are mutually exclusive at build time. Key highlights of this patch: - Introduces a new Kconfig split: `CONFIG_KVM_ARM` for native support, and a variant for Gunyah-backed virtualization. - Adds `gunyah.c`, a new arch backend file that implements the minimal KVM architecture callbacks and stub interfaces required by the KVM core to build and boot. - Refactors Makefile and build rules to support mutually exclusive builds of `CONFIG_KVM_ARM` and `CONFIG_GUNYAH`. - Introduces a dummy implementation of required KVM stubs such as: `kvm_arch_init_vm()`, `kvm_arch_vcpu_create()`, `kvm_age_gfn()`, etc. This serves as a starting point for developing virtualization support for guests running under the Gunyah hypervisor. Subsequent patches in the series will add support for memory mapping, virtual CPUs, IRQ injection, and other guest lifecycle mechanisms. CONFIG_GUNYAH is going to be introduced in the next patch imlpementing the Gunyah driver. [1] https://www.qualcomm.com/developer/blog/2024/01/gunyah-hypervisor-softw= are-supporting-protected-vms-android-virtualization-framework Signed-off-by: Karim Manaouil --- arch/arm64/include/asm/kvm_host.h | 2 +- arch/arm64/include/asm/virt.h | 7 + arch/arm64/kernel/cpufeature.c | 4 + arch/arm64/kernel/image-vars.h | 2 +- arch/arm64/kvm/Kconfig | 22 +- arch/arm64/kvm/Makefile | 14 +- arch/arm64/kvm/gunyah.c | 736 ++++++++++++++++++++++++++ include/kvm/arm_pmu.h | 2 +- include/linux/irqchip/arm-vgic-info.h | 2 +- include/linux/perf/arm_pmu.h | 2 +- 10 files changed, 781 insertions(+), 12 deletions(-) create mode 100644 arch/arm64/kvm/gunyah.c diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm= _host.h index e98cfe7855a6..efbfe31d262d 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -1417,7 +1417,7 @@ static inline bool kvm_pmu_counter_deferred(struct pe= rf_event_attr *attr) return (!has_vhe() && attr->exclude_host); } =20 -#ifdef CONFIG_KVM +#ifdef CONFIG_KVM_ARM void kvm_set_pmu_events(u64 set, struct perf_event_attr *attr); void kvm_clr_pmu_events(u64 clr); bool kvm_set_pmuserenr(u64 val); diff --git a/arch/arm64/include/asm/virt.h b/arch/arm64/include/asm/virt.h index ebf4a9f943ed..80ddb409e0cb 100644 --- a/arch/arm64/include/asm/virt.h +++ b/arch/arm64/include/asm/virt.h @@ -82,11 +82,18 @@ bool is_kvm_arm_initialised(void); =20 DECLARE_STATIC_KEY_FALSE(kvm_protected_mode_initialized); =20 +#ifndef CONFIG_KVM_ARM +static inline bool is_pkvm_initialized(void) +{ + return false; +} +#else static inline bool is_pkvm_initialized(void) { return IS_ENABLED(CONFIG_KVM) && static_branch_likely(&kvm_protected_mode_initialized); } +#endif =20 /* Reports the availability of HYP mode */ static inline bool is_hyp_mode_available(void) diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index 9c4d6d552b25..50a251c28a48 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -3548,6 +3548,9 @@ static void verify_sme_features(void) cpacr_restore(cpacr); } =20 +#ifndef CONFIG_KVM_ARM +static void verify_hyp_capabilities(void) { } +#else static void verify_hyp_capabilities(void) { u64 safe_mmfr1, mmfr0, mmfr1; @@ -3578,6 +3581,7 @@ static void verify_hyp_capabilities(void) cpu_die_early(); } } +#endif =20 static void verify_mpam_capabilities(void) { diff --git a/arch/arm64/kernel/image-vars.h b/arch/arm64/kernel/image-vars.h index 5e3c4b58f279..7688f53b55bd 100644 --- a/arch/arm64/kernel/image-vars.h +++ b/arch/arm64/kernel/image-vars.h @@ -72,7 +72,7 @@ PROVIDE(__pi__data =3D _data); PROVIDE(__pi___bss_start =3D __bss_start); PROVIDE(__pi__end =3D _end); =20 -#ifdef CONFIG_KVM +#ifdef CONFIG_KVM_ARM =20 /* * KVM nVHE code has its own symbol namespace prefixed with __kvm_nvhe_, to diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig index 096e45acadb2..eb43eabcf61b 100644 --- a/arch/arm64/kvm/Kconfig +++ b/arch/arm64/kvm/Kconfig @@ -17,9 +17,25 @@ menuconfig VIRTUALIZATION =20 if VIRTUALIZATION =20 +###################### Gunyah-based KVM ###################### +menuconfig KVM + bool "Kernel-based Virtual Machine (KVM) under Gunyah" + depends on GUNYAH + select KVM_COMMON + select KVM_GENERIC_MMU_NOTIFIER + select KVM_MMIO + select HAVE_KVM_IRQCHIP + select HAVE_KVM_READONLY_MEM + +###################### Native ARM KVM ###################### +config KVM_ARM + bool + depends on !GUNYAH + menuconfig KVM bool "Kernel-based Virtual Machine (KVM) support" - depends on AS_HAS_ARMV8_4 + depends on !GUNYAH && AS_HAS_ARMV8_4 + select KVM_ARM select KVM_COMMON select KVM_GENERIC_HARDWARE_ENABLING select KVM_GENERIC_MMU_NOTIFIER @@ -43,6 +59,8 @@ menuconfig KVM =20 If unsure, say N. =20 +if KVM_ARM + config NVHE_EL2_DEBUG bool "Debug mode for non-VHE EL2 object" depends on KVM @@ -82,5 +100,5 @@ config PTDUMP_STAGE2_DEBUGFS kernel. =20 If in doubt, say N. - +endif # KVM_ARM endif # VIRTUALIZATION diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile index 209bc76263f1..5b54eb329ce2 100644 --- a/arch/arm64/kvm/Makefile +++ b/arch/arm64/kvm/Makefile @@ -8,12 +8,14 @@ ccflags-y +=3D -I $(src) include $(srctree)/virt/kvm/Makefile.kvm =20 obj-$(CONFIG_KVM) +=3D kvm.o -obj-$(CONFIG_KVM) +=3D hyp/ +obj-$(CONFIG_KVM_ARM) +=3D hyp/ =20 CFLAGS_sys_regs.o +=3D -Wno-override-init CFLAGS_handle_exit.o +=3D -Wno-override-init =20 -kvm-y +=3D arm.o mmu.o mmio.o psci.o hypercalls.o pvtime.o \ +kvm-$(CONFIG_GUNYAH) +=3D gunyah.o + +kvm-$(CONFIG_KVM_ARM) +=3D arm.o mmu.o mmio.o psci.o hypercalls.o pvtime.o= \ inject_fault.o va_layout.o handle_exit.o \ guest.o debug.o reset.o sys_regs.o stacktrace.o \ vgic-sys-reg-v3.o fpsimd.o pkvm.o \ @@ -25,9 +27,11 @@ kvm-y +=3D arm.o mmu.o mmio.o psci.o hypercalls.o pvtime= .o \ vgic/vgic-mmio-v3.o vgic/vgic-kvm-device.o \ vgic/vgic-its.o vgic/vgic-debug.o vgic/vgic-v3-nested.o =20 -kvm-$(CONFIG_HW_PERF_EVENTS) +=3D pmu-emul.o pmu.o -kvm-$(CONFIG_ARM64_PTR_AUTH) +=3D pauth.o -kvm-$(CONFIG_PTDUMP_STAGE2_DEBUGFS) +=3D ptdump.o +ifeq ($(CONFIG_KVM_ARM),y) + kvm-$(CONFIG_HW_PERF_EVENTS) +=3D pmu-emul.o pmu.o + kvm-$(CONFIG_ARM64_PTR_AUTH) +=3D pauth.o + kvm-$(CONFIG_PTDUMP_STAGE2_DEBUGFS) +=3D ptdump.o +endif =20 always-y :=3D hyp_constants.h hyp-constants.s =20 diff --git a/arch/arm64/kvm/gunyah.c b/arch/arm64/kvm/gunyah.c new file mode 100644 index 000000000000..0095610166ad --- /dev/null +++ b/arch/arm64/kvm/gunyah.c @@ -0,0 +1,736 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * KVM port to Qualcomm's Gunyah Hypervisor + * + * Copyright (C) 2024-2025 Linaro Ltd. + * + * Author: Karim Manaouil + * + */ +#include +#include +#include +#include +#include + +static enum kvm_mode kvm_mode =3D KVM_MODE_DEFAULT; + +enum kvm_mode kvm_get_mode(void) +{ + return kvm_mode; +} + +const struct _kvm_stats_desc kvm_vm_stats_desc[] =3D { + KVM_GENERIC_VM_STATS() +}; + +const struct kvm_stats_header kvm_vm_stats_header =3D { + .name_size =3D KVM_STATS_NAME_SIZE, + .num_desc =3D ARRAY_SIZE(kvm_vm_stats_desc), + .id_offset =3D sizeof(struct kvm_stats_header), + .desc_offset =3D sizeof(struct kvm_stats_header) + KVM_STATS_NAME_SIZE, + .data_offset =3D sizeof(struct kvm_stats_header) + KVM_STATS_NAME_SIZE + + sizeof(kvm_vm_stats_desc), +}; + +const struct _kvm_stats_desc kvm_vcpu_stats_desc[] =3D { + KVM_GENERIC_VCPU_STATS(), + STATS_DESC_COUNTER(VCPU, hvc_exit_stat), + STATS_DESC_COUNTER(VCPU, wfe_exit_stat), + STATS_DESC_COUNTER(VCPU, wfi_exit_stat), + STATS_DESC_COUNTER(VCPU, mmio_exit_user), + STATS_DESC_COUNTER(VCPU, mmio_exit_kernel), + STATS_DESC_COUNTER(VCPU, signal_exits), + STATS_DESC_COUNTER(VCPU, exits) +}; + +const struct kvm_stats_header kvm_vcpu_stats_header =3D { + .name_size =3D KVM_STATS_NAME_SIZE, + .num_desc =3D ARRAY_SIZE(kvm_vcpu_stats_desc), + .id_offset =3D sizeof(struct kvm_stats_header), + .desc_offset =3D sizeof(struct kvm_stats_header) + KVM_STATS_NAME_SIZE, + .data_offset =3D sizeof(struct kvm_stats_header) + KVM_STATS_NAME_SIZE + + sizeof(kvm_vcpu_stats_desc), +}; + +static bool core_reg_offset_is_vreg(u64 off) +{ + return off >=3D KVM_REG_ARM_CORE_REG(fp_regs.vregs) && + off < KVM_REG_ARM_CORE_REG(fp_regs.fpsr); +} + +static u64 core_reg_offset_from_id(u64 id) +{ + return id & ~(KVM_REG_ARCH_MASK | KVM_REG_SIZE_MASK | KVM_REG_ARM_CORE); +} + +static int core_reg_size_from_offset(const struct kvm_vcpu *vcpu, u64 off) +{ + int size; + + switch (off) { + case KVM_REG_ARM_CORE_REG(regs.regs[0]) ... + KVM_REG_ARM_CORE_REG(regs.regs[30]): + case KVM_REG_ARM_CORE_REG(regs.sp): + case KVM_REG_ARM_CORE_REG(regs.pc): + case KVM_REG_ARM_CORE_REG(regs.pstate): + case KVM_REG_ARM_CORE_REG(sp_el1): + case KVM_REG_ARM_CORE_REG(elr_el1): + case KVM_REG_ARM_CORE_REG(spsr[0]) ... + KVM_REG_ARM_CORE_REG(spsr[KVM_NR_SPSR - 1]): + size =3D sizeof(__u64); + break; + + case KVM_REG_ARM_CORE_REG(fp_regs.vregs[0]) ... + KVM_REG_ARM_CORE_REG(fp_regs.vregs[31]): + size =3D sizeof(__uint128_t); + break; + + case KVM_REG_ARM_CORE_REG(fp_regs.fpsr): + case KVM_REG_ARM_CORE_REG(fp_regs.fpcr): + size =3D sizeof(__u32); + break; + + default: + return -EINVAL; + } + + if (!IS_ALIGNED(off, size / sizeof(__u32))) + return -EINVAL; + + /* + * The KVM_REG_ARM64_SVE regs must be used instead of + * KVM_REG_ARM_CORE for accessing the FPSIMD V-registers on + * SVE-enabled vcpus: + */ + if (vcpu_has_sve(vcpu) && core_reg_offset_is_vreg(off)) + return -EINVAL; + + return size; +} + +static void *core_reg_addr(struct kvm_vcpu *vcpu, const struct kvm_one_reg= *reg) +{ + u64 off =3D core_reg_offset_from_id(reg->id); + int size =3D core_reg_size_from_offset(vcpu, off); + + if (size < 0) + return NULL; + + if (KVM_REG_SIZE(reg->id) !=3D size) + return NULL; + + switch (off) { + case KVM_REG_ARM_CORE_REG(regs.regs[0]) ... + KVM_REG_ARM_CORE_REG(regs.regs[30]): + off -=3D KVM_REG_ARM_CORE_REG(regs.regs[0]); + off /=3D 2; + return &vcpu->arch.ctxt.regs.regs[off]; + + case KVM_REG_ARM_CORE_REG(regs.sp): + return &vcpu->arch.ctxt.regs.sp; + + case KVM_REG_ARM_CORE_REG(regs.pc): + return &vcpu->arch.ctxt.regs.pc; + + case KVM_REG_ARM_CORE_REG(regs.pstate): + return &vcpu->arch.ctxt.regs.pstate; + + case KVM_REG_ARM_CORE_REG(sp_el1): + return __ctxt_sys_reg(&vcpu->arch.ctxt, SP_EL1); + + case KVM_REG_ARM_CORE_REG(elr_el1): + return __ctxt_sys_reg(&vcpu->arch.ctxt, ELR_EL1); + + case KVM_REG_ARM_CORE_REG(spsr[KVM_SPSR_EL1]): + return __ctxt_sys_reg(&vcpu->arch.ctxt, SPSR_EL1); + + case KVM_REG_ARM_CORE_REG(spsr[KVM_SPSR_ABT]): + return &vcpu->arch.ctxt.spsr_abt; + + case KVM_REG_ARM_CORE_REG(spsr[KVM_SPSR_UND]): + return &vcpu->arch.ctxt.spsr_und; + + case KVM_REG_ARM_CORE_REG(spsr[KVM_SPSR_IRQ]): + return &vcpu->arch.ctxt.spsr_irq; + + case KVM_REG_ARM_CORE_REG(spsr[KVM_SPSR_FIQ]): + return &vcpu->arch.ctxt.spsr_fiq; + + case KVM_REG_ARM_CORE_REG(fp_regs.vregs[0]) ... + KVM_REG_ARM_CORE_REG(fp_regs.vregs[31]): + off -=3D KVM_REG_ARM_CORE_REG(fp_regs.vregs[0]); + off /=3D 4; + return &vcpu->arch.ctxt.fp_regs.vregs[off]; + + case KVM_REG_ARM_CORE_REG(fp_regs.fpsr): + return &vcpu->arch.ctxt.fp_regs.fpsr; + + case KVM_REG_ARM_CORE_REG(fp_regs.fpcr): + return &vcpu->arch.ctxt.fp_regs.fpcr; + + default: + return NULL; + } +} + +static int get_core_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *r= eg) +{ + /* + * Because the kvm_regs structure is a mix of 32, 64 and + * 128bit fields, we index it as if it was a 32bit + * array. Hence below, nr_regs is the number of entries, and + * off the index in the "array". + */ + __u32 __user *uaddr =3D (__u32 __user *)(unsigned long)reg->addr; + int nr_regs =3D sizeof(struct kvm_regs) / sizeof(__u32); + void *addr; + u32 off; + + /* Our ID is an index into the kvm_regs struct. */ + off =3D core_reg_offset_from_id(reg->id); + if (off >=3D nr_regs || + (off + (KVM_REG_SIZE(reg->id) / sizeof(__u32))) >=3D nr_regs) + return -ENOENT; + + addr =3D core_reg_addr(vcpu, reg); + if (!addr) + return -EINVAL; + + if (copy_to_user(uaddr, addr, KVM_REG_SIZE(reg->id))) + return -EFAULT; + + return 0; +} + +static int set_core_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *r= eg) +{ + __u32 __user *uaddr =3D (__u32 __user *)(unsigned long)reg->addr; + int nr_regs =3D sizeof(struct kvm_regs) / sizeof(__u32); + __uint128_t tmp; + void *valp =3D &tmp, *addr; + u64 off; + int err =3D 0; + + /* Our ID is an index into the kvm_regs struct. */ + off =3D core_reg_offset_from_id(reg->id); + if (off >=3D nr_regs || + (off + (KVM_REG_SIZE(reg->id) / sizeof(__u32))) >=3D nr_regs) + return -ENOENT; + + addr =3D core_reg_addr(vcpu, reg); + if (!addr) + return -EINVAL; + + if (KVM_REG_SIZE(reg->id) > sizeof(tmp)) + return -EINVAL; + + if (copy_from_user(valp, uaddr, KVM_REG_SIZE(reg->id))) { + err =3D -EFAULT; + goto out; + } + + memcpy(addr, valp, KVM_REG_SIZE(reg->id)); +out: + return err; +} + +static int get_sys_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *re= g) +{ + __u32 __user *uaddr =3D (__u32 __user *)(unsigned long)reg->addr; + u64 dummy_val =3D 0; + + if (copy_to_user(uaddr, &dummy_val, KVM_REG_SIZE(reg->id))) + return -EFAULT; + + return 0; +} + +static int copy_core_reg_indices(const struct kvm_vcpu *vcpu, + u64 __user *uindices) +{ + unsigned int i; + int n =3D 0; + + for (i =3D 0; i < sizeof(struct kvm_regs) / sizeof(__u32); i++) { + u64 reg =3D KVM_REG_ARM64 | KVM_REG_ARM_CORE | i; + int size =3D core_reg_size_from_offset(vcpu, i); + + if (size < 0) + continue; + + switch (size) { + case sizeof(__u32): + reg |=3D KVM_REG_SIZE_U32; + break; + + case sizeof(__u64): + reg |=3D KVM_REG_SIZE_U64; + break; + + case sizeof(__uint128_t): + reg |=3D KVM_REG_SIZE_U128; + break; + + default: + WARN_ON(1); + continue; + } + + if (uindices) { + if (put_user(reg, uindices)) + return -EFAULT; + uindices++; + } + + n++; + } + + return n; +} + +static unsigned long num_core_regs(const struct kvm_vcpu *vcpu) +{ + return copy_core_reg_indices(vcpu, NULL); +} + +int kvm_arm_get_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg) +{ + /* We currently use nothing arch-specific in upper 32 bits */ + if ((reg->id & ~KVM_REG_SIZE_MASK) >> 32 !=3D KVM_REG_ARM64 >> 32) + return -EINVAL; + + switch (reg->id & KVM_REG_ARM_COPROC_MASK) { + case KVM_REG_ARM_CORE: + return get_core_reg(vcpu, reg); + case KVM_REG_ARM64_SYSREG: + return get_sys_reg(vcpu, reg); + default: + return -ENOENT; + } +} + +int kvm_arm_set_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg) +{ + /* We currently use nothing arch-specific in upper 32 bits */ + if ((reg->id & ~KVM_REG_SIZE_MASK) >> 32 !=3D KVM_REG_ARM64 >> 32) + return -EINVAL; + + switch (reg->id & KVM_REG_ARM_COPROC_MASK) { + case KVM_REG_ARM_CORE: + return set_core_reg(vcpu, reg); + default: + return -ENOENT; + } +} + +int kvm_arch_vcpu_should_kick(struct kvm_vcpu *vcpu) +{ + return kvm_vcpu_exiting_guest_mode(vcpu) =3D=3D IN_GUEST_MODE; +} + +vm_fault_t kvm_arch_vcpu_fault(struct kvm_vcpu *vcpu, struct vm_fault *vmf) +{ + return VM_FAULT_SIGBUS; +} + +void kvm_arch_create_vm_debugfs(struct kvm *kvm) +{ +} + +void kvm_arch_destroy_vm(struct kvm *kvm) +{ + kvm_destroy_vcpus(kvm); + return; +} + +long kvm_arch_dev_ioctl(struct file *filp, + unsigned int ioctl, unsigned long arg) +{ + return -EINVAL; +} + +bool kvm_arch_irqchip_in_kernel(struct kvm *kvm) +{ + return false; +} + +bool kvm_arch_intc_initialized(struct kvm *kvm) +{ + return true; +} + +struct kvm_vcpu *kvm_arch_vcpu_alloc(void) +{ + return NULL; +} + +int kvm_arch_vcpu_precreate(struct kvm *kvm, unsigned int id) +{ + return 0; +} + +int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu) +{ + return -EINVAL; +} + +void kvm_arch_vcpu_postcreate(struct kvm_vcpu *vcpu) +{ +} + +void kvm_arch_vcpu_destroy(struct kvm_vcpu *vcpu) +{ +} + +void kvm_arch_vcpu_blocking(struct kvm_vcpu *vcpu) +{ +} + +void kvm_arch_vcpu_unblocking(struct kvm_vcpu *vcpu) +{ +} + +void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu) +{ +} + +void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu) +{ +} + +int kvm_arch_vcpu_ioctl_get_mpstate(struct kvm_vcpu *vcpu, + struct kvm_mp_state *mp_state) +{ + return -EINVAL; +} + +int kvm_arch_vcpu_ioctl_set_mpstate(struct kvm_vcpu *vcpu, + struct kvm_mp_state *mp_state) +{ + return -EINVAL; +} + +int kvm_arch_vcpu_runnable(struct kvm_vcpu *v) +{ + return 0; +} + +bool kvm_arch_vcpu_in_kernel(struct kvm_vcpu *vcpu) +{ + return false; +} + +int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu) +{ + return -EINVAL; +} + +long kvm_arch_vcpu_ioctl(struct file *filp, + unsigned int ioctl, unsigned long arg) +{ + struct kvm_vcpu *vcpu =3D filp->private_data; + void __user *argp =3D (void __user *)arg; + long r; + + switch (ioctl) { + case KVM_ARM_VCPU_INIT: { + struct kvm_vcpu_init init; + + r =3D -EFAULT; + if (copy_from_user(&init, argp, sizeof(init))) + break; + + vcpu_set_flag(vcpu, VCPU_INITIALIZED); + r =3D 0; + break; + } + case KVM_SET_ONE_REG: + case KVM_GET_ONE_REG: { + struct kvm_one_reg reg; + + r =3D -ENOEXEC; + if (unlikely(!kvm_vcpu_initialized(vcpu))) + break; + + r =3D -EFAULT; + if (copy_from_user(®, argp, sizeof(reg))) + break; + + if (ioctl =3D=3D KVM_SET_ONE_REG) + r =3D kvm_arm_set_reg(vcpu, ®); + else + r =3D kvm_arm_get_reg(vcpu, ®); + break; + } + case KVM_GET_REG_LIST: { + struct kvm_reg_list __user *user_list =3D argp; + struct kvm_reg_list reg_list; + unsigned n; + + r =3D -ENOEXEC; + if (unlikely(!kvm_vcpu_initialized(vcpu))) + break; + + r =3D -EFAULT; + if (copy_from_user(®_list, user_list, sizeof(reg_list))) + break; + + n =3D reg_list.n; + reg_list.n =3D num_core_regs(vcpu); + if (copy_to_user(user_list, ®_list, sizeof(reg_list))) + break; + r =3D -E2BIG; + if (n < reg_list.n) + break; + + r =3D 0; + copy_core_reg_indices(vcpu, user_list->reg); + break; + } + case KVM_ARM_VCPU_FINALIZE: { + return 0; + } + default: + pr_info("gunyah: %s: unrecognised vcpu ioctl %u\n", __func__, ioctl); + r =3D -EINVAL; + } + + return r; +} + +int kvm_arch_vcpu_ioctl_get_regs(struct kvm_vcpu *vcpu, struct kvm_regs *r= egs) +{ + return -EINVAL; +} + +int kvm_arch_vcpu_ioctl_set_regs(struct kvm_vcpu *vcpu, struct kvm_regs *r= egs) +{ + return -EINVAL; +} + +int kvm_arch_vcpu_ioctl_get_sregs(struct kvm_vcpu *vcpu, + struct kvm_sregs *sregs) +{ + return -EINVAL; +} + +int kvm_arch_vcpu_ioctl_set_sregs(struct kvm_vcpu *vcpu, + struct kvm_sregs *sregs) +{ + return -EINVAL; +} + +int kvm_arch_vcpu_ioctl_get_fpu(struct kvm_vcpu *vcpu, struct kvm_fpu *fpu) +{ + return -EINVAL; +} + +int kvm_arch_vcpu_ioctl_set_fpu(struct kvm_vcpu *vcpu, struct kvm_fpu *fpu) +{ + return -EINVAL; +} + +int kvm_arch_vcpu_ioctl_translate(struct kvm_vcpu *vcpu, + struct kvm_translation *tr) +{ + return -EINVAL; +} + +int kvm_arch_vcpu_ioctl_set_guest_debug(struct kvm_vcpu *vcpu, + struct kvm_guest_debug *dbg) +{ + return -EINVAL; +} + +void kvm_arch_vcpu_put_debug_state_flags(struct kvm_vcpu *vcpu) +{ +} + +int kvm_vm_ioctl_enable_cap(struct kvm *kvm, + struct kvm_enable_cap *cap) +{ + return -EINVAL; +} + +int kvm_arch_vm_ioctl(struct file *filp, unsigned int ioctl, unsigned long= arg) +{ + void __user *argp =3D (void __user *)arg; + + switch (ioctl) { + case KVM_ARM_PREFERRED_TARGET: { + struct kvm_vcpu_init init =3D { + .target =3D KVM_ARM_TARGET_GENERIC_V8, + }; + + if (copy_to_user(argp, &init, sizeof(init))) + return -EFAULT; + + return 0; + } + case KVM_ARM_SET_COUNTER_OFFSET: { + return -ENXIO; + } + case KVM_ARM_SET_DEVICE_ADDR: { + struct kvm_arm_device_addr dev_addr; + + if (copy_from_user(&dev_addr, argp, sizeof(dev_addr))) + return -EFAULT; + + return -ENODEV; + } + case KVM_HAS_DEVICE_ATTR: { + return -ENXIO; + } + case KVM_SET_DEVICE_ATTR: { + return -ENXIO; + } + case KVM_ARM_GET_REG_WRITABLE_MASKS: { + return -ENXIO; + } + default: + return -EINVAL; + } +} + +int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) +{ + int r; + + switch (ext) { + case KVM_CAP_IOEVENTFD: + case KVM_CAP_USER_MEMORY: + case KVM_CAP_SYNC_MMU: + case KVM_CAP_ONE_REG: + case KVM_CAP_READONLY_MEM: + case KVM_CAP_VCPU_ATTRIBUTES: + case KVM_CAP_ARM_USER_IRQ: + case KVM_CAP_ARM_SET_DEVICE_ADDR: + r =3D 1; + break; + case KVM_CAP_NR_VCPUS: + /* + * ARM64 treats KVM_CAP_NR_CPUS differently from all other + * architectures, as it does not always bound it to + * KVM_CAP_MAX_VCPUS. It should not matter much because + * this is just an advisory value. + */ + r =3D min_t(unsigned int, num_online_cpus(), KVM_MAX_VCPUS); + break; + case KVM_CAP_MAX_VCPUS: + case KVM_CAP_MAX_VCPU_ID: + r =3D KVM_MAX_VCPUS; + break; + default: + r =3D 0; + } + + return r; +} + +int kvm_vm_ioctl_irq_line(struct kvm *kvm, struct kvm_irq_level *irq_level, + bool line_status) +{ + return -ENXIO; +} + +int kvm_arch_init_vm(struct kvm *kvm, unsigned long type) +{ + return 0; +} + +void kvm_arch_flush_shadow_all(struct kvm *kvm) +{ +} + +int kvm_arch_flush_remote_tlbs(struct kvm *kvm) +{ + return -EINVAL; +} + +int kvm_arch_flush_remote_tlbs_range(struct kvm *kvm, + gfn_t gfn, u64 nr_pages) +{ + return -EINVAL; +} + +void kvm_arch_mmu_enable_log_dirty_pt_masked(struct kvm *kvm, + struct kvm_memory_slot *slot, + gfn_t gfn_offset, unsigned long mask) +{ +} + +void kvm_arch_commit_memory_region(struct kvm *kvm, + struct kvm_memory_slot *old, + const struct kvm_memory_slot *new, + enum kvm_mr_change change) +{ +} + +int kvm_arch_prepare_memory_region(struct kvm *kvm, + const struct kvm_memory_slot *old, + struct kvm_memory_slot *new, + enum kvm_mr_change change) +{ + return 0; +} + +void kvm_arch_free_memslot(struct kvm *kvm, struct kvm_memory_slot *slot) +{ +} + +void kvm_arch_memslots_updated(struct kvm *kvm, u64 gen) +{ +} + +void kvm_arch_flush_shadow_memslot(struct kvm *kvm, + struct kvm_memory_slot *slot) +{ +} + +bool kvm_age_gfn(struct kvm *kvm, struct kvm_gfn_range *range) +{ + return false; +} + + +bool kvm_test_age_gfn(struct kvm *kvm, struct kvm_gfn_range *range) +{ + return false; +} + +bool kvm_unmap_gfn_range(struct kvm *kvm, struct kvm_gfn_range *range) +{ + return false; +} + +int kvm_vm_ioctl_get_dirty_log(struct kvm *kvm, struct kvm_dirty_log *log) +{ + return -EINVAL; +} + +__init void kvm_compute_layout(void) +{ +} + +__init void kvm_apply_hyp_relocations(void) +{ +} + +void __init kvm_hyp_reserve(void) +{ +} + +void kvm_arch_sync_dirty_log(struct kvm *kvm, struct kvm_memory_slot *mems= lot) +{ +} + +int kvm_cpu_has_pending_timer(struct kvm_vcpu *vcpu) +{ + return -EINVAL; +} + +struct kvm *kvm_arch_alloc_vm(void) +{ + return NULL; +} diff --git a/include/kvm/arm_pmu.h b/include/kvm/arm_pmu.h index 96754b51b411..575864e93f79 100644 --- a/include/kvm/arm_pmu.h +++ b/include/kvm/arm_pmu.h @@ -12,7 +12,7 @@ =20 #define KVM_ARMV8_PMU_MAX_COUNTERS 32 =20 -#if IS_ENABLED(CONFIG_HW_PERF_EVENTS) && IS_ENABLED(CONFIG_KVM) +#if IS_ENABLED(CONFIG_HW_PERF_EVENTS) && IS_ENABLED(CONFIG_KVM_ARM) struct kvm_pmc { u8 idx; /* index into the pmu->pmc array */ struct perf_event *perf_event; diff --git a/include/linux/irqchip/arm-vgic-info.h b/include/linux/irqchip/= arm-vgic-info.h index a75b2c7de69d..7a4a6051ffa6 100644 --- a/include/linux/irqchip/arm-vgic-info.h +++ b/include/linux/irqchip/arm-vgic-info.h @@ -36,7 +36,7 @@ struct gic_kvm_info { bool no_hw_deactivation; }; =20 -#ifdef CONFIG_KVM +#ifdef CONFIG_KVM_ARM void vgic_set_kvm_info(const struct gic_kvm_info *info); #else static inline void vgic_set_kvm_info(const struct gic_kvm_info *info) {} diff --git a/include/linux/perf/arm_pmu.h b/include/linux/perf/arm_pmu.h index 6dc5e0cd76ca..c6cb2db9402a 100644 --- a/include/linux/perf/arm_pmu.h +++ b/include/linux/perf/arm_pmu.h @@ -170,7 +170,7 @@ int arm_pmu_acpi_probe(armpmu_init_fn init_fn); static inline int arm_pmu_acpi_probe(armpmu_init_fn init_fn) { return 0; } #endif =20 -#ifdef CONFIG_KVM +#ifdef CONFIG_KVM_ARM void kvm_host_pmu_init(struct arm_pmu *pmu); #else #define kvm_host_pmu_init(x) do { } while(0) --=20 2.39.5 From nobody Thu Dec 18 00:34:59 2025 Received: from mail-wr1-f50.google.com (mail-wr1-f50.google.com [209.85.221.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 729C41A2381 for ; Thu, 24 Apr 2025 14:13:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.50 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504039; cv=none; b=pk0pJ8DaY9A7uhWhi8z3v5yk1jzrart8HbiivjzpQpfjVkJJB6ny1D9UmXLVgG2oVvjgVPx1sAAOfigjiuKbGDGttTwI5Z0yKtRut5qOkqkLKY4Q/bhV2cBJdJnf8cmWYEds5+/xV6viTt6Pj7/ycAfHHCsmnk+cD68UieOgai4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504039; c=relaxed/simple; bh=pMsi9MMq97g7CjVTvZZzMUKV739tetlHWJOghEinxps=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=CMsHz6Zead52Xo5KEWo4QVVmvfuSpT/HSZ1ttyRUmK/LnafopDHcsFBHMezkCtf8pr41hbmkDhGXRSZXQlg9udHaZRrGeOvAhzEkLq3JogvAkTB4KmHFXrTj7cnXdoJkqCFlBV6NESsyf9NSuSx3yUkmDTDnnPJsmOiDrIzJx7s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=sTk4m9/r; arc=none smtp.client-ip=209.85.221.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="sTk4m9/r" Received: by mail-wr1-f50.google.com with SMTP id ffacd0b85a97d-39ee682e0ddso731303f8f.1 for ; Thu, 24 Apr 2025 07:13:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504034; x=1746108834; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=S74NKpGFaVYOESsCH+rK4kh5DkFdas6Gq4YwtRp9OLI=; b=sTk4m9/r/YqM6gAkyB9+f+Cf9v2YjMXe9/V7HSO7QbIMep386kIqnkxl+8b5ZgmaMi vq11UPzDeTnTokkrMaJkq1om7bX7cplUE72LWmmRp55xsdwa8xhIOHfkaFCBWupg7bWu r7rCnHfm8rpJMFtqLaSRxYaEy7UlfKA0KEbr52xdM6miVQnel9dDetTl4FMpxMkuR0dM QFswRunRyDAjTnUH1ma3M3jX4DJL1/2E4yj7mgGDCP0h6+LhtvAXMqxx6zZjfdR+TDB3 ymPhFpTwlgSj4rRMQ7FTLfCHMZ2gFbLb8agEcLvCUWlIUGhnsRVx/KzpLAIKXO+7aBGD mElA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504034; x=1746108834; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=S74NKpGFaVYOESsCH+rK4kh5DkFdas6Gq4YwtRp9OLI=; b=ce2hofbIwSr1Sc+LmsPv3yz8xxM6Qkqk8WHUOPmh6D5bCijaDsVLpNyDFHlLpl4FK3 a4lcJb4l+QuKZfpz6uMFx+wDeeDRbOfatFBLqy6MvdnPUYQ7GG58YyRhtzDowPXIXtwb Q+gtfnEbdyftiKV3MfhOOHOcFaouvmRvK3jWkuFS5HjGfnArs1y2ckxRiHfe17Hk31eh sLNsu9rx7J17o4uCCjWi3G71ilzNmnevYdiYdP5cHgeDeG+gW2czMWX4FBo0jDPPwn7a qNNzT5JK+hJXjfmTizmjjVOkkatdVj7WMZwsaFpctZhk1Qfqf2fpe+vAIhDAzZoLjY/C Aa+w== X-Gm-Message-State: AOJu0Yw59bcZT2uAtYqFYc7ESH8KOLptD8wQIX8Cztohub0APGef3Nxy zvYzLP2ZI2VrYv9A+mzoNTgOOxN6yc6vTAYd4jOso9mEEeNgVeeUeMflDSK5hrZ8XtUtBqx+Vw5 9PYs= X-Gm-Gg: ASbGnctL/UE2Lx5CSFPJTISEVyMPQEZLZdTRkCnDvCCarsxh2hrSX961sRqUI8jhN1A Ik3YTZXKwfXxaAqSq2vqRqefq//5SmCcrCalPlLx/eJuSk0heKKJWlKGJaJjqRLd1W/tQ8zWN/s KbxTFnEwQXe7v2xxt5PSocHKpTTodDHgYhp3DfYKYWE+ttc6N5iCCaYGMwngHoPLj3k82g+cj+0 xP5Ixc1HSeHcdK65IzA/wNZ3FQAQbJ4rfu1+9bECjzayn9WMi1n3xfgrZzGCnNEnna17UM6tRzI XGHSgEeG3/7Nr03CctnzCas//yCH4dLxm4bOjSg45xD1Vcjxct9RcvwGD+Tsyb2ZHYnwyjm7+b9 hQhuUS6tsUEoeG8d8 X-Google-Smtp-Source: AGHT+IHx9bXGbc/XwzdBX2sEvhGOWsNMsxK7elP9ZHfuoFqJa71fFuvenAr2zCrjXuMcvE5wd1GoWA== X-Received: by 2002:a05:6000:290d:b0:391:3b11:d604 with SMTP id ffacd0b85a97d-3a06cfa9d47mr2143588f8f.54.1745504034133; Thu, 24 Apr 2025 07:13:54 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.13.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:13:53 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt Subject: [RFC PATCH 07/34] KVM: gunyah: Pin guest memory Date: Thu, 24 Apr 2025 15:13:14 +0100 Message-Id: <20250424141341.841734-8-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Qualcomm's Gunyah hypervisor allows to implement protected VMs, in which private memory given to the guest is no more accessible to the host (an access violation will be raised if the host tries to read/write that memory). In the context of protected VMs (aka confidential computing), the consensus to manage this memory is via guest_memfd. We would like this port to be be based on guest_memfd. However, for this RFC, the port allocates anonymous pages, which are subject to migration and swap out. That will trigger a violation if the memory is private, which is the case for most of the guest's main memory. Since the memory is allocated and given to the guest for possibly an unbounded amount of time, we longterm pin the pages to prevent the kernel from touching and possibly swapping or migrating those pages. In upcoming versions of this port, we intend to move to guest_memfd. Signed-off-by: Karim Manaouil --- arch/arm64/include/asm/kvm_host.h | 3 ++ arch/arm64/kvm/gunyah.c | 68 ++++++++++++++++++++++++++++--- 2 files changed, 66 insertions(+), 5 deletions(-) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm= _host.h index efbfe31d262d..9c8e173fc9c1 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -229,6 +229,9 @@ struct kvm_s2_mmu { }; =20 struct kvm_arch_memory_slot { +#ifdef CONFIG_GUNYAH + struct page **pages; +#endif }; =20 /** diff --git a/arch/arm64/kvm/gunyah.c b/arch/arm64/kvm/gunyah.c index 0095610166ad..9c37ab20d7e2 100644 --- a/arch/arm64/kvm/gunyah.c +++ b/arch/arm64/kvm/gunyah.c @@ -660,11 +660,47 @@ void kvm_arch_mmu_enable_log_dirty_pt_masked(struct k= vm *kvm, { } =20 -void kvm_arch_commit_memory_region(struct kvm *kvm, - struct kvm_memory_slot *old, - const struct kvm_memory_slot *new, - enum kvm_mr_change change) +static int gunyah_pin_user_memory(struct kvm *kvm, struct kvm_memory_slot = *memslot) { + unsigned int gup_flags =3D FOLL_WRITE | FOLL_LONGTERM; + unsigned long start =3D memslot->userspace_addr; + struct vm_area_struct *vma; + struct page **pages; + int ret; + + if (!memslot->npages) + return 0; + + /* It needs to be a valid VMA-backed region */ + mmap_read_lock(current->mm); + vma =3D find_vma(current->mm, start); + if (!vma || start < vma->vm_start) { + mmap_read_unlock(current->mm); + return 0; + } + if (!(vma->vm_flags & VM_READ) || !(vma->vm_flags & VM_WRITE)) { + mmap_read_unlock(current->mm); + return 0; + } + mmap_read_unlock(current->mm); + + pages =3D kvcalloc(memslot->npages, sizeof(*pages), GFP_KERNEL); + if (!pages) + return -ENOMEM; + + ret =3D pin_user_pages_fast(start, memslot->npages, gup_flags, pages); + if (ret < 0) { + goto err; + } else if (ret !=3D memslot->npages) { + ret =3D -EIO; + goto err; + } else { + memslot->arch.pages =3D pages; + return 0; + } +err: + kvfree(pages); + return ret; } =20 int kvm_arch_prepare_memory_region(struct kvm *kvm, @@ -672,11 +708,33 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm, struct kvm_memory_slot *new, enum kvm_mr_change change) { - return 0; + int ret; + + switch (change) { + case KVM_MR_CREATE: + ret =3D gunyah_pin_user_memory(kvm, new); + break; + default: + return 0; + } + return ret; +} + +void kvm_arch_commit_memory_region(struct kvm *kvm, + struct kvm_memory_slot *old, + const struct kvm_memory_slot *new, + enum kvm_mr_change change) +{ } =20 void kvm_arch_free_memslot(struct kvm *kvm, struct kvm_memory_slot *slot) { + if (!slot->arch.pages) + return; + + unpin_user_pages(slot->arch.pages, slot->npages); + + kvfree(slot->arch.pages); } =20 void kvm_arch_memslots_updated(struct kvm *kvm, u64 gen) --=20 2.39.5 From nobody Thu Dec 18 00:34:59 2025 Received: from mail-wr1-f53.google.com (mail-wr1-f53.google.com [209.85.221.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DE1CD27FD70 for ; Thu, 24 Apr 2025 14:13:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.53 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504040; cv=none; b=U9VhZRlZuAlx1bS5786VxSSWS9OEqqsLKhsPVnCqmUOSYlsjKXb3LYeEuu3CVYUFB3JZKfawRQOgXcxAKK8/zExY2Jo5Q1J+1ZfLlzdx8OWst2HA42jjQ3el4E1I2H5aP4seBTiqx8V+cR4hkgbedk6lOvoHiTqP1/GdiOmNREY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504040; c=relaxed/simple; bh=DSNuQ7AXSuhIK9F2t1tEvg2Y4H20TKEQaYmf+C1AL10=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=cbVE8GHxhL+QqCBxph0S/8R6fyR9Ue/GHEn7Pndn6Q1vOhMtAgtM6ZPCKj1S70RfZWltp/PJRlfqDlgxTh8Kv1A7UurkmFMkkF9seK3H0nItruyynndZPddMYHo0rc7UUtRsexoXTY2Ijav9jXBdxFWM9Cqq9g4hSgVT8ekcT6g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=C6+9rZe2; arc=none smtp.client-ip=209.85.221.53 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="C6+9rZe2" Received: by mail-wr1-f53.google.com with SMTP id ffacd0b85a97d-39ee57c0b8cso1148397f8f.0 for ; Thu, 24 Apr 2025 07:13:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504036; x=1746108836; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Ys+r4b/bN1N9MasjKmdqEss1ssV3C7RnEHLMtxbroxU=; b=C6+9rZe2ytaKfy31PFB1wcxiH/emj9kOFZMA/GobiGttgsHV8DN2J7qHDUtLWyOGce BZ6rSf+K57Qgh7g99IxoF1d8LhSoFkNNOwOv0KHH6IBHd36eNHKmENIzmmCPxmxuy2Qm 8OKsThyJvgNswpgkfF+BqG9t1F6VdQcsCVOx5vbJ/FMHIlKtHrQDQFjmeHz4HK2zXwDY 0XAgA9WCObJbYTrt4fxANbZARkUmbaSfoSkV8qAT7wuBwtu7n8YStylxfe2wreoFuOHv oX+JtpLTDBp4PZnRUTtZghzd6v3+YbuUWRhbkZWrxBiwO6bgKLTMLXzfehZrz72T+edk ZiJQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504036; x=1746108836; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Ys+r4b/bN1N9MasjKmdqEss1ssV3C7RnEHLMtxbroxU=; b=fAufCjUxhxB6Uk7xbIP4HpJcjHYl2m8lE8xlJ3DrtaoyUimJhObQbBJ6myQShN1IeM oxxoAFnhdFjOOiorPaIk33+LGjAKeQheUMxr5JBAf0pboD9uqwQoIA4iZfsBb62tLMi4 QpdMFTER6WdK6TKBOfXWbNyHYpgv9ocTwO/hno1DfcS3xRRkN/EkcZaWxddIzLmB9s6Q RwRbzTw1qk/saFFl5mrEU3n7nWOxjbhtnSRHP7tDSrnZ8tiCu1a0Yj/D4DRT+eJYdhlR rMpkiP5vEcnu8r23n3gDKAhX1ArF1SNV26AVqYPHe2mzhlO5C8T2KXzvKrlHEmMrWhFv oiYg== X-Gm-Message-State: AOJu0YzDp+D//wyl7tu0BP1Gb3R4PTpjCAlv0PKGLPRCnyk8pfGVxxbX pyc9rfnla0pbXyZD/i6LbjI/kFScQEnrYKPeokW6AkifOBWmsjatLWK5WFxDc9af1r9eENYX5uX B+KY= X-Gm-Gg: ASbGncvKhNVwAAITZyNtv6oxeaJFsdBwSiVkXAJRUwGq3b85lrGuQ8EVRtMv8z88c9v g1cLhi3XqwiZaWr/ouaRYTTxl7EwFfnHN2ci2Nkao+4ED5D/uHbbkD6Ay/L67+Ye5ftqflxrCec i0jW6OZjPKU8ZpoweC3fvCvxFSZsbWACPSeW5LfIRQ5aTULJ95exfSn96OPpN2GevmHOKyfJdnd xA4qKJ+O/VgGnymj76ed7ZCmRatQ6xJDVyoJWJmaHw2rRm0RnQn6iSUdMzyhU4KLNzS5hbpj5HX OSV1b7gNmGGfWMa6g+2VW3wBzapON+n0WURDnmnlYrlba3EGEWfp9rzHJVNOFKZeBvTOVGx0/a5 pg+5sGxcr3YTQ0MwX X-Google-Smtp-Source: AGHT+IEV5wfX7DpJOSMs66ZIGfzUTJLqW6iFySId3MuH6gASyxb/C/SH9jqJ4yi7vysyGZwOnlgesg== X-Received: by 2002:a5d:47cb:0:b0:391:3fa7:bf77 with SMTP id ffacd0b85a97d-3a06cf63697mr2421146f8f.31.1745504035418; Thu, 24 Apr 2025 07:13:55 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.13.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:13:54 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt , Elliot Berman Subject: [RFC PATCH 08/34] docs: gunyah: Introduce Gunyah Hypervisor Date: Thu, 24 Apr 2025 15:13:15 +0100 Message-Id: <20250424141341.841734-9-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Elliot Berman Gunyah is an open-source Type-1 hypervisor developed by Qualcomm. It does not depend on any lower-privileged OS/kernel code for its core functionality. This increases its security and can support a smaller trusted computing based when compared to Type-2 hypervisors. Add documentation describing the Gunyah hypervisor and the main components of the Gunyah hypervisor which are of interest to Linux virtualization development. Signed-off-by: Elliot Berman Reviewed-by: Srivatsa Vaddagiri Signed-off-by: Karim Manaouil --- Documentation/virt/gunyah/index.rst | 135 ++++++++++++++++++++ Documentation/virt/gunyah/message-queue.rst | 68 ++++++++++ Documentation/virt/index.rst | 1 + 3 files changed, 204 insertions(+) create mode 100644 Documentation/virt/gunyah/index.rst create mode 100644 Documentation/virt/gunyah/message-queue.rst diff --git a/Documentation/virt/gunyah/index.rst b/Documentation/virt/gunya= h/index.rst new file mode 100644 index 000000000000..fba2c7a11d0f --- /dev/null +++ b/Documentation/virt/gunyah/index.rst @@ -0,0 +1,135 @@ +.. SPDX-License-Identifier: GPL-2.0 + +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +Gunyah Hypervisor +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +.. toctree:: + :maxdepth: 1 + + message-queue + +Gunyah is a Type-1 hypervisor which is independent of any OS kernel, and r= uns in +a more privileged CPU level (EL2 on Aarch64). It does not depend on a less +privileged operating system for its core functionality. This increases its +security and can support a much smaller trusted computing base than a Type= -2 +hypervisor. + +Gunyah is an open source hypervisor. The source repository is available at +https://github.com/quic/gunyah-hypervisor. + +Gunyah provides these following features. + +- Scheduling: + + A scheduler for virtual CPUs (vCPUs) on physical CPUs enables time-shari= ng + of the CPUs. Gunyah supports two models of scheduling which can coexist = on + a running system: + + 1. Hypervisor vCPU scheduling in which Gunyah hypervisor schedules vCP= US on + its own. The default is a real-time priority with round-robin sched= uler. + 2. "Proxy" scheduling in which an owner-VM can donate the remainder of= its + own vCPU's time slice to an owned-VM's vCPU via a hypercall. + +- Memory Management: + + APIs handling memory, abstracted as objects, limiting direct use of phys= ical + addresses. Memory ownership and usage tracking of all memory under its c= ontrol. + Memory partitioning between VMs is a fundamental security feature. + +- Interrupt Virtualization: + + Interrupt ownership is tracked and interrupt delivery is directly to the + assigned VM. Gunyah makes use of hardware interrupt virtualization where + possible. + +- Inter-VM Communication: + + There are several different mechanisms provided for communicating betwee= n VMs. + + 1. Message queues + 2. Doorbells + 3. Virtio MMIO transport + 4. Shared memory + +- Virtual platform: + + Architectural devices such as interrupt controllers and CPU timers are + directly provided by the hypervisor as well as core virtual platform dev= ices + and system APIs such as ARM PSCI. + +- Device Virtualization: + + Para-virtualization of devices is supported using inter-VM communication= and + virtio transport support. Select stage 2 faults by virtual machines that= use + proxy-scheduled vCPUs can be handled directly by Linux to provide Type-2 + hypervisor style on-demand paging and/or device emulation. + +Architectures supported +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +AArch64 with a GICv3 or GICv4.1 + +Resources and Capabilities +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D + +Services/resources provided by the Gunyah hypervisor are accessible to a +virtual machine through capabilities. A capability is an access control +token granting the holder a set of permissions to operate on a specific +hypervisor object (conceptually similar to a file-descriptor). +For example, inter-VM communication using Gunyah doorbells and message que= ues +is performed using hypercalls taking Capability ID arguments for the requi= red +IPC objects. These resources are described in Linux as a struct gunyah_res= ource. + +Unlike UNIX file descriptors, there is no path-based or similar lookup of +an object to create a new Capability, meaning simpler security analysis. +Creation of a new Capability requires the holding of a set of privileged +Capabilities which are typically never given out by the Resource Manager (= RM). + +Gunyah itself provides no APIs for Capability ID discovery. Enumeration of +Capability IDs is provided by RM as a higher level service to VMs. + +Resource Manager +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +The Gunyah Resource Manager (RM) is a privileged application VM supporting= the +Gunyah Hypervisor. It provides policy enforcement aspects of the virtualiz= ation +system. The resource manager can be treated as an extension of the Hypervi= sor +but is separated to its own partition to ensure that the hypervisor layer = itself +remains small and secure and to maintain a separation of policy and mechan= ism in +the platform. The resource manager runs at arm64 NS-EL1, similar to other +virtual machines. + +Communication with the resource manager from other virtual machines happen= s as +described in message-queue.rst. Details about the specific messages can be= found +in drivers/virt/gunyah/rsc_mgr.c + +:: + + +-------+ +--------+ +--------+ + | RM | | VM_A | | VM_B | + +-.-.-.-+ +---.----+ +---.----+ + | | | | + +-.-.-----------.------------.----+ + | | \=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D/ | | + | \=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D/ | + | Gunyah | + +---------------------------------+ + +The source for the resource manager is available at +https://github.com/quic/gunyah-resource-manager. + +The resource manager provides the following features: + +- VM lifecycle management: allocating a VM, starting VMs, destruction of V= Ms +- VM access control policy, including memory sharing and lending +- Interrupt routing configuration +- Forwarding of system-level events (e.g. VM shutdown) to owner VM +- Resource (capability) discovery + +A VM requires boot configuration to establish communication with the resou= rce +manager. This is provided to VMs via a 'hypervisor' device tree node which= is +overlaid to the VMs DT by the RM. This node lets guests know they are runn= ing +as a Gunyah guest VM, how to communicate with resource manager, and basic +description and capabilities of this VM. See +Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml for a +description of this node. diff --git a/Documentation/virt/gunyah/message-queue.rst b/Documentation/vi= rt/gunyah/message-queue.rst new file mode 100644 index 000000000000..96864708f442 --- /dev/null +++ b/Documentation/virt/gunyah/message-queue.rst @@ -0,0 +1,68 @@ +.. SPDX-License-Identifier: GPL-2.0 + +Message Queues +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +Message queue is a simple low-capacity IPC channel between two virtual mac= hines. +It is intended for sending small control and configuration messages. Each +message queue is unidirectional and buffered in the hypervisor. A full-dup= lex +IPC channel requires a pair of queues. + +The size of the queue and the maximum size of the message that can be pass= ed is +fixed at creation of the message queue. Resource manager is presently the = only +use case for message queues, and creates messages queues between itself an= d VMs +with a fixed maximum message size of 240 bytes. Longer messages require a +further protocol on top of the message queue messages themselves. For inst= ance, +communication with the resource manager adds a header field for sending lo= nger +messages which are split into smaller fragments. + +The diagram below shows how message queue works. A typical configuration +involves 2 message queues. Message queue 1 allows VM_A to send messages to= VM_B. +Message queue 2 allows VM_B to send messages to VM_A. + +1. VM_A sends a message of up to 240 bytes in length. It makes a hypercall + with the message to request the hypervisor to add the message to + message queue 1's queue. The hypervisor copies memory into the internal + message queue buffer; the memory doesn't need to be shared between + VM_A and VM_B. + +2. Gunyah raises the corresponding interrupt for VM_B (Rx vIRQ) when any of + these happens: + + a. gunyah_msgq_send() has PUSH flag. This is a typical case when the me= ssage + queue is being used to implement an RPC-like interface. + b. Explicitly with gunyah_msgq_push hypercall from VM_A. + c. Message queue has reached a threshold depth. Typically, this thresho= ld + depth is the size of the queue (in other words: when queue is full, = Rx + vIRQ is raised). + +3. VM_B calls gunyah_msgq_recv() and Gunyah copies message to requested bu= ffer. + +4. Gunyah raises the corresponding interrupt for VM_A (Tx vIRQ) when the m= essage + queue falls below a watermark depth. Typically, this is when the queue = is + drained. Note the watermark depth and the threshold depth for the Rx vI= RQ are + independent values. Coincidentally, this signal is conceptually similar= to + Clear-to-Send. + +For VM_B to send a message to VM_A, the process is identical, except that +hypercalls reference message queue 2's capability ID. The IRQ will be diff= erent +for the second message queue. + +:: + + +-------------------+ +-----------------+ +---------= ----------+ + | VM_A | |Gunyah hypervisor| | V= M_B | + | | | | | = | + | | | | | = | + | | Tx | | | = | + | |-------->| | Rx vIRQ | = | + |gunyah_msgq_send() | Tx vIRQ |Message queue 1 |-------->|gunyah_ms= gq_recv() | + | |<------- | | | = | + | | | | | = | + | | | | | = | + | | | | Tx | = | + | | Rx vIRQ | |<--------| = | + |gunyah_msgq_recv() |<--------|Message queue 2 | Tx vIRQ |gunyah_ms= gq_send() | + | | | |-------->| = | + | | | | | = | + | | | | | = | + +-------------------+ +-----------------+ +---------= ------+ diff --git a/Documentation/virt/index.rst b/Documentation/virt/index.rst index 7fb55ae08598..15869ee059b3 100644 --- a/Documentation/virt/index.rst +++ b/Documentation/virt/index.rst @@ -16,6 +16,7 @@ Virtualization Support coco/sev-guest coco/tdx-guest hyperv/index + gunyah/index =20 .. only:: html and subproject =20 --=20 2.39.5 From nobody Thu Dec 18 00:34:59 2025 Received: from mail-wr1-f48.google.com (mail-wr1-f48.google.com [209.85.221.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 22971281376 for ; Thu, 24 Apr 2025 14:13:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.48 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504041; cv=none; b=n6xB4870dBZbCB0VyfE7WAtso48DVj3StJS4mGw1RU1vrCPEaO20n586fUhDSMk7cBpz2ylvtIyEXSJUjbDa86c8cjmswKA+Le9ptb09Bc5UWiiwxVCLOVlGr9wgmV030obCe++loO91PFRJFjL0O9ruRPgZlP1neIa2HHbRlPQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504041; c=relaxed/simple; bh=nHdCN2XoC/vofWX1JvKbFFkxCi8mZFPmuv+AC6Hf5jo=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=ZQpL8TcJqIiCD2J0ZDBmweEl3El0VPeexRYo2U4B/igGbIACsEKrSLHN8/sjPou/0ab6nIpltQSfupTq/IbpjKXKHcE+/NwzL4Vo40SVOt5Bt2SvWmhAjqJNO/40d62Dxxf5Jh0scsW0uAYu6J0WYsRwWVw1bIJHzmon0lxij9c= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=ntOOgl9n; arc=none smtp.client-ip=209.85.221.48 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="ntOOgl9n" Received: by mail-wr1-f48.google.com with SMTP id ffacd0b85a97d-39d83782ef6so1647984f8f.0 for ; Thu, 24 Apr 2025 07:13:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504037; x=1746108837; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=3UEu5ZzgQoIbj2nEACS31q52k6ad3AHbKAZTQ1reR+A=; b=ntOOgl9n/47oroZlPn6W85vBTzu5j6rwgBhEPREb4qe+N2tqnT4YnQz2DuoRLWKqKE OBXxavFaZDwHbMUBy6/cWhwHQ5WvLaNgJ25npQJjfp/ft17XqPt9NoiwflfuD8Zh8kVp rlKDeiU34oH6MboBQ2L0v6yiz5rpUdrX6d4/L7lsJxBJwjw3Go1zmnt2zDqnPO4bE+Ja 3rFMR1TjmrEwKb7z5vPaZ4/B9B8CB5b++pxgnG7+tJ1rSk38I32tp19nAfOLM8peApt2 krnIwGIp5rlXpsAmiWONQf5GB/0WDGINuZKsk39kuKePSkTXs789DL2q9Y66Sl6J2IQW WiEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504037; x=1746108837; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3UEu5ZzgQoIbj2nEACS31q52k6ad3AHbKAZTQ1reR+A=; b=rF/MKb7eZCd2F9YqfrVMPRJHXPCpW9aJJpBb2AUJciTnlyRseXvPQm/xoTjnhUX9XR khhLd92UeCDdjPKmsrVNHVpO+xdTGcznzN8hfnCaZ78GrnUCyVAg9307gPT9Eqa8whmm auTN3jfKLVNYMDU6JB4mSX04BS0iQMDzwckaky4kJKdGcCexb1PjMDvNK0sYgt+513K5 cG1R5Y3ndulSj83MSuWOGgiNzk/ej9XsvfmpEjBCuGhEVU1Sq6ViwOsvDXcAkGuhU7tV Tvy3MWf0yIsD0395QRFHGvhQDTByX5JbitprdhEaMHJLzx01AAcjqbUytBjA3IAIwz4p vljg== X-Gm-Message-State: AOJu0YzzlGX1Jo5953ia3Gd/wYI/vP7TIZXu/+yofIh/vtmNwSwRz7xd RpQnXyj0fksaETrysXSfP7cFPfdOrbDEjPOKqzaWwXijX9k02oAkY6CNUPMsQxUgQuDwaz/puhA mtw0= X-Gm-Gg: ASbGnctXnMqGmQR0Toj/nwW62MbykD+thVEqk9ALov6XzRzAu4e60Khx5QJRH6Kjggh HO3UNGJvYBBAKC6tSUnJ+KXAPs15GJhzqXNdNclJlp6Sm46VPPx6ylVfo7S2J4Q6MPd1cZW5oSY XfR1SQWmJVWTQ5Oudvoh1ETn8FwviamJG1+rI2OkXM4cuxIL9s96PgRWCYurm4BcBtaPEep6W0h XnEU2iChc28nRW2+d+DQhxWd/tNC/zXiLiI5YkCd98zW6VIxWE6QnLsBViOAccB8I9uAJc7QkKu yur+u7aK5+I2qgvJeN6SLAm+7vLxP7VE4k3aFpheucLJ0F/F8fXDe3xNuUCmpbbyIX+4cyszI9o XMBhBK9Mta+PBOEe7 X-Google-Smtp-Source: AGHT+IGbR1o2FMJriTxMh0pqGyENTmZYeFEOhlcuDYlWi636WYjhDM2clZgLdfLT+7JHm7l12d/JYg== X-Received: by 2002:a05:6000:22c4:b0:3a0:3d18:285 with SMTP id ffacd0b85a97d-3a06d6dd54emr2218203f8f.25.1745504036794; Thu, 24 Apr 2025 07:13:56 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.13.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:13:56 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt , Elliot Berman Subject: [RFC PATCH 09/34] dt-bindings: Add binding for gunyah hypervisor Date: Thu, 24 Apr 2025 15:13:16 +0100 Message-Id: <20250424141341.841734-10-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Elliot Berman The Gunyah Resource Manager applies a devicetree overlay describing the virtual platform configuration of the guest VM, such as the message queue capability IDs for communicating with the Resource Manager. This information is not otherwise discoverable by a VM: the Gunyah hypervisor core does not provide a direct interface to discover capability IDs nor a way to communicate with RM without having already known the corresponding message queue capability ID. Add the DT bindings that Gunyah adheres for the hypervisor node and message queues. Reviewed-by: Rob Herring Signed-off-by: Elliot Berman Signed-off-by: Karim Manaouil --- .../bindings/firmware/gunyah-hypervisor.yaml | 82 +++++++++++++++++++ 1 file changed, 82 insertions(+) create mode 100644 Documentation/devicetree/bindings/firmware/gunyah-hyper= visor.yaml diff --git a/Documentation/devicetree/bindings/firmware/gunyah-hypervisor.y= aml b/Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml new file mode 100644 index 000000000000..cdeb4885a807 --- /dev/null +++ b/Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml @@ -0,0 +1,82 @@ +# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause) +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/firmware/gunyah-hypervisor.yaml# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: Gunyah Hypervisor + +maintainers: + - Prakruthi Deepak Heragu + - Elliot Berman + +description: |+ + Gunyah virtual machines use this information to determine the capability= IDs + of the message queues used to communicate with the Gunyah Resource Manag= er. + See also: https://github.com/quic/gunyah-resource-manager/blob/develop/s= rc/vm_creation/dto_construct.c + +properties: + compatible: + const: gunyah-hypervisor + + "#address-cells": + description: Number of cells needed to represent 64-bit capability IDs. + const: 2 + + "#size-cells": + description: must be 0, because capability IDs are not memory address + ranges and do not have a size. + const: 0 + +patternProperties: + "^gunyah-resource-mgr(@.*)?": + type: object + description: + Resource Manager node which is required to communicate to Resource + Manager VM using Gunyah Message Queues. + + properties: + compatible: + const: gunyah-resource-manager + + reg: + items: + - description: Gunyah capability ID of the TX message queue + - description: Gunyah capability ID of the RX message queue + + interrupts: + items: + - description: Interrupt for the TX message queue + - description: Interrupt for the RX message queue + + additionalProperties: false + + required: + - compatible + - reg + - interrupts + +additionalProperties: false + +required: + - compatible + - "#address-cells" + - "#size-cells" + +examples: + - | + #include + + hypervisor { + #address-cells =3D <2>; + #size-cells =3D <0>; + compatible =3D "gunyah-hypervisor"; + + gunyah-resource-mgr@0 { + compatible =3D "gunyah-resource-manager"; + interrupts =3D , /* TX allowed= IRQ */ + ; /* RX requested= IRQ */ + reg =3D <0x00000000 0x00000000>, /* TX capability ID */ + <0x00000000 0x00000001>; /* RX capability ID */ + }; + }; --=20 2.39.5 From nobody Thu Dec 18 00:34:59 2025 Received: from mail-wr1-f49.google.com (mail-wr1-f49.google.com [209.85.221.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B4CD52820D8 for ; Thu, 24 Apr 2025 14:14:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.49 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504043; cv=none; b=MdMd6QtGjicLQHqYXJh/pNNGzisC8azVNvt1+x9dgmFvPD/IDItxDuyuH1wq2vHlWaGv6ipBzitym6oNlLd8TPbGUVM/eJz72HmR/ooOgYRjmH6U1rLh+v1lj+OIO+9eR/tCUlbF4upqWB4fBIY1JeDqVF02t+ySHpqf9Os/mvk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504043; c=relaxed/simple; bh=zpH5RWwn66pK/eMc3LSMNEkV/3yaevWoUA1/kvmrIcs=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=PdJP7R7IFpp77lSWIdYhnPZ+Z7HnsTfQisAHI44nIxOTzRM5ej2ryKAGXRgH2Q3BGFk3h2FpAZiJSC01BJ6aLd2pIUC7C9Z2VmC2enyzX/U/FHbvFpXvo75zQNKxJnsuRyT+o+mK2SgxkEaA+45hsqj85HcsbAv+mgWw2iAxfck= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=j9+iKa4W; arc=none smtp.client-ip=209.85.221.49 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="j9+iKa4W" Received: by mail-wr1-f49.google.com with SMTP id ffacd0b85a97d-39141ffa9fcso1250939f8f.0 for ; Thu, 24 Apr 2025 07:14:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504038; x=1746108838; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=RusfJk/Wwgs+hfa3WlHRZBdqt7Zg7qYVP8LFIn+weQg=; b=j9+iKa4WCT/mz4U6jOuEjxqBAIM17fNBuPra7vwuEWL0QXoKe6DkDeChGdfSfN2jqZ uCaRbiExMKGx6vkJmOf6uDYYMezLzUdvO3PzNablg6nTHG7sKC5ygO7dxRWP12DQb/Lo ncNWWVX00LNc9y6vT9WTBjPdHkjhCEe1ppVdr+2tyhjPTJf7tIR0lHWwMgINoRHhtrzz KBZh7idT029V7GaVEdLIlUlagIvjMV72pL5JYshgff1oMFthqajbTK4j6rmb1Xy/vP62 O6qovaWNGZ14gRNMMC0hn7jK7GW9RnF132Bax0dN4BK/6ziNhLy/woVUD4piL85of4bM P50g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504038; x=1746108838; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=RusfJk/Wwgs+hfa3WlHRZBdqt7Zg7qYVP8LFIn+weQg=; b=v6p+HV8AgzuXeHcYRY90rMqtbxJMC2w8x2OesOCLaM4VxZBNhVDBEbwakg/IQtSFQN 0c41evVfVdAPgS79HFlotDKaYFc+SfkuNl4iy7bs3qegS2hgrNMHMsLtWA9D4FmvrQgp MEIfmossjhbfECpvQJF9XGmvyT1e7guT/7RmZahP9ii6lC5/sJcZxL3Pr2UUFLQExJa/ Dfx18P3228X2jhS7RYMctIL7eWUevjCEiIRMDhKDW7FDlm391LCmgPjsH/W47rJrNcbt ZRudV5tsh3yljMurdiawtJVUL+8MhuFb9X7R0aAVROq4Zt3kk4YtoT4ngyMamQdoTANm 59RQ== X-Gm-Message-State: AOJu0YzVbywbo95DjqPd0LSKM5F9+fXkOHhrLlrL7heuusI6B3VnzHV2 DNDIlhXY7Ifv7N1lz0xZXMtLeyXqahqsbiBG2GZCNnGqGR1YRlJroMsrEpqN2gjwkHpJQJ6cKZN SjvQ= X-Gm-Gg: ASbGncvrk6f5V+FgqW7we4BC6q9/7aMG6hqtaCspgo9Wr1XBppi19G+ycQTHPMtCrwH 38C29mYiu9D7UbE3r2M9If9tgq99aZILFeRCcIIeSFbVw6RzXheOzWR6nmwMFKI7Th98RF38bg3 rUyU086MkdswAuJO0IEmkYQQiM+0UjNUP8LEQiTxwPXAMqvLth9/jLxBx3BW02l//C6eWylU+4c /jn6VZY8+lsZ17fPoxB2h7AQnE9EUwHM/hc7A6YetvspCf9j1MIExlAc60NhEr/zPbrCTvsTk9S JPdeQB0Z4dL3mnGT+cv+ZvMRJ0kLVTTqfn4DcjFn6ErOtKtmXOEiVU1kAJ0iObKsVX08+76LECT wsut1a+p7YaA/9ccCJbjYAtyCYZ0= X-Google-Smtp-Source: AGHT+IFmkqrLYBA6pxT3bgHP40xSeHaZYwIrDUExxYU5PjMxiNNE1p1s+eiHaAutjdes9ZQldE7V0g== X-Received: by 2002:a05:6000:188d:b0:39e:dbee:f644 with SMTP id ffacd0b85a97d-3a06cfa5a93mr2715557f8f.46.1745504038249; Thu, 24 Apr 2025 07:13:58 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.13.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:13:57 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt , Elliot Berman , Dmitry Baryshkov , Srinivas Kandagatla , Alex Elder Subject: [RFC PATCH 10/34] gunyah: Common types and error codes for Gunyah hypercalls Date: Thu, 24 Apr 2025 15:13:17 +0100 Message-Id: <20250424141341.841734-11-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Elliot Berman Add architecture-independent standard error codes, types, and macros for Gunyah hypercalls. Reviewed-by: Dmitry Baryshkov Reviewed-by: Srinivas Kandagatla Reviewed-by: Alex Elder Signed-off-by: Elliot Berman Signed-off-by: Karim Manaouil --- include/linux/gunyah.h | 106 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 106 insertions(+) create mode 100644 include/linux/gunyah.h diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h new file mode 100644 index 000000000000..1eab631a49b6 --- /dev/null +++ b/include/linux/gunyah.h @@ -0,0 +1,106 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights res= erved. + */ + +#ifndef _LINUX_GUNYAH_H +#define _LINUX_GUNYAH_H + +#include +#include +#include + +/* Matches resource manager's resource types for VM_GET_HYP_RESOURCES RPC = */ +enum gunyah_resource_type { + /* clang-format off */ + GUNYAH_RESOURCE_TYPE_BELL_TX =3D 0, + GUNYAH_RESOURCE_TYPE_BELL_RX =3D 1, + GUNYAH_RESOURCE_TYPE_MSGQ_TX =3D 2, + GUNYAH_RESOURCE_TYPE_MSGQ_RX =3D 3, + GUNYAH_RESOURCE_TYPE_VCPU =3D 4, + GUNYAH_RESOURCE_TYPE_MEM_EXTENT =3D 9, + GUNYAH_RESOURCE_TYPE_ADDR_SPACE =3D 10, + /* clang-format on */ +}; + +struct gunyah_resource { + enum gunyah_resource_type type; + u64 capid; + unsigned int irq; +}; + +/*************************************************************************= *****/ +/* Common arch-independent definitions for Gunyah hypercalls = */ +#define GUNYAH_CAPID_INVAL U64_MAX +#define GUNYAH_VMID_ROOT_VM 0xff + +enum gunyah_error { + /* clang-format off */ + GUNYAH_ERROR_OK =3D 0, + GUNYAH_ERROR_UNIMPLEMENTED =3D -1, + GUNYAH_ERROR_RETRY =3D -2, + + GUNYAH_ERROR_ARG_INVAL =3D 1, + GUNYAH_ERROR_ARG_SIZE =3D 2, + GUNYAH_ERROR_ARG_ALIGN =3D 3, + + GUNYAH_ERROR_NOMEM =3D 10, + + GUNYAH_ERROR_ADDR_OVFL =3D 20, + GUNYAH_ERROR_ADDR_UNFL =3D 21, + GUNYAH_ERROR_ADDR_INVAL =3D 22, + + GUNYAH_ERROR_DENIED =3D 30, + GUNYAH_ERROR_BUSY =3D 31, + GUNYAH_ERROR_IDLE =3D 32, + + GUNYAH_ERROR_IRQ_BOUND =3D 40, + GUNYAH_ERROR_IRQ_UNBOUND =3D 41, + + GUNYAH_ERROR_CSPACE_CAP_NULL =3D 50, + GUNYAH_ERROR_CSPACE_CAP_REVOKED =3D 51, + GUNYAH_ERROR_CSPACE_WRONG_OBJ_TYPE =3D 52, + GUNYAH_ERROR_CSPACE_INSUF_RIGHTS =3D 53, + GUNYAH_ERROR_CSPACE_FULL =3D 54, + + GUNYAH_ERROR_MSGQUEUE_EMPTY =3D 60, + GUNYAH_ERROR_MSGQUEUE_FULL =3D 61, + /* clang-format on */ +}; + +/** + * gunyah_error_remap() - Remap Gunyah hypervisor errors into a Linux erro= r code + * @gunyah_error: Gunyah hypercall return value + */ +static inline int gunyah_error_remap(enum gunyah_error gunyah_error) +{ + switch (gunyah_error) { + case GUNYAH_ERROR_OK: + return 0; + case GUNYAH_ERROR_NOMEM: + return -ENOMEM; + case GUNYAH_ERROR_DENIED: + case GUNYAH_ERROR_CSPACE_CAP_NULL: + case GUNYAH_ERROR_CSPACE_CAP_REVOKED: + case GUNYAH_ERROR_CSPACE_WRONG_OBJ_TYPE: + case GUNYAH_ERROR_CSPACE_INSUF_RIGHTS: + return -EACCES; + case GUNYAH_ERROR_CSPACE_FULL: + case GUNYAH_ERROR_BUSY: + case GUNYAH_ERROR_IDLE: + return -EBUSY; + case GUNYAH_ERROR_IRQ_BOUND: + case GUNYAH_ERROR_IRQ_UNBOUND: + case GUNYAH_ERROR_MSGQUEUE_FULL: + case GUNYAH_ERROR_MSGQUEUE_EMPTY: + return -EIO; + case GUNYAH_ERROR_UNIMPLEMENTED: + return -EOPNOTSUPP; + case GUNYAH_ERROR_RETRY: + return -EAGAIN; + default: + return -EINVAL; + } +} + +#endif --=20 2.39.5 From nobody Thu Dec 18 00:34:59 2025 Received: from mail-wr1-f44.google.com (mail-wr1-f44.google.com [209.85.221.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CE2D827FD6A for ; Thu, 24 Apr 2025 14:14:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.44 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504044; cv=none; b=e+4k2dP9Eh4UAO0NP3EvNa5MzaceKKoaIVDEkmc63IWVUIr9XkRqC/Four3Oov6T4DwWE2LogsdYejU+uYcYor+x6kAgvtHobI4Z5GWQjHkZXAMeFmEgUDssIZAhgNt3XKeSBKxGWtz5WOwO/o2Qb7hE9T7YxrWvJdz/XY0yOEo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504044; c=relaxed/simple; bh=sW44/p8fc9q82gGRUYbPMkfXFBjgbu9rU1+e99sjorc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=ER/Ij5/sTnDMaAnAELyIjeUke/3VyFmUlNN0lAOiC0TQZyFCsPD7xl3NC6uu37eaOILqjAzFj7aggCieducdcn7SENEYxPtUt/DuYtPOWj9FOIVQblDVKXYkRYhQxASsHQ6vGiByWAUo56xkeIaLNOjcQVCazXgCavclYNzvvjQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=qfStsnQM; arc=none smtp.client-ip=209.85.221.44 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="qfStsnQM" Received: by mail-wr1-f44.google.com with SMTP id ffacd0b85a97d-39ee5ac4321so1168527f8f.1 for ; Thu, 24 Apr 2025 07:14:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504040; x=1746108840; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=hnFOyyVscJj2zQjUVyQj//yqO74ALl7OUDNbU7iCFM0=; b=qfStsnQMn2V6Gkyza+2neBnGTryb06KyiNTsIkI4+wXFmPDOSW7W7vx20DwQvLwbKL MAGsoZPCYp4+XWPKiNC2sYM+V3wbJ/ruiWAHn5ImY61QgFJT8D0aLHhaFTdeem+3xlIj /Ztr/j4cAQOj68cMsA1UXOX+PUIr0MCQqNDJ5Pi14nB/ltSND35KsRchGPiZG7moxYRf 7KOKiXjB3sO7E4AKmSMyszIq9mZOmv7mdM4mLAIk0dBcXf/1k8WUPzvTqsz++p73EL5k OUnzS1reM7AjX55Rd478N8i2ttIRFmcjyZR6d6P2Sft4zq4h+DapfkoqkgMH2B2nnCtt voYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504040; x=1746108840; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hnFOyyVscJj2zQjUVyQj//yqO74ALl7OUDNbU7iCFM0=; b=aTHPkOlcyVYQ7s2Ed3/ky84zTpf4wbj/pubPx8yhYmv8NfQtFlRsW7asee4xmdG6RM GFSZz/5w4howiwXhkPDwB4awYNiyye7ZZ5hNy9ZnZhZAtrcNZYYs+55azgbxI2TFtew8 Y6s2OM5GInd1vXV6VnkNqxpXF1HpKPZOMSlFsnC3f9J95OE/zeWwfSnC6rxHN0DCqN3Z Vhlb1WTnNnY/2hQz6KobsDz40hbsN+Q5qqJylPqjti52cn7xTBcHBkKCJVVHGCNaPIFo kvTUyrnvvWy4iHtRB8oFH55VpDY1QFJ47bLELuRa7+nTW93b4DP70HkdVD8IxZ1SYkts OJ5A== X-Gm-Message-State: AOJu0YynW5mE46outjyutA06/knxrMiHL5CdbuKM9p9xBU5dqmmpYqL+ X+xSLgHqi8jlzUCQ/XZx6PG1zMgOwDFrJna5MdswO7S/dhxXYUrgUKtDtIMxraJeRdyqEmbEeck MJao= X-Gm-Gg: ASbGncuYA9XdVabItaf8/8VPlXZ2Blj61+u8sOMaTUUftNW2BSAjVBMTSn1zQuhPXfW DaI6rG+47vZ4rxRUdP5y6sLCrAY5WzUUA916StOhzPzXl8w7LzbAw25gP4mvYKLtot7GQiMieET CB0jZg14k/5hDX7+ZxSwFD+NROqsHvEngBiS17oe7iWX7XVGNXzODr8JAEV79T0TF93Ht0M9b3W CKqh6NQTHAnuC5Pp1Ijs/GN/P90AlNdzom6/OMB904NZQCpONPg1ItFAUQWz2jOtt31yC25w8U1 t+1MlMXblsMEV7jA9BdyOZWEwZotGwaTBHUij2ofRNNIsAwUMyAhARWBBl1h2cKRsW1Vj/6rLWx xik7vQVtjM6G0ebnq X-Google-Smtp-Source: AGHT+IFe20I1qXpRONr6or0FTVbxh/3TZQo8fNSyYvn8TOfhWs9pYDOoLZ+ij1siBFtRALlOMdvrUQ== X-Received: by 2002:a5d:648c:0:b0:39c:2669:d7f4 with SMTP id ffacd0b85a97d-3a06cfa3406mr2184828f8f.43.1745504039514; Thu, 24 Apr 2025 07:13:59 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.13.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:13:59 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt , Elliot Berman , Srinivas Kandagatla , Alex Elder Subject: [RFC PATCH 11/34] gunyah: Add hypercalls to identify Gunyah Date: Thu, 24 Apr 2025 15:13:18 +0100 Message-Id: <20250424141341.841734-12-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Elliot Berman Add hypercalls to identify when Linux is running in a virtual machine under Gunyah. There are two calls to help identify Gunyah: 1. gh_hypercall_get_uid() returns a UID when running under a Gunyah hypervisor. 2. gh_hypercall_hyp_identify() returns build information and a set of feature flags that are supported by Gunyah. Reviewed-by: Srinivas Kandagatla Reviewed-by: Alex Elder Signed-off-by: Elliot Berman Signed-off-by: Karim Manaouil --- arch/arm64/Kbuild | 1 + arch/arm64/gunyah/Makefile | 3 ++ arch/arm64/gunyah/gunyah_hypercall.c | 62 ++++++++++++++++++++++++++++ drivers/virt/Kconfig | 2 + drivers/virt/gunyah/Kconfig | 12 ++++++ include/linux/gunyah.h | 38 +++++++++++++++++ 6 files changed, 118 insertions(+) create mode 100644 arch/arm64/gunyah/Makefile create mode 100644 arch/arm64/gunyah/gunyah_hypercall.c create mode 100644 drivers/virt/gunyah/Kconfig diff --git a/arch/arm64/Kbuild b/arch/arm64/Kbuild index 5bfbf7d79c99..e4847ba0e3c9 100644 --- a/arch/arm64/Kbuild +++ b/arch/arm64/Kbuild @@ -3,6 +3,7 @@ obj-y +=3D kernel/ mm/ net/ obj-$(CONFIG_KVM) +=3D kvm/ obj-$(CONFIG_XEN) +=3D xen/ obj-$(subst m,y,$(CONFIG_HYPERV)) +=3D hyperv/ +obj-$(CONFIG_GUNYAH) +=3D gunyah/ obj-$(CONFIG_CRYPTO) +=3D crypto/ =20 # for cleaning diff --git a/arch/arm64/gunyah/Makefile b/arch/arm64/gunyah/Makefile new file mode 100644 index 000000000000..84f1e38cafb1 --- /dev/null +++ b/arch/arm64/gunyah/Makefile @@ -0,0 +1,3 @@ +# SPDX-License-Identifier: GPL-2.0 + +obj-$(CONFIG_GUNYAH) +=3D gunyah_hypercall.o diff --git a/arch/arm64/gunyah/gunyah_hypercall.c b/arch/arm64/gunyah/gunya= h_hypercall.c new file mode 100644 index 000000000000..d44663334f38 --- /dev/null +++ b/arch/arm64/gunyah/gunyah_hypercall.c @@ -0,0 +1,62 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights res= erved. + */ + +#include +#include +#include +#include + +/* {c1d58fcd-a453-5fdb-9265-ce36673d5f14} */ +static const uuid_t GUNYAH_UUID =3D UUID_INIT(0xc1d58fcd, 0xa453, 0x5fdb, = 0x92, + 0x65, 0xce, 0x36, 0x67, 0x3d, 0x5f, + 0x14); + +bool arch_is_gunyah_guest(void) +{ + struct arm_smccc_res res; + uuid_t uuid; + u32 *up; + + arm_smccc_1_1_invoke(ARM_SMCCC_VENDOR_HYP_CALL_UID_FUNC_ID, &res); + + up =3D (u32 *)&uuid.b[0]; + up[0] =3D lower_32_bits(res.a0); + up[1] =3D lower_32_bits(res.a1); + up[2] =3D lower_32_bits(res.a2); + up[3] =3D lower_32_bits(res.a3); + + return uuid_equal(&uuid, &GUNYAH_UUID); +} +EXPORT_SYMBOL_GPL(arch_is_gunyah_guest); + +#define GUNYAH_HYPERCALL(fn) \ + ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, ARM_SMCCC_SMC_64, \ + ARM_SMCCC_OWNER_VENDOR_HYP, fn) + +/* clang-format off */ +#define GUNYAH_HYPERCALL_HYP_IDENTIFY GUNYAH_HYPERCALL(0x8000) +/* clang-format on */ + +/** + * gunyah_hypercall_hyp_identify() - Returns build information and feature= flags + * supported by Gunyah. + * @hyp_identity: filled by the hypercall with the API info and feature fl= ags. + */ +void gunyah_hypercall_hyp_identify( + struct gunyah_hypercall_hyp_identify_resp *hyp_identity) +{ + struct arm_smccc_res res; + + arm_smccc_1_1_hvc(GUNYAH_HYPERCALL_HYP_IDENTIFY, &res); + + hyp_identity->api_info =3D res.a0; + hyp_identity->flags[0] =3D res.a1; + hyp_identity->flags[1] =3D res.a2; + hyp_identity->flags[2] =3D res.a3; +} +EXPORT_SYMBOL_GPL(gunyah_hypercall_hyp_identify); + +MODULE_LICENSE("GPL"); +MODULE_DESCRIPTION("Gunyah Hypervisor Hypercalls"); diff --git a/drivers/virt/Kconfig b/drivers/virt/Kconfig index d8c848cf09a6..5a6781405a60 100644 --- a/drivers/virt/Kconfig +++ b/drivers/virt/Kconfig @@ -49,4 +49,6 @@ source "drivers/virt/acrn/Kconfig" =20 source "drivers/virt/coco/Kconfig" =20 +source "drivers/virt/gunyah/Kconfig" + endif diff --git a/drivers/virt/gunyah/Kconfig b/drivers/virt/gunyah/Kconfig new file mode 100644 index 000000000000..6f4c85db80b5 --- /dev/null +++ b/drivers/virt/gunyah/Kconfig @@ -0,0 +1,12 @@ +# SPDX-License-Identifier: GPL-2.0-only + +config GUNYAH + tristate "Gunyah Virtualization drivers" + depends on ARM64 + help + The Gunyah drivers are the helper interfaces that run in a guest VM + such as basic inter-VM IPC and signaling mechanisms, and higher level + services such as memory/device sharing, IRQ sharing, and so on. + + Say Y/M here to enable the drivers needed to interact in a Gunyah + virtual environment. diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h index 1eab631a49b6..33bcbd22d39f 100644 --- a/include/linux/gunyah.h +++ b/include/linux/gunyah.h @@ -6,9 +6,11 @@ #ifndef _LINUX_GUNYAH_H #define _LINUX_GUNYAH_H =20 +#include #include #include #include +#include =20 /* Matches resource manager's resource types for VM_GET_HYP_RESOURCES RPC = */ enum gunyah_resource_type { @@ -103,4 +105,40 @@ static inline int gunyah_error_remap(enum gunyah_error= gunyah_error) } } =20 +enum gunyah_api_feature { + /* clang-format off */ + GUNYAH_FEATURE_DOORBELL =3D 1, + GUNYAH_FEATURE_MSGQUEUE =3D 2, + GUNYAH_FEATURE_VCPU =3D 5, + GUNYAH_FEATURE_MEMEXTENT =3D 6, + /* clang-format on */ +}; + +bool arch_is_gunyah_guest(void); + +#define GUNYAH_API_V1 1 + +/* Other bits reserved for future use and will be zero */ +/* clang-format off */ +#define GUNYAH_API_INFO_API_VERSION_MASK GENMASK_ULL(13, 0) +#define GUNYAH_API_INFO_BIG_ENDIAN BIT_ULL(14) +#define GUNYAH_API_INFO_IS_64BIT BIT_ULL(15) +#define GUNYAH_API_INFO_VARIANT_MASK GENMASK_ULL(63, 56) +/* clang-format on */ + +struct gunyah_hypercall_hyp_identify_resp { + u64 api_info; + u64 flags[3]; +}; + +static inline u16 +gunyah_api_version(const struct gunyah_hypercall_hyp_identify_resp *gunyah= _api) +{ + return FIELD_GET(GUNYAH_API_INFO_API_VERSION_MASK, + gunyah_api->api_info); +} + +void gunyah_hypercall_hyp_identify( + struct gunyah_hypercall_hyp_identify_resp *hyp_identity); + #endif --=20 2.39.5 From nobody Thu Dec 18 00:34:59 2025 Received: from mail-wm1-f42.google.com (mail-wm1-f42.google.com [209.85.128.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6C9B028469B for ; Thu, 24 Apr 2025 14:14:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.42 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504046; cv=none; b=P9ihOE3wwpQT9bDuEwiCzkZgpJpftSDSFmq+edPRoMhV8y5toYCHvO84wxOTb7CYi34nf1zUX/2BZB69ef53cjMaVE7zQQv+OvY4IukuevQdDbyPu4DQ2ilLMjReuPVCWgJAd8hYw10q+rzi0h3chVug03UuRBhUm8juj+dRlEU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504046; c=relaxed/simple; bh=sOc+WgCBKAU0O5bB7WH2YBqY9uNwSXL1/wb8/ZYbbhE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=KTF/dDsFepA2Vt1KYg/iUshxH/MATonU05KwNbzQowCopor9WPbTrtXyW3VQZf9sgMnZNCLuVKFvEPlZQRztV68HKwZcE901wffMUCiF+Tc+vgM5P81P2hFyu90WWLA44162qSF5gSYTT/vTPbfpGh7FsWXbAz7vUk/BxM/6pXs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=aH33Dmtg; arc=none smtp.client-ip=209.85.128.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="aH33Dmtg" Received: by mail-wm1-f42.google.com with SMTP id 5b1f17b1804b1-43ed8d32a95so9461505e9.3 for ; Thu, 24 Apr 2025 07:14:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504041; x=1746108841; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=OmKgb2P7k7ka3054x1IngydlF8noVj1FtGkMjW5J68k=; b=aH33DmtgkOHqP2hwtI+u07WChhA2Fulbri6fmZWxNnrjT9saYqe/QqBtRtVXhmy4mf Z5EXAZ1d8kblBZtjPBBu6of5eGkEaIx6RxIM9ZAoY2QreLBNLypaWtkE8SgCw+Wwy+Lq beDcf6VKK7dfLO+SfEVGqoj1kMDTW32GRytELD1r9HRvEyNagr0FTcQEYNenN8TqHsUG LFjBtao9OX7HlFxQGZdHxETlqI+oCLtPuZARLhzLK1ZCNKjga90rDDb1brJbUzGiqGbm T7SnEpgHWa6qYBz/iWUCC88dfhzIHxczzTlYdnXNwhGuJRDXB2Kp3M9orQuiHaRDkKCl QaZg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504041; x=1746108841; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=OmKgb2P7k7ka3054x1IngydlF8noVj1FtGkMjW5J68k=; b=EsUBRX2HTwKoSPfrETii5Ju7ugytofDAMnVAMC+LeRE8cfbsZsYYG/TQni5BK7l81B L9or4D2iZ7woyNzqWnuW1c4a/WupyTZ4oPN7JCaQ7zVIa/XOIROsxbv+I4pZLAht4RVS IqHMxUekPMltk7958xv7gLep/YtQR7YqMPcz4WZ7AHkqm9iyFmt/6nB6I2FrGrS17pSM 96d8XDR+PbQmIvymelj0FxmtefJQhGHCdnFfAlM4mDT9Qm7SH8IYnmny2H1BMALHk1Ss bFB1mqzSU2W5PJehwKderBvug1np+T4XkavnfTgELIAzThMAaVG9OuahgDPN1mYEzITo KzAw== X-Gm-Message-State: AOJu0YxPqkbN+Olv0PFNHP2TWiPTDVDhUjK2zVNb03rf9XCp+YBZvIqf FUvSv37jb1wTtROHkYZ72BD0jKUZrP0d5KJ9DG6B5fftYc8bs1xAujIKNiHUJP5lW7Fi236nO4o N X-Gm-Gg: ASbGncvfpKMt+9TehreELtN4wTfsk5HCNvoSWE78kIUDbn+7JUSu8qGVe9gY26SgfDF RxSHXoJDedB4mg4ez3QN0ahDQat6/5UHjEyowLCKiSnFbwmQhh71IFZ0TfGVkZIQFqjAho67Lx6 cxpWwTYxA5ZVte7WXLOX/Kx1afK+MjUvbrEA29b/CG1zo6Oap/wsrr+YO3QL4cnIxm+QY8ItXTU exlvd28Y16nVF5taAkfGZd/nE/LiaPEuZAWWO/IhVJ8mK+UZxAbQWaFpX8wF4GXClTBTz90Ftt/ HRIXA2vUweNcikmTvJlsWRwmlBd1iogY67kJoUN+id6gwgNmtCETPTp4SgqJNy14exLfUKD5car I5ftThFAIVN9dlRph1lz7NIxWocs= X-Google-Smtp-Source: AGHT+IGfkZ98OulZ0pJYU5c1ld+aBqoKjFuSpgzRjo7Jvo/4HTkJCAjHkRJEje6F6WXTnu8CxQELEw== X-Received: by 2002:a05:600c:3b0c:b0:43c:e8ba:e166 with SMTP id 5b1f17b1804b1-4409bd83db4mr21261755e9.22.1745504040937; Thu, 24 Apr 2025 07:14:00 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.13.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:14:00 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt , Elliot Berman Subject: [RFC PATCH 12/34] gunyah: Add hypervisor driver Date: Thu, 24 Apr 2025 15:13:19 +0100 Message-Id: <20250424141341.841734-13-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Elliot Berman Add driver to detect when running under Gunyah. It performs basic identification hypercall and populates the platform bus for resource manager to probe. Signed-off-by: Elliot Berman Reviewed-by: Srivatsa Vaddagiri Signed-off-by: Karim Manaouil --- drivers/virt/Makefile | 1 + drivers/virt/gunyah/Makefile | 3 +++ drivers/virt/gunyah/gunyah.c | 52 ++++++++++++++++++++++++++++++++++++ 3 files changed, 56 insertions(+) create mode 100644 drivers/virt/gunyah/Makefile create mode 100644 drivers/virt/gunyah/gunyah.c diff --git a/drivers/virt/Makefile b/drivers/virt/Makefile index f29901bd7820..ef6a3835d078 100644 --- a/drivers/virt/Makefile +++ b/drivers/virt/Makefile @@ -10,3 +10,4 @@ obj-y +=3D vboxguest/ obj-$(CONFIG_NITRO_ENCLAVES) +=3D nitro_enclaves/ obj-$(CONFIG_ACRN_HSM) +=3D acrn/ obj-y +=3D coco/ +obj-y +=3D gunyah/ diff --git a/drivers/virt/gunyah/Makefile b/drivers/virt/gunyah/Makefile new file mode 100644 index 000000000000..34f32110faf9 --- /dev/null +++ b/drivers/virt/gunyah/Makefile @@ -0,0 +1,3 @@ +# SPDX-License-Identifier: GPL-2.0 + +obj-$(CONFIG_GUNYAH) +=3D gunyah.o diff --git a/drivers/virt/gunyah/gunyah.c b/drivers/virt/gunyah/gunyah.c new file mode 100644 index 000000000000..3e795e3ba881 --- /dev/null +++ b/drivers/virt/gunyah/gunyah.c @@ -0,0 +1,52 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2023-2024 Qualcomm Innovation Center, Inc. All rights res= erved. + */ + +#include +#include +#include +#include + +static int gunyah_probe(struct platform_device *pdev) +{ + struct gunyah_hypercall_hyp_identify_resp gunyah_api; + + if (!arch_is_gunyah_guest()) + return -ENODEV; + + gunyah_hypercall_hyp_identify(&gunyah_api); + + pr_info("Running under Gunyah hypervisor %llx/v%u\n", + FIELD_GET(GUNYAH_API_INFO_VARIANT_MASK, gunyah_api.api_info), + gunyah_api_version(&gunyah_api)); + + /* Might move this out to individual drivers if there's ever an API versi= on bump */ + if (gunyah_api_version(&gunyah_api) !=3D GUNYAH_API_V1) { + pr_info("Unsupported Gunyah version: %u\n", + gunyah_api_version(&gunyah_api)); + return -ENODEV; + } + + return devm_of_platform_populate(&pdev->dev); +} + +static const struct of_device_id gunyah_of_match[] =3D { + { .compatible =3D "qcom,gunyah-hypervisor" }, + {} +}; +MODULE_DEVICE_TABLE(of, gunyah_of_match); + +/* clang-format off */ +static struct platform_driver gunyah_driver =3D { + .probe =3D gunyah_probe, + .driver =3D { + .name =3D "gunyah", + .of_match_table =3D gunyah_of_match, + } +}; +/* clang-format on */ +module_platform_driver(gunyah_driver); + +MODULE_LICENSE("GPL"); +MODULE_DESCRIPTION("Gunyah Driver"); --=20 2.39.5 From nobody Thu Dec 18 00:34:59 2025 Received: from mail-wm1-f53.google.com (mail-wm1-f53.google.com [209.85.128.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7AA34288C94 for ; Thu, 24 Apr 2025 14:14:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.53 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504046; cv=none; b=gvDDI2We2MuS7ZNDDFfU/rVf1YnEBgyf8eWHV30dD/tfbLlEpuHCFAuiE+enSdc5HXau/h9vStTit2yfhgY4GENmAAqbTIk/L+G2xGMawi2jFGm7ggHl6SMjVV9o5h/80yAAgTvkXCdBmmADP5t066iLEuy5Pcb9lbvdKAuVspM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504046; c=relaxed/simple; bh=cRJdZ/BvpG7Nf2HfvkjONlA6iN6G9YZT/kvCa6h8Pqk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=T6yRULJQ7TsQxXKV6ODxkoEUp0ixzfs63rgB6a/X7MlfGgtbZU4Ct+to6vH+2EEFkCWmJ/0N6mYXDoFdgYafYphtbcYUdPcSZDFyt1M+wrmXQNMXIJzP/42gTkCmWjapVPnHEv1Er0Q4shmV+Yc7qW5ZSMHlIJ7pjDRpTbRhmbk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=aAsLp2wi; arc=none smtp.client-ip=209.85.128.53 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="aAsLp2wi" Received: by mail-wm1-f53.google.com with SMTP id 5b1f17b1804b1-43cec5cd73bso6357635e9.3 for ; Thu, 24 Apr 2025 07:14:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504042; x=1746108842; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Iugp3VZMso7R61paVQ20sor9LWthQ+vE2iyUDGP7wsk=; b=aAsLp2wiL4SnGGfuvk26Iktqh4BJnxFfKcsSxvjPEagGx7fi/GliYmXzZdgxm3FCgY R/qwu/efZXZX7uKjiHZUoXchN/W/kcp5mMR2C5anR2hcIJ3vbCm0XaoUacmQvPV2g4gC HvHEBkPZop2ckBPqpxuors2ZNP2+rwt8rpotniV0U9YiCzMm2V0lXTsv1bEPFgXatILl U/awvhfv2MzQmr0N5FntwhNSvD/lPVBu2z7/NmOrnMklRKIOceLIopPA4Y1gRvEZFHOD 9Kr+mQE/POTKy/7qng9AwhhvaG/MFOECRAnsoxCXf/aZag79pLOhLgzi7sjdzUZ9WG4K /FNg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504042; x=1746108842; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Iugp3VZMso7R61paVQ20sor9LWthQ+vE2iyUDGP7wsk=; b=q+w+kVfFwKv28NBN0gyB2iZhcWjx08UB+ZxcWsFx95t0g/YRbArpIUUBPs63LoxurE VM2t9fzXkwouC5VgG9zwqDcoWE6Fb/h1zgoF+Vxajoaw7/M9sTRseCZdOzxQL+hBJWQ5 QjznTDQXzolMoRAOfx4qt2bI3C+7qt+II3nROYwayGSimf2d956wzJ4/NzuP9o1gshDN vu8jUdvL0szUD4wyRZJvufWuiTJbtvnIpzLN6TQ//lGAtw/5NiPEDhl9qs93bOhu6l+H Y8quChXGrwpPVAJ5ZqigiQ/dkmRXbocTT1ORzSaX4fDiZFElfrvM/2l04yoUcl7NDMII 2yxw== X-Gm-Message-State: AOJu0Yz/6Ljr1EWleJ3Gi7jtHyPY2+X2POsXN//zf2sFCpL9ufgeWVpv UnTXDn2rINPeyeUgqv99d+3iHswqJJ/YH3H2JklifS77aArqqhSVcfdOVGSVgHD07/Lu++ph4d9 x X-Gm-Gg: ASbGncvLsMeb0z4oKgepzrUlHIW1lxxPhIIW7mPhTTR74CM5g8lNNlv6KEoWAsdtm+u r8a6otn4XXdLBCExqpLIIz5g02sRoveTbfiuWAPCFKiVkegNw2oiNdtmEUC/qCJokPe2treI7nv 4fbA7jqfymZ18kOJdJzvwx1YHz3eQrexTzcypnmFaYIcwBdkq/bMGZti+ZmzoNV+SYD2NjORN69 93kVUC6uKIeJlaKh5SP0mzLbMtyZVA/qFQM4NId0uQBCGQKHPhTJvFu+tYxq/LKzuwIqp3mIqaR dYvH8cQaRpmCghjzy21yxFtdDZQDDc/T6/n11KH9/80zsRkdjkO6SVAY7pypVRVPDCPNVg7uxII q5wqvL+Mlk5L90xY+ X-Google-Smtp-Source: AGHT+IFb5JrKaNG2KfYhmzfwfaDb/IjykBP33KVSW9og9BEhgmgsW+e69hlvAOQj6I5jNodxA+AwfQ== X-Received: by 2002:a05:600c:c092:b0:43c:e2dd:98ea with SMTP id 5b1f17b1804b1-4409c3b1e1amr23970305e9.22.1745504042259; Thu, 24 Apr 2025 07:14:02 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.14.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:14:01 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt , Elliot Berman , Alex Elder , Srinivas Kandagatla Subject: [RFC PATCH 13/34] gunyah: Add hypercalls to send and receive messages Date: Thu, 24 Apr 2025 15:13:20 +0100 Message-Id: <20250424141341.841734-14-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Elliot Berman Add hypercalls to send and receive messages on a Gunyah message queue. Reviewed-by: Alex Elder Reviewed-by: Srinivas Kandagatla Signed-off-by: Elliot Berman Signed-off-by: Karim Manaouil --- arch/arm64/gunyah/gunyah_hypercall.c | 55 ++++++++++++++++++++++++++++ include/linux/gunyah.h | 8 ++++ 2 files changed, 63 insertions(+) diff --git a/arch/arm64/gunyah/gunyah_hypercall.c b/arch/arm64/gunyah/gunya= h_hypercall.c index d44663334f38..1302e128be6e 100644 --- a/arch/arm64/gunyah/gunyah_hypercall.c +++ b/arch/arm64/gunyah/gunyah_hypercall.c @@ -37,6 +37,8 @@ EXPORT_SYMBOL_GPL(arch_is_gunyah_guest); =20 /* clang-format off */ #define GUNYAH_HYPERCALL_HYP_IDENTIFY GUNYAH_HYPERCALL(0x8000) +#define GUNYAH_HYPERCALL_MSGQ_SEND GUNYAH_HYPERCALL(0x801B) +#define GUNYAH_HYPERCALL_MSGQ_RECV GUNYAH_HYPERCALL(0x801C) /* clang-format on */ =20 /** @@ -58,5 +60,58 @@ void gunyah_hypercall_hyp_identify( } EXPORT_SYMBOL_GPL(gunyah_hypercall_hyp_identify); =20 +/** + * gunyah_hypercall_msgq_send() - Send a buffer on a message queue + * @capid: capability ID of the message queue to add message + * @size: Size of @buff + * @buff: Address of buffer to send + * @tx_flags: See GUNYAH_HYPERCALL_MSGQ_TX_FLAGS_* + * @ready: If the send was successful, ready is filled with true if more + * messages can be sent on the queue. If false, then the tx IRQ wi= ll + * be raised in future when send can succeed. + */ +enum gunyah_error gunyah_hypercall_msgq_send(u64 capid, size_t size, void = *buff, + u64 tx_flags, bool *ready) +{ + struct arm_smccc_res res; + + arm_smccc_1_1_hvc(GUNYAH_HYPERCALL_MSGQ_SEND, capid, size, + (uintptr_t)buff, tx_flags, 0, &res); + + if (res.a0 =3D=3D GUNYAH_ERROR_OK) + *ready =3D !!res.a1; + + return res.a0; +} +EXPORT_SYMBOL_GPL(gunyah_hypercall_msgq_send); + +/** + * gunyah_hypercall_msgq_recv() - Send a buffer on a message queue + * @capid: capability ID of the message queue to add message + * @buff: Address of buffer to copy received data into + * @size: Size of @buff + * @recv_size: If the receive was successful, recv_size is filled with the + * size of data received. Will be <=3D size. + * @ready: If the receive was successful, ready is filled with true if more + * messages are ready to be received on the queue. If false, then = the + * rx IRQ will be raised in future when recv can succeed. + */ +enum gunyah_error gunyah_hypercall_msgq_recv(u64 capid, void *buff, size_t= size, + size_t *recv_size, bool *ready) +{ + struct arm_smccc_res res; + + arm_smccc_1_1_hvc(GUNYAH_HYPERCALL_MSGQ_RECV, capid, (uintptr_t)buff, + size, 0, &res); + + if (res.a0 =3D=3D GUNYAH_ERROR_OK) { + *recv_size =3D res.a1; + *ready =3D !!res.a2; + } + + return res.a0; +} +EXPORT_SYMBOL_GPL(gunyah_hypercall_msgq_recv); + MODULE_LICENSE("GPL"); MODULE_DESCRIPTION("Gunyah Hypervisor Hypercalls"); diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h index 33bcbd22d39f..acd70f982425 100644 --- a/include/linux/gunyah.h +++ b/include/linux/gunyah.h @@ -141,4 +141,12 @@ gunyah_api_version(const struct gunyah_hypercall_hyp_i= dentify_resp *gunyah_api) void gunyah_hypercall_hyp_identify( struct gunyah_hypercall_hyp_identify_resp *hyp_identity); =20 +/* Immediately raise RX vIRQ on receiver VM */ +#define GUNYAH_HYPERCALL_MSGQ_TX_FLAGS_PUSH BIT(0) + +enum gunyah_error gunyah_hypercall_msgq_send(u64 capid, size_t size, void = *buff, + u64 tx_flags, bool *ready); +enum gunyah_error gunyah_hypercall_msgq_recv(u64 capid, void *buff, size_t= size, + size_t *recv_size, bool *ready); + #endif --=20 2.39.5 From nobody Thu Dec 18 00:34:59 2025 Received: from mail-wr1-f53.google.com (mail-wr1-f53.google.com [209.85.221.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F08DD28936E for ; Thu, 24 Apr 2025 14:14:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.53 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504049; cv=none; b=b+aR+3gT6oRc+MPBUDOKeXEfFr3CjdmCIdlxS+Y203DTMNQmFOWGWvjTJdFJRFeqJ3iJd8Am7z5XV957an9pKWxLNs7+uTUaDxLckFHIOe/QL62w2fKOHle9kYxXaskeiO6abwrAgctcXL34+aMYRfvpMvOtzkfu5adNQa/Tcj8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504049; c=relaxed/simple; bh=m+fPEAqeKx8MI3a+lqdiH9/ulP+VSGYzhW8y82AELSw=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=GU5az0BY3FLKOXauP+uJVDVyxPm09ORxyBZq11OdaYiLYzwhWNzOLoiZfoqKGDdJabDuXYwQ34lzePcBsbSwRlIthcTtbOW6CpJ+rAJ+xEheMDTnxQW0mrE+XbtfBxOWffJ6Xq8P4w5CBF1vyTfUJTUqsiS+VYYr+rnqjut6yEw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=QSVzhmLG; arc=none smtp.client-ip=209.85.221.53 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="QSVzhmLG" Received: by mail-wr1-f53.google.com with SMTP id ffacd0b85a97d-39c266c1389so744934f8f.1 for ; Thu, 24 Apr 2025 07:14:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504044; x=1746108844; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=+XE4xl5smx9gWubImYnQvAwnhzC9G4S+d/giF3voQDM=; b=QSVzhmLG80Fx0P05gKPq2TWi04hwkFYKyLuhUtoU4bhDSfHBBssAdoNA1EsFtqRqMb krd4u2GshbDYbYBaOEvxEIh/a3rYMWfdSym4kAhgTHT9hbzXRC+cWulnVsKu9l9xe7fN pwfo/P5YhTjdzA6a0BH8jCZEuKy+4NkyumE4WtXIoY6vGSZfVuyfqGJlq7+QqyWpX5aU Vq6y+p72perGggywfEZRwaHYBC3pRyljQ6h/CBeyoONmdVAkrSzBqN1cZzIC35vbYd+t ns1rQajDhaC1yX+Zuowhp4FtD8MgA2+69cQmTHE9dQzuSZ2EqeM+fLsiCI/ov9l77cdD b7lw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504044; x=1746108844; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=+XE4xl5smx9gWubImYnQvAwnhzC9G4S+d/giF3voQDM=; b=sMyCwij/YKPVR4j5U26MunBZe6oClVkk7UFgjHVlXea0XN7gRiA/2e+Sg3hrJSP4B/ PwwaBtbKo1sRg7Wri7ur6ohMRtWFJLC5N8gayA/McLs1ILorTb2952VEDJaN2eHyT0J4 WCvRzhvqLpSl6+xFwvnU3o1p4dNzMihcbk5ohrxbM7bSf74oozkB5OL0/c98NQrnIqj3 XJaiPIPIQnfRomZlecMP9ZEfM9Q+f+Wwcrb7IK1yryFv/wLDhvxmYGRGxPIxWds4c+qb uGAQ+LUsftJaxiK5o8GtsuKmqxeYx7ES18R62roingJOP6AaIij8drS97w5lW+zgwNpM N9vQ== X-Gm-Message-State: AOJu0Yz3lqyVatHwMUVXOjLN5mTlsljWwuEKcjGGNrQ4Mh4kaq1KREsl BgdHkOnRe7XoENS53ed5dg7e1a6jR9XbQN2lkxN97o20S1iFDcKPMV3EGmTh+k38i1ALJfQ6nIr L X-Gm-Gg: ASbGncswX8NbcKGXnPmlbdh2dAMRqOpLKqM64uTm4z92RilX0hOPz5DRt/QauYgvB8O iTiUcHQaKhCg1kiRaNzJLJEUVy9nKsCCK5FXX5//vA5ptskHa8Exm5OSmfhhz8g0VBOxcGfSJz0 A71TpVdlNkvDWEzvDFSCbSbKotFhqIsHQWULFs1mLVkX2CwYSap52/hV/HQ8GJoFlKsBlkHd2oF OQ5QlPkfD/zLPjrgRRv371YKbxHU3kbRo/6CdcC/ovdi0S9IDJINbNW5YW68LYbdkbzPKbj32JJ xwLLiEE57SvFIKoH+9aKpHSFpbtkBsgtUHvQOtF0wXNaF6paI5zckbpq+PEdnCeyNHNduxfjgm3 yZzr8h1g4n03CxKAF X-Google-Smtp-Source: AGHT+IE3uJouYJ/7M/rrSd1eFvrf/qS5HF+61EQrttb/wpTbAdACHBohOTrf1XeLM5WbZ4ptiBZEjw== X-Received: by 2002:a5d:5985:0:b0:391:2ab1:d4c2 with SMTP id ffacd0b85a97d-3a06cfabac2mr2257224f8f.37.1745504043594; Thu, 24 Apr 2025 07:14:03 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.14.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:14:03 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt , Elliot Berman Subject: [RFC PATCH 14/34] gunyah: Add resource manager RPC core Date: Thu, 24 Apr 2025 15:13:21 +0100 Message-Id: <20250424141341.841734-15-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Elliot Berman The resource manager is a special virtual machine which is always running on a Gunyah system. It provides APIs for creating and destroying VMs, secure memory management, sharing/lending of memory between VMs, and setup of inter-VM communication. Calls to the resource manager are made via message queues. This patch implements the basic probing and RPC mechanism to make those API calls. Request/response calls can be made with gh_rm_call. Drivers can also register to notifications pushed by RM via gh_rm_register_notifier Specific API calls that resource manager supports will be implemented in subsequent patches. Signed-off-by: Elliot Berman Signed-off-by: Karim Manaouil --- drivers/virt/gunyah/Makefile | 4 +- drivers/virt/gunyah/rsc_mgr.c | 724 +++++++++++++++++++++++++++++++++ include/linux/gunyah_rsc_mgr.h | 27 ++ 3 files changed, 754 insertions(+), 1 deletion(-) create mode 100644 drivers/virt/gunyah/rsc_mgr.c create mode 100644 include/linux/gunyah_rsc_mgr.h diff --git a/drivers/virt/gunyah/Makefile b/drivers/virt/gunyah/Makefile index 34f32110faf9..c2308389f551 100644 --- a/drivers/virt/gunyah/Makefile +++ b/drivers/virt/gunyah/Makefile @@ -1,3 +1,5 @@ # SPDX-License-Identifier: GPL-2.0 =20 -obj-$(CONFIG_GUNYAH) +=3D gunyah.o +gunyah_rsc_mgr-y +=3D rsc_mgr.o + +obj-$(CONFIG_GUNYAH) +=3D gunyah.o gunyah_rsc_mgr.o diff --git a/drivers/virt/gunyah/rsc_mgr.c b/drivers/virt/gunyah/rsc_mgr.c new file mode 100644 index 000000000000..75fc86887868 --- /dev/null +++ b/drivers/virt/gunyah/rsc_mgr.c @@ -0,0 +1,724 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights res= erved. + */ + +#include +#include +#include +#include +#include +#include +#include + +#include + +/* clang-format off */ +#define RM_RPC_API_VERSION_MASK GENMASK(3, 0) +#define RM_RPC_HEADER_WORDS_MASK GENMASK(7, 4) +#define RM_RPC_API_VERSION FIELD_PREP(RM_RPC_API_VERSION_MASK, 1) +#define RM_RPC_HEADER_WORDS FIELD_PREP(RM_RPC_HEADER_WORDS_MASK, \ + (sizeof(struct gunyah_rm_rpc_hdr) / sizeof(u32))) +#define RM_RPC_API (RM_RPC_API_VERSION | RM_RPC_HEADER_WORDS) + +#define RM_RPC_TYPE_CONTINUATION 0x0 +#define RM_RPC_TYPE_REQUEST 0x1 +#define RM_RPC_TYPE_REPLY 0x2 +#define RM_RPC_TYPE_NOTIF 0x3 +#define RM_RPC_TYPE_MASK GENMASK(1, 0) + +#define GUNYAH_RM_MAX_NUM_FRAGMENTS 62 +#define RM_RPC_FRAGMENTS_MASK GENMASK(7, 2) +/* clang-format on */ + +struct gunyah_rm_rpc_hdr { + u8 api; + u8 type; + __le16 seq; + __le32 msg_id; +} __packed; + +struct gunyah_rm_rpc_reply_hdr { + struct gunyah_rm_rpc_hdr hdr; + __le32 err_code; /* GUNYAH_RM_ERROR_* */ +} __packed; + +#define GUNYAH_RM_MSGQ_MSG_SIZE 240 +#define GUNYAH_RM_PAYLOAD_SIZE \ + (GUNYAH_RM_MSGQ_MSG_SIZE - sizeof(struct gunyah_rm_rpc_hdr)) + +/* RM Error codes */ +enum gunyah_rm_error { + /* clang-format off */ + GUNYAH_RM_ERROR_OK =3D 0x0, + GUNYAH_RM_ERROR_UNIMPLEMENTED =3D 0xFFFFFFFF, + GUNYAH_RM_ERROR_NOMEM =3D 0x1, + GUNYAH_RM_ERROR_NORESOURCE =3D 0x2, + GUNYAH_RM_ERROR_DENIED =3D 0x3, + GUNYAH_RM_ERROR_INVALID =3D 0x4, + GUNYAH_RM_ERROR_BUSY =3D 0x5, + GUNYAH_RM_ERROR_ARGUMENT_INVALID =3D 0x6, + GUNYAH_RM_ERROR_HANDLE_INVALID =3D 0x7, + GUNYAH_RM_ERROR_VALIDATE_FAILED =3D 0x8, + GUNYAH_RM_ERROR_MAP_FAILED =3D 0x9, + GUNYAH_RM_ERROR_MEM_INVALID =3D 0xA, + GUNYAH_RM_ERROR_MEM_INUSE =3D 0xB, + GUNYAH_RM_ERROR_MEM_RELEASED =3D 0xC, + GUNYAH_RM_ERROR_VMID_INVALID =3D 0xD, + GUNYAH_RM_ERROR_LOOKUP_FAILED =3D 0xE, + GUNYAH_RM_ERROR_IRQ_INVALID =3D 0xF, + GUNYAH_RM_ERROR_IRQ_INUSE =3D 0x10, + GUNYAH_RM_ERROR_IRQ_RELEASED =3D 0x11, + /* clang-format on */ +}; + +/** + * struct gunyah_rm_message - Represents a complete message from resource = manager + * @payload: Combined payload of all the fragments (msg headers stripped o= ff). + * @size: Size of the payload received so far. + * @msg_id: Message ID from the header. + * @type: RM_RPC_TYPE_REPLY or RM_RPC_TYPE_NOTIF. + * @num_fragments: total number of fragments expected to be received. + * @fragments_received: fragments received so far. + * @reply: Fields used for request/reply sequences + */ +struct gunyah_rm_message { + void *payload; + size_t size; + u32 msg_id; + u8 type; + + u8 num_fragments; + u8 fragments_received; + + /** + * @ret: Linux return code, there was an error processing message + * @seq: Sequence ID for the main message. + * @rm_error: For request/reply sequences with standard replies + * @seq_done: Signals caller that the RM reply has been received + */ + struct { + int ret; + u16 seq; + enum gunyah_rm_error rm_error; + struct completion seq_done; + } reply; +}; + +/** + * struct gunyah_rm - private data for communicating w/Gunyah resource man= ager + * @dev: pointer to RM platform device + * @tx_ghrsc: message queue resource to TX to RM + * @rx_ghrsc: message queue resource to RX from RM + * @active_rx_message: ongoing gunyah_rm_message for which we're receiving= fragments + * @call_xarray: xarray to allocate & lookup sequence IDs for Request/Resp= onse flows + * @next_seq: next ID to allocate (for xa_alloc_cyclic) + * @recv_msg: cached allocation for Rx messages + * @send_msg: cached allocation for Tx messages. Must hold @send_lock to m= anipulate. + * @send_lock: synchronization to allow only one request to be sent at a t= ime + * @send_ready: completed when we know Tx message queue can take more mess= ages + * @nh: notifier chain for clients interested in RM notification messages + */ +struct gunyah_rm { + struct device *dev; + struct gunyah_resource tx_ghrsc; + struct gunyah_resource rx_ghrsc; + struct gunyah_rm_message *active_rx_message; + + struct xarray call_xarray; + u32 next_seq; + + unsigned char recv_msg[GUNYAH_RM_MSGQ_MSG_SIZE]; + unsigned char send_msg[GUNYAH_RM_MSGQ_MSG_SIZE]; + struct mutex send_lock; + struct completion send_ready; + struct blocking_notifier_head nh; +}; + +/* Global resource manager instance */ +struct gunyah_rm *gunyah_rm; +EXPORT_SYMBOL_GPL(gunyah_rm); + +/** + * gunyah_rm_error_remap() - Remap Gunyah resource manager errors into a L= inux error code + * @rm_error: "Standard" return value from Gunyah resource manager + */ +static inline int gunyah_rm_error_remap(enum gunyah_rm_error rm_error) +{ + switch (rm_error) { + case GUNYAH_RM_ERROR_OK: + return 0; + case GUNYAH_RM_ERROR_UNIMPLEMENTED: + return -EOPNOTSUPP; + case GUNYAH_RM_ERROR_NOMEM: + return -ENOMEM; + case GUNYAH_RM_ERROR_NORESOURCE: + return -ENODEV; + case GUNYAH_RM_ERROR_DENIED: + return -EPERM; + case GUNYAH_RM_ERROR_BUSY: + return -EBUSY; + case GUNYAH_RM_ERROR_INVALID: + case GUNYAH_RM_ERROR_ARGUMENT_INVALID: + case GUNYAH_RM_ERROR_HANDLE_INVALID: + case GUNYAH_RM_ERROR_VALIDATE_FAILED: + case GUNYAH_RM_ERROR_MAP_FAILED: + case GUNYAH_RM_ERROR_MEM_INVALID: + case GUNYAH_RM_ERROR_MEM_INUSE: + case GUNYAH_RM_ERROR_MEM_RELEASED: + case GUNYAH_RM_ERROR_VMID_INVALID: + case GUNYAH_RM_ERROR_LOOKUP_FAILED: + case GUNYAH_RM_ERROR_IRQ_INVALID: + case GUNYAH_RM_ERROR_IRQ_INUSE: + case GUNYAH_RM_ERROR_IRQ_RELEASED: + return -EINVAL; + default: + return -EBADMSG; + } +} + +static int gunyah_rm_init_message_payload(struct gunyah_rm_message *messag= e, + const void *msg, size_t hdr_size, + size_t msg_size) +{ + const struct gunyah_rm_rpc_hdr *hdr =3D msg; + size_t max_buf_size, payload_size; + + if (msg_size < hdr_size) + return -EINVAL; + + payload_size =3D msg_size - hdr_size; + + message->num_fragments =3D FIELD_GET(RM_RPC_FRAGMENTS_MASK, hdr->type); + message->fragments_received =3D 0; + + /* There's not going to be any payload, no need to allocate buffer. */ + if (!payload_size && !message->num_fragments) + return 0; + + if (message->num_fragments > GUNYAH_RM_MAX_NUM_FRAGMENTS) + return -EINVAL; + + max_buf_size =3D payload_size + + (message->num_fragments * GUNYAH_RM_PAYLOAD_SIZE); + + message->payload =3D kzalloc(max_buf_size, GFP_KERNEL); + if (!message->payload) + return -ENOMEM; + + memcpy(message->payload, msg + hdr_size, payload_size); + message->size =3D payload_size; + return 0; +} + +static void gunyah_rm_abort_message(struct gunyah_rm *rm) +{ + kfree(rm->active_rx_message->payload); + + switch (rm->active_rx_message->type) { + case RM_RPC_TYPE_REPLY: + rm->active_rx_message->reply.ret =3D -EIO; + complete(&rm->active_rx_message->reply.seq_done); + break; + case RM_RPC_TYPE_NOTIF: + fallthrough; + default: + kfree(rm->active_rx_message); + } + + rm->active_rx_message =3D NULL; +} + +static inline void gunyah_rm_try_complete_message(struct gunyah_rm *rm) +{ + struct gunyah_rm_message *message =3D rm->active_rx_message; + + if (!message || message->fragments_received !=3D message->num_fragments) + return; + + switch (message->type) { + case RM_RPC_TYPE_REPLY: + complete(&message->reply.seq_done); + break; + case RM_RPC_TYPE_NOTIF: + blocking_notifier_call_chain(&rm->nh, message->msg_id, + message->payload); + + kfree(message->payload); + kfree(message); + break; + default: + dev_err_ratelimited(rm->dev, + "Invalid message type (%u) received\n", + message->type); + gunyah_rm_abort_message(rm); + break; + } + + rm->active_rx_message =3D NULL; +} + +static void gunyah_rm_process_notif(struct gunyah_rm *rm, const void *msg, + size_t msg_size) +{ + const struct gunyah_rm_rpc_hdr *hdr =3D msg; + struct gunyah_rm_message *message; + int ret; + + if (rm->active_rx_message) { + dev_err(rm->dev, + "Unexpected new notification, still processing an active message"); + gunyah_rm_abort_message(rm); + } + + message =3D kzalloc(sizeof(*message), GFP_KERNEL); + if (!message) + return; + + message->type =3D RM_RPC_TYPE_NOTIF; + message->msg_id =3D le32_to_cpu(hdr->msg_id); + + ret =3D gunyah_rm_init_message_payload(message, msg, sizeof(*hdr), + msg_size); + if (ret) { + dev_err(rm->dev, + "Failed to initialize message for notification: %d\n", + ret); + kfree(message); + return; + } + + rm->active_rx_message =3D message; + + gunyah_rm_try_complete_message(rm); +} + +static void gunyah_rm_process_reply(struct gunyah_rm *rm, const void *msg, + size_t msg_size) +{ + const struct gunyah_rm_rpc_reply_hdr *reply_hdr =3D msg; + struct gunyah_rm_message *message; + u16 seq_id; + + seq_id =3D le16_to_cpu(reply_hdr->hdr.seq); + message =3D xa_load(&rm->call_xarray, seq_id); + + if (!message || message->msg_id !=3D le32_to_cpu(reply_hdr->hdr.msg_id)) + return; + + if (rm->active_rx_message) { + dev_err(rm->dev, + "Unexpected new reply, still processing an active message"); + gunyah_rm_abort_message(rm); + } + + if (gunyah_rm_init_message_payload(message, msg, sizeof(*reply_hdr), + msg_size)) { + dev_err(rm->dev, + "Failed to alloc message buffer for sequence %d\n", + seq_id); + /* Send message complete and error the client. */ + message->reply.ret =3D -ENOMEM; + complete(&message->reply.seq_done); + return; + } + + message->reply.rm_error =3D le32_to_cpu(reply_hdr->err_code); + rm->active_rx_message =3D message; + + gunyah_rm_try_complete_message(rm); +} + +static void gunyah_rm_process_cont(struct gunyah_rm *rm, + struct gunyah_rm_message *message, + const void *msg, size_t msg_size) +{ + const struct gunyah_rm_rpc_hdr *hdr =3D msg; + size_t payload_size =3D msg_size - sizeof(*hdr); + + if (!rm->active_rx_message) + return; + + /* + * hdr->fragments and hdr->msg_id preserves the value from first reply + * or notif message. To detect mishandling, check it's still intact. + */ + if (message->msg_id !=3D le32_to_cpu(hdr->msg_id) || + message->num_fragments !=3D + FIELD_GET(RM_RPC_FRAGMENTS_MASK, hdr->type)) { + gunyah_rm_abort_message(rm); + return; + } + + memcpy(message->payload + message->size, msg + sizeof(*hdr), + payload_size); + message->size +=3D payload_size; + message->fragments_received++; + + gunyah_rm_try_complete_message(rm); +} + +static irqreturn_t gunyah_rm_rx(int irq, void *data) +{ + enum gunyah_error gunyah_error; + struct gunyah_rm_rpc_hdr *hdr; + struct gunyah_rm *rm =3D data; + void *msg =3D &rm->recv_msg[0]; + size_t len; + bool ready; + + do { + gunyah_error =3D gunyah_hypercall_msgq_recv(rm->rx_ghrsc.capid, + msg, + sizeof(rm->recv_msg), + &len, &ready); + if (gunyah_error !=3D GUNYAH_ERROR_OK) { + if (gunyah_error !=3D GUNYAH_ERROR_MSGQUEUE_EMPTY) + dev_warn(rm->dev, + "Failed to receive data: %d\n", + gunyah_error); + return IRQ_HANDLED; + } + + if (len < sizeof(*hdr)) { + dev_err_ratelimited( + rm->dev, + "Too small message received. size=3D%ld\n", len); + continue; + } + + hdr =3D msg; + if (hdr->api !=3D RM_RPC_API) { + dev_err(rm->dev, "Unknown RM RPC API version: %x\n", + hdr->api); + return IRQ_HANDLED; + } + + switch (FIELD_GET(RM_RPC_TYPE_MASK, hdr->type)) { + case RM_RPC_TYPE_NOTIF: + gunyah_rm_process_notif(rm, msg, len); + break; + case RM_RPC_TYPE_REPLY: + gunyah_rm_process_reply(rm, msg, len); + break; + case RM_RPC_TYPE_CONTINUATION: + gunyah_rm_process_cont(rm, rm->active_rx_message, msg, + len); + break; + default: + dev_err(rm->dev, + "Invalid message type (%lu) received\n", + FIELD_GET(RM_RPC_TYPE_MASK, hdr->type)); + return IRQ_HANDLED; + } + } while (ready); + + return IRQ_HANDLED; +} + +static irqreturn_t gunyah_rm_tx(int irq, void *data) +{ + struct gunyah_rm *rm =3D data; + + complete(&rm->send_ready); + + return IRQ_HANDLED; +} + +static int gunyah_rm_msgq_send(struct gunyah_rm *rm, size_t size, bool pus= h) +{ + const u64 tx_flags =3D push ? GUNYAH_HYPERCALL_MSGQ_TX_FLAGS_PUSH : 0; + enum gunyah_error gunyah_error; + void *data =3D &rm->send_msg[0]; + bool ready; + + lockdep_assert_held(&rm->send_lock); + +again: + wait_for_completion(&rm->send_ready); + gunyah_error =3D gunyah_hypercall_msgq_send(rm->tx_ghrsc.capid, size, + data, tx_flags, &ready); + + /* Should never happen because Linux properly tracks the ready-state of t= he msgq */ + if (WARN_ON(gunyah_error =3D=3D GUNYAH_ERROR_MSGQUEUE_FULL)) + goto again; + + if (ready) + complete(&rm->send_ready); + + return gunyah_error_remap(gunyah_error); +} + +static int gunyah_rm_send_request(struct gunyah_rm *rm, u32 message_id, + const void *req_buf, size_t req_buf_size, + struct gunyah_rm_message *message) +{ + size_t buf_size_remaining =3D req_buf_size; + const void *req_buf_curr =3D req_buf; + struct gunyah_rm_rpc_hdr *hdr =3D + (struct gunyah_rm_rpc_hdr *)&rm->send_msg[0]; + struct gunyah_rm_rpc_hdr hdr_template; + void *payload =3D hdr + 1; + u32 cont_fragments =3D 0; + size_t payload_size; + bool push; + int ret; + + if (req_buf_size > + GUNYAH_RM_MAX_NUM_FRAGMENTS * GUNYAH_RM_PAYLOAD_SIZE) { + dev_warn( + rm->dev, + "Limit (%lu bytes) exceeded for the maximum message size: %lu\n", + GUNYAH_RM_MAX_NUM_FRAGMENTS * GUNYAH_RM_PAYLOAD_SIZE, + req_buf_size); + dump_stack(); + return -E2BIG; + } + + if (req_buf_size) + cont_fragments =3D (req_buf_size - 1) / GUNYAH_RM_PAYLOAD_SIZE; + + hdr_template.api =3D RM_RPC_API; + hdr_template.type =3D FIELD_PREP(RM_RPC_TYPE_MASK, RM_RPC_TYPE_REQUEST) | + FIELD_PREP(RM_RPC_FRAGMENTS_MASK, cont_fragments); + hdr_template.seq =3D cpu_to_le16(message->reply.seq); + hdr_template.msg_id =3D cpu_to_le32(message_id); + + ret =3D mutex_lock_interruptible(&rm->send_lock); + if (ret) + return ret; + + do { + *hdr =3D hdr_template; + + /* Copy payload */ + payload_size =3D min(buf_size_remaining, GUNYAH_RM_PAYLOAD_SIZE); + memcpy(payload, req_buf_curr, payload_size); + req_buf_curr +=3D payload_size; + buf_size_remaining -=3D payload_size; + + /* Only the last message should have push flag set */ + push =3D !buf_size_remaining; + ret =3D gunyah_rm_msgq_send(rm, sizeof(*hdr) + payload_size, + push); + if (ret) + break; + + hdr_template.type =3D + FIELD_PREP(RM_RPC_TYPE_MASK, RM_RPC_TYPE_CONTINUATION) | + FIELD_PREP(RM_RPC_FRAGMENTS_MASK, cont_fragments); + } while (buf_size_remaining); + + mutex_unlock(&rm->send_lock); + return ret; +} + +/** + * gunyah_rm_call: Achieve request-response type communication with RPC + * @rm: Pointer to Gunyah resource manager internal data + * @message_id: The RM RPC message-id + * @req_buf: Request buffer that contains the payload + * @req_buf_size: Total size of the payload + * @resp_buf: Pointer to a response buffer + * @resp_buf_size: Size of the response buffer + * + * Make a request to the Resource Manager and wait for reply back. For a s= uccessful + * response, the function returns the payload. The size of the payload is = set in + * resp_buf_size. The resp_buf must be freed by the caller when 0 is retur= ned + * and resp_buf_size !=3D 0. + * + * req_buf should be not NULL for req_buf_size >0. If req_buf_size =3D=3D = 0, + * req_buf *can* be NULL and no additional payload is sent. + * + * Context: Process context. Will sleep waiting for reply. + * Return: 0 on success. <0 if error. + */ +int gunyah_rm_call(struct gunyah_rm *rm, u32 message_id, const void *req_b= uf, + size_t req_buf_size, void **resp_buf, size_t *resp_buf_size) +{ + struct gunyah_rm_message message =3D { 0 }; + u32 seq_id; + int ret; + + /* message_id 0 is reserved. req_buf_size implies req_buf is not NULL */ + if (!rm || !message_id || (!req_buf && req_buf_size)) + return -EINVAL; + + message.type =3D RM_RPC_TYPE_REPLY; + message.msg_id =3D message_id; + + message.reply.seq_done =3D + COMPLETION_INITIALIZER_ONSTACK(message.reply.seq_done); + + /* Allocate a new seq number for this message */ + ret =3D xa_alloc_cyclic(&rm->call_xarray, &seq_id, &message, xa_limit_16b, + &rm->next_seq, GFP_KERNEL); + if (ret < 0) + return ret; + message.reply.seq =3D lower_16_bits(seq_id); + + /* Send the request to the Resource Manager */ + ret =3D gunyah_rm_send_request(rm, message_id, req_buf, req_buf_size, + &message); + if (ret < 0) { + dev_warn(rm->dev, "Failed to send request. Error: %d\n", ret); + goto out; + } + + /* + * Wait for response. Uninterruptible because rollback based on what RM d= id to VM + * requires us to know how RM handled the call. + */ + wait_for_completion(&message.reply.seq_done); + + /* Check for internal (kernel) error waiting for the response */ + if (message.reply.ret) { + ret =3D message.reply.ret; + goto out; + } + + /* Got a response, did resource manager give us an error? */ + if (message.reply.rm_error !=3D GUNYAH_RM_ERROR_OK) { + dev_warn(rm->dev, "RM rejected message %08x. Error: %d\n", + message_id, message.reply.rm_error); + ret =3D gunyah_rm_error_remap(message.reply.rm_error); + kfree(message.payload); + goto out; + } + + /* Everything looks good, return the payload */ + if (resp_buf_size) + *resp_buf_size =3D message.size; + + if (message.size && resp_buf) { + *resp_buf =3D message.payload; + } else { + /* kfree in case RM sent us multiple fragments but never any data in + * those fragments. We would've allocated memory for it, but message.siz= e =3D=3D 0 + */ + kfree(message.payload); + } + +out: + xa_erase(&rm->call_xarray, message.reply.seq); + return ret; +} + +int gunyah_rm_notifier_register(struct gunyah_rm *rm, struct notifier_bloc= k *nb) +{ + return blocking_notifier_chain_register(&rm->nh, nb); +} +EXPORT_SYMBOL_GPL(gunyah_rm_notifier_register); + +int gunyah_rm_notifier_unregister(struct gunyah_rm *rm, + struct notifier_block *nb) +{ + return blocking_notifier_chain_unregister(&rm->nh, nb); +} +EXPORT_SYMBOL_GPL(gunyah_rm_notifier_unregister); + +static int gunyah_platform_probe_capability(struct platform_device *pdev, + int idx, + struct gunyah_resource *ghrsc) +{ + int ret; + + ghrsc->irq =3D platform_get_irq(pdev, idx); + if (ghrsc->irq < 0) { + dev_err(&pdev->dev, "Failed to get %s irq: %d\n", + idx ? "rx" : "tx", ghrsc->irq); + return ghrsc->irq; + } + + ret =3D of_property_read_u64_index(pdev->dev.of_node, "reg", idx, + &ghrsc->capid); + if (ret) { + dev_err(&pdev->dev, "Failed to get %s capid: %d\n", + idx ? "rx" : "tx", ret); + return ret; + } + + return 0; +} + +static int gunyah_rm_probe_tx_msgq(struct gunyah_rm *rm, + struct platform_device *pdev) +{ + int ret; + + rm->tx_ghrsc.type =3D GUNYAH_RESOURCE_TYPE_MSGQ_TX; + ret =3D gunyah_platform_probe_capability(pdev, 0, &rm->tx_ghrsc); + if (ret) + return ret; + + enable_irq_wake(rm->tx_ghrsc.irq); + + return devm_request_irq(rm->dev, rm->tx_ghrsc.irq, gunyah_rm_tx, 0, + "gunyah_rm_tx", rm); +} + +static int gunyah_rm_probe_rx_msgq(struct gunyah_rm *rm, + struct platform_device *pdev) +{ + int ret; + + rm->rx_ghrsc.type =3D GUNYAH_RESOURCE_TYPE_MSGQ_RX; + ret =3D gunyah_platform_probe_capability(pdev, 1, &rm->rx_ghrsc); + if (ret) + return ret; + + enable_irq_wake(rm->rx_ghrsc.irq); + + return devm_request_threaded_irq(rm->dev, rm->rx_ghrsc.irq, NULL, + gunyah_rm_rx, IRQF_ONESHOT, + "gunyah_rm_rx", rm); +} + +static int gunyah_rm_probe(struct platform_device *pdev) +{ + int ret; + + gunyah_rm =3D devm_kzalloc(&pdev->dev, sizeof(*gunyah_rm), GFP_KERNEL); + if (!gunyah_rm) + return -ENOMEM; + + platform_set_drvdata(pdev, gunyah_rm); + gunyah_rm->dev =3D &pdev->dev; + + mutex_init(&gunyah_rm->send_lock); + init_completion(&gunyah_rm->send_ready); + BLOCKING_INIT_NOTIFIER_HEAD(&gunyah_rm->nh); + xa_init_flags(&gunyah_rm->call_xarray, XA_FLAGS_ALLOC); + + device_init_wakeup(&pdev->dev, true); + + ret =3D gunyah_rm_probe_tx_msgq(gunyah_rm, pdev); + if (ret) + return ret; + /* assume RM is ready to receive messages from us */ + complete(&gunyah_rm->send_ready); + + ret =3D gunyah_rm_probe_rx_msgq(gunyah_rm, pdev); + if (ret) + return ret; + + return 0; +} + +static const struct of_device_id gunyah_rm_of_match[] =3D { + { .compatible =3D "gunyah-resource-manager" }, + {} +}; +MODULE_DEVICE_TABLE(of, gunyah_rm_of_match); + +static struct platform_driver gunyah_rm_driver =3D { + .probe =3D gunyah_rm_probe, + .driver =3D { + .name =3D "gunyah_rsc_mgr", + .of_match_table =3D gunyah_rm_of_match, + }, +}; +module_platform_driver(gunyah_rm_driver); + +MODULE_LICENSE("GPL"); +MODULE_DESCRIPTION("Gunyah Resource Manager Driver"); diff --git a/include/linux/gunyah_rsc_mgr.h b/include/linux/gunyah_rsc_mgr.h new file mode 100644 index 000000000000..87e919cc1e28 --- /dev/null +++ b/include/linux/gunyah_rsc_mgr.h @@ -0,0 +1,27 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights res= erved. + */ +#ifndef __GUNYAH_RSC_MGR_H +#define __GUNYAH_RSC_MGR_H + +#include +#include + +#define GUNYAH_VMID_INVAL U16_MAX + +struct gunyah_rm; + +extern struct gunyah_rm *gunyah_rm; + +int gunyah_rm_notifier_register(struct gunyah_rm *rm, + struct notifier_block *nb); +int gunyah_rm_notifier_unregister(struct gunyah_rm *rm, + struct notifier_block *nb); + + +int gunyah_rm_call(struct gunyah_rm *rsc_mgr, u32 message_id, + const void *req_buf, size_t req_buf_size, void **resp_buf, + size_t *resp_buf_size); + +#endif --=20 2.39.5 From nobody Thu Dec 18 00:34:59 2025 Received: from mail-wm1-f41.google.com (mail-wm1-f41.google.com [209.85.128.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3C2F428A414 for ; Thu, 24 Apr 2025 14:14:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.41 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504050; cv=none; b=M0N+IzJ1ZWZ22ICZmmRupvVSinxosfINAmelkRwCmxwf50JT0pG40c8Ie82V47Fkf2m+uW8l6e7eYf+YjFmdlFyrgLHbpPv5P061BnCKyEpVB70Ot5d+Mjla5Pp+wD3Kbh95u1XKLXwYofLBYtQHbln4tp5tj2P4AS7oUZDniXE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504050; c=relaxed/simple; bh=r3nd9vghu4s4uBkV+N+7fBpy1dkBUZqwpUsMh9tj/Uw=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=fMxo2pE52rA9FZFRAcQYg9STwwn6j3gBGmFoVQUqECLZuCXxihl1kj4QvpKGZPbiLyUgCvGrsafYyoihGpH3/B2KVNifUPWvdOOKDTVZoDdxxnk7bW/hU9iqnij7Bzd/TOoi1iHQxq1Ldq35G3/q/Hwts/NAigTk6/57U59FE8g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=VQsAJJKu; arc=none smtp.client-ip=209.85.128.41 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="VQsAJJKu" Received: by mail-wm1-f41.google.com with SMTP id 5b1f17b1804b1-43cfe574976so7692675e9.1 for ; Thu, 24 Apr 2025 07:14:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504045; x=1746108845; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=XrxJtuJG442F2RUwTFuDHGt6qjNm8WZW6h3yhfXqCrI=; b=VQsAJJKu0h2qk9A9xzNKMZK6r7K8M27J7ZmUtQbNAP1wZYFB/nBWgwA/ggE5n+Qd0I TAXhGrBlm1ThQgmwgmbNAZv+Yo9Ay/NQMbTr8OY4Nptw/Bewli5OGnagneHTc/Wjotm9 AJn7JH7o4LC3fD6XeYIMbpPwyxx8UCMxwPcHCgpQXXD+BNlzCmkt/71/U4tNSiUyG4xk Rll0CsbyHj5CQEjj6X4hZHyoFYjVUJzJBfOjvNIasG3FXHLXEcGxcewR++AvyQPTVTDP qHEIzbgBpFkjq7gi12n50ADGWVrJWD9RwThMWmj+qfGF3o2bPCldXWF5lBMrjjtVuQYm QUgg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504045; x=1746108845; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=XrxJtuJG442F2RUwTFuDHGt6qjNm8WZW6h3yhfXqCrI=; b=Dubh1QPOaSup9NiK1A8EIkc7nYqwegwOkfpnn7kZm7r95GAyno9l/Enx4ogOcsY/iv CYmFnc1MxkuSbKD3LJdtafzn62/V2nZ74BI2GN/1cpOfkgZMs5x79VIxmmIGUAGQrn8A r/mo7unoAG4Rs57nGmFGfszgoDJV6kfU3Q7Ys4g+OJb2nhPwCnMimNylLgEpmGL/GSTn zPC23G2YD2WXhNZPyYJ2JYvkNbpjH2wYUrfsgwJGey6u8a7YzEQDwpiCcbkMBU/uE4y+ 7+CZnMoOLjExEoQl4GYz7fZkBpYwVp/zJcgSh7yctPJQLdru9cnTEoJgUF8DrTfUouOF Wg2g== X-Gm-Message-State: AOJu0YxuTXWoNwQY1UUxCqKQ1sy5sWXFC/7rm/aI3hiA4J0qNMXJi15j rcf7ISnLvXvUY5RP/s8ZZQbdsr8Pb7BCUBsDz2C/fMsrkSVUkj1HVQ/G5oIbqZuTdBbK8/t+ob7 k X-Gm-Gg: ASbGncszm2JUyNiRmmLpygm+ExJj8LdeecvGl5RiS3ZDAaawfqCgwUt1Vf9lyQIXGq+ iJPW1EscUmPHLzIWzY4JkLtrdQ7jDVGEzS6MfSTew7zXq/hMfcXsIW3e0aSXdE2YIoHKYx9HJUM 84ppR9K4PgLc6wvMqsbygBFv5rqdj72l6EzDReCBBAxIh1mtkLE4AE1PdG+2QQJgpFMsuVLlH7+ j5sjJebSjccc7JPjqqsQJxz0WOoAnkLOSRvE6KsnhEIYtCtdmc6RH8vp92qYYp/0LrxqadVA/mc 28IHHHkXpyYxtBbOw3po0tfrbK/00o4FVu3OnoJkBXCxiLM+qW3vWQ5hKyMNbRKY3CSRZmSP+nJ coZ6w+GMyY4eutH2E X-Google-Smtp-Source: AGHT+IF2yQw22hjehK9elALveIYUJReJwgCD95mE9PZgbazIybojHN298a9GDoCEMK47GVkaXYd77A== X-Received: by 2002:a05:600c:4e48:b0:439:8c80:6af4 with SMTP id 5b1f17b1804b1-4409bd23f55mr24505185e9.19.1745504045045; Thu, 24 Apr 2025 07:14:05 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.14.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:14:04 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt , Elliot Berman Subject: [RFC PATCH 15/34] gunyah: Add VM lifecycle RPC Date: Thu, 24 Apr 2025 15:13:22 +0100 Message-Id: <20250424141341.841734-16-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Elliot Berman Add Gunyah Resource Manager RPC interfaces to launch an unauthenticated virtual machine. Signed-off-by: Elliot Berman Reviewed-by: Srivatsa Vaddagiri Signed-off-by: Karim Manaouil --- drivers/virt/gunyah/Makefile | 2 +- drivers/virt/gunyah/rsc_mgr_rpc.c | 252 ++++++++++++++++++++++++++++++ include/linux/gunyah_rsc_mgr.h | 79 +++++++++- 3 files changed, 331 insertions(+), 2 deletions(-) create mode 100644 drivers/virt/gunyah/rsc_mgr_rpc.c diff --git a/drivers/virt/gunyah/Makefile b/drivers/virt/gunyah/Makefile index c2308389f551..b1bdf3e84155 100644 --- a/drivers/virt/gunyah/Makefile +++ b/drivers/virt/gunyah/Makefile @@ -1,5 +1,5 @@ # SPDX-License-Identifier: GPL-2.0 =20 -gunyah_rsc_mgr-y +=3D rsc_mgr.o +gunyah_rsc_mgr-y +=3D rsc_mgr.o rsc_mgr_rpc.o =20 obj-$(CONFIG_GUNYAH) +=3D gunyah.o gunyah_rsc_mgr.o diff --git a/drivers/virt/gunyah/rsc_mgr_rpc.c b/drivers/virt/gunyah/rsc_mg= r_rpc.c new file mode 100644 index 000000000000..626ad2565548 --- /dev/null +++ b/drivers/virt/gunyah/rsc_mgr_rpc.c @@ -0,0 +1,252 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights res= erved. + */ + +#include + +#include + +/* Message IDs: VM Management */ +/* clang-format off */ +#define GUNYAH_RM_RPC_VM_ALLOC_VMID 0x56000001 +#define GUNYAH_RM_RPC_VM_DEALLOC_VMID 0x56000002 +#define GUNYAH_RM_RPC_VM_START 0x56000004 +#define GUNYAH_RM_RPC_VM_STOP 0x56000005 +#define GUNYAH_RM_RPC_VM_RESET 0x56000006 +#define GUNYAH_RM_RPC_VM_CONFIG_IMAGE 0x56000009 +#define GUNYAH_RM_RPC_VM_INIT 0x5600000B +#define GUNYAH_RM_RPC_VM_GET_HYP_RESOURCES 0x56000020 +#define GUNYAH_RM_RPC_VM_GET_VMID 0x56000024 +#define GUNYAH_RM_RPC_VM_SET_BOOT_CONTEXT 0x56000031 +#define GUNYAH_RM_RPC_VM_SET_DEMAND_PAGING 0x56000033 +#define GUNYAH_RM_RPC_VM_SET_ADDRESS_LAYOUT 0x56000034 +/* clang-format on */ + +struct gunyah_rm_vm_common_vmid_req { + __le16 vmid; + __le16 _padding; +} __packed; + +/* Call: VM_ALLOC */ +struct gunyah_rm_vm_alloc_vmid_resp { + __le16 vmid; + __le16 _padding; +} __packed; + +/* Call: VM_STOP */ +#define GUNYAH_RM_VM_STOP_FLAG_FORCE_STOP BIT(0) + +#define GUNYAH_RM_VM_STOP_REASON_FORCE_STOP 3 + +struct gunyah_rm_vm_stop_req { + __le16 vmid; + u8 flags; + u8 _padding; + __le32 stop_reason; +} __packed; + +/* Call: VM_CONFIG_IMAGE */ +struct gunyah_rm_vm_config_image_req { + __le16 vmid; + __le16 auth_mech; + __le32 mem_handle; + __le64 image_offset; + __le64 image_size; + __le64 dtb_offset; + __le64 dtb_size; +} __packed; + +/* + * Several RM calls take only a VMID as a parameter and give only standard + * response back. Deduplicate boilerplate code by using this common call. + */ +static int gunyah_rm_common_vmid_call(struct gunyah_rm *rm, u32 message_id, + u16 vmid) +{ + struct gunyah_rm_vm_common_vmid_req req_payload =3D { + .vmid =3D cpu_to_le16(vmid), + }; + + return gunyah_rm_call(rm, message_id, &req_payload, sizeof(req_payload), + NULL, NULL); +} + +/** + * gunyah_rm_alloc_vmid() - Allocate a new VM in Gunyah. Returns the VM id= entifier. + * @rm: Handle to a Gunyah resource manager + * @vmid: Use 0 to dynamically allocate a VM. A reserved VMID can be suppl= ied + * to request allocation of a platform-defined VM. + * + * Return: the allocated VMID or negative value on error + */ +int gunyah_rm_alloc_vmid(struct gunyah_rm *rm, u16 vmid) +{ + struct gunyah_rm_vm_common_vmid_req req_payload =3D { + .vmid =3D cpu_to_le16(vmid), + }; + struct gunyah_rm_vm_alloc_vmid_resp *resp_payload; + size_t resp_size; + void *resp; + int ret; + + ret =3D gunyah_rm_call(rm, GUNYAH_RM_RPC_VM_ALLOC_VMID, &req_payload, + sizeof(req_payload), &resp, &resp_size); + if (ret) + return ret; + + if (!vmid) { + resp_payload =3D resp; + ret =3D le16_to_cpu(resp_payload->vmid); + kfree(resp); + } + + return ret; +} +ALLOW_ERROR_INJECTION(gunyah_rm_alloc_vmid, ERRNO); + +/** + * gunyah_rm_dealloc_vmid() - Dispose of a VMID + * @rm: Handle to a Gunyah resource manager + * @vmid: VM identifier allocated with gunyah_rm_alloc_vmid + */ +int gunyah_rm_dealloc_vmid(struct gunyah_rm *rm, u16 vmid) +{ + return gunyah_rm_common_vmid_call(rm, GUNYAH_RM_RPC_VM_DEALLOC_VMID, + vmid); +} +ALLOW_ERROR_INJECTION(gunyah_rm_dealloc_vmid, ERRNO); + +/** + * gunyah_rm_vm_reset() - Reset a VM's resources + * @rm: Handle to a Gunyah resource manager + * @vmid: VM identifier allocated with gunyah_rm_alloc_vmid + * + * As part of tearing down the VM, request RM to clean up all the VM resou= rces + * associated with the VM. Only after this, Linux can clean up all the + * references it maintains to resources. + */ +int gunyah_rm_vm_reset(struct gunyah_rm *rm, u16 vmid) +{ + return gunyah_rm_common_vmid_call(rm, GUNYAH_RM_RPC_VM_RESET, vmid); +} +ALLOW_ERROR_INJECTION(gunyah_rm_vm_reset, ERRNO); + +/** + * gunyah_rm_vm_start() - Move a VM into "ready to run" state + * @rm: Handle to a Gunyah resource manager + * @vmid: VM identifier allocated with gunyah_rm_alloc_vmid + * + * On VMs which use proxy scheduling, vcpu_run is needed to actually run t= he VM. + * On VMs which use Gunyah's scheduling, the vCPUs start executing in acco= rdance with Gunyah + * scheduling policies. + */ +int gunyah_rm_vm_start(struct gunyah_rm *rm, u16 vmid) +{ + return gunyah_rm_common_vmid_call(rm, GUNYAH_RM_RPC_VM_START, vmid); +} +ALLOW_ERROR_INJECTION(gunyah_rm_vm_start, ERRNO); + +/** + * gunyah_rm_vm_stop() - Send a request to Resource Manager VM to forcibly= stop a VM. + * @rm: Handle to a Gunyah resource manager + * @vmid: VM identifier allocated with gunyah_rm_alloc_vmid + */ +int gunyah_rm_vm_stop(struct gunyah_rm *rm, u16 vmid) +{ + struct gunyah_rm_vm_stop_req req_payload =3D { + .vmid =3D cpu_to_le16(vmid), + .flags =3D GUNYAH_RM_VM_STOP_FLAG_FORCE_STOP, + .stop_reason =3D cpu_to_le32(GUNYAH_RM_VM_STOP_REASON_FORCE_STOP), + }; + + return gunyah_rm_call(rm, GUNYAH_RM_RPC_VM_STOP, &req_payload, + sizeof(req_payload), NULL, NULL); +} +ALLOW_ERROR_INJECTION(gunyah_rm_vm_stop, ERRNO); + +/** + * gunyah_rm_vm_configure() - Prepare a VM to start and provide the common + * configuration needed by RM to configure a VM + * @rm: Handle to a Gunyah resource manager + * @vmid: VM identifier allocated with gunyah_rm_alloc_vmid + * @auth_mechanism: Authentication mechanism used by resource manager to v= erify + * the virtual machine + * @mem_handle: Handle to a previously shared memparcel that contains all = parts + * of the VM image subject to authentication. + * @image_offset: Start address of VM image, relative to the start of memp= arcel + * @image_size: Size of the VM image + * @dtb_offset: Start address of the devicetree binary with VM configurati= on, + * relative to start of memparcel. + * @dtb_size: Maximum size of devicetree binary. + */ +int gunyah_rm_vm_configure(struct gunyah_rm *rm, u16 vmid, + enum gunyah_rm_vm_auth_mechanism auth_mechanism, + u32 mem_handle, u64 image_offset, u64 image_size, + u64 dtb_offset, u64 dtb_size) +{ + struct gunyah_rm_vm_config_image_req req_payload =3D { + .vmid =3D cpu_to_le16(vmid), + .auth_mech =3D cpu_to_le16(auth_mechanism), + .mem_handle =3D cpu_to_le32(mem_handle), + .image_offset =3D cpu_to_le64(image_offset), + .image_size =3D cpu_to_le64(image_size), + .dtb_offset =3D cpu_to_le64(dtb_offset), + .dtb_size =3D cpu_to_le64(dtb_size), + }; + + return gunyah_rm_call(rm, GUNYAH_RM_RPC_VM_CONFIG_IMAGE, &req_payload, + sizeof(req_payload), NULL, NULL); +} +ALLOW_ERROR_INJECTION(gunyah_rm_vm_configure, ERRNO); + +/** + * gunyah_rm_vm_init() - Move the VM to initialized state. + * @rm: Handle to a Gunyah resource manager + * @vmid: VM identifier + * + * RM will allocate needed resources for the VM. + */ +int gunyah_rm_vm_init(struct gunyah_rm *rm, u16 vmid) +{ + return gunyah_rm_common_vmid_call(rm, GUNYAH_RM_RPC_VM_INIT, vmid); +} +ALLOW_ERROR_INJECTION(gunyah_rm_vm_init, ERRNO); + +/** + * gunyah_rm_get_hyp_resources() - Retrieve hypervisor resources (capabili= ties) associated with a VM + * @rm: Handle to a Gunyah resource manager + * @vmid: VMID of the other VM to get the resources of + * @resources: Set by gunyah_rm_get_hyp_resources and contains the returne= d hypervisor resources. + * Caller must free the resources pointer if successful. + */ +int gunyah_rm_get_hyp_resources(struct gunyah_rm *rm, u16 vmid, + struct gunyah_rm_hyp_resources **resources) +{ + struct gunyah_rm_vm_common_vmid_req req_payload =3D { + .vmid =3D cpu_to_le16(vmid), + }; + struct gunyah_rm_hyp_resources *resp; + size_t resp_size; + int ret; + + ret =3D gunyah_rm_call(rm, GUNYAH_RM_RPC_VM_GET_HYP_RESOURCES, + &req_payload, sizeof(req_payload), (void **)&resp, + &resp_size); + if (ret) + return ret; + + if (!resp_size) + return -EBADMSG; + + if (resp_size < struct_size(resp, entries, 0) || + resp_size !=3D + struct_size(resp, entries, le32_to_cpu(resp->n_entries))) { + kfree(resp); + return -EBADMSG; + } + + *resources =3D resp; + return 0; +} +ALLOW_ERROR_INJECTION(gunyah_rm_get_hyp_resources, ERRNO); diff --git a/include/linux/gunyah_rsc_mgr.h b/include/linux/gunyah_rsc_mgr.h index 87e919cc1e28..294e847c27ed 100644 --- a/include/linux/gunyah_rsc_mgr.h +++ b/include/linux/gunyah_rsc_mgr.h @@ -19,9 +19,86 @@ int gunyah_rm_notifier_register(struct gunyah_rm *rm, int gunyah_rm_notifier_unregister(struct gunyah_rm *rm, struct notifier_block *nb); =20 +struct gunyah_rm_vm_exited_payload { + __le16 vmid; + __le16 exit_type; + __le32 exit_reason_size; + u8 exit_reason[]; +} __packed; + +enum gunyah_rm_notification_id { + /* clang-format off */ + GUNYAH_RM_NOTIFICATION_VM_EXITED =3D 0x56100001, + GUNYAH_RM_NOTIFICATION_VM_STATUS =3D 0x56100008, + /* clang-format on */ +}; + +enum gunyah_rm_vm_status { + /* clang-format off */ + GUNYAH_RM_VM_STATUS_NO_STATE =3D 0, + GUNYAH_RM_VM_STATUS_INIT =3D 1, + GUNYAH_RM_VM_STATUS_READY =3D 2, + GUNYAH_RM_VM_STATUS_RUNNING =3D 3, + GUNYAH_RM_VM_STATUS_PAUSED =3D 4, + GUNYAH_RM_VM_STATUS_LOAD =3D 5, + GUNYAH_RM_VM_STATUS_AUTH =3D 6, + GUNYAH_RM_VM_STATUS_INIT_FAILED =3D 8, + GUNYAH_RM_VM_STATUS_EXITED =3D 9, + GUNYAH_RM_VM_STATUS_RESETTING =3D 10, + GUNYAH_RM_VM_STATUS_RESET =3D 11, + /* clang-format on */ +}; + +struct gunyah_rm_vm_status_payload { + __le16 vmid; + u16 reserved; + u8 vm_status; + u8 os_status; + __le16 app_status; +} __packed; + +int gunyah_rm_alloc_vmid(struct gunyah_rm *rm, u16 vmid); +int gunyah_rm_dealloc_vmid(struct gunyah_rm *rm, u16 vmid); +int gunyah_rm_vm_reset(struct gunyah_rm *rm, u16 vmid); +int gunyah_rm_vm_start(struct gunyah_rm *rm, u16 vmid); +int gunyah_rm_vm_stop(struct gunyah_rm *rm, u16 vmid); + +enum gunyah_rm_vm_auth_mechanism { + /* clang-format off */ + GUNYAH_RM_VM_AUTH_NONE =3D 0, + GUNYAH_RM_VM_AUTH_QCOM_PIL_ELF =3D 1, + GUNYAH_RM_VM_AUTH_QCOM_ANDROID_PVM =3D 2, + /* clang-format on */ +}; + +int gunyah_rm_vm_configure(struct gunyah_rm *rm, u16 vmid, + enum gunyah_rm_vm_auth_mechanism auth_mechanism, + u32 mem_handle, u64 image_offset, u64 image_size, + u64 dtb_offset, u64 dtb_size); +int gunyah_rm_vm_init(struct gunyah_rm *rm, u16 vmid); + +struct gunyah_rm_hyp_resource { + u8 type; + u8 reserved; + __le16 partner_vmid; + __le32 resource_handle; + __le32 resource_label; + __le64 cap_id; + __le32 virq_handle; + __le32 virq; + __le64 base; + __le64 size; +} __packed; + +struct gunyah_rm_hyp_resources { + __le32 n_entries; + struct gunyah_rm_hyp_resource entries[]; +} __packed; + +int gunyah_rm_get_hyp_resources(struct gunyah_rm *rm, u16 vmid, + struct gunyah_rm_hyp_resources **resources); =20 int gunyah_rm_call(struct gunyah_rm *rsc_mgr, u32 message_id, const void *req_buf, size_t req_buf_size, void **resp_buf, size_t *resp_buf_size); - #endif --=20 2.39.5 From nobody Thu Dec 18 00:34:59 2025 Received: from mail-wr1-f45.google.com (mail-wr1-f45.google.com [209.85.221.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6989228BA9E for ; Thu, 24 Apr 2025 14:14:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.45 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504052; cv=none; b=OQ8fyeIaK534w//vxcjgDVOew6TvQk6bm2VKKLd+O8ELGLFGzXLNhd88lS8rsd+0VdkwMVYGRs4CeYvohONr3xU9l2fbD/WMCw7ngYivomSfFJCp1dh9dbvdAuh9mc6rKA9iUIXlbfhui+6TVA5McN0/UBh6pV/t3jsJ2hww+YE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504052; c=relaxed/simple; bh=o5jUxv0nZ8uJl1blXMpoQdyQe0eMlKyDBmLFSdRHE2s=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=BXRhMx3+qvhgDPUt5KqlH5vqc2xb26PBHZCmwrSdHDxsxD0EvRoZK2rbpgbzmu9JiSt4sq9fNIlPO3sCea/vGmZxr00Xp6r0ac4DC01GNCChd5fEg8uoQ0F+KX73cSozL0SR4bu7ynQR/OQOOxD0xWji5t94FurJHDV3Tb2sBQI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=b3AS8B4F; arc=none smtp.client-ip=209.85.221.45 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="b3AS8B4F" Received: by mail-wr1-f45.google.com with SMTP id ffacd0b85a97d-39ee651e419so654024f8f.3 for ; Thu, 24 Apr 2025 07:14:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504046; x=1746108846; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=UoVyA4T/XR+PrBla9t1/dUC6ibbVMr9f/wkTq4NWm5c=; b=b3AS8B4FhCvDA56c2cSPpxuBmsrjqdga0bD6lbyS8PvkIA8cf7zlRq6z9fzmPupRgA /YnSM9O7e/PrryqZAq6/G3a4y/TkQBZFSIAc02ai8aEvNP4AvSxYq1D4Sl5VPOHgZGYc 2LrFTQeSnpc1eMY5/9YihDnNsyuSakGuC/dtwQxrU8BZx3iK+9FKcx8IIrQM2z2l6p4O MD8VKCRcCSBpAlWE6T371f7EkZZpH/XyyM6Wv36SIm7EmyZeQweTmv0Jw0Q+kAzlnh1y eju6ObBdHz0TqpVVqjmihnPbh1sJDmyc6gvm/YBsXQr9tZC87D4B2YdvUpou/brTXAIh GK6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504046; x=1746108846; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=UoVyA4T/XR+PrBla9t1/dUC6ibbVMr9f/wkTq4NWm5c=; b=LorF7VT2aPWpNRTihZBqsvLdVaqYq4qGI4YhiCbNYTkL+GitmUbLCnfVK4+8JfKDId eVyZbsiVtTd6/THiZLEJr3yvNgUBY5xDaGsHFeGm9usk72BfbQ3g7JQtTIHYgkun/AfN lYQLg+g8w9v63aLLMROpjT9mFNHX83gT0uCHrFpoeeoNV+hUpJVk8YKfH4kwG1JlMKRp MhKV/Ewzw+RMgRfMDhCDZZ9+u4u6sxryF6ecH/SWqnDtIt4/s5AWq/XXC/dhKyXYOgjX +iIEp83Xkn3dzCwcc+avC8/ShTg0vdST3yoesnUGiypXnzKEUsmmZXdsERaTRRVKtZVd slCw== X-Gm-Message-State: AOJu0Yws2TVKrhxmL0RYDa+FHHQqwFIYJwyrUGiIBiDDu4XJJW108fxd VlEZZrxFyhzvKEcNC447wnkMlvgF2UDjpADZePFVEnG8KGY6DRoN6ZLP3sAiEQwlulhxMMncJFS q X-Gm-Gg: ASbGncu0fAxfPnmslA/4EYE/9SvBn5YHq1gPUweEGxnO97bQQpRUjXXYLN5gSeUCYDc MC4U3Axv1UBGnDg89lRiRMjk0pudd/svgFVAOGsb2w0e+yMB+Odja93WnijoPIRB7ahvaGkSw2Q Z6o/7Ha5IhqoIaVLMIZjJGOonjJh5A2mVBl9JY6uNNt1M4WbjzdQAA5FztSmYwIwkvch+ZVlp37 8XnUUAZz+GhXkuidE8lTtg/9BxEUJ/Hdtj50ek+Di8zbTlqAC9Fl4cGjfwWcury2JGaABSYL2J9 t66an6PCDDztGqmhnxlL2pSmxXpb+o8L4sPSclyf7boCiVjRlkJAh+1hqJe5TVPqUPbHogEjNTB F0OjDUCKwbAwiciZj X-Google-Smtp-Source: AGHT+IFaUaoW58kCoH3cGCkS9884/3gN3P0U2I1fqPQYARfS16RUhRZKrV74ekkuGWYubSRTquJKGQ== X-Received: by 2002:a5d:64ab:0:b0:391:2a9f:2fcb with SMTP id ffacd0b85a97d-3a06cfa8400mr2501557f8f.36.1745504046320; Thu, 24 Apr 2025 07:14:06 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.14.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:14:05 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt , Elliot Berman Subject: [RFC PATCH 16/34] gunyah: Add basic VM lifecycle management Date: Thu, 24 Apr 2025 15:13:23 +0100 Message-Id: <20250424141341.841734-17-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable This patch hooks up Gunyah virtual machine lifecycle management with the KVM backend by implementing the kvm_arch_alloc_vm(), kvm_arch_destroy_vm(), and kvm_arch_free_vm() hooks. The Gunyah VM management logic=E2=80=94VMID allocation, configuration, initialization, start/stop, teardown, and notifier handling=E2=80=94is base= d on the implementation introduced in [1], authored by Elliot Berman and Prakruthi Deepak Heragu. The original code added a special ioctl interface to support userspace initialization of guest VMs. This patch reuses the same logic, but ported to KVM, allowing to use KVM's ioctl interface to create Gunyah-based guests. [1] Commit: 532788ce71c9 ("gunyah: vm_mgr: Add VM start/stop") Link: https://lore.kernel.org/lkml/20240222-gunyah-v17-10-1e9da6763d38@= quicinc.com/ Co-developed-by: Elliot Berman Co-developed-by: Prakruthi Deepak Heragu Signed-off-by: Karim Manaouil --- arch/arm64/include/asm/kvm_host.h | 5 + arch/arm64/kvm/gunyah.c | 196 ++++++++++++++++++++++++++++-- drivers/virt/gunyah/rsc_mgr_rpc.c | 2 +- include/linux/gunyah.h | 32 +++++ 4 files changed, 227 insertions(+), 8 deletions(-) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm= _host.h index 9c8e173fc9c1..53358d3f5fa8 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -1591,4 +1591,9 @@ void kvm_set_vm_id_reg(struct kvm *kvm, u32 reg, u64 = val); #define kvm_has_s1poe(k) \ (kvm_has_feat((k), ID_AA64MMFR3_EL1, S1POE, IMP)) =20 +#ifndef CONFIG_KVM_ARM +#define __KVM_HAVE_ARCH_VM_FREE +void kvm_arch_free_vm(struct kvm *kvm); +#endif + #endif /* __ARM64_KVM_HOST_H__ */ diff --git a/arch/arm64/kvm/gunyah.c b/arch/arm64/kvm/gunyah.c index 9c37ab20d7e2..a3c29ae985c9 100644 --- a/arch/arm64/kvm/gunyah.c +++ b/arch/arm64/kvm/gunyah.c @@ -13,6 +13,12 @@ #include #include =20 +#include +#include + +#undef pr_fmt +#define pr_fmt(fmt) "gunyah: " fmt + static enum kvm_mode kvm_mode =3D KVM_MODE_DEFAULT; =20 enum kvm_mode kvm_get_mode(void) @@ -338,12 +344,6 @@ void kvm_arch_create_vm_debugfs(struct kvm *kvm) { } =20 -void kvm_arch_destroy_vm(struct kvm *kvm) -{ - kvm_destroy_vcpus(kvm); - return; -} - long kvm_arch_dev_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg) { @@ -788,7 +788,189 @@ int kvm_cpu_has_pending_timer(struct kvm_vcpu *vcpu) return -EINVAL; } =20 +static int gunyah_vm_rm_notification_status(struct gunyah_vm *ghvm, void *= data) +{ + struct gunyah_rm_vm_status_payload *payload =3D data; + + if (le16_to_cpu(payload->vmid) !=3D ghvm->vmid) + return NOTIFY_OK; + + /* All other state transitions are synchronous to a corresponding RM call= */ + if (payload->vm_status =3D=3D GUNYAH_RM_VM_STATUS_RESET) { + down_write(&ghvm->status_lock); + ghvm->vm_status =3D payload->vm_status; + up_write(&ghvm->status_lock); + wake_up(&ghvm->vm_status_wait); + } + + return NOTIFY_DONE; +} + +static int gunyah_vm_rm_notification_exited(struct gunyah_vm *ghvm, void *= data) +{ + struct gunyah_rm_vm_exited_payload *payload =3D data; + + if (le16_to_cpu(payload->vmid) !=3D ghvm->vmid) + return NOTIFY_OK; + + down_write(&ghvm->status_lock); + ghvm->vm_status =3D GUNYAH_RM_VM_STATUS_EXITED; + up_write(&ghvm->status_lock); + wake_up(&ghvm->vm_status_wait); + + return NOTIFY_DONE; +} + +static int gunyah_vm_rm_notification(struct notifier_block *nb, + unsigned long action, void *data) +{ + struct gunyah_vm *ghvm =3D container_of(nb, struct gunyah_vm, nb); + + switch (action) { + case GUNYAH_RM_NOTIFICATION_VM_STATUS: + return gunyah_vm_rm_notification_status(ghvm, data); + case GUNYAH_RM_NOTIFICATION_VM_EXITED: + return gunyah_vm_rm_notification_exited(ghvm, data); + default: + return NOTIFY_OK; + } +} + +static void gunyah_vm_stop(struct gunyah_vm *ghvm) +{ + int ret; + + if (ghvm->vm_status =3D=3D GUNYAH_RM_VM_STATUS_RUNNING) { + ret =3D gunyah_rm_vm_stop(ghvm->rm, ghvm->vmid); + if (ret) + pr_warn("Failed to stop VM: %d\n", ret); + } + + wait_event(ghvm->vm_status_wait, + ghvm->vm_status !=3D GUNYAH_RM_VM_STATUS_RUNNING); +} + +static int gunyah_vm_start(struct gunyah_vm *ghvm) +{ + int ret; + + down_write(&ghvm->status_lock); + if (ghvm->vm_status !=3D GUNYAH_RM_VM_STATUS_NO_STATE) { + up_write(&ghvm->status_lock); + return 0; + } + + ghvm->nb.notifier_call =3D gunyah_vm_rm_notification; + ret =3D gunyah_rm_notifier_register(ghvm->rm, &ghvm->nb); + if (ret) + goto err; + + ret =3D gunyah_rm_alloc_vmid(ghvm->rm, 0); + if (ret < 0) { + gunyah_rm_notifier_unregister(ghvm->rm, &ghvm->nb); + goto err; + } + ghvm->vmid =3D ret; + ghvm->vm_status =3D GUNYAH_RM_VM_STATUS_LOAD; + + ret =3D gunyah_rm_vm_configure(ghvm->rm, ghvm->vmid, ghvm->auth, 0, 0, 0,= 0, 0); + if (ret) { + pr_warn("Failed to configure VM: %d\n", ret); + goto err; + } + + ret =3D gunyah_rm_vm_init(ghvm->rm, ghvm->vmid); + if (ret) { + ghvm->vm_status =3D GUNYAH_RM_VM_STATUS_INIT_FAILED; + pr_warn("Failed to initialize VM: %d\n", ret); + goto err; + } + ghvm->vm_status =3D GUNYAH_RM_VM_STATUS_READY; + + ret =3D gunyah_rm_vm_start(ghvm->rm, ghvm->vmid); + if (ret) { + pr_warn("Failed to start VM: %d\n", ret); + goto err; + } + + ghvm->vm_status =3D GUNYAH_RM_VM_STATUS_RUNNING; + up_write(&ghvm->status_lock); + return 0; +err: + up_write(&ghvm->status_lock); + return ret; +} + +static struct gunyah_vm *gunyah_vm_alloc(struct gunyah_rm *rm) +{ + struct gunyah_vm *ghvm; + + ghvm =3D kzalloc(sizeof(*ghvm), GFP_KERNEL); + if (!ghvm) + return ERR_PTR(-ENOMEM); + + ghvm->vmid =3D GUNYAH_VMID_INVAL; + ghvm->rm =3D rm; + + init_rwsem(&ghvm->status_lock); + init_waitqueue_head(&ghvm->vm_status_wait); + ghvm->vm_status =3D GUNYAH_RM_VM_STATUS_NO_STATE; + + return ghvm; +} + +static void gunyah_destroy_vm(struct gunyah_vm *ghvm) +{ + int ret; + + /** + * We might race with a VM exit notification, but that's ok: + * gh_rm_vm_stop() will just return right away. + */ + if (ghvm->vm_status =3D=3D GUNYAH_RM_VM_STATUS_RUNNING) + gunyah_vm_stop(ghvm); + + if (ghvm->vm_status =3D=3D GUNYAH_RM_VM_STATUS_EXITED || + ghvm->vm_status =3D=3D GUNYAH_RM_VM_STATUS_READY || + ghvm->vm_status =3D=3D GUNYAH_RM_VM_STATUS_INIT_FAILED) { + ret =3D gunyah_rm_vm_reset(ghvm->rm, ghvm->vmid); + if (!ret) + wait_event(ghvm->vm_status_wait, + ghvm->vm_status =3D=3D GUNYAH_RM_VM_STATUS_RESET); + else + pr_warn("Failed to reset the vm: %d\n", ret); + } + + if (ghvm->vm_status > GUNYAH_RM_VM_STATUS_NO_STATE) { + gunyah_rm_notifier_unregister(ghvm->rm, &ghvm->nb); + ret =3D gunyah_rm_dealloc_vmid(ghvm->rm, ghvm->vmid); + if (ret) + pr_warn("Failed to deallocate vmid: %d\n", ret); + } +} + struct kvm *kvm_arch_alloc_vm(void) { - return NULL; + struct gunyah_vm *ghvm; + + ghvm =3D gunyah_vm_alloc(gunyah_rm); + if (IS_ERR(ghvm)) + return NULL; + + return &ghvm->kvm; +} + +void kvm_arch_destroy_vm(struct kvm *kvm) +{ + struct gunyah_vm *ghvm =3D kvm_to_gunyah(kvm); + + kvm_destroy_vcpus(kvm); + gunyah_destroy_vm(ghvm); +} + +void kvm_arch_free_vm(struct kvm *kvm) +{ + struct gunyah_vm *ghvm =3D kvm_to_gunyah(kvm); + + kfree(ghvm); } diff --git a/drivers/virt/gunyah/rsc_mgr_rpc.c b/drivers/virt/gunyah/rsc_mg= r_rpc.c index 626ad2565548..936592177ddb 100644 --- a/drivers/virt/gunyah/rsc_mgr_rpc.c +++ b/drivers/virt/gunyah/rsc_mgr_rpc.c @@ -3,8 +3,8 @@ * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights res= erved. */ =20 +#include #include - #include =20 /* Message IDs: VM Management */ diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h index acd70f982425..1f4389eb21fb 100644 --- a/include/linux/gunyah.h +++ b/include/linux/gunyah.h @@ -11,6 +11,12 @@ #include #include #include +#include + +#include + +#define kvm_to_gunyah(kvm_ptr) \ + container_of(kvm_ptr, struct gunyah_vm, kvm) =20 /* Matches resource manager's resource types for VM_GET_HYP_RESOURCES RPC = */ enum gunyah_resource_type { @@ -31,6 +37,32 @@ struct gunyah_resource { unsigned int irq; }; =20 +/** + * struct gunyah_vm - Main representation of a Gunyah Virtual machine + memory shared with the guest. + * @vmid: Gunyah's VMID for this virtual machine + * @kvm: kvm instance for this VM + * @rm: Pointer to the resource manager struct to make RM calls + * @nb: Notifier block for RM notifications + * @vm_status: Current state of the VM, as last reported by RM + * @vm_status_wait: Wait queue for status @vm_status changes + * @status_lock: Serializing state transitions + * @auth: Authentication mechanism to be used by resource manager when + * launching the VM + */ +struct gunyah_vm { + u16 vmid; + struct kvm kvm; + struct gunyah_rm *rm; + + struct notifier_block nb; + enum gunyah_rm_vm_status vm_status; + wait_queue_head_t vm_status_wait; + struct rw_semaphore status_lock; + + enum gunyah_rm_vm_auth_mechanism auth; +}; + /*************************************************************************= *****/ /* Common arch-independent definitions for Gunyah hypercalls = */ #define GUNYAH_CAPID_INVAL U64_MAX --=20 2.39.5 From nobody Thu Dec 18 00:34:59 2025 Received: from mail-wr1-f48.google.com (mail-wr1-f48.google.com [209.85.221.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6B8B127A919 for ; Thu, 24 Apr 2025 14:14:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.48 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504053; cv=none; b=hiMDixOxaby/xif2DeoVG2VchvZT1FORpHhEx6HWxo7ru0D9txcW0gB9giC/FsB75tRRBtDpwp+pOAZHpnwW9YW5GxfCk/y3v0Di+VqUkPcJpUyajNAjQe51JtT13q4zpmxy7ifJNNgxAGdg23amCbdPg2aDJ6rcBjRQkujwasw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504053; c=relaxed/simple; bh=S+aezV66gXf4D+s9yS2EKNbYmZv94XHap0Nf+ZRRUHk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Qq+oLS4Mq9AdCdZnIbtyyCTm5pwzS5HOTXx7J835U7tNewBUEn678j0mTRW4oky+IiDrTbg6+cE1c1XkdXW4B0PoaoizbhcZDlLPy8RYpUz53BvaK6jy+oKU1yBKQrMA+1RqfgqW+NOs/OtNw7N3KH8OBkHbJpuwwS/WO+yHpD4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=ZVrYkVnP; arc=none smtp.client-ip=209.85.221.48 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="ZVrYkVnP" Received: by mail-wr1-f48.google.com with SMTP id ffacd0b85a97d-3913d129c1aso753472f8f.0 for ; Thu, 24 Apr 2025 07:14:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504048; x=1746108848; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=98GtKQowyHwUXJ3/oi0BeOIZGrWAP4CATPU0kL4zC8c=; b=ZVrYkVnPnl9Ys6EiWjO0nW/ZKL9GZPhxZM8Jdq4yhpWJfZHOL7zyEOxMrBZQgE4ywb S/dHcmgPS1jkO5i8DXxJ+fn8JUsYf3LS8ffoJXnN9Obvw7bF4KK95uYdIdxYSk3FOog7 4gZdjcJDvoxbNYrVrTnD6TfC3gXwFj0Q7fKu28uJZMdpJQQ9WazFoPCxhnuMNwow6cLm SwXd2GQgnZH64avWP2TPUZEJ5EEpzYL2FwLsoeVkaOXNR0FrskyMcsvHbDAlTyEBhCCH fBNLGXr2xfE/z7LpOxCBiHFnskchBvSkkplqymZ6IXk/VkZsoT4gAZf1MYFXX32dFOmo V/hA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504048; x=1746108848; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=98GtKQowyHwUXJ3/oi0BeOIZGrWAP4CATPU0kL4zC8c=; b=YI09iL+EGmE6kIw/nftAcGmRbisRe3GdSpd52rE9SbGNRXweveiDStF5csNtF7JKJc btkJv5D3buAtsAPvcUNRv+g8EF8ePglVBIYwGP7Ijn0E0lvxZry0Oxj9K5y4R54N5wXC 7HU+hBbWTH709lFE1MPS7DuigrS3nrk3M4j2R9JFfJL2vcmg0dUMI9a3TIpohtFVokx+ K1Cf/B3DfdEEfwkkgPW4+Jp2HoPWyESQ6IgSL1Ccj2dvHzkgMAttSN1317+ljUy6MabS YE1IVvb63qEyQmEh0nYtzqcZ2ZvSeDUi31Z/eKJF6Te1UG4Bjxgnd8SG2U8dKojRQn2q RGUQ== X-Gm-Message-State: AOJu0YyagTMwalhC3zp8eWH83RoOIsFPN1QFraAMMrbrss5wE06IS2sl LUnBGrw6/hSN37G6Und+2sVvpRGkp04s166RszbkLu0qCgYuE+YJJjy46Af8J9REMR+gmuRjz/v F X-Gm-Gg: ASbGncs0KRckg0tj83wvpWexGKlbDR6y4F7YRrJ1ahWhJpO86gUBhY1V7Ls7IJ6PYCA r+s12CMnkIVb+bW7UboEmyjdOvOzqik7OKP2ywyktcRps4WniTHUeriwE24K026JyHWfcG3fQ2O 3T168/SQ3TgJnkJ9wrbciJ1zBLpkHDPFvxVJLKqNNZcKSRJlPV1U0RNRrvTeuNvuz5Y3vjyP+sO i2gHMUr+kIIap5uMmsbFZLs2VtbvPzjbUnYfoOjc/R19O1dXqAzEwUOU711TPB0Rpize/R0M8J1 L4yhlCq2a6d13NMUMqDZdaQVeLFAn2M1dR0foBA4GXllTk2al3w27usCcdDr7Qh27S6Ee3lbMv3 Nc8eBcgX72hgJ9eYE X-Google-Smtp-Source: AGHT+IFQAg02MXbpqUbddZDz1Nf5W/i1x6IwVsYQPoOcChQDsuxBB7Db2uzIfhJZTYe0HZGVw3TbTg== X-Received: by 2002:a05:6000:18ae:b0:39f:cd2:1fa1 with SMTP id ffacd0b85a97d-3a06d647026mr2922759f8f.3.1745504048036; Thu, 24 Apr 2025 07:14:08 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.14.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:14:07 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt , Elliot Berman Subject: [RFC PATCH 17/34] gunyah: Translate gh_rm_hyp_resource into gunyah_resource Date: Thu, 24 Apr 2025 15:13:24 +0100 Message-Id: <20250424141341.841734-18-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Elliot Berman When booting a Gunyah virtual machine, the host VM may gain capabilities to interact with resources for the guest virtual machine. Examples of such resources are vCPUs or message queues. To use those resources, we need to translate the RM response into a gunyah_resource structure which are useful to Linux drivers. Presently, Linux drivers need only to know the type of resource, the capability ID, and an interrupt. On ARM64 systems, the interrupt reported by Gunyah is the GIC interrupt ID number and always a SPI or extended SPI. Signed-off-by: Elliot Berman Reviewed-by: Srivatsa Vaddagiri Signed-off-by: Karim Manaouil --- arch/arm64/include/asm/gunyah.h | 35 +++++++++++++++++ drivers/virt/gunyah/rsc_mgr.c | 68 +++++++++++++++++++++++++++++++++ include/linux/gunyah.h | 2 + include/linux/gunyah_rsc_mgr.h | 4 ++ 4 files changed, 109 insertions(+) create mode 100644 arch/arm64/include/asm/gunyah.h diff --git a/arch/arm64/include/asm/gunyah.h b/arch/arm64/include/asm/gunya= h.h new file mode 100644 index 000000000000..29079d1a4df2 --- /dev/null +++ b/arch/arm64/include/asm/gunyah.h @@ -0,0 +1,35 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights res= erved. + */ +#ifndef _ASM_GUNYAH_H +#define _ASM_GUNYAH_H + +#include +#include + +static inline int arch_gunyah_fill_irq_fwspec_params(u32 virq, + struct irq_fwspec *fwspec) +{ + /* Assume that Gunyah gave us an SPI or ESPI; defensively check it */ + if (WARN(virq < 32, "Unexpected virq: %d\n", virq)) { + return -EINVAL; + } else if (virq <=3D 1019) { + fwspec->param_count =3D 3; + fwspec->param[0] =3D 0; /* GIC_SPI */ + fwspec->param[1] =3D virq - 32; /* virq 32 -> SPI 0 */ + fwspec->param[2] =3D IRQ_TYPE_EDGE_RISING; + } else if (WARN(virq < 4096, "Unexpected virq: %d\n", virq)) { + return -EINVAL; + } else if (virq < 5120) { + fwspec->param_count =3D 3; + fwspec->param[0] =3D 2; /* GIC_ESPI */ + fwspec->param[1] =3D virq - 4096; /* virq 4096 -> ESPI 0 */ + fwspec->param[2] =3D IRQ_TYPE_EDGE_RISING; + } else { + WARN(1, "Unexpected virq: %d\n", virq); + return -EINVAL; + } + return 0; +} +#endif diff --git a/drivers/virt/gunyah/rsc_mgr.c b/drivers/virt/gunyah/rsc_mgr.c index 75fc86887868..2d34b0ba98b2 100644 --- a/drivers/virt/gunyah/rsc_mgr.c +++ b/drivers/virt/gunyah/rsc_mgr.c @@ -9,8 +9,10 @@ #include #include #include +#include #include =20 +#include #include =20 /* clang-format off */ @@ -118,6 +120,7 @@ struct gunyah_rm_message { * @send_lock: synchronization to allow only one request to be sent at a t= ime * @send_ready: completed when we know Tx message queue can take more mess= ages * @nh: notifier chain for clients interested in RM notification messages + * @parent_fwnode: Parent IRQ fwnode to translate Gunyah hwirqs to Linux i= rqs */ struct gunyah_rm { struct device *dev; @@ -133,6 +136,8 @@ struct gunyah_rm { struct mutex send_lock; struct completion send_ready; struct blocking_notifier_head nh; + + struct fwnode_handle *parent_fwnode; }; =20 /* Global resource manager instance */ @@ -177,6 +182,53 @@ static inline int gunyah_rm_error_remap(enum gunyah_rm= _error rm_error) } } =20 +struct gunyah_resource * +gunyah_rm_alloc_resource(struct gunyah_rm *rm, + struct gunyah_rm_hyp_resource *hyp_resource) +{ + struct gunyah_resource *ghrsc; + int ret; + + ghrsc =3D kzalloc(sizeof(*ghrsc), GFP_KERNEL); + if (!ghrsc) + return NULL; + + ghrsc->type =3D hyp_resource->type; + ghrsc->capid =3D le64_to_cpu(hyp_resource->cap_id); + ghrsc->irq =3D IRQ_NOTCONNECTED; + ghrsc->rm_label =3D le32_to_cpu(hyp_resource->resource_label); + if (hyp_resource->virq) { + struct irq_fwspec fwspec; + + + fwspec.fwnode =3D rm->parent_fwnode; + ret =3D arch_gunyah_fill_irq_fwspec_params(le32_to_cpu(hyp_resource->vir= q), &fwspec); + if (ret) { + dev_err(rm->dev, + "Failed to translate interrupt for resource %d label: %d: %d\n", + ghrsc->type, ghrsc->rm_label, ret); + } + + ret =3D irq_create_fwspec_mapping(&fwspec); + if (ret < 0) { + dev_err(rm->dev, + "Failed to allocate interrupt for resource %d label: %d: %d\n", + ghrsc->type, ghrsc->rm_label, ret); + kfree(ghrsc); + return NULL; + } + ghrsc->irq =3D ret; + } + + return ghrsc; +} + +void gunyah_rm_free_resource(struct gunyah_resource *ghrsc) +{ + irq_dispose_mapping(ghrsc->irq); + kfree(ghrsc); +} + static int gunyah_rm_init_message_payload(struct gunyah_rm_message *messag= e, const void *msg, size_t hdr_size, size_t msg_size) @@ -676,6 +728,7 @@ static int gunyah_rm_probe_rx_msgq(struct gunyah_rm *rm, =20 static int gunyah_rm_probe(struct platform_device *pdev) { + struct device_node *parent_irq_node; int ret; =20 gunyah_rm =3D devm_kzalloc(&pdev->dev, sizeof(*gunyah_rm), GFP_KERNEL); @@ -695,6 +748,21 @@ static int gunyah_rm_probe(struct platform_device *pde= v) ret =3D gunyah_rm_probe_tx_msgq(gunyah_rm, pdev); if (ret) return ret; + + parent_irq_node =3D of_irq_find_parent(pdev->dev.of_node); + if (!parent_irq_node) { + dev_err(&pdev->dev, + "Failed to find interrupt parent of resource manager\n"); + return -ENODEV; + } + + gunyah_rm->parent_fwnode =3D of_node_to_fwnode(parent_irq_node); + if (!gunyah_rm->parent_fwnode) { + dev_err(&pdev->dev, + "Failed to find interrupt parent domain of resource manager\n"); + return -ENODEV; + } + /* assume RM is ready to receive messages from us */ complete(&gunyah_rm->send_ready); =20 diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h index 1f4389eb21fb..40ea21b17195 100644 --- a/include/linux/gunyah.h +++ b/include/linux/gunyah.h @@ -35,6 +35,8 @@ struct gunyah_resource { enum gunyah_resource_type type; u64 capid; unsigned int irq; + + u32 rm_label; }; =20 /** diff --git a/include/linux/gunyah_rsc_mgr.h b/include/linux/gunyah_rsc_mgr.h index 294e847c27ed..c0fe516d54a8 100644 --- a/include/linux/gunyah_rsc_mgr.h +++ b/include/linux/gunyah_rsc_mgr.h @@ -97,6 +97,10 @@ struct gunyah_rm_hyp_resources { =20 int gunyah_rm_get_hyp_resources(struct gunyah_rm *rm, u16 vmid, struct gunyah_rm_hyp_resources **resources); +struct gunyah_resource * +gunyah_rm_alloc_resource(struct gunyah_rm *rm, + struct gunyah_rm_hyp_resource *hyp_resource); +void gunyah_rm_free_resource(struct gunyah_resource *ghrsc); =20 int gunyah_rm_call(struct gunyah_rm *rsc_mgr, u32 message_id, const void *req_buf, size_t req_buf_size, void **resp_buf, --=20 2.39.5 From nobody Thu Dec 18 00:34:59 2025 Received: from mail-wr1-f48.google.com (mail-wr1-f48.google.com [209.85.221.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1B88028D85F for ; Thu, 24 Apr 2025 14:14:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.48 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504054; cv=none; b=GySm+36LgTjgJSZKzeYcIwP79AgNxC4JsLa7Y2t7aorOgiwqTCU7EqdWNQY01pnUi7e9dX5CVU2E8/69QgwYzXJ6A7pRavR0hwq4LFd13YhE6ws1V3c/rciGpRXP6Wh4cWwXx3Kcf+XJ45TQoRTP/ORWuhCggmkWM+YwQUMSUNQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504054; c=relaxed/simple; bh=DcGv+45qXx8VGU67Z7HeTl3hoEPjIE7jytPc3+geiuE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=IDRbZRaukg9ZREpJgEo4iNuf3jN8/E4gfMBxKD99yO2PZ1NU9qYXOSzV56i92KJ0Jjdq8HDuhE1DbLcHGVzu+YkxmWKsJVXMK4tNBqGha+upJCYJevoNN2qAs6zmY5GfVlIoxICnMbwABYOUOYp4liJs2ynLsljvIgJQN9dAfdk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=YIRJC4+s; arc=none smtp.client-ip=209.85.221.48 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="YIRJC4+s" Received: by mail-wr1-f48.google.com with SMTP id ffacd0b85a97d-39c1efbefc6so802686f8f.1 for ; Thu, 24 Apr 2025 07:14:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504050; x=1746108850; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=um+352HlfSoMWZjyUPYbdpZGkULscMwPKC2qBZ9EnFo=; b=YIRJC4+sfP9K7ZxYZWGxTQMjNFQy2JI0tIeeK/BfvqfqS1GKAfbm5Fr5c0XV+tMqwW 9vVk0Ng3BuugZHjkFlsmW21qafmVfN99B3w4ye3xbHNzRJvmrEp8PVmeQQjPcgB231Kw MEisY8bi3PgsLmrLOV0dsPGFF2gOQDmDs20JrwveUJuu3vnofJlkHVCBncOYpfYTIFik TKNKBOwtDhKLWFH5cAPv06B/e4FIi8IoFcPWtqrnOcdHMeH90XTTy92j+BJCGdBQF/8v sIr/HhwK+QNWj3y2FNuztGVfZ7lwO8R4ARngY8kmLmNjLUTHLQBg/Jl1IrMqzMveSjNo uRgw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504050; x=1746108850; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=um+352HlfSoMWZjyUPYbdpZGkULscMwPKC2qBZ9EnFo=; b=JhuyoSGc4QJ4STdYP+nXwCjw7YTc5qxGJJkrmTsur9tFrikREaei7bsIjz+AgNIsSG XJ0/b1/0y+LNvdb7p/Op3BVE6yvtIg1lY3FUay7ODo6dNe+W2KBOCwtt/3g2pUO9nebh 4B4JSwuidGE5ktEJ4UrCTLUz68uPnV+0Z/4QSRdT4dy7SPnPQYdKdRBQH2/GIszskxdt BrxjsHPDxQPWgzulKPETwWDI1yntJ5SlRNrJ1BNSsZcBYqF2CM/ZU8IUmsqXfodU5Mmu 5XVpJ9ISTnzQnYuoGWWswbjhVj5mlOBrBGIzqLIumQGljXvhCy6h9d9sOKJ0zntruOmf 62XA== X-Gm-Message-State: AOJu0YzlgnT82ETA7dQQEaryvrKJ7Tyx+dEGu1YGrUpLm3lutQQ3Opds gPtNs9wdhg2qKJbXaMy0H1Md05iv2tZBpIJKzEFIPmbww00v/3lZbTLkIyKC2zHLmBrxlWMXfyl O X-Gm-Gg: ASbGncs9pj6qqTG8gGAmYT2UwYZ2B7/oHjgpBW7gahUQ9+Oc142SZZu4YgqIDlmGcFd m6aXD4ud/Be2KcNgRkieDHbG84lYyI/4Ir1tZ9pK+M7DTd2n6WHxVi2IWisU0FFrBxP7X/CIBgp zENz1Rf3YIguJqH6PfiTlPE3YWzGK8l1GICzDQtyneuVWeYN+sWyqj+6xQ3/J4glVl+blDSbeh6 rSPHjTVW0dJS7TN+8uZ3rVskPrHlSTRI1lDVLC/16ju+ufIVMmslqLWIakV73A1RJ87XYQ4FNzL zt7oxcYWG0DZole4lQJHTN4i34V9Bp+67u9wuvqYRkmu9TVMClQX/ln0tAMtvUXXbeKELhrmTby R8ynPScBk0UStbKMF X-Google-Smtp-Source: AGHT+IEvmLar5qQKwS4wz8XFX51INY2Yy6tUFq9mAIas/T9L785Up8KKwGDWzGpkxhc6ebHp9w3CBA== X-Received: by 2002:a5d:64c7:0:b0:391:4389:f36a with SMTP id ffacd0b85a97d-3a06cfaf773mr2225643f8f.48.1745504049421; Thu, 24 Apr 2025 07:14:09 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.14.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:14:08 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt , Elliot Berman , Alex Elder Subject: [RFC PATCH 18/34] gunyah: Add resource tickets Date: Thu, 24 Apr 2025 15:13:25 +0100 Message-Id: <20250424141341.841734-19-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Elliot Berman Some VM functions need to acquire Gunyah resources. For instance, Gunyah vCPUs are exposed to the host as a resource. The Gunyah vCPU function will register a resource ticket and be able to interact with the hypervisor once the resource ticket is filled. Resource tickets are the mechanism for functions to acquire ownership of Gunyah resources. Gunyah functions can be created before the VM's resources are created and made available to Linux. A resource ticket identifies a type of resource and a label of a resource which the ticket holder is interested in. Resources are created by Gunyah as configured in the VM's devicetree configuration. Gunyah doesn't process the label and that makes it possible for userspace to create multiple resources with the same label. Resource ticket owners need to be prepared for populate to be called multiple times if userspace created multiple resources with the same label. Reviewed-by: Alex Elder Signed-off-by: Elliot Berman Signed-off-by: Karim Manaouil --- arch/arm64/kvm/gunyah.c | 123 +++++++++++++++++++++++++++++++++++++++- include/linux/gunyah.h | 42 +++++++++++++- 2 files changed, 161 insertions(+), 4 deletions(-) diff --git a/arch/arm64/kvm/gunyah.c b/arch/arm64/kvm/gunyah.c index a3c29ae985c9..084ee1091770 100644 --- a/arch/arm64/kvm/gunyah.c +++ b/arch/arm64/kvm/gunyah.c @@ -330,6 +330,104 @@ int kvm_arm_set_reg(struct kvm_vcpu *vcpu, const stru= ct kvm_one_reg *reg) } } =20 +static int gunyah_vm_add_resource_ticket(struct gunyah_vm *ghvm, + struct gunyah_vm_resource_ticket *ticket) +{ + struct gunyah_vm_resource_ticket *iter; + struct gunyah_resource *ghrsc, *rsc_iter; + int ret =3D 0; + + mutex_lock(&ghvm->resources_lock); + list_for_each_entry(iter, &ghvm->resource_tickets, vm_list) { + if (iter->resource_type =3D=3D ticket->resource_type && + iter->label =3D=3D ticket->label) { + ret =3D -EEXIST; + goto out; + } + } + + list_add(&ticket->vm_list, &ghvm->resource_tickets); + INIT_LIST_HEAD(&ticket->resources); + + list_for_each_entry_safe(ghrsc, rsc_iter, &ghvm->resources, list) { + if (ghrsc->type =3D=3D ticket->resource_type && + ghrsc->rm_label =3D=3D ticket->label) { + if (ticket->populate(ticket, ghrsc)) + list_move(&ghrsc->list, &ticket->resources); + } + } +out: + mutex_unlock(&ghvm->resources_lock); + return ret; +} + +static void __gunyah_vm_remove_resource_ticket(struct gunyah_vm *ghvm, + struct gunyah_vm_resource_ticket *ticket) +{ + struct gunyah_resource *ghrsc, *iter; + + list_for_each_entry_safe(ghrsc, iter, &ticket->resources, list) { + ticket->unpopulate(ticket, ghrsc); + list_move(&ghrsc->list, &ghvm->resources); + } + list_del(&ticket->vm_list); +} + +static void gunyah_vm_remove_resource_ticket(struct gunyah_vm *ghvm, + struct gunyah_vm_resource_ticket *ticket) +{ + + mutex_lock(&ghvm->resources_lock); + __gunyah_vm_remove_resource_ticket(ghvm, ticket); + mutex_unlock(&ghvm->resources_lock); +} + +static void gunyah_vm_add_resource(struct gunyah_vm *ghvm, + struct gunyah_resource *ghrsc) +{ + struct gunyah_vm_resource_ticket *ticket; + + mutex_lock(&ghvm->resources_lock); + list_for_each_entry(ticket, &ghvm->resource_tickets, vm_list) { + if (ghrsc->type =3D=3D ticket->resource_type && + ghrsc->rm_label =3D=3D ticket->label) { + if (ticket->populate(ticket, ghrsc)) + list_add(&ghrsc->list, &ticket->resources); + else + list_add(&ghrsc->list, &ghvm->resources); + /* unconditonal -- we prevent multiple identical + * resource tickets so there will not be some other + * ticket elsewhere in the list if populate() failed. + */ + goto found; + } + } + list_add(&ghrsc->list, &ghvm->resources); +found: + mutex_unlock(&ghvm->resources_lock); +} + +static void gunyah_vm_clean_resources(struct gunyah_vm *ghvm) +{ + struct gunyah_vm_resource_ticket *ticket, *titer; + struct gunyah_resource *ghrsc, *riter; + + mutex_lock(&ghvm->resources_lock); + if (!list_empty(&ghvm->resource_tickets)) { + pr_warn("Dangling resource tickets:\n"); + list_for_each_entry_safe(ticket, titer, &ghvm->resource_tickets, + vm_list) { + pr_warn(" %pS\n", ticket->populate); + __gunyah_vm_remove_resource_ticket(ghvm, ticket); + } + } + + list_for_each_entry_safe(ghrsc, riter, &ghvm->resources, list) { + gunyah_rm_free_resource(ghrsc); + } + mutex_unlock(&ghvm->resources_lock); +} + int kvm_arch_vcpu_should_kick(struct kvm_vcpu *vcpu) { return kvm_vcpu_exiting_guest_mode(vcpu) =3D=3D IN_GUEST_MODE; @@ -852,7 +950,9 @@ static void gunyah_vm_stop(struct gunyah_vm *ghvm) =20 static int gunyah_vm_start(struct gunyah_vm *ghvm) { - int ret; + struct gunyah_rm_hyp_resources *resources; + struct gunyah_resource *ghrsc; + int i, n, ret; =20 down_write(&ghvm->status_lock); if (ghvm->vm_status !=3D GUNYAH_RM_VM_STATUS_NO_STATE) { @@ -887,6 +987,22 @@ static int gunyah_vm_start(struct gunyah_vm *ghvm) } ghvm->vm_status =3D GUNYAH_RM_VM_STATUS_READY; =20 + ret =3D gunyah_rm_get_hyp_resources(ghvm->rm, ghvm->vmid, &resources); + if (ret) { + pr_warn("Failed to get hyp resources for VM: %d\n", ret); + goto err; + } + + for (i =3D 0, n =3D le32_to_cpu(resources->n_entries); i < n; i++) { + ghrsc =3D gunyah_rm_alloc_resource(ghvm->rm, + &resources->entries[i]); + if (!ghrsc) { + ret =3D -ENOMEM; + goto err; + } + gunyah_vm_add_resource(ghvm, ghrsc); + } + ret =3D gunyah_rm_vm_start(ghvm->rm, ghvm->vmid); if (ret) { pr_warn("Failed to start VM: %d\n", ret); @@ -915,6 +1031,9 @@ static struct gunyah_vm *gunyah_vm_alloc(struct gunyah= _rm *rm) init_rwsem(&ghvm->status_lock); init_waitqueue_head(&ghvm->vm_status_wait); ghvm->vm_status =3D GUNYAH_RM_VM_STATUS_NO_STATE; + mutex_init(&ghvm->resources_lock); + INIT_LIST_HEAD(&ghvm->resources); + INIT_LIST_HEAD(&ghvm->resource_tickets); =20 return ghvm; } @@ -930,6 +1049,8 @@ static void gunyah_destroy_vm(struct gunyah_vm *ghvm) if (ghvm->vm_status =3D=3D GUNYAH_RM_VM_STATUS_RUNNING) gunyah_vm_stop(ghvm); =20 + gunyah_vm_clean_resources(ghvm); + if (ghvm->vm_status =3D=3D GUNYAH_RM_VM_STATUS_EXITED || ghvm->vm_status =3D=3D GUNYAH_RM_VM_STATUS_READY || ghvm->vm_status =3D=3D GUNYAH_RM_VM_STATUS_INIT_FAILED) { diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h index 40ea21b17195..573e3bbd4cb6 100644 --- a/include/linux/gunyah.h +++ b/include/linux/gunyah.h @@ -10,6 +10,7 @@ #include #include #include +#include #include #include =20 @@ -18,6 +19,8 @@ #define kvm_to_gunyah(kvm_ptr) \ container_of(kvm_ptr, struct gunyah_vm, kvm) =20 +struct gunyah_vm; + /* Matches resource manager's resource types for VM_GET_HYP_RESOURCES RPC = */ enum gunyah_resource_type { /* clang-format off */ @@ -35,10 +38,39 @@ struct gunyah_resource { enum gunyah_resource_type type; u64 capid; unsigned int irq; - + struct list_head list; u32 rm_label; }; =20 +/** + * struct gunyah_vm_resource_ticket - Represents a ticket to reserve acces= s to VM resource(s) + * @label: Label of the resource from resource manager this ticket reserve= s. + * @vm_list: for @gunyah_vm->resource_tickets + * @resources: List of resource(s) associated with this ticket + * (members are from @gunyah_resource->list) + * @resource_type: Type of resource this ticket reserves + * @populate: callback provided by the ticket owner and called when a reso= urce is found that + * matches @resource_type and @label. Note that this callback c= ould be called + * multiple times if userspace created mutliple resources with = the same type/label. + * This callback may also have significant delay after gunyah_v= m_add_resource_ticket() + * since gunyah_vm_add_resource_ticket() could be called before= the VM starts. + * @unpopulate: callback provided by the ticket owner and called when the = ticket owner should no + * longer use the resource provided in the argument. When unp= opulate() returns, + * the ticket owner should not be able to use the resource an= y more as the resource + * might being freed. + */ +struct gunyah_vm_resource_ticket { + u32 label; + struct list_head vm_list; + struct list_head resources; + enum gunyah_resource_type resource_type; + bool (*populate)(struct gunyah_vm_resource_ticket *ticket, + struct gunyah_resource *ghrsc); + void (*unpopulate)(struct gunyah_vm_resource_ticket *ticket, + struct gunyah_resource *ghrsc); +}; + + /** * struct gunyah_vm - Main representation of a Gunyah Virtual machine memory shared with the guest. @@ -49,6 +81,9 @@ struct gunyah_resource { * @vm_status: Current state of the VM, as last reported by RM * @vm_status_wait: Wait queue for status @vm_status changes * @status_lock: Serializing state transitions + * @resource_lock: Serializing addition of resources and resource tickets + * @resources: List of &struct gunyah_resource that are associated with th= is VM + * @resource_tickets: List of &struct gunyah_vm_resource_ticket * @auth: Authentication mechanism to be used by resource manager when * launching the VM */ @@ -56,12 +91,13 @@ struct gunyah_vm { u16 vmid; struct kvm kvm; struct gunyah_rm *rm; - struct notifier_block nb; enum gunyah_rm_vm_status vm_status; wait_queue_head_t vm_status_wait; struct rw_semaphore status_lock; - + struct mutex resources_lock; + struct list_head resources; + struct list_head resource_tickets; enum gunyah_rm_vm_auth_mechanism auth; }; =20 --=20 2.39.5 From nobody Thu Dec 18 00:34:59 2025 Received: from mail-wm1-f45.google.com (mail-wm1-f45.google.com [209.85.128.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1A14B288C94 for ; Thu, 24 Apr 2025 14:14:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.45 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504055; cv=none; b=WJXIUbJXA7DP1GKi99gmu+eJUKXAXWzgY5H3DpTrgdT3EaUfw2Wv2ACSnZ3vqRm92YauSoiZ+G5QTnh+FLMGOwCWKi/UF/ek3WQlYP2FfhikCWXY82VzydOFW/+IFhVCOb+MuE74m21S/wd0U599GdfkzXnxIjrd4BHUBBdzhrY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504055; c=relaxed/simple; bh=2aIlJTxyajv9w1JNq2QUWNBaFE1l1Iz4MgPOHzkWSHs=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=UtciNqWdDn0WGwoRbdsS9O6BssjVZDaiD2XIxtULcr6SjM9/kVSx7oPpLoZfPgYCHPiEhDOox5lRBnr52hzTCZZcYv0uvMYYdgSec3pqvKhkHECpjqvRRMNWmk3LRV2WetLIkyyoFscYyPfBGnWkkz366jOuyX5XtrS/JaxAh6E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=PJZRrhye; arc=none smtp.client-ip=209.85.128.45 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="PJZRrhye" Received: by mail-wm1-f45.google.com with SMTP id 5b1f17b1804b1-43cfba466b2so10630655e9.3 for ; Thu, 24 Apr 2025 07:14:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504051; x=1746108851; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=mnnNrRFux5qVcGjuNFA0c9YQRdgVNUximYQurTlnUC4=; b=PJZRrhyeB9kFuh5fnr6Nbxc6hBWs4a64QdPEFmN6EjippJ8/wFu+CIKUVczzDUBqh3 afTa0zEhR+philf+002ackoL6VTNxbBgqwzH3tecbFIHxuzwa93EtCfPNNnTilwq7gm6 Q5ZOQ52nyt5CUsXzxFumP4ORZN0zxiWGoAx1odVSisLG4VW79iVky/ct8IuYjx3ziW0A NB1P3pxFsx/voIgr8Fn9gjhRZAk+/21vaiehhkayNW42Fqk6NyJaexztNuffnx+/BqOc qigxJdZ2AUmPsYD6qeYmRUQhvRWkn8h0xx33NFpO8sBwTFvmDtZI0e4/CPbM9C+izJzZ xVlQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504051; x=1746108851; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=mnnNrRFux5qVcGjuNFA0c9YQRdgVNUximYQurTlnUC4=; b=sf9jFgb/MTL4uNJj/2JM5XtbRhStzNjhL0ZuNup++p7t3T4HSAiiZGUTUdTOxUaT7b 9ZRSL7kpRNEc9CQsh+8oCabq7MqFzp9U5ysP9MR1ewQUU87jquGgB9ZvTe5Oj+I3VH/y S2GcwMlGHt2vjg7OsUtiy7Kr7rRrE/j3zEBi5/1GkQdNArpqTTNp7VAd5hrHlafRf+fi 811KMFwwO6KAxtwvVCkrHowbrL4npiX8G6B9hfjnETo2Kc1hHnFy7OSqFRZ5haczvMQx 2EZnfEvfrGBDv9yB70g7LFIyfQ7nTSEoxiID2+tMuRgU7Qq8G/3bkMzxqw57Ym0UIGXc DFcA== X-Gm-Message-State: AOJu0YwLyLwbnva7TZUf5dypX0HA1it+5/l2gGqoe52Q5PPI1nqoghVY mSCtGcUb1IUrinjp4222rt4o1gxaIu+r9KC0haRJJLQ7dP26PnlNhDen46GQzDqrxDIV9pYdOR1 T X-Gm-Gg: ASbGnctH9gMEQbN36Tzi3IW3O2aaYPSez51np6opSis2LzG0ky/+g8sjKr7/WA3i6q1 uIyfNmFwy0LeJHzhYb9JuFsXJjxDdEREvI+ZwcF6cGDaySxPyeVRLeYBL31yrF3POMZ92p0EP74 fIGZAPFuyCUPF7GMqYI22BqMEdHLRX9nXvj6ANmwm6nXAIqXPv33vs7PCjJa5+CYomNOSYx5tkm BfrLuX0Z2JR0tQZry2zKDGhaCt3C+dJCwHBakVyHruVYd9IArVqkpNM6VlQ+z3xuos07Eph6ire iz6A25K44CheiECrJ6f7rvQP7aFqW9AweG26yO1Q3MB/lV+mzxye9LhS0pDjpjlehWDeihi5bPF MPRGnKPLiPAjLh7g2 X-Google-Smtp-Source: AGHT+IElBFJDMQbW748Xhn6uR3Gy7UiwZwfaHD4iyiwy3AqrKyTZYkKWwIgzmc9HZX/EcWP9CEo9Lw== X-Received: by 2002:a05:6000:18a2:b0:391:47d8:de2d with SMTP id ffacd0b85a97d-3a06cf5ed4cmr2483522f8f.23.1745504050693; Thu, 24 Apr 2025 07:14:10 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.14.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:14:10 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt , Elliot Berman Subject: [RFC PATCH 19/34] gunyah: Add hypercalls for running a vCPU Date: Thu, 24 Apr 2025 15:13:26 +0100 Message-Id: <20250424141341.841734-20-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Elliot Berman Add hypercall to donate CPU time to a vCPU. Signed-off-by: Elliot Berman Reviewed-by: Srivatsa Vaddagiri Signed-off-by: Karim Manaouil --- arch/arm64/gunyah/gunyah_hypercall.c | 37 ++++++++++++++++++++++++++++ include/linux/gunyah.h | 35 ++++++++++++++++++++++++++ 2 files changed, 72 insertions(+) diff --git a/arch/arm64/gunyah/gunyah_hypercall.c b/arch/arm64/gunyah/gunya= h_hypercall.c index 1302e128be6e..fee21df42c17 100644 --- a/arch/arm64/gunyah/gunyah_hypercall.c +++ b/arch/arm64/gunyah/gunyah_hypercall.c @@ -39,6 +39,7 @@ EXPORT_SYMBOL_GPL(arch_is_gunyah_guest); #define GUNYAH_HYPERCALL_HYP_IDENTIFY GUNYAH_HYPERCALL(0x8000) #define GUNYAH_HYPERCALL_MSGQ_SEND GUNYAH_HYPERCALL(0x801B) #define GUNYAH_HYPERCALL_MSGQ_RECV GUNYAH_HYPERCALL(0x801C) +#define GUNYAH_HYPERCALL_VCPU_RUN GUNYAH_HYPERCALL(0x8065) /* clang-format on */ =20 /** @@ -113,5 +114,41 @@ enum gunyah_error gunyah_hypercall_msgq_recv(u64 capid= , void *buff, size_t size, } EXPORT_SYMBOL_GPL(gunyah_hypercall_msgq_recv); =20 +/** + * gunyah_hypercall_vcpu_run() - Donate CPU time to a vcpu + * @capid: capability ID of the vCPU to run + * @resume_data: Array of 3 state-specific resume data + * @resp: Filled reason why vCPU exited when return value is GUNYAH_ERROR_= OK + * + * See also: + * https://github.com/quic/gunyah-hypervisor/blob/develop/docs/api/gunyah_= api.md#run-a-proxy-scheduled-vcpu-thread + */ +enum gunyah_error +gunyah_hypercall_vcpu_run(u64 capid, unsigned long *resume_data, + struct gunyah_hypercall_vcpu_run_resp *resp) +{ + struct arm_smccc_1_2_regs args =3D { + .a0 =3D GUNYAH_HYPERCALL_VCPU_RUN, + .a1 =3D capid, + .a2 =3D resume_data[0], + .a3 =3D resume_data[1], + .a4 =3D resume_data[2], + /* C language says this will be implictly zero. Gunyah requires 0, so be= explicit */ + .a5 =3D 0, + }; + struct arm_smccc_1_2_regs res; + + arm_smccc_1_2_hvc(&args, &res); + if (res.a0 =3D=3D GUNYAH_ERROR_OK) { + resp->sized_state =3D res.a1; + resp->state_data[0] =3D res.a2; + resp->state_data[1] =3D res.a3; + resp->state_data[2] =3D res.a4; + } + + return res.a0; +} +EXPORT_SYMBOL_GPL(gunyah_hypercall_vcpu_run); + MODULE_LICENSE("GPL"); MODULE_DESCRIPTION("Gunyah Hypervisor Hypercalls"); diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h index 573e3bbd4cb6..f86f14018734 100644 --- a/include/linux/gunyah.h +++ b/include/linux/gunyah.h @@ -219,4 +219,39 @@ enum gunyah_error gunyah_hypercall_msgq_send(u64 capid= , size_t size, void *buff, enum gunyah_error gunyah_hypercall_msgq_recv(u64 capid, void *buff, size_t= size, size_t *recv_size, bool *ready); =20 +struct gunyah_hypercall_vcpu_run_resp { + union { + enum { + /* clang-format off */ + /* VCPU is ready to run */ + GUNYAH_VCPU_STATE_READY =3D 0, + /* VCPU is sleeping until an interrupt arrives */ + GUNYAH_VCPU_STATE_EXPECTS_WAKEUP =3D 1, + /* VCPU is powered off */ + GUNYAH_VCPU_STATE_POWERED_OFF =3D 2, + /* VCPU is blocked in EL2 for unspecified reason */ + GUNYAH_VCPU_STATE_BLOCKED =3D 3, + /* VCPU has returned for MMIO READ */ + GUNYAH_VCPU_ADDRSPACE_VMMIO_READ =3D 4, + /* VCPU has returned for MMIO WRITE */ + GUNYAH_VCPU_ADDRSPACE_VMMIO_WRITE =3D 5, + /* VCPU blocked on fault where we can demand page */ + GUNYAH_VCPU_ADDRSPACE_PAGE_FAULT =3D 7, + /* clang-format on */ + } state; + u64 sized_state; + }; + u64 state_data[3]; +}; + +enum { + GUNYAH_ADDRSPACE_VMMIO_ACTION_EMULATE =3D 0, + GUNYAH_ADDRSPACE_VMMIO_ACTION_RETRY =3D 1, + GUNYAH_ADDRSPACE_VMMIO_ACTION_FAULT =3D 2, +}; + +enum gunyah_error +gunyah_hypercall_vcpu_run(u64 capid, unsigned long *resume_data, + struct gunyah_hypercall_vcpu_run_resp *resp); + #endif --=20 2.39.5 From nobody Thu Dec 18 00:34:59 2025 Received: from mail-wm1-f41.google.com (mail-wm1-f41.google.com [209.85.128.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6B5F528DF04 for ; Thu, 24 Apr 2025 14:14:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.41 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504058; cv=none; b=dMm5ugS4PUbdlNqhQWsCuZXVjCch3ZHi2p/HWQYY/8cyaZMvteqFmKguImGlHXZAdj6VWhsycIQEwwRz+mR7JYxpYgU8kTzqQcZDHGXDPAaOitjOHJ0bzD7j2f5cEEfisq6ihs/ONFiZQeCgRpepD/fYTcFgxzgBzJ6DTNnS77k= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504058; c=relaxed/simple; bh=v1GZAOzGYATKFr8YmGOLT0nlJPt5FIGEotq+V+hCZbE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=QPVuiINoVPsaxh35kHmuhc9NV758wdRSO/0Rle2xozqUOdXimZa/PdDjUzgPXPQC1xKNryhnADD6aXzsFbWWlSst+/dNC6w3ls9vU+qOXv/+RSu6B2xRzK7+UX4kQ3MjRasxQK3tnUXCKxWOzYoWB1gjnvGL2O1wTZ26RpsxnHA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=uiibvrls; arc=none smtp.client-ip=209.85.128.41 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="uiibvrls" Received: by mail-wm1-f41.google.com with SMTP id 5b1f17b1804b1-43d0c18e84eso5332505e9.3 for ; Thu, 24 Apr 2025 07:14:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504052; x=1746108852; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Npq0NQtrgJlAV8FqtQiHtQ8TrFptNSWm5BxQWT8PEM0=; b=uiibvrlsxKeiO53O7KMvUTq6V5pxeqYhQonnUHBTJ3lNoajeqVhT/CrQeOmBs76CJJ 3dW1Mfe9W3IL29grfp4eFoqb9IEinywSh88118AlS2XUfe5/2f4q9lwj7wicMEgBTZGA A7BvFOCBgQd5oFs1xsVSt5/KgqIuphAHU1cd4to0vS8BwqGMKrZccWuxQ3VkfvT+E04s TX5puDb0Pm/QJ57M4LcXEQBgTcYN0jIE66UJi8Buxh3YB/D9zNymkWtF1k0T+EdWBRCL Hq+xoMrBRScN45umw1ADddkPHOL3SlIAma7TU2fkoE54wXBK8tgWzQrnCwWMXxNgvbK4 Ln9A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504052; x=1746108852; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Npq0NQtrgJlAV8FqtQiHtQ8TrFptNSWm5BxQWT8PEM0=; b=NXLcuhnb+C+wDysBAGDd/LTfkcW8XvHzzxMqI5IS6Kvf4dxpLRk49IBZpykeK5ZobF YVrbxJjemrospqowXn03DfQMTgQq7kbwK3FwsEbLdOgTVgCV875UfUe6nKnvqaqi+zgO tIsfI+1Pkgm71qf8K8F5/XeH50QyPtZcucmYxMZarVGmYd//KaRSPoPJjGIVQGY/jN3l MrP0Mc75hj+av14B5f8FvhNkKNOKsgxq4aNilLg+mvXNkdbWnEyhDvUdAkBVHB2ItWWv aQzBNn90ZOjxf8ocVr+lx6lqAvYsPbdv+p8gMy0Hpzo004qgp/LqeGO/+6RYylg51AYu OqTw== X-Gm-Message-State: AOJu0YwM4HgCOYrg7kBvZpmcF0hNOL/lwXOyf5Nd1Su1NIp/Vi/1nDLV hlUjeQj4110KjLZ8VzhXWbxxTLaJZ7l0c8SeJAW87oIUcuLP6kbPkCSwvUAwBP3a05hsWeMNvHz 0 X-Gm-Gg: ASbGncvIlhlsOGSMsiM0H8KQ4gFR4nP1YuXaStTqb0ugFnk44i93beFWh2c0cg/fxyI 01/vo9PE1Gk4h3dlPfW1qmo8H91fefwDCvFMHpXkSErU8qcMPXinzHo54zNhnkQH/wUmMQLS3ro nSVaf+6OSXm0yOfHFPOl+auLjNN1kWBPjrgTQKx8JrxghcMFR7tEK1mkUrqsvsGADAEuJymQWZl pCFAMrtgeeAgqq5PXNCDKHZFuzoHxfjX8L04IS8488PqMJTuOFUOYCoh5KOZnq78aJh1rJ8cuej CFFjyL1rHxC6OBd3Myj0+pxaAjnh0fRcXaoSkwJuCAePuK18HpDy0vEt3lBmAnp7DcqlZOFu1M0 XcnmcxPOftHhuTvxU X-Google-Smtp-Source: AGHT+IGDNQni2+JFXMwQWuIVE9mUFnHRCikRQcusA6Pgtq8l9mbgJyFlCKPQ4I6jOpeZedp+Ki0cWw== X-Received: by 2002:a5d:5f4f:0:b0:39c:1f10:c736 with SMTP id ffacd0b85a97d-3a06cfab8cbmr2032162f8f.43.1745504052080; Thu, 24 Apr 2025 07:14:12 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.14.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:14:11 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt , Elliot Berman Subject: [RFC PATCH 20/34] gunyah: add proxy-scheduled vCPUs Date: Thu, 24 Apr 2025 15:13:27 +0100 Message-Id: <20250424141341.841734-21-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable This patch is based heavily on the original Gunyah vCPU support from Elliot Berman and Prakruthi Deepak Heragu: https://lore.kernel.org/lkml/20240222-gunyah-v17-14-1e9da6763d38@quicinc.co= m/ The original implementation had its own character device interface. This pa= tch ports Gunyah vCPU management to KVM (e.g., `kvm_arch_vcpu_run()` calls Guny= ah hypervisor, running as firmware, via hypercalls, which then runs the vCPU). This enables Gunyah vCPUs to be driven through the standard KVM userspace interface (e.g., via QEMU), while transparently using Gunyah=E2=80=99s proxy-scheduled vCPU mechanisms under the hood. Co-developed-by: Elliot Berman Co-developed-by: Prakruthi Deepak Heragu Signed-off-by: Karim Manaouil --- arch/arm64/kvm/gunyah.c | 348 +++++++++++++++++++++++++++++++++++++++- include/linux/gunyah.h | 51 ++++++ 2 files changed, 395 insertions(+), 4 deletions(-) diff --git a/arch/arm64/kvm/gunyah.c b/arch/arm64/kvm/gunyah.c index 084ee1091770..e066482c2e71 100644 --- a/arch/arm64/kvm/gunyah.c +++ b/arch/arm64/kvm/gunyah.c @@ -19,6 +19,8 @@ #undef pr_fmt #define pr_fmt(fmt) "gunyah: " fmt =20 +static int gunyah_vm_start(struct gunyah_vm *ghvm); + static enum kvm_mode kvm_mode =3D KVM_MODE_DEFAULT; =20 enum kvm_mode kvm_get_mode(void) @@ -458,9 +460,311 @@ bool kvm_arch_intc_initialized(struct kvm *kvm) return true; } =20 -struct kvm_vcpu *kvm_arch_vcpu_alloc(void) +/* + * When hypervisor allows us to schedule vCPU again, it gives us an interr= upt + */ +static irqreturn_t gunyah_vcpu_irq_handler(int irq, void *data) +{ + struct gunyah_vcpu *vcpu =3D data; + + complete(&vcpu->ready); + return IRQ_HANDLED; +} + +static int gunyah_vcpu_rm_notification(struct notifier_block *nb, + unsigned long action, void *data) { - return NULL; + struct gunyah_vcpu *vcpu =3D container_of(nb, struct gunyah_vcpu, nb); + struct gunyah_rm_vm_exited_payload *exit_payload =3D data; + + /* Wake up userspace waiting for the vCPU to be runnable again */ + if (action =3D=3D GUNYAH_RM_NOTIFICATION_VM_EXITED && + le16_to_cpu(exit_payload->vmid) =3D=3D vcpu->ghvm->vmid) + complete(&vcpu->ready); + + return NOTIFY_OK; +} + +static int gunyah_handle_page_fault( + struct gunyah_vcpu *vcpu, + const struct gunyah_hypercall_vcpu_run_resp *vcpu_run_resp) +{ + return -EINVAL; +} + +static bool gunyah_kvm_handle_mmio(struct gunyah_vcpu *vcpu, + unsigned long resume_data[3], + const struct gunyah_hypercall_vcpu_run_resp *vcpu_run_resp) +{ + struct kvm_vcpu *kvm_vcpu =3D &vcpu->kvm_vcpu; + struct kvm_run *run =3D kvm_vcpu->run; + u64 addr =3D vcpu_run_resp->state_data[0]; + u64 len =3D vcpu_run_resp->state_data[1]; + u64 data =3D vcpu_run_resp->state_data[2]; + bool write; + + if (WARN_ON(len > sizeof(u64))) + len =3D sizeof(u64); + + if (vcpu_run_resp->state =3D=3D GUNYAH_VCPU_ADDRSPACE_VMMIO_READ) { + write =3D false; + /* + * Record that we need to give vCPU user's supplied + * value next gunyah_vcpu_run() + */ + vcpu->state =3D GUNYAH_VCPU_RUN_STATE_MMIO_READ; + } else { + /* TODO: HANDLE IOEVENTFD !! */ + write =3D true; + vcpu->state =3D GUNYAH_VCPU_RUN_STATE_MMIO_WRITE; + } + + if (write) + memcpy(run->mmio.data, &data, len); + + run->mmio.is_write =3D write; + run->mmio.phys_addr =3D addr; + run->mmio.len =3D len; + kvm_vcpu->mmio_needed =3D 1; + + kvm_vcpu->stat.mmio_exit_user++; + run->exit_reason =3D KVM_EXIT_MMIO; + + return false; +} + +static int gunyah_handle_mmio_resume(struct gunyah_vcpu *vcpu, + unsigned long resume_data[3]) +{ + struct kvm_vcpu *kvm_vcpu =3D &vcpu->kvm_vcpu; + struct kvm_run *run =3D kvm_vcpu->run; + + resume_data[1] =3D GUNYAH_ADDRSPACE_VMMIO_ACTION_EMULATE; + if (vcpu->state =3D=3D GUNYAH_VCPU_RUN_STATE_MMIO_READ) + memcpy(&resume_data[0], run->mmio.data, run->mmio.len); + return 0; +} + +/** + * gunyah_vcpu_check_system() - Check whether VM as a whole is running + * @vcpu: Pointer to gunyah_vcpu + * + * Returns true if the VM is alive. + * Returns false if the vCPU is the VM is not alive (can only be that VM i= s shutting down). + */ +static bool gunyah_vcpu_check_system(struct gunyah_vcpu *vcpu) + __must_hold(&vcpu->lock) +{ + bool ret =3D true; + + down_read(&vcpu->ghvm->status_lock); + if (likely(vcpu->ghvm->vm_status =3D=3D GUNYAH_RM_VM_STATUS_RUNNING)) + goto out; + + vcpu->state =3D GUNYAH_VCPU_RUN_STATE_SYSTEM_DOWN; + ret =3D false; +out: + up_read(&vcpu->ghvm->status_lock); + return ret; +} + +static int gunyah_vcpu_run(struct gunyah_vcpu *vcpu) +{ + struct gunyah_hypercall_vcpu_run_resp vcpu_run_resp; + struct kvm_vcpu *kvm_vcpu =3D &vcpu->kvm_vcpu; + struct kvm_run *run =3D kvm_vcpu->run; + unsigned long resume_data[3] =3D { 0 }; + enum gunyah_error gunyah_error; + int ret =3D 0; + + if (mutex_lock_interruptible(&vcpu->lock)) + return -ERESTARTSYS; + + if (!vcpu->rsc) { + ret =3D -ENODEV; + goto out; + } + + switch (vcpu->state) { + case GUNYAH_VCPU_RUN_STATE_UNKNOWN: + if (vcpu->ghvm->vm_status !=3D GUNYAH_RM_VM_STATUS_RUNNING) { + /** + * Check if VM is up. If VM is starting, will block + * until VM is fully up since that thread does + * down_write. + */ + if (!gunyah_vcpu_check_system(vcpu)) + goto out; + } + vcpu->state =3D GUNYAH_VCPU_RUN_STATE_READY; + break; + case GUNYAH_VCPU_RUN_STATE_MMIO_READ: + case GUNYAH_VCPU_RUN_STATE_MMIO_WRITE: + ret =3D gunyah_handle_mmio_resume(vcpu, resume_data); + if (ret) + goto out; + vcpu->state =3D GUNYAH_VCPU_RUN_STATE_READY; + break; + case GUNYAH_VCPU_RUN_STATE_SYSTEM_DOWN: + goto out; + default: + break; + } + + run->exit_reason =3D KVM_EXIT_UNKNOWN; + + while (!ret && !signal_pending(current)) { + if (vcpu->immediate_exit) { + ret =3D -EINTR; + goto out; + } + gunyah_error =3D gunyah_hypercall_vcpu_run( + vcpu->rsc->capid, resume_data, &vcpu_run_resp); + + if (gunyah_error =3D=3D GUNYAH_ERROR_OK) { + memset(resume_data, 0, sizeof(resume_data)); + + switch (vcpu_run_resp.state) { + case GUNYAH_VCPU_STATE_READY: + if (need_resched()) + schedule(); + break; + case GUNYAH_VCPU_STATE_POWERED_OFF: + /** + * vcpu might be off because the VM is shut down + * If so, it won't ever run again + */ + if (!gunyah_vcpu_check_system(vcpu)) + goto out; + /** + * Otherwise, another vcpu will turn it on (e.g. + * by PSCI) and hyp sends an interrupt to wake + * Linux up. + */ + fallthrough; + case GUNYAH_VCPU_STATE_EXPECTS_WAKEUP: + ret =3D wait_for_completion_interruptible( + &vcpu->ready); + /** + * reinitialize completion before next + * hypercall. If we reinitialize after the + * hypercall, interrupt may have already come + * before re-initializing the completion and + * then end up waiting for event that already + * happened. + */ + reinit_completion(&vcpu->ready); + /** + * Check VM status again. Completion + * might've come from VM exiting + */ + if (!ret && !gunyah_vcpu_check_system(vcpu)) + goto out; + break; + case GUNYAH_VCPU_STATE_BLOCKED: + schedule(); + break; + case GUNYAH_VCPU_ADDRSPACE_VMMIO_READ: + case GUNYAH_VCPU_ADDRSPACE_VMMIO_WRITE: + if (!gunyah_kvm_handle_mmio(vcpu, resume_data, + &vcpu_run_resp)) + goto out; + break; + case GUNYAH_VCPU_ADDRSPACE_PAGE_FAULT: + ret =3D gunyah_handle_page_fault(vcpu, &vcpu_run_resp); + if (ret) + goto out; + break; + default: + pr_warn( + "Unknown vCPU state: %llx\n", + vcpu_run_resp.sized_state); + schedule(); + break; + } + } else if (gunyah_error =3D=3D GUNYAH_ERROR_RETRY) { + schedule(); + } else { + ret =3D gunyah_error_remap(gunyah_error); + } + } + +out: + mutex_unlock(&vcpu->lock); + + if (signal_pending(current)) + return -ERESTARTSYS; + + return ret; +} + +static bool gunyah_vcpu_populate(struct gunyah_vm_resource_ticket *ticket, + struct gunyah_resource *ghrsc) +{ + struct gunyah_vcpu *vcpu =3D + container_of(ticket, struct gunyah_vcpu, ticket); + int ret; + + mutex_lock(&vcpu->lock); + if (vcpu->rsc) { + pr_warn("vcpu%d already got a Gunyah resource", vcpu->ticket.label); + ret =3D -EEXIST; + goto out; + } + vcpu->rsc =3D ghrsc; + + ret =3D request_irq(vcpu->rsc->irq, gunyah_vcpu_irq_handler, + IRQF_TRIGGER_RISING, "gunyah_vcpu", vcpu); + if (ret) { + pr_warn("Failed to request vcpu irq %d: %d", vcpu->rsc->irq, + ret); + goto out; + } + + enable_irq_wake(vcpu->rsc->irq); +out: + mutex_unlock(&vcpu->lock); + return !ret; +} + +static void gunyah_vcpu_unpopulate(struct gunyah_vm_resource_ticket *ticke= t, + struct gunyah_resource *ghrsc) +{ + struct gunyah_vcpu *vcpu =3D + container_of(ticket, struct gunyah_vcpu, ticket); + + vcpu->immediate_exit =3D true; + complete_all(&vcpu->ready); + mutex_lock(&vcpu->lock); + free_irq(vcpu->rsc->irq, vcpu); + vcpu->rsc =3D NULL; + mutex_unlock(&vcpu->lock); +} + +static int gunyah_vcpu_create(struct gunyah_vm *ghvm, struct gunyah_vcpu *= vcpu, int id) +{ + int r; + + mutex_init(&vcpu->lock); + init_completion(&vcpu->ready); + + vcpu->ghvm =3D ghvm; + vcpu->nb.notifier_call =3D gunyah_vcpu_rm_notification; + /** + * Ensure we run after the vm_mgr handles the notification and does + * any necessary state changes. + */ + vcpu->nb.priority =3D -1; + r =3D gunyah_rm_notifier_register(ghvm->rm, &vcpu->nb); + if (r) + return r; + + vcpu->ticket.resource_type =3D GUNYAH_RESOURCE_TYPE_VCPU; + vcpu->ticket.label =3D id; + vcpu->ticket.populate =3D gunyah_vcpu_populate; + vcpu->ticket.unpopulate =3D gunyah_vcpu_unpopulate; + + return gunyah_vm_add_resource_ticket(ghvm, &vcpu->ticket); } =20 int kvm_arch_vcpu_precreate(struct kvm *kvm, unsigned int id) @@ -470,7 +774,8 @@ int kvm_arch_vcpu_precreate(struct kvm *kvm, unsigned i= nt id) =20 int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu) { - return -EINVAL; + GUNYAH_STATE(vcpu); + return gunyah_vcpu_create(ghvm, ghvcpu, vcpu->vcpu_id); } =20 void kvm_arch_vcpu_postcreate(struct kvm_vcpu *vcpu) @@ -479,6 +784,28 @@ void kvm_arch_vcpu_postcreate(struct kvm_vcpu *vcpu) =20 void kvm_arch_vcpu_destroy(struct kvm_vcpu *vcpu) { + GUNYAH_STATE(vcpu); + + gunyah_rm_notifier_unregister(ghvcpu->ghvm->rm, &ghvcpu->nb); + gunyah_vm_remove_resource_ticket(ghvcpu->ghvm, &ghvcpu->ticket); + kfree(ghvcpu); +} + +struct kvm_vcpu *kvm_arch_vcpu_alloc(void) +{ + struct gunyah_vcpu *vcpu; + + vcpu =3D kzalloc(sizeof(*vcpu), GFP_KERNEL_ACCOUNT); + if (!vcpu) + return NULL; + return &vcpu->kvm_vcpu; +} + +void kvm_arch_vcpu_free(struct kvm_vcpu *kvm_vcpu) +{ + struct gunyah_vcpu *vcpu =3D gunyah_vcpu(kvm_vcpu); + + kfree(vcpu); } =20 void kvm_arch_vcpu_blocking(struct kvm_vcpu *vcpu) @@ -521,7 +848,20 @@ bool kvm_arch_vcpu_in_kernel(struct kvm_vcpu *vcpu) =20 int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu) { - return -EINVAL; + GUNYAH_STATE(vcpu); + int ret; + + if (!xchg(&ghvm->started, 1)) { + ret =3D gunyah_vm_start(ghvm); + if (ret) { + xchg(&ghvm->started, 0); + goto out; + } + } + ret =3D gunyah_vcpu_run(ghvcpu); +out: + return ret; + } =20 long kvm_arch_vcpu_ioctl(struct file *filp, diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h index f86f14018734..fa6e3fd4bee1 100644 --- a/include/linux/gunyah.h +++ b/include/linux/gunyah.h @@ -16,9 +16,16 @@ =20 #include =20 +#define gunyah_vcpu(kvm_vcpu_ptr) \ + container_of(kvm_vcpu_ptr, struct gunyah_vcpu, kvm_vcpu) + #define kvm_to_gunyah(kvm_ptr) \ container_of(kvm_ptr, struct gunyah_vm, kvm) =20 +#define GUNYAH_STATE(kvm_vcpu) \ + struct gunyah_vm __maybe_unused *ghvm =3D kvm_to_gunyah(kvm_vcpu->kvm); \ + struct gunyah_vcpu __maybe_unused *ghvcpu =3D gunyah_vcpu(kvm_vcpu) + struct gunyah_vm; =20 /* Matches resource manager's resource types for VM_GET_HYP_RESOURCES RPC = */ @@ -89,6 +96,7 @@ struct gunyah_vm_resource_ticket { */ struct gunyah_vm { u16 vmid; + bool started; struct kvm kvm; struct gunyah_rm *rm; struct notifier_block nb; @@ -101,6 +109,49 @@ struct gunyah_vm { enum gunyah_rm_vm_auth_mechanism auth; }; =20 +/** + * struct gunyah_vcpu - Track an instance of gunyah vCPU + * @kvm_vcpu: kvm instance + * @rsc: Pointer to the Gunyah vCPU resource, will be NULL until VM starts + * @lock: One userspace thread at a time should run the vCPU + * @ghvm: Pointer to the main VM struct; quicker look up than going through + * @f->ghvm + * @state: Our copy of the state of the vCPU, since userspace could trick + * kernel to behave incorrectly if we relied on @vcpu_run + * @ready: if vCPU goes to sleep, hypervisor reports to us that it's sleep= ing + * and will signal interrupt (from @rsc) when it's time to wake up. + * This completion signals that we can run vCPU again. + * @nb: When VM exits, the status of VM is reported via @vcpu_run->status. + * We need to track overall VM status, and the nb gives us the update= s from + * Resource Manager. + * @ticket: resource ticket to claim vCPU# for the VM + */ +struct gunyah_vcpu { + struct kvm_vcpu kvm_vcpu; + struct gunyah_resource *rsc; + struct mutex lock; + struct gunyah_vm *ghvm; + + /** + * Track why the vcpu_run hypercall returned. This mirrors the vcpu_run + * structure shared with userspace, except is used internally to avoid + * trusting userspace to not modify the vcpu_run structure. + */ + enum { + GUNYAH_VCPU_RUN_STATE_UNKNOWN =3D 0, + GUNYAH_VCPU_RUN_STATE_READY, + GUNYAH_VCPU_RUN_STATE_MMIO_READ, + GUNYAH_VCPU_RUN_STATE_MMIO_WRITE, + GUNYAH_VCPU_RUN_STATE_SYSTEM_DOWN, + } state; + + bool immediate_exit; + struct completion ready; + + struct notifier_block nb; + struct gunyah_vm_resource_ticket ticket; +}; + /*************************************************************************= *****/ /* Common arch-independent definitions for Gunyah hypercalls = */ #define GUNYAH_CAPID_INVAL U64_MAX --=20 2.39.5 From nobody Thu Dec 18 00:34:59 2025 Received: from mail-wr1-f53.google.com (mail-wr1-f53.google.com [209.85.221.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9779128EA50 for ; Thu, 24 Apr 2025 14:14:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.53 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504058; cv=none; b=P1WQfEUnS5dqcaw2uKR9PGbLMS22pAsBtUHEWBWsL9Y1GI13pUL476ZVU4/kQDchqobMQn8uYZxDuIAoscFL5QZbAh1eBgGmR7UFCwvqJgqhFOVBnlb4it07du1ivyD1QQbEyu464cleXlH/9X5hR5RuqC02bcAmK0vCZsn1ZoM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504058; c=relaxed/simple; bh=stGjEAXhqZzqZDA4Zy4FUNW4LEqqvQTzHgd1ovyETPw=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=uvogchsra3HzCWmBUt0A82pV6p9gfWKorpkfkexxBusLiCl0n/N0g6l+WnYiumugOt3kaU80npDfREaE/piukCRMkaIqfJSx1VDSA8NyZJwkzfo4SQTlz5K/9j0nWClO8sgL+qGB1QiAAN+5/OpI4NRlXlRnePregiAGPE3g89w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=c+72HnD5; arc=none smtp.client-ip=209.85.221.53 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="c+72HnD5" Received: by mail-wr1-f53.google.com with SMTP id ffacd0b85a97d-39c0dfba946so793810f8f.3 for ; Thu, 24 Apr 2025 07:14:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504053; x=1746108853; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=XbjrAsGjkzHb8tCUfbOmumB9MeDC4bmSUl/RmrD2cWk=; b=c+72HnD5/D8MG9G0SFJbb1Xi+Ip2linbgO0v5+sPWmdtg1xKGVY7/JTb31iY2U6w/P aZcID0zyLXbxvVi+6sg2TtcXxCuM5XboTHBF7kUs70fVR9oaO7HME799SG71ZxFgG7ZN yR4fh4IYnHQUH7kZ+7ooo+vuG4xLT7Wf/7r8WDXkSX2j0WDMln6VE3MPhoKJDCLDjPvr YN13BfURiLm/PPd0CtVivdhZ8tGPV/YZUglcfVPiVZh2vyXBRMTopr5eY0uNbdSUsmHO Q1SmJPQsl3ozkuSBpqvOradrt/k/hxe1iiTTzCH8w2SZVTew6fGj+GPLahoyvvlqpw8F 732w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504053; x=1746108853; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=XbjrAsGjkzHb8tCUfbOmumB9MeDC4bmSUl/RmrD2cWk=; b=WNihF4nzyFCSKLnrhYkAqekMrLXSUzTT93GA+v3pPeOASOalPPWJhuS7RKlqTyDPDM zPgB8EyjExM2OVKF92MP3y1aWOr2aEyaTDnBaFDxNMFv51IxOOyQOiaBb12vAlpDHjza nj/ftlIFx//mLKnmeiJzSfsigI+tgaIENIoo8EuzjVW0mcZUqRWcK0LDJZqSzUgUt1KK xAcnR9on51S1dw2JA6X7FFnBvOhQLTJdTanzg6ISunB9KshJv9WaUFHDoJIz6YRqk/db l8DQybVOqUfXSQDyCuEAuFbBOSF9IBwVf/dZ17GGjnvVfzT4elXY/K2ZyjMIyMitxlJu 70ng== X-Gm-Message-State: AOJu0Yz7JpQowQUCF3SEoNN6o8YaJ3I9J3E8QhVVEeiAj+h1BejXK+U6 vKQr9g3+abuzznqt6/OJk+17OV6sva+djl0KkYG3JGfaQe6XQwdssd+4DLUIUpHOmKQg/IS/Sns 3 X-Gm-Gg: ASbGncuuwWzPyhL4XCsJez63aKaJXVvEDxMPyr9l1J89/lCXomCPcm/1dyA7KlTU+cy JU66GF5LYiHm7U4oB5rvyi8MJVIFnRqvLOaTr6IcHcStpK6z80KSrPEo6sHs4U1Bo0QV3yFiu/x MLKcI1erKYRlFjvMBTzYLLX7QDQ1lf0/JgtmTNYQF9ncURVHvlGy3ZPwGf00dv67SZoUJy5pzQ6 +BQjU0v+BXMNsL3HzTDSANaZlaM9zrjA6C5Wb7uIyOrd627CFb47sh7r3VQzvPpM6c/S9mhgyPZ sOl0ip5I4QKYxcjoMooeUmFtiwJ/uy2kFu0joqO2kNgvdw9MnFMamL+sAuwt+uz3+bAZglaWRjq o8JgtwaqvvY19z44P X-Google-Smtp-Source: AGHT+IGxrh9LsxSTvY+9a9cBpLnjZelejrN6GQ8nRlvgc6+sGPLvZ+HV3GV5bjEoSe2VS8MsJ3m6Aw== X-Received: by 2002:a05:6000:4202:b0:391:40bd:6222 with SMTP id ffacd0b85a97d-3a06cf5c719mr2302063f8f.22.1745504053364; Thu, 24 Apr 2025 07:14:13 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.14.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:14:12 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt , Elliot Berman , Srivatsa Vaddagiri Subject: [RFC PATCH 21/34] gunyah: Add hypercalls for demand paging Date: Thu, 24 Apr 2025 15:13:28 +0100 Message-Id: <20250424141341.841734-22-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Elliot Berman Three hypercalls are needed to support demand paging. In create page mappings for a virtual machine's address space, memory must be moved to a memory extent that is allowed to be mapped into that address space. Memory extents are Gunyah's implementation of access control. Once the memory is moved to the proper memory extent, the memory can be mapped into the VM's address space. Implement the bindings to perform those hypercalls. Signed-off-by: Elliot Berman Reviewed-by: Srivatsa Vaddagiri Signed-off-by: Karim Manaouil --- arch/arm64/gunyah/gunyah_hypercall.c | 87 ++++++++++++++++++++++++++++ arch/arm64/include/asm/gunyah.h | 22 +++++++ include/linux/gunyah.h | 56 ++++++++++++++++++ 3 files changed, 165 insertions(+) diff --git a/arch/arm64/gunyah/gunyah_hypercall.c b/arch/arm64/gunyah/gunya= h_hypercall.c index fee21df42c17..38403dc28c66 100644 --- a/arch/arm64/gunyah/gunyah_hypercall.c +++ b/arch/arm64/gunyah/gunyah_hypercall.c @@ -39,6 +39,9 @@ EXPORT_SYMBOL_GPL(arch_is_gunyah_guest); #define GUNYAH_HYPERCALL_HYP_IDENTIFY GUNYAH_HYPERCALL(0x8000) #define GUNYAH_HYPERCALL_MSGQ_SEND GUNYAH_HYPERCALL(0x801B) #define GUNYAH_HYPERCALL_MSGQ_RECV GUNYAH_HYPERCALL(0x801C) +#define GUNYAH_HYPERCALL_ADDRSPACE_MAP GUNYAH_HYPERCALL(0x802B) +#define GUNYAH_HYPERCALL_ADDRSPACE_UNMAP GUNYAH_HYPERCALL(0x802C) +#define GUNYAH_HYPERCALL_MEMEXTENT_DONATE GUNYAH_HYPERCALL(0x8061) #define GUNYAH_HYPERCALL_VCPU_RUN GUNYAH_HYPERCALL(0x8065) /* clang-format on */ =20 @@ -114,6 +117,90 @@ enum gunyah_error gunyah_hypercall_msgq_recv(u64 capid= , void *buff, size_t size, } EXPORT_SYMBOL_GPL(gunyah_hypercall_msgq_recv); =20 +/** + * gunyah_hypercall_addrspace_map() - Add memory to an address space from = a memory extent + * @capid: Address space capability ID + * @extent_capid: Memory extent capability ID + * @vbase: location in address space + * @extent_attrs: Attributes for the memory + * @flags: Flags for address space mapping + * @offset: Offset into memory extent (physical address of memory) + * @size: Size of memory to map; must be page-aligned + */ +enum gunyah_error gunyah_hypercall_addrspace_map(u64 capid, u64 extent_cap= id, u64 vbase, + u32 extent_attrs, u32 flags, u64 offset, u64 size) +{ + struct arm_smccc_1_2_regs args =3D { + .a0 =3D GUNYAH_HYPERCALL_ADDRSPACE_MAP, + .a1 =3D capid, + .a2 =3D extent_capid, + .a3 =3D vbase, + .a4 =3D extent_attrs, + .a5 =3D flags, + .a6 =3D offset, + .a7 =3D size, + /* C language says this will be implictly zero. Gunyah requires 0, so be= explicit */ + .a8 =3D 0, + }; + struct arm_smccc_1_2_regs res; + + arm_smccc_1_2_hvc(&args, &res); + + return res.a0; +} +EXPORT_SYMBOL_GPL(gunyah_hypercall_addrspace_map); + +/** + * gunyah_hypercall_addrspace_unmap() - Remove memory from an address space + * @capid: Address space capability ID + * @extent_capid: Memory extent capability ID + * @vbase: location in address space + * @flags: Flags for address space mapping + * @offset: Offset into memory extent (physical address of memory) + * @size: Size of memory to map; must be page-aligned + */ +enum gunyah_error gunyah_hypercall_addrspace_unmap(u64 capid, u64 extent_c= apid, u64 vbase, + u32 flags, u64 offset, u64 size) +{ + struct arm_smccc_1_2_regs args =3D { + .a0 =3D GUNYAH_HYPERCALL_ADDRSPACE_UNMAP, + .a1 =3D capid, + .a2 =3D extent_capid, + .a3 =3D vbase, + .a4 =3D flags, + .a5 =3D offset, + .a6 =3D size, + /* C language says this will be implictly zero. Gunyah requires 0, so be= explicit */ + .a7 =3D 0, + }; + struct arm_smccc_1_2_regs res; + + arm_smccc_1_2_hvc(&args, &res); + + return res.a0; +} +EXPORT_SYMBOL_GPL(gunyah_hypercall_addrspace_unmap); + +/** + * gunyah_hypercall_memextent_donate() - Donate memory from one memory ext= ent to another + * @options: donate options + * @from_capid: Memory extent capability ID to donate from + * @to_capid: Memory extent capability ID to donate to + * @offset: Offset into memory extent (physical address of memory) + * @size: Size of memory to donate; must be page-aligned + */ +enum gunyah_error gunyah_hypercall_memextent_donate(u32 options, u64 from_= capid, u64 to_capid, + u64 offset, u64 size) +{ + struct arm_smccc_res res; + + arm_smccc_1_1_hvc(GUNYAH_HYPERCALL_MEMEXTENT_DONATE, options, from_capid,= to_capid, + offset, size, 0, &res); + + return res.a0; +} +EXPORT_SYMBOL_GPL(gunyah_hypercall_memextent_donate); + /** * gunyah_hypercall_vcpu_run() - Donate CPU time to a vcpu * @capid: capability ID of the vCPU to run diff --git a/arch/arm64/include/asm/gunyah.h b/arch/arm64/include/asm/gunya= h.h index 29079d1a4df2..4adf24977fd1 100644 --- a/arch/arm64/include/asm/gunyah.h +++ b/arch/arm64/include/asm/gunyah.h @@ -32,4 +32,26 @@ static inline int arch_gunyah_fill_irq_fwspec_params(u32= virq, } return 0; } + +enum arch_gunyah_memtype { + /* clang-format off */ + GUNYAH_MEMTYPE_DEVICE_nGnRnE =3D 0, + GUNYAH_DEVICE_nGnRE =3D 1, + GUNYAH_DEVICE_nGRE =3D 2, + GUNYAH_DEVICE_GRE =3D 3, + + GUNYAH_NORMAL_NC =3D 0b0101, + GUNYAH_NORMAL_ONC_IWT =3D 0b0110, + GUNYAH_NORMAL_ONC_IWB =3D 0b0111, + GUNYAH_NORMAL_OWT_INC =3D 0b1001, + GUNYAH_NORMAL_WT =3D 0b1010, + GUNYAH_NORMAL_OWT_IWB =3D 0b1011, + GUNYAH_NORMAL_OWB_INC =3D 0b1101, + GUNYAH_NORMAL_OWB_IWT =3D 0b1110, + GUNYAH_NORMAL_WB =3D 0b1111, + /* clang-format on */ +}; + +#define ARCH_GUNYAH_DEFAULT_MEMTYPE GUNYAH_NORMAL_WB + #endif diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h index fa6e3fd4bee1..2648d3a623de 100644 --- a/include/linux/gunyah.h +++ b/include/linux/gunyah.h @@ -270,6 +270,62 @@ enum gunyah_error gunyah_hypercall_msgq_send(u64 capid= , size_t size, void *buff, enum gunyah_error gunyah_hypercall_msgq_recv(u64 capid, void *buff, size_t= size, size_t *recv_size, bool *ready); =20 +#define GUNYAH_ADDRSPACE_SELF_CAP 0 + +enum gunyah_pagetable_access { + /* clang-format off */ + GUNYAH_PAGETABLE_ACCESS_NONE =3D 0, + GUNYAH_PAGETABLE_ACCESS_X =3D 1, + GUNYAH_PAGETABLE_ACCESS_W =3D 2, + GUNYAH_PAGETABLE_ACCESS_R =3D 4, + GUNYAH_PAGETABLE_ACCESS_RX =3D 5, + GUNYAH_PAGETABLE_ACCESS_RW =3D 6, + GUNYAH_PAGETABLE_ACCESS_RWX =3D 7, + /* clang-format on */ +}; + +/* clang-format off */ +#define GUNYAH_MEMEXTENT_MAPPING_USER_ACCESS GENMASK_ULL(2, 0) +#define GUNYAH_MEMEXTENT_MAPPING_KERNEL_ACCESS GENMASK_ULL(6, 4) +#define GUNYAH_MEMEXTENT_MAPPING_TYPE GENMASK_ULL(23, 16) +/* clang-format on */ + +enum gunyah_memextent_donate_type { + /* clang-format off */ + GUNYAH_MEMEXTENT_DONATE_TO_CHILD =3D 0, + GUNYAH_MEMEXTENT_DONATE_TO_PARENT =3D 1, + GUNYAH_MEMEXTENT_DONATE_TO_SIBLING =3D 2, + GUNYAH_MEMEXTENT_DONATE_TO_PROTECTED =3D 3, + GUNYAH_MEMEXTENT_DONATE_FROM_PROTECTED =3D 4, + /* clang-format on */ +}; + +enum gunyah_addrspace_map_flag_bits { + /* clang-format off */ + GUNYAH_ADDRSPACE_MAP_FLAG_PARTIAL =3D 0, + GUNYAH_ADDRSPACE_MAP_FLAG_PRIVATE =3D 1, + GUNYAH_ADDRSPACE_MAP_FLAG_VMMIO =3D 2, + GUNYAH_ADDRSPACE_MAP_FLAG_NOSYNC =3D 31, + /* clang-format on */ +}; + +enum gunyah_error gunyah_hypercall_addrspace_map(u64 capid, u64 extent_cap= id, + u64 vbase, u32 extent_attrs, + u32 flags, u64 offset, + u64 size); +enum gunyah_error gunyah_hypercall_addrspace_unmap(u64 capid, u64 extent_c= apid, + u64 vbase, u32 flags, + u64 offset, u64 size); + +/* clang-format off */ +#define GUNYAH_MEMEXTENT_OPTION_TYPE_MASK GENMASK_ULL(7, 0) +#define GUNYAH_MEMEXTENT_OPTION_NOSYNC BIT(31) +/* clang-format on */ + +enum gunyah_error gunyah_hypercall_memextent_donate(u32 options, u64 from_= capid, + u64 to_capid, u64 offset, + u64 size); + struct gunyah_hypercall_vcpu_run_resp { union { enum { --=20 2.39.5 From nobody Thu Dec 18 00:34:59 2025 Received: from mail-wr1-f52.google.com (mail-wr1-f52.google.com [209.85.221.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EFDF128FFD4 for ; Thu, 24 Apr 2025 14:14:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.52 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504060; cv=none; b=szHr0Y1Dfn58m6lDUNOrkC/46QuIkKGeZVj9NYL0SV0oCxPHOZoKwsTLx+ITMIRlLgm3Ms3KcYxsgj2BKkDJSFFlHOGRXUAq+VJMnxl0tx8f2xnzR8dniF+I7GWaOEEb7wGA0mv67bEkZ7J0m38TzXTbHQFd4y7xHsDQb/YTRb8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504060; c=relaxed/simple; bh=5x04Nc6aN+Qwp5pR10UBtd/47cBSvolTTTY6q9NX8X0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=jr4MAXGjD9knltUe0sa60YwujYeqftwJ8zTK6lrpSi8FT/LIibsRksVm3ifXwMaKU9tfs77ADNzeCbACHp+FwDEVXCPIuoeBSx2BN/r3dOA/GLlCJ7PZgpd9J6ym3QKvRe8B671PBYD51ZKtyxDxpElxgFCYCNQCi6baXfXHHoU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=Vo+qX0tx; arc=none smtp.client-ip=209.85.221.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="Vo+qX0tx" Received: by mail-wr1-f52.google.com with SMTP id ffacd0b85a97d-39ee5ac4321so1168838f8f.1 for ; Thu, 24 Apr 2025 07:14:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504055; x=1746108855; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=2+5ILmQ9VnpjXFSHOfGNmys0dMzibsKAG6xUYDKquGI=; b=Vo+qX0txD0lSEyG/0ftOH91Av7zp+QzKC4xemOoWPqXd3LVq48luY27dYzPzgaecnC ygbLJUhxOjaEmrUhZ5hp0Q9pUQ7vJQSRZAF6/BmdFSLTmSnyY39ETKz6qJlMVZD1BxJ2 DIMk9D+fvMxFe0f2TrsVSYnd2l9iLVm7OYDaZrqAxSXncjnApI+DAnIJRsx7T15xixPt 1dCmpKe/rcwUJ6zSy2S8Eo8+glDn+5e58dx1ZhRtQ7FwvzhLS0e/zkURm+LprL4vr8AV V25Hvqi0ZQ74zVBbBCUfyWBNQrJZhjS1yEove6AlxwCHDRAevfhFIcG6oIgss9SnzdLm VOcw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504055; x=1746108855; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2+5ILmQ9VnpjXFSHOfGNmys0dMzibsKAG6xUYDKquGI=; b=OEKV8zSwJbR6yRtVxBv6HEcOoqf/2sWK9HXQPEtqsSPGX4YRgudVLQVBokEumvzM2F EkSGRLILSnc78B2P6zEEYnHOjHIqfiP9bSaixO2sdlxGXOnBrqDOkynE43jWBazXZxUj bnQRUXcek+GXB8sewS3qEMXueF4VfSQk1u8ttCbfXevZrYH6SLz2kHmfVcF0IPmjOoMT qbDhYsYtEOHV6haAmdkIRBMwNLRO/zj+5ZIRtEwchTaZRgr/meUlPF4vyRsRVzkOLcuZ X3H7cstog3z5eLCgqM+N3GI2BoTZEWq7PYEGhs8ohTT5Z7VpNpdiDMbQRHgioQ1CSn2M P4mQ== X-Gm-Message-State: AOJu0YzALj1i0jmfnln2MKdJuTC4nH6VmXGdjUv8m/nORMt47kcyAeuy ioMoJcjDjGxtxnNjM1UtGflLVhpsFIBhuCpgyOoLRn85FkYgCRTRoeJc2nxdZa3PbKJGux3n960 q X-Gm-Gg: ASbGncveFQmRp34uHlt1NN0Dg5gbCRIVVEaRHomd3jQOYtfNDl5Wmj4gT7BmobHMdpl tMweGQrydwCfnhhm3ZaZNhFSLxMkPZBzu2uXji4rIjMgZmhVG3hNG0p6ESwg3nYeB3CutbCZhJq Al+eRl8N9Oka98OM+lxCgoNAzPnbqyzJWdrQUHd8KEHmSILH+iUGomcu5jeV5QLdgVucJ+YsKuI 6UGhiCIAEdtbqRCTBvIKdZI8XQ32HefsFoN7XOKkb1bJToX6ELcFgfsaBTYjiGjFyEu7WYY+S2w vk+H9/mq3HTwFNB/dIG5RxGaf1LwY4dXSOmK0SsTSHwNeDQcvhhM1vl/b9z2yIbXBoVxCjUebgQ itful/NtuVjC/JTt9eAV8YwqKgv0= X-Google-Smtp-Source: AGHT+IFKyLoVojvk9QX3IrdV6JaO3B9fcEr46gjxxC0HMPFKU10zHQ1xzS7LtC4gCESH366Qprg02w== X-Received: by 2002:a5d:64e6:0:b0:39c:dfa:ca71 with SMTP id ffacd0b85a97d-3a06cfab61dmr2330540f8f.49.1745504054653; Thu, 24 Apr 2025 07:14:14 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.14.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:14:14 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt , Elliot Berman Subject: [RFC PATCH 22/34] gunyah: Add memory parcel RPC Date: Thu, 24 Apr 2025 15:13:29 +0100 Message-Id: <20250424141341.841734-23-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Elliot Berman In a Gunyah hypervisor system using the Gunyah Resource Manager, the "standard" unit of donating, lending and sharing memory is called a memory parcel (memparcel). A memparcel is an abstraction used by the resource manager for securely managing donating, lending and sharing memory, which may be physically and virtually fragmented, without dealing directly with physical memory addresses. Memparcels are created and managed through the RM RPC functions for lending, sharing and reclaiming memory from VMs. When creating a new VM the initial VM memory containing the VM image and the VM's device tree blob must be provided as a memparcel. The memparcel must be created using the RM RPC for lending and mapping the memory to the VM. Signed-off-by: Elliot Berman Reviewed-by: Srivatsa Vaddagiri Signed-off-by: Karim Manaouil --- drivers/virt/gunyah/rsc_mgr_rpc.c | 204 ++++++++++++++++++++++++++++++ include/linux/gunyah_rsc_mgr.h | 50 ++++++++ 2 files changed, 254 insertions(+) diff --git a/drivers/virt/gunyah/rsc_mgr_rpc.c b/drivers/virt/gunyah/rsc_mg= r_rpc.c index 936592177ddb..0266c2a8d583 100644 --- a/drivers/virt/gunyah/rsc_mgr_rpc.c +++ b/drivers/virt/gunyah/rsc_mgr_rpc.c @@ -7,6 +7,12 @@ #include #include =20 +/* Message IDs: Memory Management */ +#define GUNYAH_RM_RPC_MEM_LEND 0x51000012 +#define GUNYAH_RM_RPC_MEM_SHARE 0x51000013 +#define GUNYAH_RM_RPC_MEM_RECLAIM 0x51000015 +#define GUNYAH_RM_RPC_MEM_APPEND 0x51000018 + /* Message IDs: VM Management */ /* clang-format off */ #define GUNYAH_RM_RPC_VM_ALLOC_VMID 0x56000001 @@ -23,6 +29,49 @@ #define GUNYAH_RM_RPC_VM_SET_ADDRESS_LAYOUT 0x56000034 /* clang-format on */ =20 +/* Call: MEM_LEND, MEM_SHARE */ +#define GUNYAH_RM_MAX_MEM_ENTRIES 512 + +#define GUNYAH_MEM_SHARE_REQ_FLAGS_APPEND BIT(1) + +struct gunyah_rm_mem_share_req_header { + u8 mem_type; + u8 _padding0; + u8 flags; + u8 _padding1; + __le32 label; +} __packed; + +struct gunyah_rm_mem_share_req_acl_section { + __le16 n_entries; + __le16 _padding; + struct gunyah_rm_mem_acl_entry entries[]; +} __packed; + +struct gunyah_rm_mem_share_req_mem_section { + __le16 n_entries; + __le16 _padding; + struct gunyah_rm_mem_entry entries[]; +} __packed; + +/* Call: MEM_RELEASE */ +struct gunyah_rm_mem_release_req { + __le32 mem_handle; + u8 flags; /* currently not used */ + u8 _padding0; + __le16 _padding1; +} __packed; + +/* Call: MEM_APPEND */ +#define GUNYAH_MEM_APPEND_REQ_FLAGS_END BIT(0) + +struct gunyah_rm_mem_append_req_header { + __le32 mem_handle; + u8 flags; + u8 _padding0; + __le16 _padding1; +} __packed; + struct gunyah_rm_vm_common_vmid_req { __le16 vmid; __le16 _padding; @@ -72,6 +121,161 @@ static int gunyah_rm_common_vmid_call(struct gunyah_rm= *rm, u32 message_id, NULL, NULL); } =20 +static int gunyah_rm_mem_append(struct gunyah_rm *rm, u32 mem_handle, + struct gunyah_rm_mem_entry *entries, + size_t n_entries) +{ + struct gunyah_rm_mem_append_req_header *req __free(kfree) =3D NULL; + struct gunyah_rm_mem_share_req_mem_section *mem; + int ret =3D 0; + size_t n; + + req =3D kzalloc(sizeof(*req) + struct_size(mem, entries, GUNYAH_RM_MAX_ME= M_ENTRIES), + GFP_KERNEL); + if (!req) + return -ENOMEM; + + req->mem_handle =3D cpu_to_le32(mem_handle); + mem =3D (void *)(req + 1); + + while (n_entries) { + req->flags =3D 0; + if (n_entries > GUNYAH_RM_MAX_MEM_ENTRIES) { + n =3D GUNYAH_RM_MAX_MEM_ENTRIES; + } else { + req->flags |=3D GUNYAH_MEM_APPEND_REQ_FLAGS_END; + n =3D n_entries; + } + + mem->n_entries =3D cpu_to_le16(n); + memcpy(mem->entries, entries, sizeof(*entries) * n); + + ret =3D gunyah_rm_call(rm, GUNYAH_RM_RPC_MEM_APPEND, req, + sizeof(*req) + struct_size(mem, entries, n), + NULL, NULL); + if (ret) + break; + + entries +=3D n; + n_entries -=3D n; + } + + return ret; +} + +/** + * gunyah_rm_mem_share() - Share memory with other virtual machines. + * @rm: Handle to a Gunyah resource manager + * @p: Information about the memory to be shared. + * + * Sharing keeps Linux's access to the memory while the memory parcel is s= hared. + */ +int gunyah_rm_mem_share(struct gunyah_rm *rm, struct gunyah_rm_mem_parcel = *p) +{ + u32 message_id =3D p->n_acl_entries =3D=3D 1 ? GUNYAH_RM_RPC_MEM_LEND : + GUNYAH_RM_RPC_MEM_SHARE; + size_t msg_size, initial_mem_entries =3D p->n_mem_entries, resp_size; + struct gunyah_rm_mem_share_req_acl_section *acl; + struct gunyah_rm_mem_share_req_mem_section *mem; + struct gunyah_rm_mem_share_req_header *req_header; + size_t acl_size, mem_size; + u32 *attr_section; + bool need_append =3D false; + __le32 *resp; + void *msg; + int ret; + + if (!p->acl_entries || !p->n_acl_entries || !p->mem_entries || + !p->n_mem_entries || p->n_acl_entries > U8_MAX || + p->mem_handle !=3D GUNYAH_MEM_HANDLE_INVAL) + return -EINVAL; + + if (initial_mem_entries > GUNYAH_RM_MAX_MEM_ENTRIES) { + initial_mem_entries =3D GUNYAH_RM_MAX_MEM_ENTRIES; + need_append =3D true; + } + + acl_size =3D struct_size(acl, entries, p->n_acl_entries); + mem_size =3D struct_size(mem, entries, initial_mem_entries); + + /* The format of the message goes: + * request header + * ACL entries (which VMs get what kind of access to this memory parcel) + * Memory entries (list of memory regions to share) + * Memory attributes (currently unused, we'll hard-code the size to 0) + */ + msg_size =3D sizeof(struct gunyah_rm_mem_share_req_header) + acl_size + + mem_size + + sizeof(u32); /* for memory attributes, currently unused */ + + msg =3D kzalloc(msg_size, GFP_KERNEL); + if (!msg) + return -ENOMEM; + + req_header =3D msg; + acl =3D (void *)req_header + sizeof(*req_header); + mem =3D (void *)acl + acl_size; + attr_section =3D (void *)mem + mem_size; + + req_header->mem_type =3D p->mem_type; + if (need_append) + req_header->flags |=3D GUNYAH_MEM_SHARE_REQ_FLAGS_APPEND; + req_header->label =3D cpu_to_le32(p->label); + + acl->n_entries =3D cpu_to_le32(p->n_acl_entries); + memcpy(acl->entries, p->acl_entries, + flex_array_size(acl, entries, p->n_acl_entries)); + + mem->n_entries =3D cpu_to_le16(initial_mem_entries); + memcpy(mem->entries, p->mem_entries, + flex_array_size(mem, entries, initial_mem_entries)); + + /* Set n_entries for memory attribute section to 0 */ + *attr_section =3D 0; + + ret =3D gunyah_rm_call(rm, message_id, msg, msg_size, (void **)&resp, + &resp_size); + kfree(msg); + + if (ret) + return ret; + + p->mem_handle =3D le32_to_cpu(*resp); + kfree(resp); + + if (need_append) { + ret =3D gunyah_rm_mem_append( + rm, p->mem_handle, &p->mem_entries[initial_mem_entries], + p->n_mem_entries - initial_mem_entries); + if (ret) { + gunyah_rm_mem_reclaim(rm, p); + p->mem_handle =3D GUNYAH_MEM_HANDLE_INVAL; + } + } + + return ret; +} +ALLOW_ERROR_INJECTION(gunyah_rm_mem_share, ERRNO); + +/** + * gunyah_rm_mem_reclaim() - Reclaim a memory parcel + * @rm: Handle to a Gunyah resource manager + * @parcel: Information about the memory to be reclaimed. + * + * RM maps the associated memory back into the stage-2 page tables of the = owner VM. + */ +int gunyah_rm_mem_reclaim(struct gunyah_rm *rm, + struct gunyah_rm_mem_parcel *parcel) +{ + struct gunyah_rm_mem_release_req req =3D { + .mem_handle =3D cpu_to_le32(parcel->mem_handle), + }; + + return gunyah_rm_call(rm, GUNYAH_RM_RPC_MEM_RECLAIM, &req, sizeof(req), + NULL, NULL); +} +ALLOW_ERROR_INJECTION(gunyah_rm_mem_reclaim, ERRNO); + /** * gunyah_rm_alloc_vmid() - Allocate a new VM in Gunyah. Returns the VM id= entifier. * @rm: Handle to a Gunyah resource manager diff --git a/include/linux/gunyah_rsc_mgr.h b/include/linux/gunyah_rsc_mgr.h index c0fe516d54a8..c42a0cb42ba6 100644 --- a/include/linux/gunyah_rsc_mgr.h +++ b/include/linux/gunyah_rsc_mgr.h @@ -9,6 +9,7 @@ #include =20 #define GUNYAH_VMID_INVAL U16_MAX +#define GUNYAH_MEM_HANDLE_INVAL U32_MAX =20 struct gunyah_rm; =20 @@ -57,6 +58,55 @@ struct gunyah_rm_vm_status_payload { __le16 app_status; } __packed; =20 +#define GUNYAH_RM_ACL_X BIT(0) +#define GUNYAH_RM_ACL_W BIT(1) +#define GUNYAH_RM_ACL_R BIT(2) + +struct gunyah_rm_mem_acl_entry { + __le16 vmid; + u8 perms; + u8 reserved; +} __packed; + +struct gunyah_rm_mem_entry { + __le64 phys_addr; + __le64 size; +} __packed; + +enum gunyah_rm_mem_type { + GUNYAH_RM_MEM_TYPE_NORMAL =3D 0, + GUNYAH_RM_MEM_TYPE_IO =3D 1, +}; + +/* + * struct gunyah_rm_mem_parcel - Info about memory to be lent/shared/donat= ed/reclaimed + * @mem_type: The type of memory: normal (DDR) or IO + * @label: An client-specified identifier which can be used by the other V= Ms to identify the purpose + * of the memory parcel. + * @n_acl_entries: Count of the number of entries in the @acl_entries arra= y. + * @acl_entries: An array of access control entries. Each entry specifies = a VM and what access + * is allowed for the memory parcel. + * @n_mem_entries: Count of the number of entries in the @mem_entries arra= y. + * @mem_entries: An array of regions to be associated with the memory parc= el. Addresses should be + * (intermediate) physical addresses from Linux's perspectiv= e. + * @mem_handle: On success, filled with memory handle that RM allocates fo= r this memory parcel + */ +struct gunyah_rm_mem_parcel { + enum gunyah_rm_mem_type mem_type; + u32 label; + size_t n_acl_entries; + struct gunyah_rm_mem_acl_entry *acl_entries; + size_t n_mem_entries; + struct gunyah_rm_mem_entry *mem_entries; + u32 mem_handle; +}; + +/* RPC Calls */ +int gunyah_rm_mem_share(struct gunyah_rm *rm, + struct gunyah_rm_mem_parcel *parcel); +int gunyah_rm_mem_reclaim(struct gunyah_rm *rm, + struct gunyah_rm_mem_parcel *parcel); + int gunyah_rm_alloc_vmid(struct gunyah_rm *rm, u16 vmid); int gunyah_rm_dealloc_vmid(struct gunyah_rm *rm, u16 vmid); int gunyah_rm_vm_reset(struct gunyah_rm *rm, u16 vmid); --=20 2.39.5 From nobody Thu Dec 18 00:34:59 2025 Received: from mail-wm1-f42.google.com (mail-wm1-f42.google.com [209.85.128.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4A0D628FFF5 for ; Thu, 24 Apr 2025 14:14:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.42 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504061; cv=none; b=hHOTXXXHjc2/apYp3SBTE/RGAQKXnkcmoRSnDD8IFQAX1CpuWKh1vfnSw/bojnuIaqGtlshWTl6n1CsWdDYdMHA5+fbvnLTmrmWNIOewoNKJkEQjdejXPrl8q9pZYhTLlMXofW5hwhkK3SqRLMsiwCXyZM/S5LyFoYCHlZN+0e0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504061; c=relaxed/simple; bh=NPLvNPRTGZlQyPSKuchNVhThznQp0bCS99QwrEGsnQg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=nrUW5Jqk4ToQgqaLZGkHx/vtOz8TqvoY0YuJ1jKHOjwUizVl8QNeBGcqD7azkH6vYz6SvhxZA3vh7eT2/D77bie5RF1vNyCSQ4EEahLe0GAiFE+gj17Csf13buJGxROfP7MDcq6Lk0tOuLFWkoLmiZwL8LN2Wq6iDqRTKLS7gl4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=LGaJZa+G; arc=none smtp.client-ip=209.85.128.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="LGaJZa+G" Received: by mail-wm1-f42.google.com with SMTP id 5b1f17b1804b1-43ea40a6e98so10390175e9.1 for ; Thu, 24 Apr 2025 07:14:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504056; x=1746108856; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=2LhbI7ata+xuMo4MQ3LDVVCBYF7iuDckOVgkFelFFVk=; b=LGaJZa+Gqb1A6S6wHwdOIGAHjxh7A1/kh4PeILy9RALRqV113f7vzYy1odu2l7vDad EGO/7UIQ+8a/vpk9fqX4WLk9mgwDX8Iqcgh5Po46E6eTHRQqJi+ePdSBD97nvLHLh2lV +ayjuxpSwfFQuHTTzrZACtNRtbsOMvO7svd1tlUGmopAuMl7qbpMEZxhTvCGdqVdatIC ZumxSbQ7yTOs0+CipPSOO+VKWmdUw3/IaxRWuKiox9SJWFDazZyqw5SujqsRLLpGx54c 5yS0QvcZM5+Ac8QSB3TW3YT/3YhvunOXPS3Php1i2WecpS6diPIMdBBEQPF2Oj71erem 0cNg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504056; x=1746108856; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2LhbI7ata+xuMo4MQ3LDVVCBYF7iuDckOVgkFelFFVk=; b=F1O37dE5Mf459JHOCFeBQHtaniwjHw0vce3bhFPv8Cfqsdjyh47x2Ta21LzIvnvQF8 /91ckjY7zeqN1TGfAGZOzKlM6CZeb7s+90bjHMQjBDCihwO85Utk08KyCbMVSzqV7pOF SEK1Xmc0jiXI2CnbR/WBneP9mgO9buihX99pEGLzqWeGZf5BIfYp41wzs3J9vJdwlTSS eo587B7jgtfEAzqy2/IYm2SCBn+Ke9G++7D6LHC6h6UzWqfo4OQQF8Y6CAI2ife9A6UD exEBU8qXSycaxgfBKUJdJI++qeKFII/HXdPmGHBX0I9mR3RsiDhwdvsykpaRCNond4ji FVZA== X-Gm-Message-State: AOJu0YwSdkHWU7bwVaVCID39JKkjjenRbagZsC20Vo6xD3+N7AT2xk1/ 39jLbsnPAGpJ28TOBMNAGqCvbCUERI7RM01txsD/14chaFlOLmXBaXHf4EuhnDV2X+FYRdRRtSR X X-Gm-Gg: ASbGncu0LpgFmSZZpsquJl9E9gtqFGDosDhLZSDXL9ci5wdVZ+OCF67S67HVPU3chSs hxrsONJTRdvSeg5hch4L5BZZBcV7g0OsNkBgmwGBydJLDJ8sLoUPm7ejLfBf5ThrMfrjhbev88z OkzvN4nYfL/iq5pGhcYMRjbhmjd4Ef/CroZT8o3P9RnOnNmHwdtSmm+Q3U9Rnx7ZpdBoqSO+AIC IMb2kCE15L1AygxyWMVSG7CrAUVaVkqgWwOc3EzlPjaTL/UAQH+FrOVr+Q2NZQhfMsBHsxdZi3e TF+P43FZ1PXcJBgq1wRdJ9EeDBbdiKQtGV19WcLIH9mr4MStI0XxMrJ2AqGOHMP+u2JhDIUnRS5 bHx9T3jtUAbXAqhfZ X-Google-Smtp-Source: AGHT+IHqcrFkop2AMczZB7InA+Ahpw3ClIzGk7Xop+jnijaBOC/Y4QzTBvOvzogG1ZcwEVc5x1XCjQ== X-Received: by 2002:a05:600c:8106:b0:440:6852:5b31 with SMTP id 5b1f17b1804b1-4409bd0d902mr31333525e9.10.1745504055954; Thu, 24 Apr 2025 07:14:15 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.14.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:14:15 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt , Elliot Berman Subject: [RFC PATCH 23/34] gunyah: Add interfaces to map memory into guest address space Date: Thu, 24 Apr 2025 15:13:30 +0100 Message-Id: <20250424141341.841734-24-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Elliot Berman This patch is a subset of [1], without gunyah guest-memfd parts. I added the original commit message below. [1] https://lore.kernel.org/lkml/20240222-gunyah-v17-20-1e9da6763d38@quicin= c.com/ --- arch/arm64/kvm/gunyah.c | 212 +++++++++++++++++++++++++++++++++++++++- include/linux/gunyah.h | 6 ++ 2 files changed, 217 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kvm/gunyah.c b/arch/arm64/kvm/gunyah.c index e066482c2e71..5d18d133df50 100644 --- a/arch/arm64/kvm/gunyah.c +++ b/arch/arm64/kvm/gunyah.c @@ -10,8 +10,8 @@ #include #include #include -#include #include +#include =20 #include #include @@ -19,6 +19,15 @@ #undef pr_fmt #define pr_fmt(fmt) "gunyah: " fmt =20 +#define GUNYAH_VM_ADDRSPACE_LABEL 0 +#define GUNYAH_VM_MEM_EXTENT_GUEST_PRIVATE_LABEL 0 +#define GUNYAH_VM_MEM_EXTENT_HOST_SHARED_LABEL 1 +#define GUNYAH_VM_MEM_EXTENT_GUEST_SHARED_LABEL 3 +#define GUNYAH_VM_MEM_EXTENT_HOST_PRIVATE_LABEL 2 + +#define WRITE_TAG (1 << 0) +#define SHARE_TAG (1 << 1) + static int gunyah_vm_start(struct gunyah_vm *ghvm); =20 static enum kvm_mode kvm_mode =3D KVM_MODE_DEFAULT; @@ -332,6 +341,23 @@ int kvm_arm_set_reg(struct kvm_vcpu *vcpu, const struc= t kvm_one_reg *reg) } } =20 +static bool gunyah_vm_resource_ticket_populate_noop( + struct gunyah_vm_resource_ticket *ticket, struct gunyah_resource *ghrsc) +{ + return true; +} +static void gunyah_vm_resource_ticket_unpopulate_noop( + struct gunyah_vm_resource_ticket *ticket, struct gunyah_resource *ghrsc) +{ +} + +static inline struct gunyah_resource * +__first_resource(struct gunyah_vm_resource_ticket *ticket) +{ + return list_first_entry_or_null(&ticket->resources, + struct gunyah_resource, list); +} + static int gunyah_vm_add_resource_ticket(struct gunyah_vm *ghvm, struct gunyah_vm_resource_ticket *ticket) { @@ -430,6 +456,159 @@ static void gunyah_vm_clean_resources(struct gunyah_v= m *ghvm) mutex_unlock(&ghvm->resources_lock); } =20 +static inline u32 donate_flags(bool share) +{ + if (share) + return FIELD_PREP_CONST(GUNYAH_MEMEXTENT_OPTION_TYPE_MASK, + GUNYAH_MEMEXTENT_DONATE_TO_SIBLING); + else + return FIELD_PREP_CONST(GUNYAH_MEMEXTENT_OPTION_TYPE_MASK, + GUNYAH_MEMEXTENT_DONATE_TO_PROTECTED); +} + +static inline u32 reclaim_flags(bool share) +{ + if (share) + return FIELD_PREP_CONST(GUNYAH_MEMEXTENT_OPTION_TYPE_MASK, + GUNYAH_MEMEXTENT_DONATE_TO_SIBLING); + else + return FIELD_PREP_CONST(GUNYAH_MEMEXTENT_OPTION_TYPE_MASK, + GUNYAH_MEMEXTENT_DONATE_FROM_PROTECTED); +} + +static int gunyah_memory_provide_folio(struct gunyah_vm *ghvm, + struct folio *folio, gfn_t gfn, bool share, bool write) +{ + struct gunyah_resource *guest_extent, *host_extent, *addrspace; + u32 map_flags =3D BIT(GUNYAH_ADDRSPACE_MAP_FLAG_PARTIAL); + u64 extent_attrs; + gfn_t gpa =3D gfn_to_gpa(gfn); + phys_addr_t pa =3D PFN_PHYS(folio_pfn(folio)); + enum gunyah_pagetable_access access; + size_t size =3D folio_size(folio); + enum gunyah_error gunyah_error; + unsigned long tag =3D 0; + int ret; + + if (share) { + guest_extent =3D __first_resource(&ghvm->guest_shared_extent_ticket); + host_extent =3D __first_resource(&ghvm->host_shared_extent_ticket); + } else { + guest_extent =3D __first_resource(&ghvm->guest_private_extent_ticket); + host_extent =3D __first_resource(&ghvm->host_private_extent_ticket); + } + addrspace =3D __first_resource(&ghvm->addrspace_ticket); + + if (!addrspace || !guest_extent || !host_extent) + return -ENODEV; + + if (share) { + map_flags |=3D BIT(GUNYAH_ADDRSPACE_MAP_FLAG_VMMIO); + tag |=3D SHARE_TAG; + } else { + map_flags |=3D BIT(GUNYAH_ADDRSPACE_MAP_FLAG_PRIVATE); + } + + if (write) + tag |=3D WRITE_TAG; + + if (share && write) + access =3D GUNYAH_PAGETABLE_ACCESS_RW; + else if (share && !write) + access =3D GUNYAH_PAGETABLE_ACCESS_R; + else if (!share && write) + access =3D GUNYAH_PAGETABLE_ACCESS_RWX; + else /* !share && !write */ + access =3D GUNYAH_PAGETABLE_ACCESS_RX; + + gunyah_error =3D gunyah_hypercall_memextent_donate(donate_flags(share), + host_extent->capid, + guest_extent->capid, + pa, size); + if (gunyah_error !=3D GUNYAH_ERROR_OK) { + pr_err("Failed to donate memory for guest address 0x%016llx: %d\n", + gpa, gunyah_error); + return gunyah_error_remap(gunyah_error); + } + + extent_attrs =3D + FIELD_PREP_CONST(GUNYAH_MEMEXTENT_MAPPING_TYPE, + ARCH_GUNYAH_DEFAULT_MEMTYPE) | + FIELD_PREP(GUNYAH_MEMEXTENT_MAPPING_USER_ACCESS, access) | + FIELD_PREP(GUNYAH_MEMEXTENT_MAPPING_KERNEL_ACCESS, access); + gunyah_error =3D gunyah_hypercall_addrspace_map(addrspace->capid, + guest_extent->capid, gpa, + extent_attrs, map_flags, + pa, size); + if (gunyah_error !=3D GUNYAH_ERROR_OK) { + pr_err("Failed to map guest address 0x%016llx: %d\n", gpa, + gunyah_error); + ret =3D gunyah_error_remap(gunyah_error); + goto memextent_reclaim; + } + + return 0; +memextent_reclaim: + gunyah_error =3D gunyah_hypercall_memextent_donate(reclaim_flags(share), + guest_extent->capid, + host_extent->capid, pa, + size); + if (gunyah_error !=3D GUNYAH_ERROR_OK) + pr_err("Failed to reclaim memory donation for guest address 0x%016llx: %= d\n", + gpa, gunyah_error); + return ret; +} + +static int gunyah_memory_reclaim_folio(struct gunyah_vm *ghvm, + struct folio *folio, gfn_t gfn, bool share) +{ + u32 map_flags =3D BIT(GUNYAH_ADDRSPACE_MAP_FLAG_PARTIAL); + struct gunyah_resource *guest_extent, *host_extent, *addrspace; + enum gunyah_error gunyah_error; + phys_addr_t pa; + size_t size; + int ret; + + addrspace =3D __first_resource(&ghvm->addrspace_ticket); + if (!addrspace) + return -ENODEV; + + guest_extent =3D __first_resource(&ghvm->guest_private_extent_ticket); + host_extent =3D __first_resource(&ghvm->host_private_extent_ticket); + map_flags |=3D BIT(GUNYAH_ADDRSPACE_MAP_FLAG_PRIVATE); + + pa =3D PFN_PHYS(folio_pfn(folio)); + size =3D folio_size(folio); + + gunyah_error =3D gunyah_hypercall_addrspace_unmap(addrspace->capid, + guest_extent->capid, + gfn_to_gpa(gfn), + map_flags, pa, size); + if (gunyah_error !=3D GUNYAH_ERROR_OK) { + pr_err_ratelimited( + "Failed to unmap guest address 0x%016llx: %d\n", + gfn_to_gpa(gfn), gunyah_error); + ret =3D gunyah_error_remap(gunyah_error); + goto err; + } + + gunyah_error =3D gunyah_hypercall_memextent_donate(reclaim_flags(share), + guest_extent->capid, + host_extent->capid, pa, + size); + if (gunyah_error !=3D GUNYAH_ERROR_OK) { + pr_err_ratelimited( + "Failed to reclaim memory donation for guest address 0x%016llx: %d\n", + gfn_to_gpa(gfn), gunyah_error); + ret =3D gunyah_error_remap(gunyah_error); + goto err; + } + + return 0; +err: + return ret; +} + int kvm_arch_vcpu_should_kick(struct kvm_vcpu *vcpu) { return kvm_vcpu_exiting_guest_mode(vcpu) =3D=3D IN_GUEST_MODE; @@ -1357,6 +1536,17 @@ static int gunyah_vm_start(struct gunyah_vm *ghvm) return ret; } =20 +static inline void setup_extent_ticket(struct gunyah_vm *ghvm, + struct gunyah_vm_resource_ticket *ticket, + u32 label) +{ + ticket->resource_type =3D GUNYAH_RESOURCE_TYPE_MEM_EXTENT; + ticket->label =3D label; + ticket->populate =3D gunyah_vm_resource_ticket_populate_noop; + ticket->unpopulate =3D gunyah_vm_resource_ticket_unpopulate_noop; + gunyah_vm_add_resource_ticket(ghvm, ticket); +} + static struct gunyah_vm *gunyah_vm_alloc(struct gunyah_rm *rm) { struct gunyah_vm *ghvm; @@ -1375,6 +1565,20 @@ static struct gunyah_vm *gunyah_vm_alloc(struct guny= ah_rm *rm) INIT_LIST_HEAD(&ghvm->resources); INIT_LIST_HEAD(&ghvm->resource_tickets); =20 + ghvm->addrspace_ticket.resource_type =3D GUNYAH_RESOURCE_TYPE_ADDR_SPACE; + ghvm->addrspace_ticket.label =3D GUNYAH_VM_ADDRSPACE_LABEL; + ghvm->addrspace_ticket.populate =3D gunyah_vm_resource_ticket_populate_no= op; + ghvm->addrspace_ticket.unpopulate =3D gunyah_vm_resource_ticket_unpopulat= e_noop; + gunyah_vm_add_resource_ticket(ghvm, &ghvm->addrspace_ticket); + + setup_extent_ticket(ghvm, &ghvm->host_private_extent_ticket, + GUNYAH_VM_MEM_EXTENT_HOST_PRIVATE_LABEL); + setup_extent_ticket(ghvm, &ghvm->host_shared_extent_ticket, + GUNYAH_VM_MEM_EXTENT_HOST_SHARED_LABEL); + setup_extent_ticket(ghvm, &ghvm->guest_private_extent_ticket, + GUNYAH_VM_MEM_EXTENT_GUEST_PRIVATE_LABEL); + setup_extent_ticket(ghvm, &ghvm->guest_shared_extent_ticket, + GUNYAH_VM_MEM_EXTENT_GUEST_SHARED_LABEL); return ghvm; } =20 @@ -1389,6 +1593,12 @@ static void gunyah_destroy_vm(struct gunyah_vm *ghvm) if (ghvm->vm_status =3D=3D GUNYAH_RM_VM_STATUS_RUNNING) gunyah_vm_stop(ghvm); =20 + gunyah_vm_remove_resource_ticket(ghvm, &ghvm->addrspace_ticket); + gunyah_vm_remove_resource_ticket(ghvm, &ghvm->host_shared_extent_ticket); + gunyah_vm_remove_resource_ticket(ghvm, &ghvm->host_private_extent_ticket); + gunyah_vm_remove_resource_ticket(ghvm, &ghvm->guest_shared_extent_ticket); + gunyah_vm_remove_resource_ticket(ghvm, &ghvm->guest_private_extent_ticket= ); + gunyah_vm_clean_resources(ghvm); =20 if (ghvm->vm_status =3D=3D GUNYAH_RM_VM_STATUS_EXITED || diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h index 2648d3a623de..d1048d0fc246 100644 --- a/include/linux/gunyah.h +++ b/include/linux/gunyah.h @@ -14,6 +14,7 @@ #include #include =20 +#include #include =20 #define gunyah_vcpu(kvm_vcpu_ptr) \ @@ -107,6 +108,11 @@ struct gunyah_vm { struct list_head resources; struct list_head resource_tickets; enum gunyah_rm_vm_auth_mechanism auth; + struct gunyah_vm_resource_ticket addrspace_ticket; + struct gunyah_vm_resource_ticket host_private_extent_ticket; + struct gunyah_vm_resource_ticket host_shared_extent_ticket; + struct gunyah_vm_resource_ticket guest_private_extent_ticket; + struct gunyah_vm_resource_ticket guest_shared_extent_ticket; }; =20 /** --=20 2.39.5 From nobody Thu Dec 18 00:34:59 2025 Received: from mail-wr1-f44.google.com (mail-wr1-f44.google.com [209.85.221.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C9907290BC8 for ; Thu, 24 Apr 2025 14:14:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.44 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504063; cv=none; b=QMzFzNdx/cAcTslCkgBV2d49tDCY5Ul+ET5qZVQd+4YcsWf1VJVk/7697rZgYprc+/4xgjHVf6mi3JK1w7A7XJ096+ct/UTae7Bs1daK5F5JGl7Ywiz/ZURU82o41J+c8MRnx+qQQzHKhqsg/C7wmfD+/OcI1gf6Zvy6targZCk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504063; c=relaxed/simple; bh=vLbxwMoJK25jtdiinc5JboXzA9vEq1NIw2shYX4meSY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Rg77hKB/arHA4NQK/6/j5oArSd/EZ2UUzUFjbaklNckZAgNQxDsqv1aN3rsP4wLkZiMG+UeZF/HrBgSrPSzKK5kSlnAFbxDTydH7bVennwVEeYo3pwfG2FZ5NBpwZKgZkdeWPO1esR9iUc8lFtTH402mJMI+iI/OFIFWJrS2D7g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=V1AW5rAW; arc=none smtp.client-ip=209.85.221.44 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="V1AW5rAW" Received: by mail-wr1-f44.google.com with SMTP id ffacd0b85a97d-39ee651e419so654220f8f.3 for ; Thu, 24 Apr 2025 07:14:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504057; x=1746108857; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=YXeykB8FkcDuL2Q2W63/7k8ahxVIXL64cFUzN9R5114=; b=V1AW5rAWrfLG1Wkq/87KzJUariKgRAJdGvyfb3BPRdNwLW0FK72L3c9c0jBBXVAWyJ 3uhlbzwZO5P44RGLoNPqy6hqRo+K6BS+8X+oJbGEDcD2u5ziLSPhQyl6froafzR5x2Pj W56JpMYuWxH8iQ5NCPwma7xAfz7VjRJ+HUjGqdBgBP7o14nB3mY3ntsB/13uvlIF0t7v PeeB6LCB5wS0t5MD6QUXJSTQ1A3UVHSnxnlyXgSvMqVW181kFj2gc3OrTjwm/qNIhhVG Ks6f5LSXc2a9yUO5eRMQBc4wXwkuVnnHcCmvMbP3O6hkh/aGjrC9ZXgu4iyndXiS5YbQ ZwRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504057; x=1746108857; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=YXeykB8FkcDuL2Q2W63/7k8ahxVIXL64cFUzN9R5114=; b=rSr3Os+23ppNDePKgg92EIJPq+aTY2Jd0ozZnImsxgkysq8/cX/wWcu4AsrFWwVDpZ yaW5JQBdVm33nXxVl0XTOdinUUZHmEJm9o/4ELMF3sR1TdUdSP3LgvzOMEKsf2E5hW0u rH/UfgyV9/XLkoPdkw3yKDnBkx7hp2u6lm4SadaX54UcXM6O9YAvf75E4ZO/KqR7VE3F Ye7tKA9j4ppnwZCO8bfnvANFiKUF7jdguExyox3oct1HXhXAYkDGepKlFC4KE8VHuiyz 8tPRXga5B9LUpgGQdf6WmlSIBb6xtIdBE7oXYMcA/CQ19lzNPftzpjd+dZ4Wlfb10Ec7 QqZQ== X-Gm-Message-State: AOJu0YxfqbYUPRczqCfO9j2d33X9VORjTLy/NBIyX7zFyPbhye7qhFj+ vt085/Es9wh2M8Hf8as6JofSp6SiVsl8Xuu7TO/6ma8+bRV/URMkugajlc1VDo//BQsJqb+9UP1 o X-Gm-Gg: ASbGnctEJhRl0LS21spucgIkLm/Y7QI+7iow/nZLU4dpTFoIf08M/uhYskAaUC5j/GD I4bBW4BAZLBlG7d1QZoX+zwfJX873ZBK48IZPy0QcNvpQwzcebrJEsrDhL93BB/dM6uTBtAjz2M gpytpJiao5UmETfcYFsx22Dw0H1YpP5qQ3KYdCUnbCNY9yO262NTUXRk5DR0bqSdSQb/QyiuRmH SnVKXtMjn2jIVuE9IGTGLOtpTAhXSjUIIbexcLZ4i0+Wk43TGy1O9Iz5eVhziC9p2JDwQ10EilG 6FlPIvkLLvnvz3bduSUFC6v5jGDdgEiPFWXbXWgz+27hUUpxuasE791eOEgkn9SCIvn56vNBVP2 pvhyWN/zEhqQx0MJ0HfTzIZlEYzM= X-Google-Smtp-Source: AGHT+IFg8XOlOSbfkRtBuJWxisCALodsFHPa4IUTZp3Kj17cI2U8pp461yntnlLPPaPA3d0u/jWMRw== X-Received: by 2002:a05:6000:420f:b0:399:71d4:a9 with SMTP id ffacd0b85a97d-3a06cfb2391mr2162635f8f.52.1745504057229; Thu, 24 Apr 2025 07:14:17 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.14.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:14:16 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt , Elliot Berman Subject: [RFC PATCH 24/34] gunyah: Add platform ops on mem_lend/mem_reclaim Date: Thu, 24 Apr 2025 15:13:31 +0100 Message-Id: <20250424141341.841734-25-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Elliot Berman On Qualcomm platforms, there is a firmware entity which controls access to physical pages. In order to share memory with another VM, this entity needs to be informed that the guest VM should have access to the memory. Co-developed-by: Prakruthi Deepak Heragu Signed-off-by: Prakruthi Deepak Heragu Signed-off-by: Elliot Berman Signed-off-by: Karim Manaouil --- arch/arm64/kvm/gunyah.c | 29 ++++- drivers/virt/gunyah/Kconfig | 4 + drivers/virt/gunyah/Makefile | 1 + drivers/virt/gunyah/gunyah_platform_hooks.c | 117 ++++++++++++++++++++ drivers/virt/gunyah/rsc_mgr_rpc.c | 19 +++- include/linux/gunyah.h | 63 +++++++++-- include/linux/gunyah_rsc_mgr.h | 10 ++ 7 files changed, 226 insertions(+), 17 deletions(-) create mode 100644 drivers/virt/gunyah/gunyah_platform_hooks.c diff --git a/arch/arm64/kvm/gunyah.c b/arch/arm64/kvm/gunyah.c index 5d18d133df50..7216db642174 100644 --- a/arch/arm64/kvm/gunyah.c +++ b/arch/arm64/kvm/gunyah.c @@ -488,7 +488,7 @@ static int gunyah_memory_provide_folio(struct gunyah_vm= *ghvm, size_t size =3D folio_size(folio); enum gunyah_error gunyah_error; unsigned long tag =3D 0; - int ret; + int ret, tmp; =20 if (share) { guest_extent =3D __first_resource(&ghvm->guest_shared_extent_ticket); @@ -521,6 +521,11 @@ static int gunyah_memory_provide_folio(struct gunyah_v= m *ghvm, else /* !share && !write */ access =3D GUNYAH_PAGETABLE_ACCESS_RX; =20 + ret =3D gunyah_rm_platform_pre_demand_page(ghvm->rm, ghvm->vmid, access, + folio); + if (ret) + return ret; + gunyah_error =3D gunyah_hypercall_memextent_donate(donate_flags(share), host_extent->capid, guest_extent->capid, @@ -528,7 +533,8 @@ static int gunyah_memory_provide_folio(struct gunyah_vm= *ghvm, if (gunyah_error !=3D GUNYAH_ERROR_OK) { pr_err("Failed to donate memory for guest address 0x%016llx: %d\n", gpa, gunyah_error); - return gunyah_error_remap(gunyah_error); + ret =3D gunyah_error_remap(gunyah_error); + goto platform_release; } =20 extent_attrs =3D @@ -556,6 +562,14 @@ static int gunyah_memory_provide_folio(struct gunyah_v= m *ghvm, if (gunyah_error !=3D GUNYAH_ERROR_OK) pr_err("Failed to reclaim memory donation for guest address 0x%016llx: %= d\n", gpa, gunyah_error); +platform_release: + tmp =3D gunyah_rm_platform_reclaim_demand_page(ghvm->rm, ghvm->vmid, + access, folio); + if (tmp) { + pr_err("Platform failed to reclaim memory for guest address 0x%016llx: %= d", + gpa, tmp); + return ret; + } return ret; } =20 @@ -565,6 +579,7 @@ static int gunyah_memory_reclaim_folio(struct gunyah_vm= *ghvm, u32 map_flags =3D BIT(GUNYAH_ADDRSPACE_MAP_FLAG_PARTIAL); struct gunyah_resource *guest_extent, *host_extent, *addrspace; enum gunyah_error gunyah_error; + enum gunyah_pagetable_access access; phys_addr_t pa; size_t size; int ret; @@ -604,6 +619,16 @@ static int gunyah_memory_reclaim_folio(struct gunyah_v= m *ghvm, goto err; } =20 + access =3D GUNYAH_PAGETABLE_ACCESS_RWX; + + ret =3D gunyah_rm_platform_reclaim_demand_page(ghvm->rm, ghvm->vmid, acce= ss, folio); + if (ret) { + pr_err_ratelimited( + "Platform failed to reclaim memory for guest address 0x%016llx: %d", + gfn_to_gpa(gfn), ret); + goto err; + } + return 0; err: return ret; diff --git a/drivers/virt/gunyah/Kconfig b/drivers/virt/gunyah/Kconfig index 6f4c85db80b5..23ba523d25dc 100644 --- a/drivers/virt/gunyah/Kconfig +++ b/drivers/virt/gunyah/Kconfig @@ -3,6 +3,7 @@ config GUNYAH tristate "Gunyah Virtualization drivers" depends on ARM64 + select GUNYAH_PLATFORM_HOOKS help The Gunyah drivers are the helper interfaces that run in a guest VM such as basic inter-VM IPC and signaling mechanisms, and higher level @@ -10,3 +11,6 @@ config GUNYAH =20 Say Y/M here to enable the drivers needed to interact in a Gunyah virtual environment. + +config GUNYAH_PLATFORM_HOOKS + tristate diff --git a/drivers/virt/gunyah/Makefile b/drivers/virt/gunyah/Makefile index b1bdf3e84155..45cabba3110c 100644 --- a/drivers/virt/gunyah/Makefile +++ b/drivers/virt/gunyah/Makefile @@ -3,3 +3,4 @@ gunyah_rsc_mgr-y +=3D rsc_mgr.o rsc_mgr_rpc.o =20 obj-$(CONFIG_GUNYAH) +=3D gunyah.o gunyah_rsc_mgr.o +obj-$(CONFIG_GUNYAH_PLATFORM_HOOKS) +=3D gunyah_platform_hooks.o diff --git a/drivers/virt/gunyah/gunyah_platform_hooks.c b/drivers/virt/gun= yah/gunyah_platform_hooks.c new file mode 100644 index 000000000000..8a1af171e4c9 --- /dev/null +++ b/drivers/virt/gunyah/gunyah_platform_hooks.c @@ -0,0 +1,117 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights res= erved. + */ + +#include +#include +#include +#include + +#include + +static const struct gunyah_rm_platform_ops *rm_platform_ops; +static DECLARE_RWSEM(rm_platform_ops_lock); + +int gunyah_rm_platform_pre_mem_share(struct gunyah_rm *rm, + struct gunyah_rm_mem_parcel *mem_parcel) +{ + int ret =3D 0; + + down_read(&rm_platform_ops_lock); + if (rm_platform_ops && rm_platform_ops->pre_mem_share) + ret =3D rm_platform_ops->pre_mem_share(rm, mem_parcel); + up_read(&rm_platform_ops_lock); + return ret; +} +EXPORT_SYMBOL_GPL(gunyah_rm_platform_pre_mem_share); + +int gunyah_rm_platform_post_mem_reclaim(struct gunyah_rm *rm, + struct gunyah_rm_mem_parcel *mem_parcel) +{ + int ret =3D 0; + + down_read(&rm_platform_ops_lock); + if (rm_platform_ops && rm_platform_ops->post_mem_reclaim) + ret =3D rm_platform_ops->post_mem_reclaim(rm, mem_parcel); + up_read(&rm_platform_ops_lock); + return ret; +} +EXPORT_SYMBOL_GPL(gunyah_rm_platform_post_mem_reclaim); + +int gunyah_rm_platform_pre_demand_page(struct gunyah_rm *rm, u16 vmid, + enum gunyah_pagetable_access access, + struct folio *folio) +{ + int ret =3D 0; + + down_read(&rm_platform_ops_lock); + if (rm_platform_ops && rm_platform_ops->pre_demand_page) + ret =3D rm_platform_ops->pre_demand_page(rm, vmid, access, folio); + up_read(&rm_platform_ops_lock); + return ret; +} +EXPORT_SYMBOL_GPL(gunyah_rm_platform_pre_demand_page); + +int gunyah_rm_platform_reclaim_demand_page(struct gunyah_rm *rm, u16 vmid, + enum gunyah_pagetable_access access, + struct folio *folio) +{ + int ret =3D 0; + + down_read(&rm_platform_ops_lock); + if (rm_platform_ops && rm_platform_ops->pre_demand_page) + ret =3D rm_platform_ops->release_demand_page(rm, vmid, access, + folio); + up_read(&rm_platform_ops_lock); + return ret; +} +EXPORT_SYMBOL_GPL(gunyah_rm_platform_reclaim_demand_page); + +int gunyah_rm_register_platform_ops( + const struct gunyah_rm_platform_ops *platform_ops) +{ + int ret =3D 0; + + down_write(&rm_platform_ops_lock); + if (!rm_platform_ops) + rm_platform_ops =3D platform_ops; + else + ret =3D -EEXIST; + up_write(&rm_platform_ops_lock); + return ret; +} +EXPORT_SYMBOL_GPL(gunyah_rm_register_platform_ops); + +void gunyah_rm_unregister_platform_ops( + const struct gunyah_rm_platform_ops *platform_ops) +{ + down_write(&rm_platform_ops_lock); + if (rm_platform_ops =3D=3D platform_ops) + rm_platform_ops =3D NULL; + up_write(&rm_platform_ops_lock); +} +EXPORT_SYMBOL_GPL(gunyah_rm_unregister_platform_ops); + +static void _devm_gunyah_rm_unregister_platform_ops(void *data) +{ + gunyah_rm_unregister_platform_ops( + (const struct gunyah_rm_platform_ops *)data); +} + +int devm_gunyah_rm_register_platform_ops( + struct device *dev, const struct gunyah_rm_platform_ops *ops) +{ + int ret; + + ret =3D gunyah_rm_register_platform_ops(ops); + if (ret) + return ret; + + return devm_add_action(dev, _devm_gunyah_rm_unregister_platform_ops, + (void *)ops); +} +EXPORT_SYMBOL_GPL(devm_gunyah_rm_register_platform_ops); + +MODULE_LICENSE("GPL"); +MODULE_DESCRIPTION("Gunyah Platform Hooks"); diff --git a/drivers/virt/gunyah/rsc_mgr_rpc.c b/drivers/virt/gunyah/rsc_mg= r_rpc.c index 0266c2a8d583..ec187d116dd7 100644 --- a/drivers/virt/gunyah/rsc_mgr_rpc.c +++ b/drivers/virt/gunyah/rsc_mgr_rpc.c @@ -212,6 +212,12 @@ int gunyah_rm_mem_share(struct gunyah_rm *rm, struct g= unyah_rm_mem_parcel *p) if (!msg) return -ENOMEM; =20 + ret =3D gunyah_rm_platform_pre_mem_share(rm, p); + if (ret) { + kfree(msg); + return ret; + } + req_header =3D msg; acl =3D (void *)req_header + sizeof(*req_header); mem =3D (void *)acl + acl_size; @@ -237,8 +243,10 @@ int gunyah_rm_mem_share(struct gunyah_rm *rm, struct g= unyah_rm_mem_parcel *p) &resp_size); kfree(msg); =20 - if (ret) + if (ret) { + gunyah_rm_platform_post_mem_reclaim(rm, p); return ret; + } =20 p->mem_handle =3D le32_to_cpu(*resp); kfree(resp); @@ -270,9 +278,14 @@ int gunyah_rm_mem_reclaim(struct gunyah_rm *rm, struct gunyah_rm_mem_release_req req =3D { .mem_handle =3D cpu_to_le32(parcel->mem_handle), }; + int ret; + + ret =3D gunyah_rm_call(rm, GUNYAH_RM_RPC_MEM_RECLAIM, &req, sizeof(req), + NULL, NULL); + if (ret) + return ret; =20 - return gunyah_rm_call(rm, GUNYAH_RM_RPC_MEM_RECLAIM, &req, sizeof(req), - NULL, NULL); + return gunyah_rm_platform_post_mem_reclaim(rm, parcel); } ALLOW_ERROR_INJECTION(gunyah_rm_mem_reclaim, ERRNO); =20 diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h index d1048d0fc246..1d363ab8967a 100644 --- a/include/linux/gunyah.h +++ b/include/linux/gunyah.h @@ -158,6 +158,57 @@ struct gunyah_vcpu { struct gunyah_vm_resource_ticket ticket; }; =20 +enum gunyah_pagetable_access { + /* clang-format off */ + GUNYAH_PAGETABLE_ACCESS_NONE =3D 0, + GUNYAH_PAGETABLE_ACCESS_X =3D 1, + GUNYAH_PAGETABLE_ACCESS_W =3D 2, + GUNYAH_PAGETABLE_ACCESS_R =3D 4, + GUNYAH_PAGETABLE_ACCESS_RX =3D 5, + GUNYAH_PAGETABLE_ACCESS_RW =3D 6, + GUNYAH_PAGETABLE_ACCESS_RWX =3D 7, + /* clang-format on */ +}; + +struct gunyah_rm_platform_ops { + int (*pre_mem_share)(struct gunyah_rm *rm, + struct gunyah_rm_mem_parcel *mem_parcel); + int (*post_mem_reclaim)(struct gunyah_rm *rm, + struct gunyah_rm_mem_parcel *mem_parcel); + + int (*pre_demand_page)(struct gunyah_rm *rm, u16 vmid, + enum gunyah_pagetable_access access, + struct folio *folio); + int (*release_demand_page)(struct gunyah_rm *rm, u16 vmid, + enum gunyah_pagetable_access access, + struct folio *folio); +}; + +#if IS_ENABLED(CONFIG_GUNYAH_PLATFORM_HOOKS) +int gunyah_rm_register_platform_ops( + const struct gunyah_rm_platform_ops *platform_ops); +void gunyah_rm_unregister_platform_ops( + const struct gunyah_rm_platform_ops *platform_ops); +int devm_gunyah_rm_register_platform_ops( + struct device *dev, const struct gunyah_rm_platform_ops *ops); +#else +static inline int gunyah_rm_register_platform_ops( + const struct gunyah_rm_platform_ops *platform_ops) +{ + return 0; +} +static inline void gunyah_rm_unregister_platform_ops( + const struct gunyah_rm_platform_ops *platform_ops) +{ +} +static inline int +devm_gunyah_rm_register_platform_ops(struct device *dev, + const struct gunyah_rm_platform_ops *ops) +{ + return 0; +} +#endif + /*************************************************************************= *****/ /* Common arch-independent definitions for Gunyah hypercalls = */ #define GUNYAH_CAPID_INVAL U64_MAX @@ -278,18 +329,6 @@ enum gunyah_error gunyah_hypercall_msgq_recv(u64 capid= , void *buff, size_t size, =20 #define GUNYAH_ADDRSPACE_SELF_CAP 0 =20 -enum gunyah_pagetable_access { - /* clang-format off */ - GUNYAH_PAGETABLE_ACCESS_NONE =3D 0, - GUNYAH_PAGETABLE_ACCESS_X =3D 1, - GUNYAH_PAGETABLE_ACCESS_W =3D 2, - GUNYAH_PAGETABLE_ACCESS_R =3D 4, - GUNYAH_PAGETABLE_ACCESS_RX =3D 5, - GUNYAH_PAGETABLE_ACCESS_RW =3D 6, - GUNYAH_PAGETABLE_ACCESS_RWX =3D 7, - /* clang-format on */ -}; - /* clang-format off */ #define GUNYAH_MEMEXTENT_MAPPING_USER_ACCESS GENMASK_ULL(2, 0) #define GUNYAH_MEMEXTENT_MAPPING_KERNEL_ACCESS GENMASK_ULL(6, 4) diff --git a/include/linux/gunyah_rsc_mgr.h b/include/linux/gunyah_rsc_mgr.h index c42a0cb42ba6..fb3feee73490 100644 --- a/include/linux/gunyah_rsc_mgr.h +++ b/include/linux/gunyah_rsc_mgr.h @@ -155,4 +155,14 @@ void gunyah_rm_free_resource(struct gunyah_resource *g= hrsc); int gunyah_rm_call(struct gunyah_rm *rsc_mgr, u32 message_id, const void *req_buf, size_t req_buf_size, void **resp_buf, size_t *resp_buf_size); + +int gunyah_rm_platform_pre_mem_share(struct gunyah_rm *rm, + struct gunyah_rm_mem_parcel *mem_parcel); +int gunyah_rm_platform_post_mem_reclaim( + struct gunyah_rm *rm, struct gunyah_rm_mem_parcel *mem_parcel); + +int gunyah_rm_platform_pre_demand_page(struct gunyah_rm *rm, u16 vmid, + u32 flags, struct folio *folio); +int gunyah_rm_platform_reclaim_demand_page(struct gunyah_rm *rm, u16 vmid, + u32 flags, struct folio *folio); #endif --=20 2.39.5 From nobody Thu Dec 18 00:34:59 2025 Received: from mail-wr1-f49.google.com (mail-wr1-f49.google.com [209.85.221.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DE043291152 for ; Thu, 24 Apr 2025 14:14:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.49 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504063; cv=none; b=up0aAtk32nauABzT4o81rUg63LszY49LijPjqJUi+VWWKEBAsL4BGM1RJHZuyPpeiRPvIZ5viSLqe817nkEgFAJIMf6h2PpyI0Chr2WoCuNnYDAMjTVGUBo2ChYvngouuUJTzSJyZ6utVtm8ptDhHsNufg+SlLCnwE7Adv1VePM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504063; c=relaxed/simple; bh=jJJ9JgoTw3N1VJhubYw/1Fl9AFSEgg6AWN3vhACh320=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=SKfW8/vsRN+hX4Dc2XIcnROGTrnYNBeia2qEyksEEDxo9/0FiqEBjdDFOo+345G3zOoPfnRl124PHvgseUjBGHB+8rgdTfC6xvtt/fP2RCYDwbe/MiqPetd4zEVyUyjDq8TmilDWaKpB+7QU481osnodiyIfGePO9bxLMM9/qTs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=l2sDkHvc; arc=none smtp.client-ip=209.85.221.49 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="l2sDkHvc" Received: by mail-wr1-f49.google.com with SMTP id ffacd0b85a97d-39141ffa9fcso1251495f8f.0 for ; Thu, 24 Apr 2025 07:14:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504059; x=1746108859; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=JWj7uDN15UXCYW87dO7IIzOLdNiL+f7UvEUy/qTIHd4=; b=l2sDkHvcTVBFdmyOzriXRy7V+POCWjRrzn/Rwj7DOrq3S1JzIBaq3FrRtBcpA8OwMp Hir/SBUQIEXfJUrgAdjUlyNF563afxrDlJRhSr4WOfzLXQ2rK5TArXvAwDzZult/FV65 r6DDBHUY/qeAzFATXGba6ZBggzU9VoaB3DAyorTkS/mCC3BTqRiL8DiG6m53eEtZrwFJ l89nyucBkjfbwad8dYxizW4TuSDIOCmCHCx4QVWDoSWdokMSBungKMUDhFXwiwfG4Xhn CZOnmrDGcRcaau6dmfP0X/dN4oXzEgXEfA9EON6B3BMxVTgXNZST8w+6qss5uRZmoLmm 5KKg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504059; x=1746108859; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=JWj7uDN15UXCYW87dO7IIzOLdNiL+f7UvEUy/qTIHd4=; b=jIG3/niumuy1UWHXqiXnxH/6fD1Km6u0PDiO4pPLb9cmm35+ZA0KFD9FD/otRTsbo2 37+TbmniXQ5brhTzhk49dBu9iGBTVX9WQK658PdoCMvronr/+UYGBf8atsYLgWkiqJ4R ThsqvoZwYpB7XmFJhlAG/ayTDpb1u1MSv/iIVz/J1O1goBX9SlbZmdnhG/qsv6BAWKP9 2lskQRq2e/Gf+BwKu9+YRNeEDaWlNg6ToIZFZ+9o9pF3nf0wEaYsj4ao8RCooD7/yeEA po12r2REglYYjaFsqQBJo0LKjVtkR71vsKKFtaVXWjrhnMbLptZ8qQdIMQWAiAPIbZ5u 0s8A== X-Gm-Message-State: AOJu0Yx8NAhqrBj943bCDte+46Z20imLMLQPck7YisyvEEuipmV5pq8A 7wkRh2b27AO1FVGUlE80x0K+LFwH6ZZbGBNGU8HkTlVricwzMqrsFUSyRvynC/rccBc+F6GZIjl + X-Gm-Gg: ASbGncvvrahwepnJS7ShAT8U5t9chOSMWW1v25r7/FChXAFU7YcPEXoD8/iuj+XXlf1 eoWT1Ek2Su0H/fBd8PccruE1qJWZVBf0iFwt6nlLIX+iaSnWVcW9jSEVeEydYg9TooaVvvPqrHs HU4kKQOOsMESfUBaju7c8KpsIQ/B2Ynqshwg9v901e4GHPWIPdvwLgqURuzrEVUZ6sP9pA/Z5/v HCSZyYgY5+FR+hDCe/NZJxR77SHeRdrRUjV0+82RUaDy/hML+ZTIY4Xv6hTiAbMN1xA+HzBwWoi BjsYT2E5ucfp1R4SOpJM1jN1OHKsbQ6SGGiL6MN5ikSKSJVQtMI8YEnM0G5Qxv69zWDTDL2bL+e 0Lzz1sBliKx0bUyAHcwsDZCvur/4= X-Google-Smtp-Source: AGHT+IEZ62CGRLn02mw6Sp1FYoWsMMZ/LoDmb0ZQtBfZ0JQt0jIozyM3jnvRsN2MwUnOvlF4h+jCiw== X-Received: by 2002:a05:6000:184c:b0:39f:64ff:668d with SMTP id ffacd0b85a97d-3a06cfa5a65mr2546775f8f.40.1745504058677; Thu, 24 Apr 2025 07:14:18 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.14.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:14:18 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt , Elliot Berman , Alex Elder Subject: [RFC PATCH 25/34] gunyah: Add Qualcomm Gunyah platform ops Date: Thu, 24 Apr 2025 15:13:32 +0100 Message-Id: <20250424141341.841734-26-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Elliot Berman Qualcomm platforms have a firmware entity which performs access control to physical pages. Dynamically started Gunyah virtual machines use the QCOM_SCM_RM_MANAGED_VMID for access. Linux thus needs to assign access to the memory used by guest VMs. Gunyah doesn't do this operation for us since it is the current VM (typically VMID_HLOS) delegating the access and not Gunyah itself. Use the Gunyah platform ops to achieve this so that only Qualcomm platforms attempt to make the needed SCM calls. Reviewed-by: Alex Elder Co-developed-by: Prakruthi Deepak Heragu Signed-off-by: Prakruthi Deepak Heragu Signed-off-by: Elliot Berman Signed-off-by: Karim Manaouil --- drivers/virt/gunyah/Kconfig | 13 ++ drivers/virt/gunyah/Makefile | 1 + drivers/virt/gunyah/gunyah_qcom.c | 220 ++++++++++++++++++++++++++++++ 3 files changed, 234 insertions(+) create mode 100644 drivers/virt/gunyah/gunyah_qcom.c diff --git a/drivers/virt/gunyah/Kconfig b/drivers/virt/gunyah/Kconfig index 23ba523d25dc..fe2823dc48ba 100644 --- a/drivers/virt/gunyah/Kconfig +++ b/drivers/virt/gunyah/Kconfig @@ -4,6 +4,7 @@ config GUNYAH tristate "Gunyah Virtualization drivers" depends on ARM64 select GUNYAH_PLATFORM_HOOKS + imply GUNYAH_QCOM_PLATFORM if ARCH_QCOM help The Gunyah drivers are the helper interfaces that run in a guest VM such as basic inter-VM IPC and signaling mechanisms, and higher level @@ -14,3 +15,15 @@ config GUNYAH =20 config GUNYAH_PLATFORM_HOOKS tristate + +config GUNYAH_QCOM_PLATFORM + tristate "Support for Gunyah on Qualcomm platforms" + depends on GUNYAH + select GUNYAH_PLATFORM_HOOKS + select QCOM_SCM + help + Enable support for interacting with Gunyah on Qualcomm + platforms. Interaction with Qualcomm firmware requires + extra platform-specific support. + + Say Y/M here to use Gunyah on Qualcomm platforms. diff --git a/drivers/virt/gunyah/Makefile b/drivers/virt/gunyah/Makefile index 45cabba3110c..349c37e9f0ad 100644 --- a/drivers/virt/gunyah/Makefile +++ b/drivers/virt/gunyah/Makefile @@ -4,3 +4,4 @@ gunyah_rsc_mgr-y +=3D rsc_mgr.o rsc_mgr_rpc.o =20 obj-$(CONFIG_GUNYAH) +=3D gunyah.o gunyah_rsc_mgr.o obj-$(CONFIG_GUNYAH_PLATFORM_HOOKS) +=3D gunyah_platform_hooks.o +obj-$(CONFIG_GUNYAH_QCOM_PLATFORM) +=3D gunyah_qcom.o diff --git a/drivers/virt/gunyah/gunyah_qcom.c b/drivers/virt/gunyah/gunyah= _qcom.c new file mode 100644 index 000000000000..f2342d51a018 --- /dev/null +++ b/drivers/virt/gunyah/gunyah_qcom.c @@ -0,0 +1,220 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2023-2024 Qualcomm Innovation Center, Inc. All rights res= erved. + */ + +#include +#include +#include +#include +#include +#include +#include + +#define QCOM_SCM_RM_MANAGED_VMID 0x3A +#define QCOM_SCM_MAX_MANAGED_VMID 0x3F + +static int +qcom_scm_gunyah_rm_pre_mem_share(struct gunyah_rm *rm, + struct gunyah_rm_mem_parcel *mem_parcel) +{ + struct qcom_scm_vmperm *new_perms __free(kfree) =3D NULL; + u64 src, src_cpy; + int ret =3D 0, i, n; + u16 vmid; + + new_perms =3D kcalloc(mem_parcel->n_acl_entries, sizeof(*new_perms), + GFP_KERNEL); + if (!new_perms) + return -ENOMEM; + + for (n =3D 0; n < mem_parcel->n_acl_entries; n++) { + vmid =3D le16_to_cpu(mem_parcel->acl_entries[n].vmid); + if (vmid <=3D QCOM_SCM_MAX_MANAGED_VMID) + new_perms[n].vmid =3D vmid; + else + new_perms[n].vmid =3D QCOM_SCM_RM_MANAGED_VMID; + if (mem_parcel->acl_entries[n].perms & GUNYAH_RM_ACL_X) + new_perms[n].perm |=3D QCOM_SCM_PERM_EXEC; + if (mem_parcel->acl_entries[n].perms & GUNYAH_RM_ACL_W) + new_perms[n].perm |=3D QCOM_SCM_PERM_WRITE; + if (mem_parcel->acl_entries[n].perms & GUNYAH_RM_ACL_R) + new_perms[n].perm |=3D QCOM_SCM_PERM_READ; + } + + src =3D BIT_ULL(QCOM_SCM_VMID_HLOS); + + for (i =3D 0; i < mem_parcel->n_mem_entries; i++) { + src_cpy =3D src; + ret =3D qcom_scm_assign_mem( + le64_to_cpu(mem_parcel->mem_entries[i].phys_addr), + le64_to_cpu(mem_parcel->mem_entries[i].size), &src_cpy, + new_perms, mem_parcel->n_acl_entries); + if (ret) + break; + } + + /* Did it work ok? */ + if (!ret) + return 0; + + src =3D 0; + for (n =3D 0; n < mem_parcel->n_acl_entries; n++) { + vmid =3D le16_to_cpu(mem_parcel->acl_entries[n].vmid); + if (vmid <=3D QCOM_SCM_MAX_MANAGED_VMID) + src |=3D BIT_ULL(vmid); + else + src |=3D BIT_ULL(QCOM_SCM_RM_MANAGED_VMID); + } + + new_perms[0].vmid =3D QCOM_SCM_VMID_HLOS; + new_perms[0].perm =3D QCOM_SCM_PERM_EXEC | QCOM_SCM_PERM_WRITE | + QCOM_SCM_PERM_READ; + + for (i--; i >=3D 0; i--) { + src_cpy =3D src; + WARN_ON_ONCE(qcom_scm_assign_mem( + le64_to_cpu(mem_parcel->mem_entries[i].phys_addr), + le64_to_cpu(mem_parcel->mem_entries[i].size), &src_cpy, + new_perms, 1)); + } + + return ret; +} + +static int +qcom_scm_gunyah_rm_post_mem_reclaim(struct gunyah_rm *rm, + struct gunyah_rm_mem_parcel *mem_parcel) +{ + struct qcom_scm_vmperm new_perms; + u64 src =3D 0, src_cpy; + int ret =3D 0, i, n; + u16 vmid; + + new_perms.vmid =3D QCOM_SCM_VMID_HLOS; + new_perms.perm =3D QCOM_SCM_PERM_EXEC | QCOM_SCM_PERM_WRITE | + QCOM_SCM_PERM_READ; + + for (n =3D 0; n < mem_parcel->n_acl_entries; n++) { + vmid =3D le16_to_cpu(mem_parcel->acl_entries[n].vmid); + if (vmid <=3D QCOM_SCM_MAX_MANAGED_VMID) + src |=3D (1ull << vmid); + else + src |=3D (1ull << QCOM_SCM_RM_MANAGED_VMID); + } + + for (i =3D 0; i < mem_parcel->n_mem_entries; i++) { + src_cpy =3D src; + ret =3D qcom_scm_assign_mem( + le64_to_cpu(mem_parcel->mem_entries[i].phys_addr), + le64_to_cpu(mem_parcel->mem_entries[i].size), &src_cpy, + &new_perms, 1); + WARN_ON_ONCE(ret); + } + + return ret; +} + +static int +qcom_scm_gunyah_rm_pre_demand_page(struct gunyah_rm *rm, u16 vmid, + enum gunyah_pagetable_access access, + struct folio *folio) +{ + struct qcom_scm_vmperm new_perms[2]; + unsigned int n =3D 1; + u64 src; + + new_perms[0].vmid =3D QCOM_SCM_RM_MANAGED_VMID; + new_perms[0].perm =3D QCOM_SCM_PERM_EXEC | QCOM_SCM_PERM_WRITE | + QCOM_SCM_PERM_READ; + if (access !=3D GUNYAH_PAGETABLE_ACCESS_X && + access !=3D GUNYAH_PAGETABLE_ACCESS_RX && + access !=3D GUNYAH_PAGETABLE_ACCESS_RWX) { + new_perms[1].vmid =3D QCOM_SCM_VMID_HLOS; + new_perms[1].perm =3D QCOM_SCM_PERM_EXEC | QCOM_SCM_PERM_WRITE | + QCOM_SCM_PERM_READ; + n++; + } + + src =3D BIT_ULL(QCOM_SCM_VMID_HLOS); + + return qcom_scm_assign_mem(__pfn_to_phys(folio_pfn(folio)), + folio_size(folio), &src, new_perms, n); +} + +static int +qcom_scm_gunyah_rm_release_demand_page(struct gunyah_rm *rm, u16 vmid, + enum gunyah_pagetable_access access, + struct folio *folio) +{ + struct qcom_scm_vmperm new_perms; + u64 src; + + new_perms.vmid =3D QCOM_SCM_VMID_HLOS; + new_perms.perm =3D QCOM_SCM_PERM_EXEC | QCOM_SCM_PERM_WRITE | + QCOM_SCM_PERM_READ; + + src =3D BIT_ULL(QCOM_SCM_RM_MANAGED_VMID); + + if (access !=3D GUNYAH_PAGETABLE_ACCESS_X && + access !=3D GUNYAH_PAGETABLE_ACCESS_RX && + access !=3D GUNYAH_PAGETABLE_ACCESS_RWX) + src |=3D BIT_ULL(QCOM_SCM_VMID_HLOS); + + return qcom_scm_assign_mem(__pfn_to_phys(folio_pfn(folio)), + folio_size(folio), &src, &new_perms, 1); +} + +static struct gunyah_rm_platform_ops qcom_scm_gunyah_rm_platform_ops =3D { + .pre_mem_share =3D qcom_scm_gunyah_rm_pre_mem_share, + .post_mem_reclaim =3D qcom_scm_gunyah_rm_post_mem_reclaim, + .pre_demand_page =3D qcom_scm_gunyah_rm_pre_demand_page, + .release_demand_page =3D qcom_scm_gunyah_rm_release_demand_page, +}; + +/* {19bd54bd-0b37-571b-946f-609b54539de6} */ +static const uuid_t QCOM_EXT_UUID =3D UUID_INIT(0x19bd54bd, 0x0b37, 0x571b= , 0x94, + 0x6f, 0x60, 0x9b, 0x54, 0x53, + 0x9d, 0xe6); + +#define GUNYAH_QCOM_EXT_CALL_UUID_ID \ + ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, ARM_SMCCC_SMC_32, \ + ARM_SMCCC_OWNER_VENDOR_HYP, 0x3f01) + +static bool gunyah_has_qcom_extensions(void) +{ + struct arm_smccc_res res; + uuid_t uuid; + u32 *up; + + arm_smccc_1_1_smc(GUNYAH_QCOM_EXT_CALL_UUID_ID, &res); + + up =3D (u32 *)&uuid.b[0]; + up[0] =3D lower_32_bits(res.a0); + up[1] =3D lower_32_bits(res.a1); + up[2] =3D lower_32_bits(res.a2); + up[3] =3D lower_32_bits(res.a3); + + return uuid_equal(&uuid, &QCOM_EXT_UUID); +} + +static int __init qcom_gunyah_platform_hooks_register(void) +{ + if (!gunyah_has_qcom_extensions()) + return -ENODEV; + + pr_info("Enabling Gunyah hooks for Qualcomm platforms.\n"); + + return gunyah_rm_register_platform_ops( + &qcom_scm_gunyah_rm_platform_ops); +} + +static void __exit qcom_gunyah_platform_hooks_unregister(void) +{ + gunyah_rm_unregister_platform_ops(&qcom_scm_gunyah_rm_platform_ops); +} + +module_init(qcom_gunyah_platform_hooks_register); +module_exit(qcom_gunyah_platform_hooks_unregister); +MODULE_DESCRIPTION("Qualcomm Technologies, Inc. Platform Hooks for Gunyah"= ); +MODULE_LICENSE("GPL"); --=20 2.39.5 From nobody Thu Dec 18 00:34:59 2025 Received: from mail-wr1-f50.google.com (mail-wr1-f50.google.com [209.85.221.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DD4EF291177 for ; Thu, 24 Apr 2025 14:14:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.50 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504064; cv=none; b=Cs4s2PIP2+7DCVxK7zUhenV/whIBMsSpJLeDgFX4j20FBykzbGf3lKoov5p/3fH5V3krQ7lXOsqVjYLkuYAoqlJkwe1evZaWWH/tbX7Qe9j8yEePJEyiwnZ/pnFMmBMiUIra4VEQrTVIZg8O7IVMPHy/DSoxSKZoTS4ulmkSCtw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504064; c=relaxed/simple; bh=l8MXGFGw31uuwPrLVCURxj+dBhiUP5Eq/JDGE+Ioyyo=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=aTchR76HhnFxPEQVix0P4olVbiBY6+BI7eyruJd5ShmPt5Gq3a+SfVFL1gDBt3kYQU1ZQ8gYCTb5JSHhMdO7L5CouEgSKps8gpqacU7giDkXZEyFtDCrLg15WX9rThNJRk15h8e+X/DMwHV3Cz/afmp+TqdeFG5eudjdK6/q0a0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=rQRWxIZ8; arc=none smtp.client-ip=209.85.221.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="rQRWxIZ8" Received: by mail-wr1-f50.google.com with SMTP id ffacd0b85a97d-3a064a3e143so567539f8f.3 for ; Thu, 24 Apr 2025 07:14:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504060; x=1746108860; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ULK/WgQrAoOKgv7yTbnqoplbgLc/ybTV2ol4z7wD4V8=; b=rQRWxIZ8fpXO6y7A7UEoBuCu58ZSGVRY0fjpUvj5yf8BrVkOihzZTjogYnJHmrD9OG 5bBkzKMlKMuW7rijJvKvn5tzGE4ax3rinG5VQMmuwWDDuQieJiCbQmPJDAMonyD/KNBe cfZbIYruffocTqvtlrq7tyjPuoXof+RxuzdU2huHJcpAqTA1ujaVCIQRTZKoTztXvzwh qZMS1XOZsce0KykvQTP+oDgnnDxzTkiBHhsH6l3NUVVKKeZ39mJEAUhN2/65eho8OO4X IHXSFmu5YI3FeaFhWrHR1akw5UVoAhlIobKVnFlNNIkIfax5YE5OOBHoU4//QWZhg5DI vofQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504060; x=1746108860; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ULK/WgQrAoOKgv7yTbnqoplbgLc/ybTV2ol4z7wD4V8=; b=ZYu32yI2AKFxezuPgJYKdH9keDvlb0DUKGXirTfYwaXz4EWA9mmqDak9u8jHyi2xeI Rl9BkaRinpUpQU4+p4L/pqIvxDVpkxtMe3XfpW0tW7pxV+XHd6VgKjpqnqReJFzG/Pq0 wN02nYquqkPQUOYC1npZURSxL4e/zLON4VEy6Cv+Rhjl4yZ0WMlFkY5fMfkdME+n6sSD ZkkgHW77O608Dv0YoAWLSWsYeufC9VASgzqGZhLhSoiWDIWCCsKXuLQ5BHApRPJDU0v2 bfaKCJu0Ue++qf9QKxSt2nklSUt2nzmltbAeejJyy404J/lT45pI4uUMSIBGi7VxJxu3 u9BQ== X-Gm-Message-State: AOJu0Ywfo6a5tqI4W3Zp5gD2g4EHWQbUBneG8DBUoobDQAbOP0NkSXhS vg1QoKQux0iKG9YXN+BevajeInMCGGOAXlLN6Q4Udsuwrv1l0cM6GADmEVpaLO9Tc/2UlnKa6GJ a X-Gm-Gg: ASbGnctqnpEUSqTTXQ0Dpkvxew8OmhT1O7zawzAtxn0vLBXLsq+YKuz3ivAPBRDzLIs YG7Zq4DWOkp9c41eSLzZA7CrkRFVrVfpUfA9U2NVtsoHGvveKwfSjt+FMUpDYCeTYDcYd8v1L2K +bpmLSK3GKJVb+cAbcEx6x4n6GJAi92DrnAntDr8btkAl1/dxv/1UiWPwtreT3x1m/nufnuWbYB V5FHd/vOpde9TXvuSSm/qC1u0XuQZ+VVciesiA5L4rL+f5tfGDrOawwWn/imUXdyIL+tkQ6s5Gp EE3Rh/NMswToPp1BajJnQLRJnA3qTZ7MBKxruXS42Qq14Wn25culC547PBz3r6He0ONyTcf7VE6 yafi66U4wOSca8YfA X-Google-Smtp-Source: AGHT+IFE5/vk9/X0Lu9aLCAdTyRnF+oi4a9Eo+hij9nbvfLjJtzlSBwWD4MqvNxQh/Fi8Qjri674AQ== X-Received: by 2002:a05:6000:220c:b0:3a0:65bc:3543 with SMTP id ffacd0b85a97d-3a06cf65a2bmr2283431f8f.35.1745504059899; Thu, 24 Apr 2025 07:14:19 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.14.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:14:19 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt Subject: [RFC PATCH 26/34] gunyah: Share memory parcels Date: Thu, 24 Apr 2025 15:13:33 +0100 Message-Id: <20250424141341.841734-27-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Gunyah Resource Manager sets up a virtual machine based on a device tree which lives in guest memory. Resource manager requires this memory to be provided as a memory parcel for it to read and manipulate. Implement a function to construct a memory parcel from the guest's pinned memory pages. Signed-off-by: Karim Manaouil --- arch/arm64/kvm/gunyah.c | 80 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 80 insertions(+) diff --git a/arch/arm64/kvm/gunyah.c b/arch/arm64/kvm/gunyah.c index 7216db642174..ef0971146b56 100644 --- a/arch/arm64/kvm/gunyah.c +++ b/arch/arm64/kvm/gunyah.c @@ -634,6 +634,86 @@ static int gunyah_memory_reclaim_folio(struct gunyah_v= m *ghvm, return ret; } =20 +static int gunyah_share_memory_parcel(struct gunyah_vm *ghvm, + struct gunyah_rm_mem_parcel *parcel, gfn_t gfn, u64 nr) +{ + struct kvm *kvm =3D &ghvm->kvm; + struct kvm_memory_slot *memslot; + struct page **pages; + int ret; + u64 i; + + if (!nr) + return -EINVAL; + + memslot =3D gfn_to_memslot(kvm, gfn); + if (!memslot) + return -ENOENT; + + parcel->mem_entries =3D kcalloc(nr, sizeof(*parcel->mem_entries), GFP_KER= NEL); + if (!parcel->mem_entries) + return -ENOMEM; + parcel->n_mem_entries =3D nr; + + pages =3D memslot->arch.pages + (gfn - memslot->base_gfn); + + for (i =3D 0; i < nr; i++) { + parcel->mem_entries[i].size =3D cpu_to_le64(PAGE_SIZE); + parcel->mem_entries[i].phys_addr =3D cpu_to_le64(page_to_phys(pages[i])); + } + + parcel->n_acl_entries =3D 1; + parcel->acl_entries =3D kcalloc(parcel->n_acl_entries, + sizeof(*parcel->acl_entries), GFP_KERNEL); + if (!parcel->n_acl_entries) { + ret =3D -ENOMEM; + goto free_entries; + } + parcel->acl_entries[0].vmid =3D cpu_to_le16(ghvm->vmid); + parcel->acl_entries[0].perms |=3D GUNYAH_RM_ACL_R; + parcel->acl_entries[0].perms |=3D GUNYAH_RM_ACL_W; + parcel->acl_entries[0].perms |=3D GUNYAH_RM_ACL_X; + parcel->mem_handle =3D GUNYAH_MEM_HANDLE_INVAL; + + ret =3D gunyah_rm_mem_share(ghvm->rm, parcel); + if (ret) + goto free_acl; + + return ret; +free_acl: + kfree(parcel->acl_entries); + parcel->acl_entries =3D NULL; +free_entries: + kfree(parcel->mem_entries); + parcel->mem_entries =3D NULL; + parcel->n_mem_entries =3D 0; + + return ret; +} + +static int gunyah_reclaim_memory_parcel(struct gunyah_vm *ghvm, + struct gunyah_rm_mem_parcel *parcel, gfn_t gfn, u64 nr) +{ + int ret; + + if (parcel->mem_handle !=3D GUNYAH_MEM_HANDLE_INVAL) { + ret =3D gunyah_rm_mem_reclaim(ghvm->rm, parcel); + if (ret) { + dev_err(ghvm->parent, "Failed to reclaim parcel: %d\n", + ret); + /* We can't reclaim the pages -- hold onto the pages + * forever because we don't know what state the memory + * is in + */ + return ret; + } + parcel->mem_handle =3D GUNYAH_MEM_HANDLE_INVAL; + kfree(parcel->mem_entries); + kfree(parcel->acl_entries); + } + return 0; +} + int kvm_arch_vcpu_should_kick(struct kvm_vcpu *vcpu) { return kvm_vcpu_exiting_guest_mode(vcpu) =3D=3D IN_GUEST_MODE; --=20 2.39.5 From nobody Thu Dec 18 00:34:59 2025 Received: from mail-wr1-f52.google.com (mail-wr1-f52.google.com [209.85.221.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A13D52918E6 for ; Thu, 24 Apr 2025 14:14:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.52 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504066; cv=none; b=c+s5I4fozZiSQYapDowFvZ+uRRKOm4DBGnMVW6stRsqWaXPiqXP+bY07cfXW1Lv3mmL+tiFQSmX/T2NKugeHLkh8NR784TkcO1X/O0gpIT5K383lFtBid2edeByBVRzYFDRnmo/8DQlhJrUFIdOpemrNfFBzy6wMrWHtbO7DvdM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504066; c=relaxed/simple; bh=Yi4ZtnxtwEwwcGq5Pu3T2/cALo63bY4y5tVWXmp+Y3c=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=O6mrpRaYeD8jO7Frt3bTgyvaevbLJj410EEDk0Roa02xNSPV7zGtJ/q3lXNrJ0bsb1P0GqLw/h4l6XT5cI24OSo+kKS6M7IJHpao0ip0vHNhAskYXBCZPFNG00xIGzv0DaufkjSrNn5wb6m25Okvc0/0s/lqi0QW+IF7i6kjNYA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=P3uDnySM; arc=none smtp.client-ip=209.85.221.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="P3uDnySM" Received: by mail-wr1-f52.google.com with SMTP id ffacd0b85a97d-39c0dfba946so793963f8f.3 for ; Thu, 24 Apr 2025 07:14:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504061; x=1746108861; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=KqI809F7wKcnGZMELURBH0WwNUVNARNAdRqZ1Gwku8c=; b=P3uDnySMxJTWQ7/cn0c6QZfS+2mrcsS4YkITm9Jqvkwwd5JKcVX1YIHlyx536M/Icz 4zoZfnwIj3oiZI26ksGkdThqhKWvQPNuKpAtg0fkVVqb0ht0/qc+RXXvjh9+I7NEmj82 l2vILtAz+xiQF9fd9P4TADL3bYT0NkFm03GbsGx9OF16fSnpc8NRO2c8JfG9wsP2i7AG 5RmNr/QZ9Rn4X7SQjxqek8nWQOPb2h12lqjSgAFaeCHojqBNfD7Rc22d4ZYqnlClkN05 bPkc1avSLqwWPj47rGA9wVVzjqkp1NktobmDDPe/qldAM3KePajlpvrZKB615gfDflo7 4MJg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504061; x=1746108861; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=KqI809F7wKcnGZMELURBH0WwNUVNARNAdRqZ1Gwku8c=; b=btbY62sapx0ZmMqqtzwfKk+3Vz7U+Pjn/yheG4qxvotVpg+rBOfb8vT4p0CfXq9vJL Q3yKAlO4OeJ16HioPe6vmzVXVckF5tWubk+7ra4Z4c+gVk4llMBvU5Xp+dYeM5hz4NIL FDr7ZG9HWDw9M/v62yXEzGLHL7wmaiXdDavHzoLTvUX6pqdoPizSglr/VIRbHo1++WPb nx0++36Cn1YOtSdcapVBeU5lK19parxG1qM5ECHqwOG1AeXbkU3yDfMV5Ig4gc99pxmj b5Do33fvELzNLoPesGL8RvE39gePbQW0Lx5DMjVC4LQKjgfhRWGL+EW0Iia75rJSf7kk o2mQ== X-Gm-Message-State: AOJu0YyPBF6j7nH22aGHO03yAYAWOcD1689QKipB9i+84Cbe8X8GpBC0 BjiKEQNv/4dw1DDybOQohw6ntM2xx2Az+r27CRugDzlXnfaM9OIOtLWZNGMGlblIW44JYBPimtY s X-Gm-Gg: ASbGncv4soHepEka2LtvaocJNF4yvjkWcS5W/8dDJjn4GFSoB5Qe0lv6XfRrBFZ+j7H XPTcEhA1n8cFuJk0oAa4kJNVgT5wp6GCWPBIwUvNyLcHwdtk4H/uXFBiCLGbNd+0DOgCI3rYEft SVDnsMk6kulWEj8ojN/Y34+x0xNIabLGS5E2Qp8p4fkLv5N9RuyaeHQBT6U49H6McT1ulMq7rmu aaOK3qL/SgsjK81MklLZt9gD7Kv38zqb0Vkv6n2/5Zg4s7lucnLPdl4AkX9HhwVzHdgHHZ0WRt0 PRp/hGa1i0yB6X1yDPrALDJaqAIYjvGfRjuGPXq32cM+gd4El4OWFUREZsPC0D4UrIFWdFqPOEY lw+//neQ+KaLVqnXf X-Google-Smtp-Source: AGHT+IHu0xINB3AI0ORmGPEOw/RldxQp1Lf0CLnYLBWHSDl9/V/LyKw8BRQRYMCgyV0pnZHvllr1jA== X-Received: by 2002:a5d:584e:0:b0:39a:c9ac:cd3a with SMTP id ffacd0b85a97d-3a06cfa9c91mr2435184f8f.51.1745504061234; Thu, 24 Apr 2025 07:14:21 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.14.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:14:20 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt , Elliot Berman Subject: [RFC PATCH 27/34] gunyah: Share guest VM dtb configuration to Gunyah Date: Thu, 24 Apr 2025 15:13:34 +0100 Message-Id: <20250424141341.841734-28-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Elliot Berman Gunyah Resource Manager sets up a virtual machine based on a device tree which lives in guest memory. Resource manager requires this memory to be provided as a memory parcel for it to read and manipulate. Construct a memory parcel, lend it to the virtual machine, and inform resource manager about the device tree location (the memory parcel ID and offset into the memory parcel). Signed-off-by: Elliot Berman Signed-off-by: Karim Manaouil --- arch/arm64/kvm/gunyah.c | 27 ++++++++++++++++++++++++--- include/linux/gunyah.h | 10 ++++++++++ 2 files changed, 34 insertions(+), 3 deletions(-) diff --git a/arch/arm64/kvm/gunyah.c b/arch/arm64/kvm/gunyah.c index ef0971146b56..687f2beea4e7 100644 --- a/arch/arm64/kvm/gunyah.c +++ b/arch/arm64/kvm/gunyah.c @@ -699,8 +699,7 @@ static int gunyah_reclaim_memory_parcel(struct gunyah_v= m *ghvm, if (parcel->mem_handle !=3D GUNYAH_MEM_HANDLE_INVAL) { ret =3D gunyah_rm_mem_reclaim(ghvm->rm, parcel); if (ret) { - dev_err(ghvm->parent, "Failed to reclaim parcel: %d\n", - ret); + pr_err("Failed to reclaim parcel: %d\n", ret); /* We can't reclaim the pages -- hold onto the pages * forever because we don't know what state the memory * is in @@ -1574,6 +1573,7 @@ static void gunyah_vm_stop(struct gunyah_vm *ghvm) =20 static int gunyah_vm_start(struct gunyah_vm *ghvm) { + struct kvm *kvm =3D &ghvm->kvm; struct gunyah_rm_hyp_resources *resources; struct gunyah_resource *ghrsc; int i, n, ret; @@ -1597,7 +1597,18 @@ static int gunyah_vm_start(struct gunyah_vm *ghvm) ghvm->vmid =3D ret; ghvm->vm_status =3D GUNYAH_RM_VM_STATUS_LOAD; =20 - ret =3D gunyah_rm_vm_configure(ghvm->rm, ghvm->vmid, ghvm->auth, 0, 0, 0,= 0, 0); + ghvm->dtb.parcel_start =3D gpa_to_gfn(kvm->dtb.guest_phys_addr); + ghvm->dtb.parcel_pages =3D gpa_to_gfn(kvm->dtb.size); + ret =3D gunyah_share_memory_parcel(ghvm, &ghvm->dtb.parcel, + ghvm->dtb.parcel_start, + ghvm->dtb.parcel_pages); + if (ret) { + pr_warn("Failed to allocate parcel for DTB: %d\n", ret); + goto err; + } + + ret =3D gunyah_rm_vm_configure(ghvm->rm, ghvm->vmid, ghvm->auth, + ghvm->dtb.parcel.mem_handle, 0, 0, 0, kvm->dtb.size); if (ret) { pr_warn("Failed to configure VM: %d\n", ret); goto err; @@ -1698,6 +1709,16 @@ static void gunyah_destroy_vm(struct gunyah_vm *ghvm) if (ghvm->vm_status =3D=3D GUNYAH_RM_VM_STATUS_RUNNING) gunyah_vm_stop(ghvm); =20 + if (ghvm->vm_status =3D=3D GUNYAH_RM_VM_STATUS_LOAD || + ghvm->vm_status =3D=3D GUNYAH_RM_VM_STATUS_READY || + ghvm->vm_status =3D=3D GUNYAH_RM_VM_STATUS_INIT_FAILED) { + ret =3D gunyah_reclaim_memory_parcel(ghvm, &ghvm->dtb.parcel, + ghvm->dtb.parcel_start, + ghvm->dtb.parcel_pages); + if (ret) + pr_err("Failed to reclaim DTB parcel: %d\n", ret); + } + gunyah_vm_remove_resource_ticket(ghvm, &ghvm->addrspace_ticket); gunyah_vm_remove_resource_ticket(ghvm, &ghvm->host_shared_extent_ticket); gunyah_vm_remove_resource_ticket(ghvm, &ghvm->host_private_extent_ticket); diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h index 1d363ab8967a..72aafc813664 100644 --- a/include/linux/gunyah.h +++ b/include/linux/gunyah.h @@ -94,6 +94,12 @@ struct gunyah_vm_resource_ticket { * @resource_tickets: List of &struct gunyah_vm_resource_ticket * @auth: Authentication mechanism to be used by resource manager when * launching the VM + * @dtb: For tracking dtb configuration when launching the VM + * @dtb.parcel_start: Guest frame number where the memory parcel that we l= ent to + * VM (DTB could start in middle of folio; we lend enti= re + * folio; parcel_start is start of the folio) + * @dtb.parcel_pages: Number of pages lent for the memory parcel + * @dtb.parcel: Data for resource manager to lend the parcel */ struct gunyah_vm { u16 vmid; @@ -113,6 +119,10 @@ struct gunyah_vm { struct gunyah_vm_resource_ticket host_shared_extent_ticket; struct gunyah_vm_resource_ticket guest_private_extent_ticket; struct gunyah_vm_resource_ticket guest_shared_extent_ticket; + struct { + gfn_t parcel_start, parcel_pages; + struct gunyah_rm_mem_parcel parcel; + } dtb; }; =20 /** --=20 2.39.5 From nobody Thu Dec 18 00:34:59 2025 Received: from mail-wr1-f51.google.com (mail-wr1-f51.google.com [209.85.221.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 99B6429291D for ; Thu, 24 Apr 2025 14:14:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504067; cv=none; b=TK8Bz8N7pqZ38FvCsIMp7j+1rdwyxuIkAZ1h6to5DKomgCFQIT27etwK2tdCMbeTS4JbHgeC03pnxxzjmYGAOXMFcPWYpKS+0HMieYxug4NIUBXHHZow/FlYq5fllpAIoFQvnKAsNSxHJs3hSyH4mG8Rdn8ST6C7zY/LYG7V5rA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504067; c=relaxed/simple; bh=TszZQ//DMZ7Wex/EWSs2RVro6IlBnBW/flMNEyx2TWU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=h03cPScIScQogBqBqvs5yFUFvcU9aNlNc2ua2xnW+C1+H1FCaOxLF2TRlRINsV2JpGmaisywrl6zOlCVN6geiUNlLElzNXQq5zH6z/YkmeQkt9aSPb00qPTG0ADKR/+zWziDz08TNbf5umTqsLv8u2rCDEC4LgEekg/aH3+Q7og= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=x9sK5gL9; arc=none smtp.client-ip=209.85.221.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="x9sK5gL9" Received: by mail-wr1-f51.google.com with SMTP id ffacd0b85a97d-39c1efbefc6so802943f8f.1 for ; Thu, 24 Apr 2025 07:14:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504063; x=1746108863; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=88AiON8knVFtAm4gGlYsfQZYlJxkn1/VFztoyFrvZWY=; b=x9sK5gL9U0W0aBidfjPjwipqtUklIVk4o1kIne0qmrnkewk+mp+beWI7OgPH1JKmB/ e77X33ZePtK10LJpX1wgpF4Z3Dh/LIWgdKfwdSz7XOGTSrMgkNDfD9HP9pYpOk97XOjh 2yWryuObm+Pz29EgXo9Y04ybVkN8i755Yvsi4ZiZyR/5oIcg9WTSRFCy8ZlOCRwBcihU J7WLXsU5YtdUhQBh6TDZMZHJ5UeSCEzi+pL68GQgNSIh8mtoAvdiGc2tgh2cZwuEDjOa mMkJOTIMkbQb5qkfxHFt+I3J9vB0w2Zbw/VGaeBE4tJr/KilNB8IKtpgE8HnWXzRIazx qZ6Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504063; x=1746108863; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=88AiON8knVFtAm4gGlYsfQZYlJxkn1/VFztoyFrvZWY=; b=s2AemuRi20Gp0xyjimhjDCfSx8eQPCcJDEzkDRVP5BNSYKWI2sUsqKSmIEFXUyKooI UOcLI3ph48M3gKueCGsX557BMMCeCjqz941T0pfIoFxyMnchFdYPVkFFCPfXW2mYjdTE mWde8Q8t67ao+izE3GONoCHtDRF84+eCfR18ga+KuAgx/T4FxLjOGr5p+TJQMzIxYqYt hmsYfD0SLD67GL4hhiKXJtxSeQxLgK1ZQUCzUKa8Ly16NdKE+ferFrz4ZIkSKb/IDqbh +0ttUPdJjRITmrTrsTvWnWkKORsj6EESfT81TbkSqt1+RiMaDos5+K93Nf14r/mt94Da DJXQ== X-Gm-Message-State: AOJu0Yydmu9Vh8mwU9W+4FuKQmtcu55xQS2j0tXIf/ezeqiL1YTMRw/C iHehTH6XI6gRjghCdVxu6lqj3V2Bl27/9l5u0gpOrG8s/vgOgsXCL3vOiEFj41P6wBajXFPjfZL L X-Gm-Gg: ASbGnctiwzLxJnnXOZDTcj78GSczMGCCwv2elQHB2OGJaUP1B4Qc10O83UxFh44ivAb dHlixulVsuBiSVOCc5vIM3+C8jTzovoU9Tf6XVpPaGt3+NJUJUzydF/rtlFMleX73vONoh+warx tFvpkgGQcDqbtkSx1whGhijaawdiuiYkaEqt8vkZY3OLzvqvNxTody456c0qfp0yN6txNSSYbZv jmsgjRS+tcUJ1gJsA0qKt1P/YHVjfSeNDgs+TEulJt+HXt6SPXeuGCe9S3Wn1XppV75Kxa/aC76 bAlZdLlWomYidGijtunVViJs0uD1/R1svGtRxLCcKfnOwGBvMRK+KnqA6ByKsgavjQ1L0PRUDS+ us/FX5vh1mBSdPf5h X-Google-Smtp-Source: AGHT+IE6SiBau6EY9Mh/ltJdHWfjy1fT3q7qjVfQcBmH9UUhGIZ598Z7cZVBqAGGCWYaKx9pVYAqXA== X-Received: by 2002:a05:6000:43d6:10b0:3a0:7017:61f6 with SMTP id ffacd0b85a97d-3a07017626dmr1028260f8f.14.1745504062675; Thu, 24 Apr 2025 07:14:22 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.14.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:14:22 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt , Elliot Berman Subject: [RFC PATCH 28/34] gunyah: Add RPC to enable demand paging Date: Thu, 24 Apr 2025 15:13:35 +0100 Message-Id: <20250424141341.841734-29-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Elliot Berman Add Gunyah Resource Manager RPC to enable demand paging for a virtual machine. Resource manager needs to be informed of private memory regions which will be demand paged and the location where the DTB memory parcel should live in the guest's address space. Signed-off-by: Elliot Berman Signed-off-by: Karim Manaouil --- drivers/virt/gunyah/rsc_mgr_rpc.c | 71 +++++++++++++++++++++++++++++++ include/linux/gunyah_rsc_mgr.h | 12 ++++++ 2 files changed, 83 insertions(+) diff --git a/drivers/virt/gunyah/rsc_mgr_rpc.c b/drivers/virt/gunyah/rsc_mg= r_rpc.c index ec187d116dd7..7fccd871cc0b 100644 --- a/drivers/virt/gunyah/rsc_mgr_rpc.c +++ b/drivers/virt/gunyah/rsc_mgr_rpc.c @@ -106,6 +106,23 @@ struct gunyah_rm_vm_config_image_req { __le64 dtb_size; } __packed; =20 +/* Call: VM_SET_DEMAND_PAGING */ +struct gunyah_rm_vm_set_demand_paging_req { + __le16 vmid; + __le16 _padding; + __le32 range_count; + DECLARE_FLEX_ARRAY(struct gunyah_rm_mem_entry, ranges); +} __packed; + +/* Call: VM_SET_ADDRESS_LAYOUT */ +struct gunyah_rm_vm_set_address_layout_req { + __le16 vmid; + __le16 _padding; + __le32 range_id; + __le64 range_base; + __le64 range_size; +} __packed; + /* * Several RM calls take only a VMID as a parameter and give only standard * response back. Deduplicate boilerplate code by using this common call. @@ -467,3 +484,57 @@ int gunyah_rm_get_hyp_resources(struct gunyah_rm *rm, = u16 vmid, return 0; } ALLOW_ERROR_INJECTION(gunyah_rm_get_hyp_resources, ERRNO); + +/** + * gunyah_rm_vm_set_demand_paging() - Enable demand paging of memory regio= ns + * @rm: Handle to a Gunyah resource manager + * @vmid: VMID of the other VM + * @count: Number of demand paged memory regions + * @entries: Array of the regions + */ +int gunyah_rm_vm_set_demand_paging(struct gunyah_rm *rm, u16 vmid, u32 cou= nt, + struct gunyah_rm_mem_entry *entries) +{ + struct gunyah_rm_vm_set_demand_paging_req *req __free(kfree) =3D NULL; + size_t req_size; + + req_size =3D struct_size(req, ranges, count); + if (req_size =3D=3D SIZE_MAX) + return -EINVAL; + + req =3D kzalloc(req_size, GFP_KERNEL); + if (!req) + return -ENOMEM; + + req->vmid =3D cpu_to_le16(vmid); + req->range_count =3D cpu_to_le32(count); + memcpy(req->ranges, entries, sizeof(*entries) * count); + + return gunyah_rm_call(rm, GUNYAH_RM_RPC_VM_SET_DEMAND_PAGING, req, + req_size, NULL, NULL); +} +ALLOW_ERROR_INJECTION(gunyah_rm_vm_set_demand_paging, ERRNO); + +/** + * gunyah_rm_vm_set_address_layout() - Set the start address of images + * @rm: Handle to a Gunyah resource manager + * @vmid: VMID of the other VM + * @range_id: Which image to set + * @base_address: Base address + * @size: Size + */ +int gunyah_rm_vm_set_address_layout(struct gunyah_rm *rm, u16 vmid, + enum gunyah_rm_range_id range_id, + u64 base_address, u64 size) +{ + struct gunyah_rm_vm_set_address_layout_req req =3D { + .vmid =3D cpu_to_le16(vmid), + .range_id =3D cpu_to_le32(range_id), + .range_base =3D cpu_to_le64(base_address), + .range_size =3D cpu_to_le64(size), + }; + + return gunyah_rm_call(rm, GUNYAH_RM_RPC_VM_SET_ADDRESS_LAYOUT, &req, + sizeof(req), NULL, NULL); +} +ALLOW_ERROR_INJECTION(gunyah_rm_vm_set_address_layout, ERRNO); diff --git a/include/linux/gunyah_rsc_mgr.h b/include/linux/gunyah_rsc_mgr.h index fb3feee73490..f16e64af9273 100644 --- a/include/linux/gunyah_rsc_mgr.h +++ b/include/linux/gunyah_rsc_mgr.h @@ -152,6 +152,18 @@ gunyah_rm_alloc_resource(struct gunyah_rm *rm, struct gunyah_rm_hyp_resource *hyp_resource); void gunyah_rm_free_resource(struct gunyah_resource *ghrsc); =20 +int gunyah_rm_vm_set_demand_paging(struct gunyah_rm *rm, u16 vmid, u32 cou= nt, + struct gunyah_rm_mem_entry *mem_entries); +enum gunyah_rm_range_id { + GUNYAH_RM_RANGE_ID_IMAGE =3D 0, + GUNYAH_RM_RANGE_ID_FIRMWARE =3D 1, +}; + +int gunyah_rm_vm_set_address_layout(struct gunyah_rm *rm, u16 vmid, + enum gunyah_rm_range_id range_id, + u64 base_address, u64 size); + + int gunyah_rm_call(struct gunyah_rm *rsc_mgr, u32 message_id, const void *req_buf, size_t req_buf_size, void **resp_buf, size_t *resp_buf_size); --=20 2.39.5 From nobody Thu Dec 18 00:34:59 2025 Received: from mail-wr1-f42.google.com (mail-wr1-f42.google.com [209.85.221.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 29C892949E1 for ; Thu, 24 Apr 2025 14:14:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.42 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504069; cv=none; b=mDsCgJcQ4Q6ttPqpCF3tv85riFyuxg9b4F6XXhXSdZ1fKM2zJ7dg+v/T4AyCGn66MJjBb4Wsp+sso4LfbQjxYR/ECVel74q+5hSesNOqYY9llkiZWSoH8X7AIa5LOd3eEm96TSmEqx6p7VvErfIp4Um91ZL5f5b+jtQ8EALZIVU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504069; c=relaxed/simple; bh=4ABx24IRjfvVXtvRhqG0T6gWoZOLeg1qYCJYADA24es=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=r59SGc2GLZejLXgwEn0rPo/n3fxQYgj0hXx2+MMxEoWMxYS+jac+q5Wu+usr5tjqVF9GUnakZBXLVodm2r2qbnuOmw8jMj4JYlIhVlkzGW/jCC/heI5yQPulW/kvqpHe8D2pHamI/EMoPpdqiAaR1vd4gd4rbHTDZToAIhFH55U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=c+Nx3rUg; arc=none smtp.client-ip=209.85.221.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="c+Nx3rUg" Received: by mail-wr1-f42.google.com with SMTP id ffacd0b85a97d-39ac8e7688aso668071f8f.2 for ; Thu, 24 Apr 2025 07:14:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504064; x=1746108864; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=WC4sArz2tKH+wjPmCs+Bj3DeYtgEPbRq63zRADPtWtk=; b=c+Nx3rUg6FC30JCuD2Slbwo5+E/SGjmyOKEVkH02gOU9x0Hi+Z9SokAYrM0TCEBD94 /E8LdTFmJt47WiNqPE9Oo5ak510zj8YatEwCUGn1J39+r9X5CdwHRXD65/AmPorFjMsk KllBv+H0VgPpePwNqe8IC76HHXo9/M8QZ1Uatk2bGgwQX7X97snbbuyrkC0YxcAHkPMD FE6DAGYZSWu1B1Zn88kvQXvWxfU39uZDxxvq2CL8WDYMU07HBaznbPo1TP2KhyGY4uFg 1HzllzIWEJAJj3aP8e4tUFZ10cnNsI5v+yd0+BErIupk0hGOT/s2P6OsfIzIg5EFOID4 /HiQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504064; x=1746108864; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=WC4sArz2tKH+wjPmCs+Bj3DeYtgEPbRq63zRADPtWtk=; b=HZJs++36etrbNGgebo2cFvXIMRNG832O71GE9CJr43Z+m/NgbeWzXUSryyBocoQ8VE C2yC8E3VoiGGlpYfVGZzqBfzZWWp7S0oyYlnrmjbZnvkvASuJVzEjnB3k0IRqhJPqHUA knHan6hpCV2aPHeMZDvoT6LkZHqdihSOj8molVNJ5DxiM8KpDwYTpATyCVGCz9r7lwKV QRFeQcrJ27E/lUADxkwiC1L52oP0SbXp7cDQ54XAW8acqeuj7rNeKEWKKZ06QZyxBKUS upfgDfmTkz3XVJKTNSigc5tSQL+YSBm6SsCXICFA5mTAC3IfbZ/TkCT2snokta9Et5YY elFQ== X-Gm-Message-State: AOJu0YyKCj992zLZ3V8xUdm6usF2O8ElcEPifD3KH7k8t8bhkpyaWI4B hZepbUVt0+X+nw1dnOiOsmJNwOGrl3kOSKGYlBqbzsLR/7JvjJjGBl11apVHpP36zUZAyhANN7y / X-Gm-Gg: ASbGncut5ZwKdRyuPZ0MDEQsacmXOOsJN6gn600u/o8vOYOA3okPWZ4Sv5CjG/o2Lq7 Egv+RRkdQeT2jwUAaaLq8YxshheNneOzlXyHQXwfjT1Q+I5BJJIsvFJwTpHmRPeYD6hkWJwcaW+ VXf5O2XM/J/onMVRAsREw+BZ3pg+WlFIV8445z/XUnNvJAK9el08xWn7TH0k50YmF/IRc9rV5h7 SJ9gK0//hEo7LCdTpQ3L+llrSFDBxZ5qKaKp/pXybkOHhOknMLKuxbiAWT2qJd+75IT981a/PNS px5rTjcc53F4zdOwVMB17bqURgqz6RKwx/9cpK6qrEFHke64E1c2D9qeZCrjANEaYEWAdJNRb2S CcKK332Q4N6EcOUQ/ X-Google-Smtp-Source: AGHT+IGeGIUXl8vOV0aXlznupMFAAmbzGK8/bha/61z42fQMkuhHsWrpW3uTjHsZlOeGVmwfVkNdPQ== X-Received: by 2002:a05:6000:18ad:b0:39a:c9ac:cd58 with SMTP id ffacd0b85a97d-3a06cf63742mr2096999f8f.29.1745504063860; Thu, 24 Apr 2025 07:14:23 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.14.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:14:23 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt Subject: [RFC PATCH 29/34] gunyah: Enable demand paging Date: Thu, 24 Apr 2025 15:13:36 +0100 Message-Id: <20250424141341.841734-30-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Tell resource manager to enable demand paging and wire vCPU faults to provide the backing folio. When a folio is donated to the guest, PG_private is set on the folio so that we can recognize it later when taking it back (e.g. to reclaim the folio when the VM exists). Signed-off-by: Karim Manaouil --- arch/arm64/kvm/gunyah.c | 182 +++++++++++++++++++++++++++++++++++++++- 1 file changed, 179 insertions(+), 3 deletions(-) diff --git a/arch/arm64/kvm/gunyah.c b/arch/arm64/kvm/gunyah.c index 687f2beea4e7..e8037d636e8f 100644 --- a/arch/arm64/kvm/gunyah.c +++ b/arch/arm64/kvm/gunyah.c @@ -8,6 +8,7 @@ * */ #include +#include #include #include #include @@ -476,7 +477,7 @@ static inline u32 reclaim_flags(bool share) GUNYAH_MEMEXTENT_DONATE_FROM_PROTECTED); } =20 -static int gunyah_memory_provide_folio(struct gunyah_vm *ghvm, +static int __gunyah_memory_provide_folio(struct gunyah_vm *ghvm, struct folio *folio, gfn_t gfn, bool share, bool write) { struct gunyah_resource *guest_extent, *host_extent, *addrspace; @@ -573,7 +574,7 @@ static int gunyah_memory_provide_folio(struct gunyah_vm= *ghvm, return ret; } =20 -static int gunyah_memory_reclaim_folio(struct gunyah_vm *ghvm, +static int gunyah_memory_reclaim_folio_locked(struct gunyah_vm *ghvm, struct folio *folio, gfn_t gfn, bool share) { u32 map_flags =3D BIT(GUNYAH_ADDRSPACE_MAP_FLAG_PARTIAL); @@ -713,6 +714,144 @@ static int gunyah_reclaim_memory_parcel(struct gunyah= _vm *ghvm, return 0; } =20 +static int gunyah_memory_provide_folio(struct gunyah_vm *ghvm, gfn_t gfn, = bool write) +{ + struct kvm *kvm =3D &ghvm->kvm; + struct kvm_memory_slot *memslot; + struct page *page; + struct folio *folio; + int ret; + + /* Gunyah always starts guest address space at 1G */ + if (gfn < gpa_to_gfn(SZ_1G)) + return -EINVAL; + + memslot =3D gfn_to_memslot(kvm, gfn); + if (!memslot) + return -ENOENT; + + page =3D memslot->arch.pages[gfn - memslot->base_gfn]; + folio =3D page_folio(page); + + folio_lock(folio); + /* Did we race with another vCPU? */ + if (folio_test_private(folio)) { + folio_unlock(folio); + return 0; + } + + ret =3D __gunyah_memory_provide_folio(ghvm, folio, gfn, false, true); + if (ret) { + folio_unlock(folio); + return ret; + } + folio_set_private(folio); + folio_unlock(folio); + + return 0; +} + +static int gunyah_reclaim_memory_range(struct gunyah_vm *ghvm, gfn_t start= , gfn_t nr) +{ + struct kvm *kvm =3D &ghvm->kvm; + struct kvm_memory_slot *slot; + struct kvm_memslots *slots; + struct kvm_memslot_iter iter; + gfn_t end =3D start + nr; + int i, ret; + + for (i =3D 0; i < kvm_arch_nr_memslot_as_ids(kvm); i++) { + slots =3D __kvm_memslots(kvm, i); + + kvm_for_each_memslot_in_gfn_range(&iter, slots, start, end) { + struct page **pages, *page; + struct folio *folio; + unsigned long offset; + unsigned long reclaimed =3D 0; + + slot =3D iter.slot; + pages =3D slot->arch.pages; + for (offset =3D 0; offset < slot->npages;) { + page =3D pages[offset]; + folio =3D page_folio(page); + folio_lock(folio); + if (folio_test_private(folio)) { + ret =3D gunyah_memory_reclaim_folio_locked(ghvm, folio, + slot->base_gfn + offset, false); + if (ret) { + WARN_ON_ONCE(1); + return ret; + } + folio_clear_private(folio); + reclaimed++; + } + folio_unlock(folio); + offset +=3D folio_nr_pages(folio); + } + } + } + return 0; +} + +static int gunyah_memory_parcel_to_paged(struct gunyah_vm *ghvm, gfn_t sta= rt, gfn_t nr) +{ + struct kvm *kvm =3D &ghvm->kvm; + struct kvm_memory_slot *memslot; + struct page **pages, *page; + struct folio *folio; + int i; + + memslot =3D gfn_to_memslot(kvm, start); + if (!memslot) + return -ENOENT; + + if (start - memslot->base_gfn < nr) + return -EINVAL; + + pages =3D &memslot->arch.pages[start - memslot->base_gfn]; + + for (i =3D 0; i < nr;) { + page =3D pages[i]; + folio =3D page_folio(page); + VM_BUG_ON(folio_test_private(folio)); + folio_set_private(folio); + i +=3D folio_nr_pages(folio); + } + + return 0; +} + +static int gunyah_start_paging(struct gunyah_vm *ghvm) +{ + struct kvm_memslots *slots; + struct kvm_memory_slot *slot; + struct gunyah_rm_mem_entry *entries, *entry; + int count =3D 0; + int bkt, ret; + + slots =3D kvm_memslots(&ghvm->kvm); + kvm_for_each_memslot(slot, bkt, slots) { + if (slot->base_gfn >=3D PFN_DOWN(SZ_1G)) + count++; + } + + entries =3D entry =3D kcalloc(count, sizeof(*entries), GFP_KERNEL); + if (!entries) + return -ENOMEM; + + kvm_for_each_memslot(slot, bkt, slots) { + if (slot->base_gfn < PFN_DOWN(SZ_1G)) + continue; + entry->phys_addr =3D cpu_to_le64(gfn_to_gpa(slot->base_gfn)); + entry->size =3D cpu_to_le64(gfn_to_gpa(slot->npages)); + entry++; + } + + ret =3D gunyah_rm_vm_set_demand_paging(ghvm->rm, ghvm->vmid, count, entri= es); + kfree(entries); + return ret; +} + int kvm_arch_vcpu_should_kick(struct kvm_vcpu *vcpu) { return kvm_vcpu_exiting_guest_mode(vcpu) =3D=3D IN_GUEST_MODE; @@ -772,7 +911,14 @@ static int gunyah_handle_page_fault( struct gunyah_vcpu *vcpu, const struct gunyah_hypercall_vcpu_run_resp *vcpu_run_resp) { - return -EINVAL; + u64 addr =3D vcpu_run_resp->state_data[0]; + bool write =3D !!vcpu_run_resp->state_data[1]; + int ret =3D 0; + + ret =3D gunyah_memory_provide_folio(vcpu->ghvm, gpa_to_gfn(addr), write); + if (!ret || ret =3D=3D -EAGAIN) + return 0; + return ret; } =20 static bool gunyah_kvm_handle_mmio(struct gunyah_vcpu *vcpu, @@ -1614,6 +1760,28 @@ static int gunyah_vm_start(struct gunyah_vm *ghvm) goto err; } =20 + ret =3D gunyah_start_paging(ghvm); + if (ret) { + pr_warn("Failed to set up demand paging: %d\n", ret); + goto err; + } + + ret =3D gunyah_memory_parcel_to_paged(ghvm, ghvm->dtb.parcel_start, + ghvm->dtb.parcel_pages); + if (ret) { + pr_warn("Failed to set up paging for memparcel: %d\n", ret); + goto err; + } + + ret =3D gunyah_rm_vm_set_address_layout( + ghvm->rm, ghvm->vmid, GUNYAH_RM_RANGE_ID_IMAGE, + ghvm->dtb.parcel_start << PAGE_SHIFT, + ghvm->dtb.parcel_pages << PAGE_SHIFT); + if (ret) { + pr_warn("Failed to set location of DTB mem parcel: %d\n", ret); + goto err; + } + ret =3D gunyah_rm_vm_init(ghvm->rm, ghvm->vmid); if (ret) { ghvm->vm_status =3D GUNYAH_RM_VM_STATUS_INIT_FAILED; @@ -1719,6 +1887,14 @@ static void gunyah_destroy_vm(struct gunyah_vm *ghvm) pr_err("Failed to reclaim DTB parcel: %d\n", ret); } =20 + /** + * If this fails, we're going to lose the memory for good and is + * BUG_ON-worthy, but not unrecoverable (we just lose memory). + * This call should always succeed though because the VM is in not + * running and RM will let us reclaim all the memory. + */ + WARN_ON(gunyah_reclaim_memory_range(ghvm, gpa_to_gfn(SZ_1G), gpa_to_gfn(U= 64_MAX))); + gunyah_vm_remove_resource_ticket(ghvm, &ghvm->addrspace_ticket); gunyah_vm_remove_resource_ticket(ghvm, &ghvm->host_shared_extent_ticket); gunyah_vm_remove_resource_ticket(ghvm, &ghvm->host_private_extent_ticket); --=20 2.39.5 From nobody Thu Dec 18 00:34:59 2025 Received: from mail-wr1-f45.google.com (mail-wr1-f45.google.com [209.85.221.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 683E91C2335 for ; Thu, 24 Apr 2025 14:14:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.45 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504069; cv=none; b=S8bhzXya4/Ib14CPRFVFvlPBgTujxp6/q94HDwtwwW+8/SltOT+AW/CkoeFP9nUrQ3Xg+2s4W2otZf/VwZmt8Ltm99+kpOgasBFQta4JolW7MNJcy4OTipqs4sI836udx2K0o0+3AM3Hp2s3o/HZrLjo5Ud82/KsCGgkgVlk/NY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504069; c=relaxed/simple; bh=KiBOPw2VoeUP9jPtQen3QJ8+JIhOTaUQt1mQ67vK+Wk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=GGL9LHXg9+HR5h6753XVg+drvr3VT4BYcKujWGdg8GzV2oNY+fe3H2ak+U/ED0hnsD2cD3e/snvMOOyCmBBX3sXqPJ36Tfn8bmF5OfLPW6jLE+UI4+5BvIZtZ+OEzcZalCAklZnUVdUrnNQAKHeqU5h3UnTf1iHbCGkzdj5Tn50= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=BCIEvZRv; arc=none smtp.client-ip=209.85.221.45 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="BCIEvZRv" Received: by mail-wr1-f45.google.com with SMTP id ffacd0b85a97d-39c266c1389so745265f8f.1 for ; Thu, 24 Apr 2025 07:14:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504065; x=1746108865; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Fkw6jo/oeFh6qnWXDp6jlBdNEq901uF/wqE5/ZMg0xQ=; b=BCIEvZRv2R68O63R8brL2WfAmdo925iFx38XKdTQ0t+jxsjdLI1JKc/L5tWtXUgyok xDFWVlpPvWYx3toLYNTof9Xh1eEc4LiLVBxk/xTOK8U9tAr4RdU8PEjKTFoM+KdjiQWz pCiFnyymKrfV9BdBIiFf7Gf3o5zHiH04R18oZMKpjtiX9CWwCovgdK9tgLDSRknqUzkM pAupuCjzcAsPWd8wmZa1UTKIuC8oavN0xCcVzBzPMBzOVC6AKRg3Sx0z2ncIHuz4FWqM vFWHksGt5xY/giD2q9cuPyRcRZmD5RZoZP3Vb3qdL1mGwMuAsRjqhT70yzTBAX9WrW8k sZyw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504065; x=1746108865; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Fkw6jo/oeFh6qnWXDp6jlBdNEq901uF/wqE5/ZMg0xQ=; b=iiMxG3Bpff6btJs+N+sOQ1VmiwAoOW3Iywtnb9nHiJ1bj8Uy9GRFJ19zORaxKXYonr jrfvma/pxs0S2gJDGA29AW6nFkrriL2IIYnkOTo5ptRM3fGoJoDgWIBSRc/KMxNOaXVR a1IkSopfsbsel+CshjiSgYlwFMl+ALzx8zoYtXxh+YGnqBbM7VMHTNDMxOx8pKve6FUP tTB2bxQhETPIXiKN8jpJD8Ipht/eYGwIRnbTC1bBB/7//BcNzDHbzmUsaScCQ3LDV3F1 S/zDygFYGVqFySCNjaZBkncF1dvWydl9/7CNLoE+1qPx/DxGLA9xoCdlhgWR/BaepJjf qQDQ== X-Gm-Message-State: AOJu0YzZ+p6TXF085VXaHAmWU16X7FDQUL7A5RLY+j0t99hnVhGCy32K 7YWQRrP+p8v3YahpZyNmFCzTqOlpouNNaZeBjSD6hA9l5fbbH/7P8JJLHYvxBrjGz6VjdtbABKj b X-Gm-Gg: ASbGncuoYjcWCMkh7ouX6vLJgUM6UGz53oFn7bXmmdm6JDVxbZiqek0TYLcMf1NSdN/ cLRmtyOPSc3Va5osGytJ9m5rxOQ2lGjYT96YgeVOwnt+D2Awpf85srG5qgj2JzM8JGk0iWjTt/P jANQEpByjWeJPdTOwvG30Sz1awD4924Es6292K9/NymyYEKp8F+g4trnL0pQD6/A04ZR7TpFgoQ BdF5K39YcFWxgViLNGmpikzYG4yzUbFei9QfUkXYK9y54B+ChRQxbrhBRWULUKUF1OLEfI1OJql bLZC+ih6qr+Hj+R8FlAQuCV8Hti/tVdDlspBnim36kiT5mTNK9JBjQ9MO/L34QmUN+D0vgcNCoR atSjSf6xLqBEuF/wX X-Google-Smtp-Source: AGHT+IELqmSvaP6OfocDnjzMS7mUki3w51itbTq+w393mXtO1Vl290MnI9dSMs1K0cj071gNYf3OVQ== X-Received: by 2002:a5d:6da2:0:b0:39e:cc0c:9789 with SMTP id ffacd0b85a97d-3a06cf4f2f6mr2251923f8f.11.1745504065280; Thu, 24 Apr 2025 07:14:25 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.14.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:14:24 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt , Elliot Berman Subject: [RFC PATCH 30/34] gunyah: Add RPC to set VM boot context Date: Thu, 24 Apr 2025 15:13:37 +0100 Message-Id: <20250424141341.841734-31-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Elliot Berman The initial context of a the primary vCPU can be initialized by performing RM RPC calls. Signed-off-by: Elliot Berman Reviewed-by: Srivatsa Vaddagiri Signed-off-by: Karim Manaouil --- drivers/virt/gunyah/rsc_mgr_rpc.c | 32 +++++++++++++++++++++++++++++++ include/linux/gunyah_rsc_mgr.h | 2 ++ 2 files changed, 34 insertions(+) diff --git a/drivers/virt/gunyah/rsc_mgr_rpc.c b/drivers/virt/gunyah/rsc_mg= r_rpc.c index 7fccd871cc0b..78c4d1d5d42a 100644 --- a/drivers/virt/gunyah/rsc_mgr_rpc.c +++ b/drivers/virt/gunyah/rsc_mgr_rpc.c @@ -106,6 +106,15 @@ struct gunyah_rm_vm_config_image_req { __le64 dtb_size; } __packed; =20 +/* Call: VM_SET_BOOT_CONTEXT */ +struct gunyah_rm_vm_set_boot_context_req { + __le16 vmid; + u8 reg_set; + u8 reg_index; + __le32 _padding; + __le64 value; +} __packed; + /* Call: VM_SET_DEMAND_PAGING */ struct gunyah_rm_vm_set_demand_paging_req { __le16 vmid; @@ -447,6 +456,29 @@ int gunyah_rm_vm_init(struct gunyah_rm *rm, u16 vmid) } ALLOW_ERROR_INJECTION(gunyah_rm_vm_init, ERRNO); =20 +/** + * gunyah_rm_vm_set_boot_context() - set the initial boot context of the p= rimary vCPU + * @rm: Handle to a Gunyah resource manager + * @vmid: VM identifier + * @reg_set: See &enum gunyah_vm_boot_context_reg + * @reg_index: Which register to set; must be 0 for REG_SET_PC + * @value: Value to set in the register + */ +int gunyah_rm_vm_set_boot_context(struct gunyah_rm *rm, u16 vmid, u8 reg_s= et, + u8 reg_index, u64 value) +{ + struct gunyah_rm_vm_set_boot_context_req req_payload =3D { + .vmid =3D cpu_to_le16(vmid), + .reg_set =3D reg_set, + .reg_index =3D reg_index, + .value =3D cpu_to_le64(value), + }; + + return gunyah_rm_call(rm, GUNYAH_RM_RPC_VM_SET_BOOT_CONTEXT, + &req_payload, sizeof(req_payload), NULL, NULL); +} +ALLOW_ERROR_INJECTION(gunyah_rm_vm_set_boot_context, ERRNO); + /** * gunyah_rm_get_hyp_resources() - Retrieve hypervisor resources (capabili= ties) associated with a VM * @rm: Handle to a Gunyah resource manager diff --git a/include/linux/gunyah_rsc_mgr.h b/include/linux/gunyah_rsc_mgr.h index f16e64af9273..6a07fe25b0ba 100644 --- a/include/linux/gunyah_rsc_mgr.h +++ b/include/linux/gunyah_rsc_mgr.h @@ -126,6 +126,8 @@ int gunyah_rm_vm_configure(struct gunyah_rm *rm, u16 vm= id, u32 mem_handle, u64 image_offset, u64 image_size, u64 dtb_offset, u64 dtb_size); int gunyah_rm_vm_init(struct gunyah_rm *rm, u16 vmid); +int gunyah_rm_vm_set_boot_context(struct gunyah_rm *rm, u16 vmid, u8 reg_s= et, + u8 reg_index, u64 value); =20 struct gunyah_rm_hyp_resource { u8 type; --=20 2.39.5 From nobody Thu Dec 18 00:34:59 2025 Received: from mail-wr1-f46.google.com (mail-wr1-f46.google.com [209.85.221.46]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C4E7928FFF5 for ; Thu, 24 Apr 2025 14:14:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.46 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504071; cv=none; b=c800WmJPVEVP/cn9FBts+IVoR64QHRs2SkGZaXG9zs182Y7Xg5kbSP7ed/pKT3CcdtlNFxSVICXE7XinJbIZHlFxh/R7j+eUZhaupm1L8G9kPCv65ZblG7x81Kk4lB3EY/84xSfGEW5EAS4geRAv83cSCxgZLvCjDGPxw2JKQk0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504071; c=relaxed/simple; bh=lrb7r2WDY6/wiV70trrJFyhVfjh+ttGLRTnjEamSc+I=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=FbS874J4tKxkbx2GXnXOzcx6QKbRZC3i2S2oYEVpmBrVi/Urp6s0rtonTNp8xlZ8jhUxZCsx6ra2E+m0wL5gNUnBDxQfC7DxSQrND9lVTQJsorbAXgRigtaa/uiuFlCJKcnSfS6wHxPrO1n4HXd6k2acMUHg6yydb6lNqlhnk7k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=XbJe9QFr; arc=none smtp.client-ip=209.85.221.46 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="XbJe9QFr" Received: by mail-wr1-f46.google.com with SMTP id ffacd0b85a97d-39c14016868so1020558f8f.1 for ; Thu, 24 Apr 2025 07:14:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504066; x=1746108866; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Z+7xBf3XhJMHfpZe6B0obtpZq77mLhIssmTOLfcblvc=; b=XbJe9QFr/198bFBMe0kDnmRcwfggZDJWfbsXRfz0KwxrS2sZZlklMbpaxSGQxDkso+ 8ZXyoRzxGpnaCcazFtAGjzLP1pIclvwLO3OZ82s8nmcercWkwXZw8VibNb3/naT9Ch7t lg4UXSgMyYgXagUHDHFgvj01YQsKI9EPC3slYHdT8pudV5tBWXWWe/Tom8r43tad1QHM LhYy7YlW9awbC0iyAv+MA/R4vKbUctEshurVcj6pQcQIczcjshb1apYJ50NtfCBgRg9j xvoFgWgGCU4Rfum3+xIrinWqTd7Gsi9yE21RS8LOJsCn1TS3Iq/dvVZHkPePYPAwuMLX EB9A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504066; x=1746108866; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Z+7xBf3XhJMHfpZe6B0obtpZq77mLhIssmTOLfcblvc=; b=ita9MQVDhOAq8mQbFE73E4dlzGWpPEgp4+m1WQV45n8MVNCQICNKCdqHTzggIoHMh3 p3BGNaBIeiasirWIIIa53dxZjdWsrAMYkMzLH4WA4/i/oce3tMpSnt+tkDGgBcIFjR5F /9wQPCtIfRuwsbTE4ccftMo27PIa+GHOks8ceYJv4U+GH/jmmaavnxDlCQKg0uVM/sqz 7K0RNJ0gMjYB9Ni7OKW62pr6gGR/gosfzeMVI2ohbQiCSmAVy6mXybiO63eIV5tYI1xc pbDaRj113KoXFJ4GrCxkZAHnuuOQYTeTrJQjj9d97k+4B3/9/DpJku657lQsPbjt9GbC lqFg== X-Gm-Message-State: AOJu0YyLqjD+cU2Hne3XRz+iRFtghwWB0SVNBG4nNRXL7RD+WRIMRF13 vXnGDCxnIscDl4R2accUL79VQJKREitHRT5inauwB+bw6QqXgFl0lMPPEF742YAb2GjLpesfJH/ u X-Gm-Gg: ASbGncsL3Ev5SThCdxUinrkAqlbHDFnhXAk7+FqN1/hLYu7SZzofNtz78Z7olKYajQR eMfvQjJ4uZKN3zF8RDRKZUqZHVE41ewkB9ULwH980Wg4OXxZMWg/nLmo2iNQUbwQDuv0Ex9ToNE PKexAaPFEG/IWPVoDtLHPy2Nz6cH8mXmOwqEmETwrnqJSy48UMUTrYWnJuYbehYMMVEDwjX/2F3 knVzDri6VeEj84q4q77WIqJJGRJL0tkWRVdZ5VoCAnfBcyfVdn7JWCGCHudi3STfqLOp9jytog6 SIUnY54UAghIexMcoct321lNjhnoE1Lb+1txpD9BGN63BwUKNwY77aOu28jRadNUNoj4ww8maf6 VRBfWAZ69zxx73RqE X-Google-Smtp-Source: AGHT+IHRjNQ9hktsGOI0gDbs88pFfwmyof1gAXg0Su1amTrqhLAxn7c+kspUyHkoAKmkYrrR9nwrhw== X-Received: by 2002:a5d:48ca:0:b0:39c:1257:cd41 with SMTP id ffacd0b85a97d-3a06cfd462dmr1866395f8f.59.1745504066490; Thu, 24 Apr 2025 07:14:26 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.14.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:14:26 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt Subject: [RFC PATCH 31/34] gunyah: allow userspace to set boot cpu context Date: Thu, 24 Apr 2025 15:13:38 +0100 Message-Id: <20250424141341.841734-32-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Allow userspace hypervisor (e.g. Qemu) to set the context of the boot cpu. At the moment, only the program counter (PC) is needed. Signed-off-by: Karim Manaouil --- arch/arm64/kvm/gunyah.c | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/arch/arm64/kvm/gunyah.c b/arch/arm64/kvm/gunyah.c index e8037d636e8f..df922be2429e 100644 --- a/arch/arm64/kvm/gunyah.c +++ b/arch/arm64/kvm/gunyah.c @@ -1703,6 +1703,24 @@ static int gunyah_vm_rm_notification(struct notifier= _block *nb, } } =20 +/* + * We only need to set PC to start of kernel + */ +static int gunyah_vm_set_boot_ctx(struct gunyah_vm *ghvm) +{ + struct kvm_vcpu *vcpu =3D kvm_get_vcpu(&ghvm->kvm, 0); + u64 core_reg =3D KVM_REG_ARM64 | KVM_REG_SIZE_U64 | KVM_REG_ARM_CORE; + struct kvm_one_reg reg; + u64 *regaddr; + + reg.id =3D core_reg | KVM_REG_ARM_CORE_REG(regs.pc); + regaddr =3D core_reg_addr(vcpu, ®); + + /* We only need to set PC atm. regset is 1 */ + return gunyah_rm_vm_set_boot_context( + ghvm->rm, ghvm->vmid, 1, 0, *regaddr); +} + static void gunyah_vm_stop(struct gunyah_vm *ghvm) { int ret; @@ -1790,6 +1808,12 @@ static int gunyah_vm_start(struct gunyah_vm *ghvm) } ghvm->vm_status =3D GUNYAH_RM_VM_STATUS_READY; =20 + ret =3D gunyah_vm_set_boot_ctx(ghvm); + if (ret) { + pr_warn("Failed to setup boot context: %d\n", ret); + goto err; + } + ret =3D gunyah_rm_get_hyp_resources(ghvm->rm, ghvm->vmid, &resources); if (ret) { pr_warn("Failed to get hyp resources for VM: %d\n", ret); --=20 2.39.5 From nobody Thu Dec 18 00:34:59 2025 Received: from mail-wm1-f54.google.com (mail-wm1-f54.google.com [209.85.128.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B78B12957A3 for ; Thu, 24 Apr 2025 14:14:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.54 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504072; cv=none; b=Y4M8Z7s7Xz/SiHLq8UIjPOPbQ1ITyeO9YShArz3jzI2ScqccqjzwRIF+jAPfSTxp/NIp5wFfPrIEj65bA549DKDQUW05Ni9t8ga2Imk2q3V5ML4usFXXGxqj8fprstDT0/NhXNGp6X/0kJCw8oPR39aK6zfMnYoH7e0Vk4ynp1c= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504072; c=relaxed/simple; bh=D+WfMC8Jh2PxaUkmB3v59ZfRR9QkXj4ch9yqlrdrhtI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=AmnW3dnks/kylES9roW6zWhouAfuIv8iva8xUexVRJIZOo8Msy8wOlnAG2tpmBgtr6FxhhjoCNkEdSkax69rUCHPG0csS6bf3xZjk1h0ShWjt0ZOVNo63z5sUxFKfvlRVuaP7sLdBRyhDJhn8Cr2UeQ7gXnEP1x9HHY0IGXveyQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=cu5ALZQi; arc=none smtp.client-ip=209.85.128.54 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="cu5ALZQi" Received: by mail-wm1-f54.google.com with SMTP id 5b1f17b1804b1-43cef035a3bso7531735e9.1 for ; Thu, 24 Apr 2025 07:14:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504068; x=1746108868; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Oa6gfxRAjkR7wfTiIJFT8jbfGXVVwGTh4OuP6vCgukQ=; b=cu5ALZQiIvlO6N17My54QqYBjdiqAlejII9I8zQrhp2pVmzqF+hSfAfcV/5EziFzLU HmeTtCwOQjWkkrPhOQMNCO82OS8I6W+dXyRNbCFcrb0mNF7klQL+wk3qmhDVWZ9nKJxD cECOGDSYt8P9TYQ0I5tS4Vh7MWWCPiG2tjUwfDYm8YmczxC1zKKXNFsfvanK/KPrvSnE hbEuun554g2wES6pGRLaz5MU1gdlj8xo69Pa6LH9iGHhqQir7aSxe/vOHN1G1c7DhIzv 3+N4PlbC2aobhj+JT5f8pjOv7GHH4FkWZ5UcApKcq5rjRT/mUl1qL8Bmtb+XHj2aB8Vy nHlA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504068; x=1746108868; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Oa6gfxRAjkR7wfTiIJFT8jbfGXVVwGTh4OuP6vCgukQ=; b=LULfIMqwhJvEKqcx3YxoFAELNr8bCNA1IrEIXGKXV9oy+zHbmtKguidgzFmmel6GF8 q5bWEH7nzuBeyyxPK6CiXNmAeZQLOUAXJD0+7GPmuPf8L/VHYo+9WL/nu8MdAdWlklUh RFemyxLgIcyXeIUZQ10iEA6Onv7TcJEYxqS4mTVNufhd+FWMiOmf+jn1Pt6uE9y7JmxB sNkLvYAHOUj20hxNDe2sDzCAhFDpLipMChZSiXTFt5ZkAhgq7+E/Q7NOPtZ8jCESPGGH tYI7lq5p5HDuL4a6IHTKmmVvtAJNPTBy83QwUtnnkPr/D5A4xE66KId6XzzFiZP/EC/d UwmQ== X-Gm-Message-State: AOJu0Yyo2WHpF5bn3TqbFjmxP2hTQsjiQr2p3qkbhLv/WtKmRG04X1Lh F8/iPoY5STQdHxI/t7jV7x98++Vfl0AeBQFEGWBMgxsPas8wP8y4omwjP/gSUJri61VtJ8TZMDs B X-Gm-Gg: ASbGncse2nvBJ9kFdEgowMFvWup8jt4ypDWWATd6b0KXixjSDvqHsOP4jjBWs+zO+s3 u8syYfZgNikqUebWVJGa6O4r4e5jb7h3ktok7MF0/cAykTCiPzQF/G9zOt57ofRNQOGC0SpVr4X ZOOHMrxrJLwjTCsWCLN+rakxLzZEPPUfEayvJAWSZAtPR0UDyj9P+eWHBybQ4Ju9SXdyh2f0Upz 1ZM/nHbpWejkHsWylaARHV40DHbAdEU9YHhvqOoCRmVdaedMCU6zjUxzvUJSKKWg5LugQklGdwJ 4dNfLoxAJ/YeSEhHFAYx/D4I2yjZ/FM5Gftd9lKOuinf+xYX3nvSgyafz4sOtJ6YOdNhACTUuiv jv1frexuJmHTsXWQr X-Google-Smtp-Source: AGHT+IERVpNU2DILo0bvt8R4DAynThAP/fjqterv2/ng44U7A/g/KpyVK5SJtSkw42xdSMDhah0Vrw== X-Received: by 2002:a05:6000:1acb:b0:391:2f15:c1f4 with SMTP id ffacd0b85a97d-3a06cfad64amr2138772f8f.55.1745504067786; Thu, 24 Apr 2025 07:14:27 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.14.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:14:27 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt , Elliot Berman , Alex Elder Subject: [RFC PATCH 32/34] gunyah: Add hypercalls for sending doorbell Date: Thu, 24 Apr 2025 15:13:39 +0100 Message-Id: <20250424141341.841734-33-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Elliot Berman Gunyah doorbells allow a virtual machine to signal another using interrupts. Add the hypercalls needed to assert the interrupt. Reviewed-by: Alex Elder Signed-off-by: Elliot Berman Signed-off-by: Karim Manaouil --- arch/arm64/gunyah/gunyah_hypercall.c | 38 ++++++++++++++++++++++++++++ include/linux/gunyah.h | 5 ++++ 2 files changed, 43 insertions(+) diff --git a/arch/arm64/gunyah/gunyah_hypercall.c b/arch/arm64/gunyah/gunya= h_hypercall.c index 38403dc28c66..3c2672d683ae 100644 --- a/arch/arm64/gunyah/gunyah_hypercall.c +++ b/arch/arm64/gunyah/gunyah_hypercall.c @@ -37,6 +37,8 @@ EXPORT_SYMBOL_GPL(arch_is_gunyah_guest); =20 /* clang-format off */ #define GUNYAH_HYPERCALL_HYP_IDENTIFY GUNYAH_HYPERCALL(0x8000) +#define GUNYAH_HYPERCALL_BELL_SEND GUNYAH_HYPERCALL(0x8012) +#define GUNYAH_HYPERCALL_BELL_SET_MASK GUNYAH_HYPERCALL(0x8015) #define GUNYAH_HYPERCALL_MSGQ_SEND GUNYAH_HYPERCALL(0x801B) #define GUNYAH_HYPERCALL_MSGQ_RECV GUNYAH_HYPERCALL(0x801C) #define GUNYAH_HYPERCALL_ADDRSPACE_MAP GUNYAH_HYPERCALL(0x802B) @@ -64,6 +66,42 @@ void gunyah_hypercall_hyp_identify( } EXPORT_SYMBOL_GPL(gunyah_hypercall_hyp_identify); =20 +/** + * gunyah_hypercall_bell_send() - Assert a gunyah doorbell + * @capid: capability ID of the doorbell + * @new_flags: bits to set on the doorbell + * @old_flags: Filled with the bits set before the send call if return val= ue is GUNYAH_ERROR_OK + */ +enum gunyah_error gunyah_hypercall_bell_send(u64 capid, u64 new_flags, u64= *old_flags) +{ + struct arm_smccc_res res; + + arm_smccc_1_1_hvc(GUNYAH_HYPERCALL_BELL_SEND, capid, new_flags, 0, &res); + + if (res.a0 =3D=3D GUNYAH_ERROR_OK && old_flags) + *old_flags =3D res.a1; + + return res.a0; +} +EXPORT_SYMBOL_GPL(gunyah_hypercall_bell_send); + +/** + * gunyah_hypercall_bell_set_mask() - Set masks on a Gunyah doorbell + * @capid: capability ID of the doorbell + * @enable_mask: which bits trigger the receiver interrupt + * @ack_mask: which bits are automatically acknowledged when the receiver + * interrupt is ack'd + */ +enum gunyah_error gunyah_hypercall_bell_set_mask(u64 capid, u64 enable_mas= k, u64 ack_mask) +{ + struct arm_smccc_res res; + + arm_smccc_1_1_hvc(GUNYAH_HYPERCALL_BELL_SET_MASK, capid, enable_mask, ack= _mask, 0, &res); + + return res.a0; +} +EXPORT_SYMBOL_GPL(gunyah_hypercall_bell_set_mask); + /** * gunyah_hypercall_msgq_send() - Send a buffer on a message queue * @capid: capability ID of the message queue to add message diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h index 72aafc813664..26fdfa3174da 100644 --- a/include/linux/gunyah.h +++ b/include/linux/gunyah.h @@ -329,6 +329,11 @@ gunyah_api_version(const struct gunyah_hypercall_hyp_i= dentify_resp *gunyah_api) void gunyah_hypercall_hyp_identify( struct gunyah_hypercall_hyp_identify_resp *hyp_identity); =20 +enum gunyah_error gunyah_hypercall_bell_send(u64 capid, u64 new_flags, + u64 *old_flags); +enum gunyah_error gunyah_hypercall_bell_set_mask(u64 capid, u64 enable_mas= k, + u64 ack_mask); + /* Immediately raise RX vIRQ on receiver VM */ #define GUNYAH_HYPERCALL_MSGQ_TX_FLAGS_PUSH BIT(0) =20 --=20 2.39.5 From nobody Thu Dec 18 00:34:59 2025 Received: from mail-wr1-f49.google.com (mail-wr1-f49.google.com [209.85.221.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 769311C8619 for ; Thu, 24 Apr 2025 14:14:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.49 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504073; cv=none; b=bla7yu7s7NmT+TuW2C6ImMoaogU1fqrMsKt5Npv+T14GsG9B+DrK30Wy0jNqaATL1Vjnq85CR908/to2SLfI3vBIfIhE4+MP2A5tN2i3DtvBrI76KHVW1zGhSaxcH1zKYRtHnLXzVAFYiS8KlLI1C48dnHV3tomNucMNW864N+k= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504073; c=relaxed/simple; bh=fw0dWabNAHpVG50PP6u8LJhYKTqtfzfYN79tONhSgqw=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=mf1FMgBxBPxEgJnk6wivspOjWrCJJvPwVajjSye6LBKzSdwkmQm6bI0t+jkXTVIT8HWqQCSLlv2RtqsJh0wMqGBi/BYQXHUoaBTetMe7bq+HiIW45VD5VcGjYhAf3iq2Tlo1l4nLqJoqQMUevU27AK6iIPDeEua3E0ncB8+jz5U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=atJfyX5S; arc=none smtp.client-ip=209.85.221.49 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="atJfyX5S" Received: by mail-wr1-f49.google.com with SMTP id ffacd0b85a97d-39ee623fe64so1027137f8f.1 for ; Thu, 24 Apr 2025 07:14:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504069; x=1746108869; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=IXvNlcutyMXVdoUVJkL/Ta8/a2mdxRJrpPZQuCyEfUg=; b=atJfyX5SL9JWC3DgDFlzekyvtegoru7FHIeAWSer2RL8B2oqSTeEBPso/EErYNOhQw GynviIERB4+BRDCddxvnf5ZCrI8grdjdTFpReSNrTo8VEm0rZiWwjGEH99jdgs5R06Lp LeHZywumUi1xcYgi6paPqbga8qvUrr1g/fVzUT81Y3L9HboF/2Tgz1mXQ4MPE/2Wmc5i Bg0nTbpM/ouQA/kVbVZv5Fqu3zPOuOWbDbYh24xIqR6POCxXJL9suBsRwyHAc6YjawCF XL99SLseyGwrOaccFlYJkNoQCSqhmkmoinrLnp2DUtF6gMUMGvfoCTSSZhaCCfro4ix0 m69g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504069; x=1746108869; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=IXvNlcutyMXVdoUVJkL/Ta8/a2mdxRJrpPZQuCyEfUg=; b=r3uLVcs+VSoNpTwwr2Y62RQit+b36pPoYsbAqrzCNRu5incAu94uLwQKjaCG+ulUQj rRVDzHrKMRD20OWrgzcaMlgiOExoHCWSgTwPIBJoU2mO2s/rrwTIs0qRz9AcgEiXhOF4 Fh/7g4wwMOYl/F9npDprYyyYxHJkrYGltHopJWC+4o1oXaRaYc9b/X6KWBSZ/Y3r+5pb BGEwfTXiJlJNEtniufBp1sle01UpWrAw0DAVUpgjyq6lpHKMdBFe5FNbd0ocYsrl5aMl Vh8gwbG7UqWLHgLbGh8736xFV4bs1PiTrpsi/VFJ/Mzu5NF7E3tQsdB2v7+1gkeBN2t2 a8Ww== X-Gm-Message-State: AOJu0YwJv9JWOV3Fekg3BHFYqk7TPQUSK7PJ2WU4jDYoVBtwHuvkgAMK gt7yWsTF+K+3t1eKagAMQK7Nswf3hyXVoJwyjep0EORfQtn//idq2mAftlO6d1wPI9ntSdvbshG g X-Gm-Gg: ASbGncuJGxk140QeTNC33+xzyb+9gbdl85WTgJ0kvlodtDi1GnI13plXxACd/buHl9k f2k2u9XfrSWBid7MgAfojAMkKAmOARdznfUAV1jdkIRINrvPon/TCOcN1EyEN0He6Sr/Xei2unO RUkv7s97Nb0gjxcWgjXJuaOSZ8/kpH59Q1YCbO4zTtTcD6LqEyWTH8IT6fkHxESrVCkASJRwokl 9V2urQq4/ymoU0xZ/9pe5Ad4ocrbqKVb/9Ew+fxSGhLAm7PKqhg6unGlWr80sgd+lgjFlnbQ9S/ UfnEvCWEArghazbYhlyiXI9DhZ7W6HAfqKE8VDgsm2hGoAGm9pIZNp+LgpqZTsrK1NZKq2Oefow xYjlCgTzaD88EfwZV X-Google-Smtp-Source: AGHT+IH0tCAHy+56/cv4ZEG1Nkj43NDfif06A0s6QNH24+K7YCAgJ/1BfT2MGE/c41oOF+y78aq+Lw== X-Received: by 2002:a5d:4092:0:b0:39a:c80b:8288 with SMTP id ffacd0b85a97d-3a06cf635f7mr2018268f8f.33.1745504069033; Thu, 24 Apr 2025 07:14:29 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.14.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:14:28 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt Subject: [RFC PATCH 33/34] KVM: gunyah: Implement irqfd interface Date: Thu, 24 Apr 2025 15:13:40 +0100 Message-Id: <20250424141341.841734-34-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Enables support for injecting interrupts into guest vCPUs via KVM's irqfd interface. Irqfds are requested from userspace via KVM_IRQFD ioctl. kvm_arch_irqfd_init() implementation, in Gunyah, then creates a resource ticket for the irqfd. The userspace must also create a devicetree node to create a doorbell with the corresponding label. Later after VM configuration (in gunyah_vm_start()), the resource ticket will be populated and the irqfd instance will be bound to its corresponding doorbell (identified with a capability id). When userspace asserts the irq line, irqfd->set_irq() callback is called to inject an interrupt into the guest via a hypercall. Signed-off-by: Karim Manaouil --- arch/arm64/kvm/gunyah.c | 100 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 100 insertions(+) diff --git a/arch/arm64/kvm/gunyah.c b/arch/arm64/kvm/gunyah.c index df922be2429e..23b9128bf5b1 100644 --- a/arch/arm64/kvm/gunyah.c +++ b/arch/arm64/kvm/gunyah.c @@ -29,6 +29,14 @@ #define WRITE_TAG (1 << 0) #define SHARE_TAG (1 << 1) =20 +struct gunyah_irqfd { + struct gunyah_vm *ghvm; + struct gunyah_resource *ghrsc; + struct gunyah_vm_resource_ticket ticket; + struct kvm_kernel_irqfd irqfd; + bool level; +}; + static int gunyah_vm_start(struct gunyah_vm *ghvm); =20 static enum kvm_mode kvm_mode =3D KVM_MODE_DEFAULT; @@ -852,6 +860,98 @@ static int gunyah_start_paging(struct gunyah_vm *ghvm) return ret; } =20 +static bool gunyah_irqfd_populate(struct gunyah_vm_resource_ticket *ticket, + struct gunyah_resource *ghrsc) +{ + struct gunyah_irqfd *irqfd =3D + container_of(ticket, struct gunyah_irqfd, ticket); + int ret; + + if (irqfd->ghrsc) { + pr_warn("irqfd%d already got a Gunyah resource", irqfd->ticket.label); + return false; + } + + irqfd->ghrsc =3D ghrsc; + if (irqfd->level) { + /* Configure the bell to trigger when bit 0 is asserted (see + * irq_wakeup) and for bell to automatically clear bit 0 once + * received by the VM (ack_mask). need to make sure bit 0 is cleared ri= ght away, + * otherwise the line will never be deasserted. Emulating edge + * trigger interrupt does not need to set either mask + * because irq is listed only once per gunyah_hypercall_bell_send + */ + ret =3D gunyah_hypercall_bell_set_mask(irqfd->ghrsc->capid, 1, 1); + if (ret) + pr_warn("irq %d couldn't be set as level triggered." + "Might cause IRQ storm if asserted\n", + irqfd->ticket.label); + } + + return true; +} + +static void gunyah_irqfd_unpopulate(struct gunyah_vm_resource_ticket *tick= et, + struct gunyah_resource *ghrsc) +{ +} + +static int gunyah_set_irq(struct kvm_kernel_irqfd *kvm_irqfd) +{ + int ret; + struct gunyah_irqfd *irqfd =3D + container_of(kvm_irqfd, struct gunyah_irqfd, irqfd); + + if (irqfd->ghrsc) { + if (gunyah_hypercall_bell_send(irqfd->ghrsc->capid, 1, NULL)) { + pr_err_ratelimited("Failed to inject interrupt %d: %d\n", + irqfd->ticket.label, ret); + return -1; + } + } else { + pr_err_ratelimited("Premature injection of interrupt\n"); + return -1; + } + + return 1; +} + +struct kvm_kernel_irqfd *kvm_arch_irqfd_alloc(void) +{ + struct gunyah_irqfd *irqfd; + + irqfd =3D kzalloc(sizeof(struct gunyah_irqfd), GFP_KERNEL); + if (!irqfd) + return NULL; + + return &irqfd->irqfd; +} + +void kvm_arch_irqfd_free(struct kvm_kernel_irqfd *kvm_irqfd) +{ + struct gunyah_irqfd *irqfd =3D container_of(kvm_irqfd, struct gunyah_irqf= d, irqfd); + + gunyah_vm_remove_resource_ticket(irqfd->ghvm, &irqfd->ticket); + kfree(irqfd); +} + +int kvm_arch_irqfd_init(struct kvm_kernel_irqfd *kvm_irqfd) +{ + struct gunyah_vm *ghvm =3D container_of(kvm_irqfd->kvm, struct gunyah_vm,= kvm); + struct gunyah_irqfd *irqfd =3D container_of(kvm_irqfd, struct gunyah_irqf= d, irqfd); + + kvm_irqfd->set_irq =3D gunyah_set_irq; + + irqfd->ghvm =3D ghvm; + + irqfd->ticket.resource_type =3D GUNYAH_RESOURCE_TYPE_BELL_TX; + irqfd->ticket.label =3D kvm_irqfd->gsi; + irqfd->ticket.populate =3D gunyah_irqfd_populate; + irqfd->ticket.unpopulate =3D gunyah_irqfd_unpopulate; + + return gunyah_vm_add_resource_ticket(ghvm, &irqfd->ticket); +} + int kvm_arch_vcpu_should_kick(struct kvm_vcpu *vcpu) { return kvm_vcpu_exiting_guest_mode(vcpu) =3D=3D IN_GUEST_MODE; --=20 2.39.5 From nobody Thu Dec 18 00:34:59 2025 Received: from mail-wr1-f50.google.com (mail-wr1-f50.google.com [209.85.221.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7A57A2918E6 for ; Thu, 24 Apr 2025 14:14:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.50 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504074; cv=none; b=LVa/PVxTjcKqE0iK7A6maU56cj/hdqC/XxpeWeDdib6q+tIZ+YLFmSsWlGq8LCUmGXXoFZrOa+2K9NU9k8ggSrDZ7DXRC9yi7NtpL9j8hlNIA9BP2PWf1MvNHh79DZldSKhthRGUclKHHR41nWSRk+QDEWM9Gvs6wftZyMAxxvY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504074; c=relaxed/simple; bh=N53i4bFiZ0PwECXO+3O9PEZg7KbhghQli4Mhy0f7goE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=KbEsX9wB7s3VmUoKVl/JES/4gLp2OoO1+y3xK0ZNCpupiIc4SAUc/XeF8ch8SpCwtsLNJ2qmmebnShHPEHI2iOWMvw9M1qKY8qlwbGQbBSHux8FwV3MfTOoLKt8Io6HNQ/iIUu4ML9iW9UDeJgLkf1zQTLSSdV4t2CcHZ4xQzsA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=A/BNi9dr; arc=none smtp.client-ip=209.85.221.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="A/BNi9dr" Received: by mail-wr1-f50.google.com with SMTP id ffacd0b85a97d-39efc1365e4so559777f8f.1 for ; Thu, 24 Apr 2025 07:14:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504070; x=1746108870; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=3i+gLyWtr6XPiioNUTryX7U5TAHJs0bj5+QGltLz/R0=; b=A/BNi9drAEtZYzDLWDDHNkbHoLQ7G/rSzO6lWtT9wJBxxPLNRLXjqG2CV9Xf5QbQBT Sd9zeNztMZUVOCRA5OflPbZpQ0GAQKMtgYOZVkl6AYSf1iMSjRA5bELWM2/2Ro5G2jLp uJnN4FsqDjPExjGbLQI6o4GhGLVBgLi7/xUjnivg4y9khXjgA3Dmmzsi1UOyqNBUNBT8 MVi1QC4XXmiTDjqog5ZbsG6l9kZaoZlE4hgdKAohqlGXx7MczFhDs/rVxT/XoUHclq03 v+rEjUJ0/gcda3IVfMtpZfSgpUg8mpRx1XLX5Yctye5hH+r06rdmsphQr+bWu8Cqrz5t eCpQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504070; x=1746108870; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3i+gLyWtr6XPiioNUTryX7U5TAHJs0bj5+QGltLz/R0=; b=sozN6x0vsAinNR+xpuD17+oDm0cvd71FWROMvDQu3bd48sAL8RS7pi+mHI6GJzGVbT sUzuucNuBdYT5bBQoS+EtCTleDE9Hnyn+kbyxEoAWHoK906voPTsT2Scz2//E5S6O+oK Ml/4s0EM6L6sWztNGYwIaZOswVeQzvJNW2HLw8vLdXRvoeXObylazv8m4cKfi6VqwrRc 3MaCObkCLq8dki0qZkzLdE/EgnnWg00nNl6+tYKIbMW18rQyJcNOJXboNxjNs2tKoVNj cRgsALST/wvaqRLeu0nNp9UHkPec21NriWE/PXY58LMX2OGKFVmCBSdhQVBtjx554LMP 74XA== X-Gm-Message-State: AOJu0YwbykbSqiCRSKv0V9WvrmWSW8miPOKo1MhVR5tsHSpLWiAEL1/b DaOouEwDidYefkx7miGS8q+R/+wU1sk4mITaPOFxBinFlndR+ldTEDx/3vWs2PhI2/9O0V/jpWs Y X-Gm-Gg: ASbGncvcdemTnpYoYZ6f8YngAsz02lke/BISbMF0pwpAalHZnieRxvaAYaVPDHy8EBk VJ/+05RECiIvBvCM1qVQtMKMM39GVduJTnTBWl6HqLeXVcjt3x5r3dr6lx2sS8y5b7GpnP0GtaI pfZcaivWfKiJg+E+5IJINUjoAx+/7wasFUpWWR5+taTSLbLu0k+1AQf0j4Y4VC1WvMT7tZc5xEx B2cRK8o6JF+aQtbDYOzaw0M3ijfQB/0varULR2pqgJ+6/V/o55Q+h1DbyPIH7r4Vah0ULZ3OQL0 V7A1S7DvkQLZrfmr0LoZC4tcHbwIWOwP954FssLjyojAe0dKfTDxsLhb+MwWoHZN9Z+LAj3GwsL rj/G4WkGC7uaK8Btk X-Google-Smtp-Source: AGHT+IFOk65S5hPF/j31g7Of/aiN0lBptA3ImD/zObUzEDJUJE+zLunCG1fROxNheO7aYYEMJ9HW0A== X-Received: by 2002:a5d:5f87:0:b0:38d:b325:471f with SMTP id ffacd0b85a97d-3a06cf5ac50mr2557177f8f.15.1745504070317; Thu, 24 Apr 2025 07:14:30 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.14.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:14:29 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt Subject: [RFC PATCH 34/34] KVM: gunyah: enable KVM for Gunyah Date: Thu, 24 Apr 2025 15:13:41 +0100 Message-Id: <20250424141341.841734-35-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Enable KVM to act as the interface to create and manage Qualcomm's Gunyah-based virtual machines [1]. [1] https://www.qualcomm.com/developer/blog/2024/08/learn-about-gunyah--qua= lcomm-s-open-source--lightweight-hypervis Signed-off-by: Karim Manaouil --- arch/arm64/kvm/gunyah.c | 12 ++++++++++++ drivers/virt/gunyah/gunyah.c | 3 +++ include/linux/gunyah.h | 1 + 3 files changed, 16 insertions(+) diff --git a/arch/arm64/kvm/gunyah.c b/arch/arm64/kvm/gunyah.c index 23b9128bf5b1..91f0260ef3f5 100644 --- a/arch/arm64/kvm/gunyah.c +++ b/arch/arm64/kvm/gunyah.c @@ -2071,3 +2071,15 @@ void kvm_arch_free_vm(struct kvm *kvm) =20 kfree(ghvm); } + +int kvm_gunyah_init(void) +{ + int err; + + err =3D kvm_init(sizeof(struct kvm_vcpu), 0, THIS_MODULE); + if (err) + return err; + + return 0; +} +EXPORT_SYMBOL_GPL(kvm_gunyah_init); diff --git a/drivers/virt/gunyah/gunyah.c b/drivers/virt/gunyah/gunyah.c index 3e795e3ba881..629c812fbac0 100644 --- a/drivers/virt/gunyah/gunyah.c +++ b/drivers/virt/gunyah/gunyah.c @@ -28,6 +28,9 @@ static int gunyah_probe(struct platform_device *pdev) return -ENODEV; } =20 + if (!kvm_gunyah_init()) + pr_info("KVM for Gunyah is available!\n"); + return devm_of_platform_populate(&pdev->dev); } =20 diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h index 26fdfa3174da..7d43449c4547 100644 --- a/include/linux/gunyah.h +++ b/include/linux/gunyah.h @@ -421,4 +421,5 @@ enum gunyah_error gunyah_hypercall_vcpu_run(u64 capid, unsigned long *resume_data, struct gunyah_hypercall_vcpu_run_resp *resp); =20 +int kvm_gunyah_init(void); #endif --=20 2.39.5