From nobody Mon Jun 15 18:31:46 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B25E0C433F5 for ; Mon, 25 Apr 2022 13:42:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242174AbiDYNpm (ORCPT ); Mon, 25 Apr 2022 09:45:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36042 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242128AbiDYNpY (ORCPT ); Mon, 25 Apr 2022 09:45:24 -0400 Received: from mail-wm1-x34a.google.com (mail-wm1-x34a.google.com [IPv6:2a00:1450:4864:20::34a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 58CB949915 for ; Mon, 25 Apr 2022 06:42:20 -0700 (PDT) Received: by mail-wm1-x34a.google.com with SMTP id t187-20020a1c46c4000000b0038ebc45dbfcso6762251wma.2 for ; Mon, 25 Apr 2022 06:42:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=lQ5RaUoos61s9sLCarSwknlz/MZI6vMjNT8rDD3dDxM=; b=j2tmbKMNwRA2mz5km15aKH/1ZJxY9aUD6oO9vf9Dz2U8f+zNTbbWWGHa6dhbE3p9Ky mmIi/p8JieXJ64JTNvsb57X7pmUh/ridCJVfQsFARMSwcSRsFPDfsGESq2gBrsLOe1eN teHOY7k3/pW7iisAobtizS440U6wOU/MiHyLfDZeThLFc4BYXgRQ4syyb3us5dXswN16 hYYQjuaC+ry8Uvox6WngV6UmNWdPowMqWF2h72l9vx1EeyROjq+JnBHG2rRIoRnAaXor aEvKocwxNNF1c4+32kJ9cKxzpomJYJF5JQUCzleVKStdhXSiB3nbn0gDTSxvvW7bexb9 OxwA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=lQ5RaUoos61s9sLCarSwknlz/MZI6vMjNT8rDD3dDxM=; b=lhfFhKe4BoXzYoA7bYF8gd53a5MdXwUOLOAonpQ0e7Cp//AVb78AIFL4fAWpeMcyWu zb7uBoLhjaSO4rfQ5qmGx3rsNdg9SCSvIiR+t2hjkiKQFLUYUKrSRO4xlatjhwADEMRa c2PaI91y7A+Fy3pv3qtUGzZbNh4h4/7GRNVCZMd+svAde4rFL9uGUjxQ72PTJtdgrRhS cAnZljHDtL9MOod1H6lutaBz9dc4Mgmt0qM7pbMKa/m0jTzaIB33cLKXBpzwlmIAYa6v Ww2ZcJw7H8THa7sAiI9HIxMrB4LX8f/CWhRYsIe9mhmHj2XLYSzA9+mx8ikcHZrU+p5U +oxA== X-Gm-Message-State: AOAM531cgFJpnf9L503HmfHlgzCOaaf/KrRxrXk1X5+LEBNr0N7SisGu 1vuHKFYKtX/0A7U9ssQVr0LWn8EiOOB5/H+eWxk= X-Google-Smtp-Source: ABdhPJwuf63jwKxGNpR594SpPnDVD8dU5JHi8uxZovQ2z9bfNKL1JuGRfl/zvtVLxfLemm04TI0A+OUavutqXab/T+M= X-Received: from sene.c.googlers.com ([fda3:e722:ac3:cc00:28:9cb1:c0a8:27c4]) (user=sebastianene job=sendgmr) by 2002:a1c:19c3:0:b0:392:9cef:e32b with SMTP id 186-20020a1c19c3000000b003929cefe32bmr16580716wmz.116.1650894138945; Mon, 25 Apr 2022 06:42:18 -0700 (PDT) Date: Mon, 25 Apr 2022 13:42:05 +0000 In-Reply-To: <20220425134204.149042-1-sebastianene@google.com> Message-Id: <20220425134204.149042-2-sebastianene@google.com> Mime-Version: 1.0 References: <20220425134204.149042-1-sebastianene@google.com> X-Mailer: git-send-email 2.36.0.rc2.479.g8af0fa9b8e-goog Subject: [PATCH v3 1/2] dt-bindings: vm-wdt: Add qemu,vm-watchdog compatible From: Sebastian Ene To: Rob Herring , Greg Kroah-Hartman , Arnd Bergmann , Dragan Cvetic Cc: linux-kernel@vger.kernel.org, devicetree@vger.kernel.org, maz@kernel.org, will@kernel.org, qperret@google.com, Sebastian Ene Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The stall detection mechanism allows to configure the expiration duration and the internal counter clock frequency measured in Hz. Add these properties in the schema. Signed-off-by: Sebastian Ene --- .../devicetree/bindings/misc/vm-wdt.yaml | 44 +++++++++++++++++++ 1 file changed, 44 insertions(+) create mode 100644 Documentation/devicetree/bindings/misc/vm-wdt.yaml diff --git a/Documentation/devicetree/bindings/misc/vm-wdt.yaml b/Documenta= tion/devicetree/bindings/misc/vm-wdt.yaml new file mode 100644 index 000000000000..cb7665a0c5af --- /dev/null +++ b/Documentation/devicetree/bindings/misc/vm-wdt.yaml @@ -0,0 +1,44 @@ +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/misc/vm-wdt.yaml# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: VM watchdog + +description: | + This binding describes a CPU stall detector mechanism for virtual cpus. + +maintainers: + - Sebastian Ene + +properties: + compatible: + enum: + - qemu,vm-watchdog + clock: + $ref: /schemas/types.yaml#/definitions/uint32 + description: | + The watchdog internal clock measure in Hz used to decrement the + watchdog counter register on each tick. + Defaults to 10 if unset. + timeout-sec: + $ref: /schemas/types.yaml#/definitions/uint32 + description: | + The watchdog expiration timeout measured in seconds. + Defaults to 8 if unset. + +required: + - compatible + +additionalProperties: false + +examples: + - | + watchdog { + compatible =3D "qemu,vm-watchdog"; + clock =3D <10>; + timeout-sec =3D <8>; + }; + +... --=20 2.36.0.rc2.479.g8af0fa9b8e-goog From nobody Mon Jun 15 18:31:46 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 26BE2C433FE for ; Mon, 25 Apr 2022 13:43:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242198AbiDYNq3 (ORCPT ); Mon, 25 Apr 2022 09:46:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37718 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242191AbiDYNpo (ORCPT ); Mon, 25 Apr 2022 09:45:44 -0400 Received: from mail-wm1-x349.google.com (mail-wm1-x349.google.com [IPv6:2a00:1450:4864:20::349]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2AB7B4A3CF for ; Mon, 25 Apr 2022 06:42:35 -0700 (PDT) Received: by mail-wm1-x349.google.com with SMTP id c125-20020a1c3583000000b0038e3f6e871aso6750208wma.8 for ; Mon, 25 Apr 2022 06:42:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=QDUkzkor6rIG/fum/ZEDdQx1WnuBoE/Wp/SxBYeFJMY=; b=Zd7TBBwmHLJV4pk8yXPo4BcyqlnwcEU+4+5iRdnCjDv4TtBGJCnoM7A0pqMEm7XDfe uh9CRKnSsA4xfsQvKZ8JC177PEGPoJdktdcExtcJ1HiFXZXaBJyBiahh1RYQmGIh/f/7 vPJuqbIWx2d5ovYIqUWUEBJJ60yRbrHKQKynkR/1aFMbsEhLaCd/0dS6+fH5EL3HXzxA f29gwRM+djKCMYK9ZmhgSuCf+oinCEf9jqv+jSai8WkdNeNlnkiboUE8FcC97qwxdEzq MWhIujifCihz26KX4AZeG3z1v29W0JxIYmTrufjldgt/hmlpvWJXfQWtb033Gof9+m6V vPZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=QDUkzkor6rIG/fum/ZEDdQx1WnuBoE/Wp/SxBYeFJMY=; b=HtfEv51gk7/XheD7Dg2P3yTTUzM+LuH8312VwsKNu9Vulp9TWpjVUeGlvzEmv2CXli YI4KptGMCGygaAM2ZCF+mA7rvEbKpB9KYmz9o+c9tCt5XHXpCxUPvK0+ALCtVArA82JS afUTJPw2yrc2gDySYx0cZg5vGCenphODm8Y4H1RaU2+Mq7EVAxt9m7RIheVcWmJezYqX xDlt9gjG7j4/EQiZjT+StqVeAoxvuwpD7TMxo76q8/yi3i0iwIoXsz92rI2UhhEjCaX8 DeVdYhxt3zG9UK7nOwV2Er1J2Kihw1lXJ1n4R3qmFAFBPKsizoZj0KNG8uKxXySj38UX 5moA== X-Gm-Message-State: AOAM5325pxXH2RnbzweUogly3y1EW3a+YJ9F23esscHRap4sLnIgFRmJ /99ASjiSemVCuXdUAegTH0fFF07xfqiltnfHZL8= X-Google-Smtp-Source: ABdhPJx9+cCTjMDWkJ55eLyA4L1JsGnVW9FkRCc/bysbWyvvpCDnTEhIp7g5RwOMElR9R+YxdZ2drt5NiH4jlEc9Xvc= X-Received: from sene.c.googlers.com ([fda3:e722:ac3:cc00:28:9cb1:c0a8:27c4]) (user=sebastianene job=sendgmr) by 2002:a05:600c:19c6:b0:392:88e2:7426 with SMTP id u6-20020a05600c19c600b0039288e27426mr16629002wmq.132.1650894153575; Mon, 25 Apr 2022 06:42:33 -0700 (PDT) Date: Mon, 25 Apr 2022 13:42:07 +0000 In-Reply-To: <20220425134204.149042-1-sebastianene@google.com> Message-Id: <20220425134204.149042-3-sebastianene@google.com> Mime-Version: 1.0 References: <20220425134204.149042-1-sebastianene@google.com> X-Mailer: git-send-email 2.36.0.rc2.479.g8af0fa9b8e-goog Subject: [PATCH v3 2/2] misc: Add a mechanism to detect stalls on guest vCPUs From: Sebastian Ene To: Rob Herring , Greg Kroah-Hartman , Arnd Bergmann , Dragan Cvetic Cc: linux-kernel@vger.kernel.org, devicetree@vger.kernel.org, maz@kernel.org, will@kernel.org, qperret@google.com, Sebastian Ene Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" This driver creates per-cpu hrtimers which are required to do the periodic 'pet' operation. On a conventional watchdog-core driver, the userspace is responsible for delivering the 'pet' events by writing to the particular /dev/watchdogN node. In this case we require a strong thread affinity to be able to account for lost time on a per vCPU. This part of the driver is the 'frontend' which is reponsible for delivering the periodic 'pet' events, configuring the virtual peripheral and listening for cpu hotplug events. The other part of the driver handles the peripheral emulation and this part accounts for lost time by looking at the /proc/{}/task/{}/stat entries and is located here: https://chromium-review.googlesource.com/c/chromiumos/platform/crosvm/+/354= 8817 Signed-off-by: Sebastian Ene --- drivers/misc/Kconfig | 12 +++ drivers/misc/Makefile | 1 + drivers/misc/vm-wdt.c | 207 ++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 220 insertions(+) create mode 100644 drivers/misc/vm-wdt.c diff --git a/drivers/misc/Kconfig b/drivers/misc/Kconfig index 2b9572a6d114..71c173e3f064 100644 --- a/drivers/misc/Kconfig +++ b/drivers/misc/Kconfig @@ -493,6 +493,18 @@ config OPEN_DICE =20 If unsure, say N. =20 +config VM_WATCHDOG + tristate "Virtual Machine Watchdog" + select LOCKUP_DETECTOR + help + Detect CPU locks on the virtual machine. This driver relies on the + hrtimers which are CPU-binded to do the 'pet' operation. When a vCPU + has to do a 'pet', it exists the guest through MMIO write and the + backend driver takes into account the lost ticks for this particular + CPU. + To compile this driver as a module, choose M here: the + module will be called vm-wdt. + source "drivers/misc/c2port/Kconfig" source "drivers/misc/eeprom/Kconfig" source "drivers/misc/cb710/Kconfig" diff --git a/drivers/misc/Makefile b/drivers/misc/Makefile index 2ec634354cf5..fa9d644da5db 100644 --- a/drivers/misc/Makefile +++ b/drivers/misc/Makefile @@ -59,3 +59,4 @@ obj-$(CONFIG_XILINX_SDFEC) +=3D xilinx_sdfec.o obj-$(CONFIG_HISI_HIKEY_USB) +=3D hisi_hikey_usb.o obj-$(CONFIG_UID_SYS_STATS) +=3D uid_sys_stats.o obj-$(CONFIG_OPEN_DICE) +=3D open-dice.o +obj-$(CONFIG_VM_WATCHDOG) +=3D vm-wdt.o \ No newline at end of file diff --git a/drivers/misc/vm-wdt.c b/drivers/misc/vm-wdt.c new file mode 100644 index 000000000000..0c4df2fefbb9 --- /dev/null +++ b/drivers/misc/vm-wdt.c @@ -0,0 +1,207 @@ +// SPDX-License-Identifier: GPL-2.0 +// +// Virtual watchdog driver. +// Copyright (C) Google, 2022 + +#include +#include +#include +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#define DRV_NAME "vm_wdt" + +#define VMWDT_REG_STATUS (0x00) +#define VMWDT_REG_LOAD_CNT (0x04) +#define VMWDT_REG_CURRENT_CNT (0x08) +#define VMWDT_REG_CLOCK_FREQ_HZ (0x0C) +#define VMWDT_REG_LEN (0x10) + +#define VMWDT_DEFAULT_CLOCK_HZ (10) +#define VMWDT_DEFAULT_TIMEOT_SEC (8) + +struct vm_wdt_s { + void __iomem *membase; + u32 clock_freq; + u32 expiration_sec; + u32 ping_timeout_ms; + struct hrtimer per_cpu_hrtimer; + struct platform_device *dev; +}; + +#define vmwdt_reg_write(wdt, reg, value) \ + iowrite32((value), (wdt)->membase + (reg)) +#define vmwdt_reg_read(wdt, reg) \ + io32read((wdt)->membase + (reg)) + +static struct platform_device *virt_dev; + +static enum hrtimer_restart vmwdt_timer_fn(struct hrtimer *hrtimer) +{ + struct vm_wdt_s *cpu_wdt; + u32 ticks; + + cpu_wdt =3D container_of(hrtimer, struct vm_wdt_s, per_cpu_hrtimer); + ticks =3D cpu_wdt->clock_freq * cpu_wdt->expiration_sec; + vmwdt_reg_write(cpu_wdt, VMWDT_REG_LOAD_CNT, ticks); + hrtimer_forward_now(hrtimer, ms_to_ktime(cpu_wdt->ping_timeout_ms)); + + return HRTIMER_RESTART; +} + +static void vmwdt_start(void *arg) +{ + u32 ticks; + struct vm_wdt_s *cpu_wdt =3D arg; + struct hrtimer *hrtimer =3D &cpu_wdt->per_cpu_hrtimer; + + vmwdt_reg_write(cpu_wdt, VMWDT_REG_CLOCK_FREQ_HZ, + cpu_wdt->clock_freq); + + /* Compute the number of ticks required for the watchdog counter + * register based on the internal clock frequency and the watchdog + * timeout given from the device tree. + */ + ticks =3D cpu_wdt->clock_freq * cpu_wdt->expiration_sec; + vmwdt_reg_write(cpu_wdt, VMWDT_REG_LOAD_CNT, ticks); + + /* Enable the internal clock and start the watchdog */ + vmwdt_reg_write(cpu_wdt, VMWDT_REG_STATUS, 1); + + hrtimer_init(hrtimer, CLOCK_MONOTONIC, HRTIMER_MODE_REL); + hrtimer->function =3D vmwdt_timer_fn; + hrtimer_start(hrtimer, ms_to_ktime(cpu_wdt->ping_timeout_ms), + HRTIMER_MODE_REL_PINNED); +} + +static void vmwdt_stop(void *arg) +{ + struct vm_wdt_s *cpu_wdt =3D arg; + struct hrtimer *hrtimer =3D &cpu_wdt->per_cpu_hrtimer; + + hrtimer_cancel(hrtimer); + + /* Disable the watchdog */ + vmwdt_reg_write(cpu_wdt, VMWDT_REG_STATUS, 0); +} + +static int start_watchdog_on_cpu(unsigned int cpu) +{ + struct vm_wdt_s *vm_wdt =3D platform_get_drvdata(virt_dev); + + vmwdt_start(this_cpu_ptr(vm_wdt)); + return 0; +} + +static int stop_watchdog_on_cpu(unsigned int cpu) +{ + struct vm_wdt_s *vm_wdt =3D platform_get_drvdata(virt_dev); + + vmwdt_stop(this_cpu_ptr(vm_wdt)); + return 0; +} + +static int vmwdt_probe(struct platform_device *dev) +{ + int cpu, ret, err; + void __iomem *membase; + struct resource *r; + struct vm_wdt_s *vm_wdt; + u32 wdt_clock, wdt_timeout_sec =3D 0; + + r =3D platform_get_resource(dev, IORESOURCE_MEM, 0); + if (r =3D=3D NULL) + return -ENOENT; + + vm_wdt =3D alloc_percpu(typeof(struct vm_wdt_s)); + if (!vm_wdt) + return -ENOMEM; + + membase =3D ioremap(r->start, resource_size(r)); + if (!membase) { + ret =3D -ENXIO; + goto err_withmem; + } + + virt_dev =3D dev; + platform_set_drvdata(dev, vm_wdt); + if (of_property_read_u32(dev->dev.of_node, "clock", &wdt_clock)) + wdt_clock =3D VMWDT_DEFAULT_CLOCK_HZ; + + if (of_property_read_u32(dev->dev.of_node, "timeout-sec", + &wdt_timeout_sec)) + wdt_timeout_sec =3D VMWDT_DEFAULT_TIMEOT_SEC; + + for_each_cpu_and(cpu, cpu_online_mask, &watchdog_cpumask) { + struct vm_wdt_s *cpu_wdt =3D per_cpu_ptr(vm_wdt, cpu); + + cpu_wdt->membase =3D membase + cpu * VMWDT_REG_LEN; + cpu_wdt->clock_freq =3D wdt_clock; + cpu_wdt->expiration_sec =3D wdt_timeout_sec; + cpu_wdt->ping_timeout_ms =3D wdt_timeout_sec * MSEC_PER_SEC / 2; + smp_call_function_single(cpu, vmwdt_start, cpu_wdt, true); + } + + err =3D cpuhp_setup_state_nocalls(CPUHP_AP_ONLINE_DYN, + "virt/watchdog:online", + start_watchdog_on_cpu, + stop_watchdog_on_cpu); + if (err < 0) { + pr_warn("could not be initialized"); + ret =3D err; + goto err_withmem; + } + + return 0; + +err_withmem: + free_percpu(vm_wdt); + return ret; +} + +static int vmwdt_remove(struct platform_device *dev) +{ + int cpu; + struct vm_wdt_s *vm_wdt =3D platform_get_drvdata(dev); + + for_each_cpu_and(cpu, cpu_online_mask, &watchdog_cpumask) { + struct vm_wdt_s *cpu_wdt =3D per_cpu_ptr(vm_wdt, cpu); + + smp_call_function_single(cpu, vmwdt_stop, cpu_wdt, true); + } + + free_percpu(vm_wdt); + return 0; +} + +static const struct of_device_id vmwdt_of_match[] =3D { + { .compatible =3D "qemu,vm-watchdog", }, + {} +}; + +MODULE_DEVICE_TABLE(of, vm_watchdog_of_match); + +static struct platform_driver vmwdt_driver =3D { + .probe =3D vmwdt_probe, + .remove =3D vmwdt_remove, + .driver =3D { + .name =3D DRV_NAME, + .of_match_table =3D vmwdt_of_match, + }, +}; + +module_platform_driver(vmwdt_driver); + +MODULE_LICENSE("GPL"); +MODULE_AUTHOR("Sebastian Ene "); +MODULE_DESCRIPTION("Virtual watchdog driver"); --=20 2.36.0.rc2.479.g8af0fa9b8e-goog