From nobody Fri Oct 10 13:45:37 2025 Received: from mail-pg1-f202.google.com (mail-pg1-f202.google.com [209.85.215.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8AB8B8F5A for ; Sat, 14 Jun 2025 00:08:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749859691; cv=none; b=RQy4nFQLcL3i2Em89xk+FrbhGFh1G8aX8xGRA/NLkj6vP9YD+uEiJh3VtRiTQizABb0GgbXCoWtVrFNCfH3x1/cC4V1j2E8Cd+TCf1/TaksQu3UX7f5nMkM/WgNJsgM5NEauVVtKDWNyy6WFX1PuLgpXiE5IGN6JkkXoO21WCwU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749859691; c=relaxed/simple; bh=0eGNS+WVUNN50Z5AiPai93dnMlivgIU5IZZ4O2qFo7Q=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=iWZt1vu04ceHZXLrmy5AfcznPXRy9Icrl6bNAxnFE2cQw4Hg4whERJc9J1bfResaiuxTJ0j+s+plEHIaWm177GW5uL2Bit6J9pRMwXDhg61S4WIdRJl7rKCbEGAuCgMZGQUMQ+THK2e5wPvro9uXRRjbNhBN1AxtA15maoOnoiI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--hramamurthy.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=fTPdiPzh; arc=none smtp.client-ip=209.85.215.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--hramamurthy.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="fTPdiPzh" Received: by mail-pg1-f202.google.com with SMTP id 41be03b00d2f7-b3183193374so165820a12.1 for ; Fri, 13 Jun 2025 17:08:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1749859689; x=1750464489; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=ZGKQOvqZ0hMp+QDpneOSk7DbvBFQMDy0XsSuv6X7spE=; b=fTPdiPzhz2R2WUI4yUbB+qXWN0junA8YEMdP+K1xGf6/l3cHJE92wZJZsCQOhEX9lF UQKBcFq3afJ3loJzfAVMo1du8RIGTTIxAlia/gAMg5BmjgPq32Q9V5zQHrB6F3IZaMpu zuoC74qUjV7MHDRXaVKanUXXVx9e7HwZFmki1xgytJkoGhTCkxIYVTWIlLGSBNjBUS3S L3n3MnVDqUtw96oMoLBWT+8AhQsOIB5//DkB78Q9VNsz+TIL39rDuRaJ6vlXcUmzSF9b AIQeMBaCox48l/NUl1jiCjEeCMJASfdlIf4CycdH2iYTHXFbcSj10h1qS5Yn2gJP0OWN SEsQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1749859689; x=1750464489; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=ZGKQOvqZ0hMp+QDpneOSk7DbvBFQMDy0XsSuv6X7spE=; b=wBE/CY4/PV1zSDGWZXbETE8wkTB3B4RHU76LR0V0SKwsosNqrDjccUmX6fWkB7c1Ab Z2LwcaRzoUg2Zb89JDYq4tOC+bLvciJpR4prYKmB05b5I9GIkD7fescw4j+TotHhzSuT FmpHiVoQN7ZNmURxA3NmIvNqNG6wx28CKcF+Porsz63sRW7cM0Gm+58NTIHE+7sSFMOM pMyGz0YUML4Ba12xtqrRzXZvEU4sMSnIcRBNQFfkZoWt1TIC9bXZ7daYESyRjI/66Q6Y hu+u06ptbdge9xfrt9hbEDvDQSG3i2C/bNQNjIpEWVooEvYyC0/OOoH7o+ZGf1ZbOzCR wmYg== X-Forwarded-Encrypted: i=1; AJvYcCVWC7tJWssHSM9gMqp21AMmcdZXiFAjQnz6AG4Xsz7jDMdLbAOLsYq/TlUrk9zhkrG7vL26FWnfLH4F+NE=@vger.kernel.org X-Gm-Message-State: AOJu0YzbqKUAyVDBWYQMgIjIiYw1K8LZqaWQEzw+9UaultC5zY9KKOAn jxNRJ1A7W4xDpemvvRTsl7vMcMJmhR2pdtXG+b6yoibt07mWiP2OdPesBOW5pTA5ABv+5bhJEk/ fMdOZuBdkKVAJkKhBC1S7mreyTA== X-Google-Smtp-Source: AGHT+IEK16tVj5NCDJ1XkyqiOhYgV10WppbBSvcx/r1gSqLmEz4YxeEz72s6Ow1PPZWTj9XFuLo5Ez4jw2TaMeyjlg== X-Received: from pfva6.prod.google.com ([2002:a05:6a00:c86:b0:746:3244:f1e6]) (user=hramamurthy job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a20:430d:b0:1f5:7cb4:b713 with SMTP id adf61e73a8af0-21fbd55995cmr1545169637.19.1749859688859; Fri, 13 Jun 2025 17:08:08 -0700 (PDT) Date: Sat, 14 Jun 2025 00:07:51 +0000 In-Reply-To: <20250614000754.164827-1-hramamurthy@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250614000754.164827-1-hramamurthy@google.com> X-Mailer: git-send-email 2.50.0.rc1.591.g9c95f17f64-goog Message-ID: <20250614000754.164827-6-hramamurthy@google.com> Subject: [PATCH net-next v5 5/8] gve: Add support to query the nic clock From: Harshitha Ramamurthy To: netdev@vger.kernel.org Cc: davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, jeroendb@google.com, hramamurthy@google.com, andrew+netdev@lunn.ch, willemb@google.com, ziweixiao@google.com, pkaligineedi@google.com, yyd@google.com, joshwash@google.com, shailend@google.com, linux@treblig.org, thostet@google.com, jfraker@google.com, richardcochran@gmail.com, jdamato@fastly.com, vadim.fedorenko@linux.dev, horms@kernel.org, linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Kevin Yang Query the nic clock and store the results. The timestamp delivered in descriptors has a wraparound time of ~4 seconds so 250ms is chosen as the sync cadence to provide a balance between performance, and drift potential when we do start associating host time and nic time. Leverage PTP's aux_work to query the nic clock periodically. Signed-off-by: Kevin Yang Signed-off-by: John Fraker Signed-off-by: Tim Hostetler Signed-off-by: Ziwei Xiao Reviewed-by: Willem de Bruijn Signed-off-by: Harshitha Ramamurthy --- Changes in v5: - Change to register PTP when initializing the driver and keep it alive until destroying the driver. (Jakub Kicinski) Changes in v4: - release the ptp in the error path of gve_init_clock (Jakub Kicinski) Changes in v2: - Utilize the ptp's aux_work instead of delayed_work (Jakub Kicinski, Vadim Fedorenko) --- drivers/net/ethernet/google/gve/gve.h | 21 ++++++ drivers/net/ethernet/google/gve/gve_main.c | 7 +- drivers/net/ethernet/google/gve/gve_ptp.c | 84 +++++++++++++++++++++- 3 files changed, 109 insertions(+), 3 deletions(-) diff --git a/drivers/net/ethernet/google/gve/gve.h b/drivers/net/ethernet/g= oogle/gve/gve.h index 8d2aa654fd4c..527e17da60bc 100644 --- a/drivers/net/ethernet/google/gve/gve.h +++ b/drivers/net/ethernet/google/gve/gve.h @@ -882,6 +882,9 @@ struct gve_priv { /* True if the device supports reading the nic clock */ bool nic_timestamp_supported; struct gve_ptp *ptp; + struct gve_nic_ts_report *nic_ts_report; + dma_addr_t nic_ts_report_bus; + u64 last_sync_nic_counter; /* Clock counter from last NIC TS report */ }; =20 enum gve_service_task_flags_bit { @@ -1261,6 +1264,24 @@ int gve_del_flow_rule(struct gve_priv *priv, struct = ethtool_rxnfc *cmd); int gve_flow_rules_reset(struct gve_priv *priv); /* RSS config */ int gve_init_rss_config(struct gve_priv *priv, u16 num_queues); +/* PTP and timestamping */ +#if IS_ENABLED(CONFIG_PTP_1588_CLOCK) +int gve_clock_nic_ts_read(struct gve_priv *priv); +int gve_init_clock(struct gve_priv *priv); +void gve_teardown_clock(struct gve_priv *priv); +#else /* CONFIG_PTP_1588_CLOCK */ +static inline int gve_clock_nic_ts_read(struct gve_priv *priv) +{ + return -EOPNOTSUPP; +} + +static inline int gve_init_clock(struct gve_priv *priv) +{ + return 0; +} + +static inline void gve_teardown_clock(struct gve_priv *priv) { } +#endif /* CONFIG_PTP_1588_CLOCK */ /* report stats handling */ void gve_handle_report_stats(struct gve_priv *priv); /* exported by ethtool.c */ diff --git a/drivers/net/ethernet/google/gve/gve_main.c b/drivers/net/ether= net/google/gve/gve_main.c index dc35a23ec47f..bc2f36962ee8 100644 --- a/drivers/net/ethernet/google/gve/gve_main.c +++ b/drivers/net/ethernet/google/gve/gve_main.c @@ -619,9 +619,12 @@ static int gve_setup_device_resources(struct gve_priv = *priv) err =3D gve_alloc_counter_array(priv); if (err) goto abort_with_rss_config_cache; - err =3D gve_alloc_notify_blocks(priv); + err =3D gve_init_clock(priv); if (err) goto abort_with_counter; + err =3D gve_alloc_notify_blocks(priv); + if (err) + goto abort_with_clock; err =3D gve_alloc_stats_report(priv); if (err) goto abort_with_ntfy_blocks; @@ -674,6 +677,8 @@ static int gve_setup_device_resources(struct gve_priv *= priv) gve_free_stats_report(priv); abort_with_ntfy_blocks: gve_free_notify_blocks(priv); +abort_with_clock: + gve_teardown_clock(priv); abort_with_counter: gve_free_counter_array(priv); abort_with_rss_config_cache: diff --git a/drivers/net/ethernet/google/gve/gve_ptp.c b/drivers/net/ethern= et/google/gve/gve_ptp.c index 293f8dd49afe..e96247c9d68d 100644 --- a/drivers/net/ethernet/google/gve/gve_ptp.c +++ b/drivers/net/ethernet/google/gve/gve_ptp.c @@ -5,13 +5,52 @@ */ =20 #include "gve.h" +#include "gve_adminq.h" + +/* Interval to schedule a nic timestamp calibration, 250ms. */ +#define GVE_NIC_TS_SYNC_INTERVAL_MS 250 + +/* Read the nic timestamp from hardware via the admin queue. */ +int gve_clock_nic_ts_read(struct gve_priv *priv) +{ + u64 nic_raw; + int err; + + err =3D gve_adminq_report_nic_ts(priv, priv->nic_ts_report_bus); + if (err) + return err; + + nic_raw =3D be64_to_cpu(priv->nic_ts_report->nic_timestamp); + WRITE_ONCE(priv->last_sync_nic_counter, nic_raw); + + return 0; +} + +static long gve_ptp_do_aux_work(struct ptp_clock_info *info) +{ + const struct gve_ptp *ptp =3D container_of(info, struct gve_ptp, info); + struct gve_priv *priv =3D ptp->priv; + int err; + + if (gve_get_reset_in_progress(priv) || !gve_get_admin_queue_ok(priv)) + goto out; + + err =3D gve_clock_nic_ts_read(priv); + if (err && net_ratelimit()) + dev_err(&priv->pdev->dev, + "%s read err %d\n", __func__, err); + +out: + return msecs_to_jiffies(GVE_NIC_TS_SYNC_INTERVAL_MS); +} =20 static const struct ptp_clock_info gve_ptp_caps =3D { .owner =3D THIS_MODULE, .name =3D "gve clock", + .do_aux_work =3D gve_ptp_do_aux_work, }; =20 -static int __maybe_unused gve_ptp_init(struct gve_priv *priv) +static int gve_ptp_init(struct gve_priv *priv) { struct gve_ptp *ptp; int err; @@ -44,7 +83,7 @@ static int __maybe_unused gve_ptp_init(struct gve_priv *p= riv) return err; } =20 -static void __maybe_unused gve_ptp_release(struct gve_priv *priv) +static void gve_ptp_release(struct gve_priv *priv) { struct gve_ptp *ptp =3D priv->ptp; =20 @@ -57,3 +96,44 @@ static void __maybe_unused gve_ptp_release(struct gve_pr= iv *priv) kfree(ptp); priv->ptp =3D NULL; } + +int gve_init_clock(struct gve_priv *priv) +{ + int err; + + if (!priv->nic_timestamp_supported) + return 0; + + err =3D gve_ptp_init(priv); + if (err) + return err; + + priv->nic_ts_report =3D + dma_alloc_coherent(&priv->pdev->dev, + sizeof(struct gve_nic_ts_report), + &priv->nic_ts_report_bus, + GFP_KERNEL); + if (!priv->nic_ts_report) { + dev_err(&priv->pdev->dev, "%s dma alloc error\n", __func__); + err =3D -ENOMEM; + goto release_ptp; + } + + return 0; + +release_ptp: + gve_ptp_release(priv); + return err; +} + +void gve_teardown_clock(struct gve_priv *priv) +{ + gve_ptp_release(priv); + + if (priv->nic_ts_report) { + dma_free_coherent(&priv->pdev->dev, + sizeof(struct gve_nic_ts_report), + priv->nic_ts_report, priv->nic_ts_report_bus); + priv->nic_ts_report =3D NULL; + } +} --=20 2.50.0.rc1.591.g9c95f17f64-goog