From nobody Sun Apr 5 19:42:20 2026 Received: from mail-pg1-f202.google.com (mail-pg1-f202.google.com [209.85.215.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7A9283D7D77 for ; Fri, 3 Apr 2026 19:44:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775245475; cv=none; b=hSz6pNDbcxrysIapJNLY0ymQML/ng9A1ab2P1Uk8VKVwwHnSB1NWHjkzGHn+Kvaa2j5ZOvTBvPDGL3AcsPRznnyUYYg+AFBUeCEnd6h9PpcTwF95SgL5V1U2ttfn3Qwh1VP/y5TlvOk8c0OPk3I85uuU+LGEXbXJ4eZqRnBhxgs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775245475; c=relaxed/simple; bh=oYuTgHASR13wx51gCN7dp7iKyzHqtjbJmTVztcALf64=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=GDB31+G0oPBUm0HJ6GIKOsdtzHPjzywZpE6QJtaff3PiBpU0bykaZAwBsFTlB2SbiWPpy+sA40B+/ergEW4pvBwwikhpPPmNeAMxfLgPyON9f7/sfLFYwgEWjR1zTBmKc6CN36iOFi90FTmAAbwXPG3mZIXLW6XMFS0RewQfHC0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--hramamurthy.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=LqnsYswA; arc=none smtp.client-ip=209.85.215.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--hramamurthy.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="LqnsYswA" Received: by mail-pg1-f202.google.com with SMTP id 41be03b00d2f7-c70ea91bfe1so1460673a12.1 for ; Fri, 03 Apr 2026 12:44:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1775245474; x=1775850274; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=A8l16tATZuU70nyjoOFEBJeX0fkq5enqtuO7E7lUUZk=; b=LqnsYswA0gAVKtvGMC+ApU/C/HgXJxbad7w2ADWSCbSP328OhryptGAF0cmquQihzK X7Jd/msQ1xK1dw+pFvaYcfpHet4N/KQMf+VEr/gE2kU0kmwyjtM/y8dAnOcpQexGfLpC hZ3uXC186JhIlpE6188xz/7hP+xwpbb75nIDWxGIOqX9+xqq6K977e+8+wEzuw2Y+0a1 tsafuGwlcAkvWOUEijThxuYVaaltHpPsdhJXk+3wdJ6YGAsj/3ioMOYuiHh5OBSwFC88 OKR5XMofzK30abrJBDgLjfLGn7ZdrgJn9r2I2FseJVFTCdfMmz8bPsCCCcckbgzedj5I DO1w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1775245474; x=1775850274; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=A8l16tATZuU70nyjoOFEBJeX0fkq5enqtuO7E7lUUZk=; b=E9yQ7EbjhnCfQYsCiCcGoP7czsmfaqvbcNbhRRFx7cLkRX1jfWo+Fc/3ny4b0y5AZ5 eIUE7PJ1WPy2nxaWmlXX0m3MOeK+0GxGSVSt9O0VrdI50yLeyMSW2exRYt1MzHmeu2mp WXhrhbU8nGslfI9p4P0eLMRFlahKsQZfhTxICfgD8HyvpGCwRJnCEzua9b1Dgwc21gkr Gj1C/YN+tTe8msOuOjbZR19l8y+9/9dGsNhaDHKQHrL7SOaNmyuv7+LbBfjKXoxUTII/ jLvpHJFAmi4RTy8tYu02jl8Y1hzPIWeztzyPHWpLdO7cWj8hbDXmzQmZGkWeknVfJsJV 7x0g== X-Forwarded-Encrypted: i=1; AJvYcCXZScRPRlZLYYAEDd1OYeUQ8bsBW6cGeNFGkjPqc/0OGT95CK4EpqdzXOErfLT6pRuD8pRUjyfVw84eWMo=@vger.kernel.org X-Gm-Message-State: AOJu0YxScpg0FSNZS1gSO8hISfi1f0hd2O+RQ80lBhJhmfGxnba4uQlc If0PHm++TEfOb3p0hCfYC+qGGN2ukkY1TPWwq4GbGUelDZ+9++71TqqTfFyz+Ujb5QhZWMN82UU xca7h3VwNXUqvMh5XLV5l+Keflg== X-Received: from pfbfb16.prod.google.com ([2002:a05:6a00:2d90:b0:824:bb96:db0]) (user=hramamurthy job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a00:4fc6:b0:82c:eb46:aeac with SMTP id d2e1a72fcca58-82d0dbc34dcmr3946304b3a.50.1775245473310; Fri, 03 Apr 2026 12:44:33 -0700 (PDT) Date: Fri, 3 Apr 2026 19:44:26 +0000 In-Reply-To: <20260403194427.1830609-1-hramamurthy@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260403194427.1830609-1-hramamurthy@google.com> X-Mailer: git-send-email 2.53.0.1213.gd9a14994de-goog Message-ID: <20260403194427.1830609-3-hramamurthy@google.com> Subject: [PATCH net-next v3 2/3] gve: make nic clock reads thread safe From: Harshitha Ramamurthy To: netdev@vger.kernel.org Cc: joshwash@google.com, hramamurthy@google.com, andrew+netdev@lunn.ch, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, richardcochran@gmail.com, willemb@google.com, nktgrg@google.com, jfraker@google.com, ziweixiao@google.com, maolson@google.com, thostet@google.com, jordanrhee@google.com, jefrogers@google.com, alok.a.tiwari@oracle.com, yyd@google.com, linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ankit Garg Add a mutex to protect the shared DMA buffer that receives NIC timestamp reports. The NIC timestamp will be read from two different threads: the periodic worker and upcoming `gettimex64`. Move clock registration to the last step of initialization to ensure that all data needed by the clock module is initialized before the clock is exposed to usermode. Reviewed-by: Joshua Washington Signed-off-by: Ankit Garg Signed-off-by: Jordan Rhee Signed-off-by: Harshitha Ramamurthy --- Changes in v3: - Reorder init/teardown to register PTP clock last, and simplify code - Move ptp-related members from gve_priv to gve_ptp - Only assign priv->ptp after ptp module is successfully initialized --- drivers/net/ethernet/google/gve/gve.h | 12 +- drivers/net/ethernet/google/gve/gve_ethtool.c | 3 +- drivers/net/ethernet/google/gve/gve_ptp.c | 134 ++++++++---------- 3 files changed, 63 insertions(+), 86 deletions(-) diff --git a/drivers/net/ethernet/google/gve/gve.h b/drivers/net/ethernet/g= oogle/gve/gve.h index 1d66d3834f7e..7b69d0cfc0d5 100644 --- a/drivers/net/ethernet/google/gve/gve.h +++ b/drivers/net/ethernet/google/gve/gve.h @@ -792,6 +792,9 @@ struct gve_ptp { struct ptp_clock_info info; struct ptp_clock *clock; struct gve_priv *priv; + struct mutex nic_ts_read_lock; /* Protects nic_ts_report */ + struct gve_nic_ts_report *nic_ts_report; + dma_addr_t nic_ts_report_bus; }; =20 struct gve_priv { @@ -923,8 +926,6 @@ struct gve_priv { bool nic_timestamp_supported; struct gve_ptp *ptp; struct kernel_hwtstamp_config ts_config; - struct gve_nic_ts_report *nic_ts_report; - dma_addr_t nic_ts_report_bus; u64 last_sync_nic_counter; /* Clock counter from last NIC TS report */ }; =20 @@ -1201,7 +1202,7 @@ static inline bool gve_supports_xdp_xmit(struct gve_p= riv *priv) =20 static inline bool gve_is_clock_enabled(struct gve_priv *priv) { - return priv->nic_ts_report; + return priv->ptp; } =20 /* gqi napi handler defined in gve_main.c */ @@ -1321,14 +1322,9 @@ int gve_flow_rules_reset(struct gve_priv *priv); int gve_init_rss_config(struct gve_priv *priv, u16 num_queues); /* PTP and timestamping */ #if IS_ENABLED(CONFIG_PTP_1588_CLOCK) -int gve_clock_nic_ts_read(struct gve_priv *priv); int gve_init_clock(struct gve_priv *priv); void gve_teardown_clock(struct gve_priv *priv); #else /* CONFIG_PTP_1588_CLOCK */ -static inline int gve_clock_nic_ts_read(struct gve_priv *priv) -{ - return -EOPNOTSUPP; -} =20 static inline int gve_init_clock(struct gve_priv *priv) { diff --git a/drivers/net/ethernet/google/gve/gve_ethtool.c b/drivers/net/et= hernet/google/gve/gve_ethtool.c index dc2213b5ce24..4fd7e8a442c5 100644 --- a/drivers/net/ethernet/google/gve/gve_ethtool.c +++ b/drivers/net/ethernet/google/gve/gve_ethtool.c @@ -972,8 +972,7 @@ static int gve_get_ts_info(struct net_device *netdev, info->rx_filters |=3D BIT(HWTSTAMP_FILTER_NONE) | BIT(HWTSTAMP_FILTER_ALL); =20 - if (priv->ptp) - info->phc_index =3D ptp_clock_index(priv->ptp->clock); + info->phc_index =3D ptp_clock_index(priv->ptp->clock); } =20 return 0; diff --git a/drivers/net/ethernet/google/gve/gve_ptp.c b/drivers/net/ethern= et/google/gve/gve_ptp.c index 06b1cf4a5efc..ad15f1209a83 100644 --- a/drivers/net/ethernet/google/gve/gve_ptp.c +++ b/drivers/net/ethernet/google/gve/gve_ptp.c @@ -11,19 +11,20 @@ #define GVE_NIC_TS_SYNC_INTERVAL_MS 250 =20 /* Read the nic timestamp from hardware via the admin queue. */ -int gve_clock_nic_ts_read(struct gve_priv *priv) +static int gve_clock_nic_ts_read(struct gve_ptp *ptp, u64 *nic_raw) { - u64 nic_raw; int err; =20 - err =3D gve_adminq_report_nic_ts(priv, priv->nic_ts_report_bus); + mutex_lock(&ptp->nic_ts_read_lock); + err =3D gve_adminq_report_nic_ts(ptp->priv, ptp->nic_ts_report_bus); if (err) - return err; + goto out; =20 - nic_raw =3D be64_to_cpu(priv->nic_ts_report->nic_timestamp); - WRITE_ONCE(priv->last_sync_nic_counter, nic_raw); + *nic_raw =3D be64_to_cpu(ptp->nic_ts_report->nic_timestamp); =20 - return 0; +out: + mutex_unlock(&ptp->nic_ts_read_lock); + return err; } =20 static int gve_ptp_gettimex64(struct ptp_clock_info *info, @@ -41,17 +42,21 @@ static int gve_ptp_settime64(struct ptp_clock_info *inf= o, =20 static long gve_ptp_do_aux_work(struct ptp_clock_info *info) { - const struct gve_ptp *ptp =3D container_of(info, struct gve_ptp, info); + struct gve_ptp *ptp =3D container_of(info, struct gve_ptp, info); struct gve_priv *priv =3D ptp->priv; + u64 nic_raw; int err; =20 if (gve_get_reset_in_progress(priv) || !gve_get_admin_queue_ok(priv)) goto out; =20 - err =3D gve_clock_nic_ts_read(priv); - if (err && net_ratelimit()) - dev_err(&priv->pdev->dev, - "%s read err %d\n", __func__, err); + err =3D gve_clock_nic_ts_read(ptp, &nic_raw); + if (err) { + dev_err_ratelimited(&priv->pdev->dev, "%s read err %d\n", + __func__, err); + goto out; + } + WRITE_ONCE(priv->last_sync_nic_counter, nic_raw); =20 out: return msecs_to_jiffies(GVE_NIC_TS_SYNC_INTERVAL_MS); @@ -65,94 +70,71 @@ static const struct ptp_clock_info gve_ptp_caps =3D { .do_aux_work =3D gve_ptp_do_aux_work, }; =20 -static int gve_ptp_init(struct gve_priv *priv) +int gve_init_clock(struct gve_priv *priv) { struct gve_ptp *ptp; + u64 nic_raw; int err; =20 - priv->ptp =3D kzalloc_obj(*priv->ptp); - if (!priv->ptp) + ptp =3D kzalloc_obj(*priv->ptp); + if (!ptp) return -ENOMEM; =20 - ptp =3D priv->ptp; ptp->info =3D gve_ptp_caps; - ptp->clock =3D ptp_clock_register(&ptp->info, &priv->pdev->dev); - - if (IS_ERR(ptp->clock)) { - dev_err(&priv->pdev->dev, "PTP clock registration failed\n"); - err =3D PTR_ERR(ptp->clock); - goto free_ptp; - } - ptp->priv =3D priv; - return 0; - -free_ptp: - kfree(ptp); - priv->ptp =3D NULL; - return err; -} - -static void gve_ptp_release(struct gve_priv *priv) -{ - struct gve_ptp *ptp =3D priv->ptp; - - if (!ptp) - return; - - if (ptp->clock) - ptp_clock_unregister(ptp->clock); - - kfree(ptp); - priv->ptp =3D NULL; -} - -int gve_init_clock(struct gve_priv *priv) -{ - int err; - - err =3D gve_ptp_init(priv); - if (err) - return err; - - priv->nic_ts_report =3D + mutex_init(&ptp->nic_ts_read_lock); + ptp->nic_ts_report =3D dma_alloc_coherent(&priv->pdev->dev, sizeof(struct gve_nic_ts_report), - &priv->nic_ts_report_bus, - GFP_KERNEL); - if (!priv->nic_ts_report) { + &ptp->nic_ts_report_bus, GFP_KERNEL); + if (!ptp->nic_ts_report) { dev_err(&priv->pdev->dev, "%s dma alloc error\n", __func__); err =3D -ENOMEM; - goto release_ptp; + goto free_ptp; } - err =3D gve_clock_nic_ts_read(priv); + + err =3D gve_clock_nic_ts_read(ptp, &nic_raw); if (err) { dev_err(&priv->pdev->dev, "failed to read NIC clock %d\n", err); - goto release_nic_ts_report; + goto free_dma_mem; } - ptp_schedule_worker(priv->ptp->clock, + WRITE_ONCE(priv->last_sync_nic_counter, nic_raw); + + ptp->clock =3D ptp_clock_register(&ptp->info, &priv->pdev->dev); + if (IS_ERR(ptp->clock)) { + dev_err(&priv->pdev->dev, "PTP clock registration failed\n"); + err =3D PTR_ERR(ptp->clock); + goto free_dma_mem; + } + + priv->ptp =3D ptp; + ptp_schedule_worker(ptp->clock, msecs_to_jiffies(GVE_NIC_TS_SYNC_INTERVAL_MS)); =20 return 0; =20 -release_nic_ts_report: - dma_free_coherent(&priv->pdev->dev, - sizeof(struct gve_nic_ts_report), - priv->nic_ts_report, priv->nic_ts_report_bus); - priv->nic_ts_report =3D NULL; -release_ptp: - gve_ptp_release(priv); +free_dma_mem: + dma_free_coherent(&priv->pdev->dev, sizeof(struct gve_nic_ts_report), + ptp->nic_ts_report, ptp->nic_ts_report_bus); + ptp->nic_ts_report =3D NULL; +free_ptp: + mutex_destroy(&ptp->nic_ts_read_lock); + kfree(ptp); return err; } =20 void gve_teardown_clock(struct gve_priv *priv) { - gve_ptp_release(priv); + struct gve_ptp *ptp =3D priv->ptp; =20 - if (priv->nic_ts_report) { - dma_free_coherent(&priv->pdev->dev, - sizeof(struct gve_nic_ts_report), - priv->nic_ts_report, priv->nic_ts_report_bus); - priv->nic_ts_report =3D NULL; - } + if (!ptp) + return; + + priv->ptp =3D NULL; + ptp_clock_unregister(ptp->clock); + dma_free_coherent(&priv->pdev->dev, sizeof(struct gve_nic_ts_report), + ptp->nic_ts_report, ptp->nic_ts_report_bus); + ptp->nic_ts_report =3D NULL; + mutex_destroy(&ptp->nic_ts_read_lock); + kfree(ptp); } --=20 2.53.0.1213.gd9a14994de-goog