From nobody Wed Jun 17 01:50:47 2026 Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BB30918A6D4 for ; Wed, 29 Apr 2026 01:28:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777426107; cv=none; b=Gy0yOprBc3N5mdL/wepB3vILOahxSwTjiD5z2/YrprHdZxJfLHHd9u1iN7V4ACknyEXtahMgDZSvXqcBhSx2TWnIxPY+jig9/GEC9HT/LDnakB1uDol65kvwvE3il67WAKU9qk5agYoNffuHA/LJ7jQsXYgx5bAnZYbsgTprtmg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777426107; c=relaxed/simple; bh=xGCwW9AJ00w0E9A81uL8RSVHIPPhwZ/YeQ2rEJ++s8I=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=FgJfWPesMDUUobRdphuiggmuqttQUc4yGOLs9CRisL+mDBMqU0by2rE87ZSx9Tigr6WzJPrdX7nBHyfpAlY6SI0gxlQesbDtAbNy7WoAIYfIUVfSBCB0HDKU2377j3Pig7IcdpjhpDyb5LKm2GZP69DvlCsVTx1H4p4RyGVc5/s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--hramamurthy.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=NTZrDMnw; arc=none smtp.client-ip=209.85.214.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--hramamurthy.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="NTZrDMnw" Received: by mail-pl1-f201.google.com with SMTP id d9443c01a7336-2b2ed279eedso83886765ad.0 for ; Tue, 28 Apr 2026 18:28:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1777426105; x=1778030905; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=B/sqZJ2nakkLZrxrJSKax1gOUohwsZLRlvLL9dCFNUk=; b=NTZrDMnwap6d8k5krbMSUDLKYe8Jx4JvAplxIKNJPhnkS7ZN7ApdUbQRcbFu+7yI9A kOwNUFQcOiHCq3o2U7kTALKbH/zWad4+n006f24obW/0X3KkrQKXGYrDWZRCRTLD9zGu KbXncm9ManJsiP7GY7U5vyhNCk1FK1KXwhIz4Oh46vIknBd+p/p5TP1jU1S1x3zxA5vm hANvn8AqaRVsqyzobh0ThTykRQ0zYShh/T1vIvr8ZR/WhWmvpYFKcUZANtDGKTtF9/kI RF2O4I58NyR/ECbFpmUz6coy5nIg35vmdTH1R7mlywz7hsQ0XOsOzAkz6xRCmSOGABVX mvYQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1777426105; x=1778030905; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=B/sqZJ2nakkLZrxrJSKax1gOUohwsZLRlvLL9dCFNUk=; b=NQZm2BkDAGQwarm1huaKLfXDlsv8t59/wzvR4nKFifCnShLsEIRMhwMrYhl1Pg5aCx i0q3aDCKhzY07uNPWXu7wP8+sjDUJaXnCD8zEUU3Ew3CJtEiigHYlnHsQS2lvuTmcpwr 2gMkRHgS5wcFrLiD+Li+duQjSBzh7m+WIou+jrtyUNC1/T0SJe6JTznVMgH6YbQlramM WL2RMMgqYkFf+soe69XDsxvi672AvcKmB3znwCly9NVJsh1nnhCllEiXNi2VV6nisifp 88t1mpVip905dFTXS5Gp42wNqjzJJJxHQ1WsxAOlygfvp3Owu1sWfRDB837mRMMksOnu G3JA== X-Forwarded-Encrypted: i=1; AFNElJ+6FZ2FGzhoKJm4Xhs6fCeLhvbl5JWXKEmMRHsH9uRm/9RDhQt62ZFkTgaoSVMKZ5IKKMq2scCbGrV5Lx8=@vger.kernel.org X-Gm-Message-State: AOJu0YwhiY7TPLprab2Slnc3znYiB7nuAlDPJ1YSYLUgPDaw3RJKiHFX qknun8kgyySf4VbRkwDJ167pguwXd/0G81cJV7YG/qCiWAZK6m8VF1NYWe7pU3cRh5QGzqEc7/E nME8vV6q3mpN4VJ4Mcy3HJFNLig== X-Received: from plbkk13.prod.google.com ([2002:a17:903:70d:b0:2b2:506e:862d]) (user=hramamurthy job=prod-delivery.src-stubby-dispatcher) by 2002:a17:903:1a2b:b0:2b0:9101:1b77 with SMTP id d9443c01a7336-2b987391b1bmr17075435ad.17.1777426104885; Tue, 28 Apr 2026 18:28:24 -0700 (PDT) Date: Wed, 29 Apr 2026 01:28:17 +0000 In-Reply-To: <20260429012819.3102675-1-hramamurthy@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260429012819.3102675-1-hramamurthy@google.com> X-Mailer: git-send-email 2.54.0.545.g6539524ca2-goog Message-ID: <20260429012819.3102675-2-hramamurthy@google.com> Subject: [PATCH net-next v5 1/3] gve: skip error logging for retryable AdminQ commands From: Harshitha Ramamurthy To: netdev@vger.kernel.org Cc: joshwash@google.com, hramamurthy@google.com, andrew+netdev@lunn.ch, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, richardcochran@gmail.com, jstultz@google.com, tglx@kernel.org, sboyd@kernel.org, willemb@google.com, nktgrg@google.com, jfraker@google.com, ziweixiao@google.com, maolson@google.com, jordanrhee@google.com, thostet@google.com, alok.a.tiwari@oracle.com, pkaligineedi@google.com, horms@kernel.org, dwmw2@infradead.org, jacob.e.keller@intel.com, yyd@google.com, linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Jordan Rhee AdminQ commands may return -EAGAIN under certain transient conditions. These commands are intended to be retried by the driver, so logging a formal error to the system log is misleading and creates unnecessary noise. Modify the logging logic to skip the error message when the result is -EAGAIN, and move logging to dev_err_ratelimited() to avoid spamming the log. Reviewed-by: Joshua Washington Signed-off-by: Jordan Rhee Signed-off-by: Harshitha Ramamurthy --- Changes in v4: - call out change to dev_err_ratelimited() in the commit message (Jacob Kel= ler) - remove extra print when adminQ status is GVE_ADMINQ_COMMAND_UNSET (Jacob = Keller) --- drivers/net/ethernet/google/gve/gve_adminq.c | 27 +++++++++++++++----- 1 file changed, 20 insertions(+), 7 deletions(-) diff --git a/drivers/net/ethernet/google/gve/gve_adminq.c b/drivers/net/eth= ernet/google/gve/gve_adminq.c index 08587bf40ed4..a65b14835aa0 100644 --- a/drivers/net/ethernet/google/gve/gve_adminq.c +++ b/drivers/net/ethernet/google/gve/gve_adminq.c @@ -416,16 +416,10 @@ static bool gve_adminq_wait_for_cmd(struct gve_priv *= priv, u32 prod_cnt) =20 static int gve_adminq_parse_err(struct gve_priv *priv, u32 status) { - if (status !=3D GVE_ADMINQ_COMMAND_PASSED && - status !=3D GVE_ADMINQ_COMMAND_UNSET) { - dev_err(&priv->pdev->dev, "AQ command failed with status %d\n", status); - priv->adminq_cmd_fail++; - } switch (status) { case GVE_ADMINQ_COMMAND_PASSED: return 0; case GVE_ADMINQ_COMMAND_UNSET: - dev_err(&priv->pdev->dev, "parse_aq_err: err and status both unset, this= should not be possible.\n"); return -EINVAL; case GVE_ADMINQ_COMMAND_ERROR_ABORTED: case GVE_ADMINQ_COMMAND_ERROR_CANCELLED: @@ -455,6 +449,16 @@ static int gve_adminq_parse_err(struct gve_priv *priv,= u32 status) } } =20 +static bool gve_adminq_is_retryable(enum gve_adminq_opcodes opcode) +{ + switch (opcode) { + case GVE_ADMINQ_REPORT_NIC_TIMESTAMP: + return true; + default: + return false; + } +} + /* Flushes all AQ commands currently queued and waits for them to complete. * If there are failures, it will return the first error. */ @@ -482,9 +486,18 @@ static int gve_adminq_kick_and_wait(struct gve_priv *p= riv) cmd =3D &priv->adminq[i & priv->adminq_mask]; status =3D be32_to_cpu(READ_ONCE(cmd->status)); err =3D gve_adminq_parse_err(priv, status); - if (err) + if (err) { + enum gve_adminq_opcodes opcode =3D + be32_to_cpu(READ_ONCE(cmd->opcode)); + priv->adminq_cmd_fail++; + if (!gve_adminq_is_retryable(opcode) || err !=3D -EAGAIN) + dev_err_ratelimited(&priv->pdev->dev, + "AQ command %d failed with status %d\n", + opcode, status); + // Return the first error if we failed. return err; + } } =20 return 0; --=20 2.54.0.545.g6539524ca2-goog From nobody Wed Jun 17 01:50:47 2026 Received: from mail-pg1-f202.google.com (mail-pg1-f202.google.com [209.85.215.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6C8DD25B2FA for ; Wed, 29 Apr 2026 01:28:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777426109; cv=none; b=N0eF1xrG44OeVETk6Fz1nXjkdXSM0IzwZ9pnl+InPpHu+BzkyH8WU8ZoOyaSnfCdQje2wugZIo5/LUvQCsvxRaTf7CP3iPu0Ty82FQVEkfxzv/C33lAxRyFFiBT9PDWZmpoSG7E/m5150DIZp9lOZnRpRJZub8WeoE5t3ia/448= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777426109; c=relaxed/simple; bh=i9L8XnvSFiPpXIC0LouXBHFGUfPyALs9LPvamcJaDKk=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=CEtOISz8WhWlpE2qXIHso+WLG3eyiCvRedReDYaLS9EvTd2z7NOSsH8ETiZzy4ImAEFWWYaq2lh9d8qXpzMvBcGfVPZUlhfgYqflrIpHUIlbV75MZGb7Km1uWZWcsWyje0budpvy7IuEzjOrLzjq3sNMKcHMZLAIttxdXoFh9Sk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--hramamurthy.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=XZ2KQC4G; arc=none smtp.client-ip=209.85.215.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--hramamurthy.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="XZ2KQC4G" Received: by mail-pg1-f202.google.com with SMTP id 41be03b00d2f7-c7993b17335so4549384a12.0 for ; Tue, 28 Apr 2026 18:28:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1777426107; x=1778030907; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=RlzKRi1235IEdxnPrVCGp9XETwfEZvpjn5soN3/viWU=; b=XZ2KQC4GGLB3H0htF4O5zmfS29N9LdaoVILKhNgtzKNpT67Nz9UC8Sgp+awQhol5gO f1szoMSTnUo8BbI5QkCJFcdvvr/x0zwy8vJEyl/2GvORPVo7hulNxcARiBYctGlbtzNx X+h12IrnctV+n/2q9s1ITafwXy/2BVV33KouzS38Ft09bW+/xgLAyayixOvYDA3nIUka FGgb966khvw6s7gXo8VgMHcdHIRSyj5Hrms7kc7zxE904SM0Gf5Y66vsErK0abO4zf5t FYROPv1oPpah2xWH4/t4/TLULjRbYyK9wgJdFOJKwoMJGaPLj9GN2tBdONJ1+0ARwvzw R2NQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1777426107; x=1778030907; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=RlzKRi1235IEdxnPrVCGp9XETwfEZvpjn5soN3/viWU=; b=M5lyGxlU7IvuArDi1tyTSUCfmwyZ6pvOhvHIybu231+/w/I99ZK9MxJxrgGziYBTyV sp5qXOaOdCKqgGHVLafcqhnlL+3T/3DmOX9VWvxVCzlo6FVNdmQXmcCtberjOOMHUfBV zJUlnnTnAd7WkLQIQs0hFpWaauvL+fA+RMsSXyQ1jMRw2n6WkLoUp1uygYvm3/40Uh5n nTkTgNm106GaWy/szCWftkksvoyJNNOkoZtPiciitWHObJrHOMBMkMxAFGiE2lK5NVvZ 4FNxDt2O7YL3GAUfYXwvDGArWjERWIQzy6ks0hLkiGkL5Q4bmfm7PYPgTS8iD71d0NSe umsg== X-Forwarded-Encrypted: i=1; AFNElJ9kA3Uzz7SyeWkxI50iBP0k+z82NY4+6AfkercoQAPJ+1Q8RWSqdXfEteAC47VSL1GEemFUXQQ+8CRIY/w=@vger.kernel.org X-Gm-Message-State: AOJu0YzwVd7ed6l6uykLv+spKuq0dTzD4OVi1eBJqLqUF6Ma3pXhcfhp 2FyXgh4Fa4ZFIKASAiLO58JeURG/rhJwC7vqIe82efyogkq1ZpfE8yXLOKJgZ3QokNCnEqp5UWk CyKWIJfG7/Y8oJyPtXDGBDrHHOA== X-Received: from pfck21-n2.prod.google.com ([2002:a05:6a00:a055:20b0:833:3a86:a8c8]) (user=hramamurthy job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a20:ed01:10b0:3a2:e0d3:37ea with SMTP id adf61e73a8af0-3a39c049fcdmr4461147637.11.1777426106594; Tue, 28 Apr 2026 18:28:26 -0700 (PDT) Date: Wed, 29 Apr 2026 01:28:18 +0000 In-Reply-To: <20260429012819.3102675-1-hramamurthy@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260429012819.3102675-1-hramamurthy@google.com> X-Mailer: git-send-email 2.54.0.545.g6539524ca2-goog Message-ID: <20260429012819.3102675-3-hramamurthy@google.com> Subject: [PATCH net-next v5 2/3] gve: make nic clock reads thread safe From: Harshitha Ramamurthy To: netdev@vger.kernel.org Cc: joshwash@google.com, hramamurthy@google.com, andrew+netdev@lunn.ch, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, richardcochran@gmail.com, jstultz@google.com, tglx@kernel.org, sboyd@kernel.org, willemb@google.com, nktgrg@google.com, jfraker@google.com, ziweixiao@google.com, maolson@google.com, jordanrhee@google.com, thostet@google.com, alok.a.tiwari@oracle.com, pkaligineedi@google.com, horms@kernel.org, dwmw2@infradead.org, jacob.e.keller@intel.com, yyd@google.com, linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ankit Garg Add a mutex to protect the shared DMA buffer that receives NIC timestamp reports. The NIC timestamp will be read from two different threads: the periodic worker and upcoming `gettimex64`. Move clock registration to the last step of initialization to ensure that all data needed by the clock module is initialized before the clock is exposed to usermode. Reviewed-by: Joshua Washington Signed-off-by: Ankit Garg Signed-off-by: Jordan Rhee Signed-off-by: Harshitha Ramamurthy --- Changes in v3: - Reorder init/teardown to register PTP clock last, and simplify code - Move ptp-related members from gve_priv to gve_ptp - Only assign priv->ptp after ptp module is successfully initialized --- drivers/net/ethernet/google/gve/gve.h | 12 +- drivers/net/ethernet/google/gve/gve_ethtool.c | 3 +- drivers/net/ethernet/google/gve/gve_ptp.c | 134 ++++++++---------- 3 files changed, 63 insertions(+), 86 deletions(-) diff --git a/drivers/net/ethernet/google/gve/gve.h b/drivers/net/ethernet/g= oogle/gve/gve.h index 1d66d3834f7e..7b69d0cfc0d5 100644 --- a/drivers/net/ethernet/google/gve/gve.h +++ b/drivers/net/ethernet/google/gve/gve.h @@ -792,6 +792,9 @@ struct gve_ptp { struct ptp_clock_info info; struct ptp_clock *clock; struct gve_priv *priv; + struct mutex nic_ts_read_lock; /* Protects nic_ts_report */ + struct gve_nic_ts_report *nic_ts_report; + dma_addr_t nic_ts_report_bus; }; =20 struct gve_priv { @@ -923,8 +926,6 @@ struct gve_priv { bool nic_timestamp_supported; struct gve_ptp *ptp; struct kernel_hwtstamp_config ts_config; - struct gve_nic_ts_report *nic_ts_report; - dma_addr_t nic_ts_report_bus; u64 last_sync_nic_counter; /* Clock counter from last NIC TS report */ }; =20 @@ -1201,7 +1202,7 @@ static inline bool gve_supports_xdp_xmit(struct gve_p= riv *priv) =20 static inline bool gve_is_clock_enabled(struct gve_priv *priv) { - return priv->nic_ts_report; + return priv->ptp; } =20 /* gqi napi handler defined in gve_main.c */ @@ -1321,14 +1322,9 @@ int gve_flow_rules_reset(struct gve_priv *priv); int gve_init_rss_config(struct gve_priv *priv, u16 num_queues); /* PTP and timestamping */ #if IS_ENABLED(CONFIG_PTP_1588_CLOCK) -int gve_clock_nic_ts_read(struct gve_priv *priv); int gve_init_clock(struct gve_priv *priv); void gve_teardown_clock(struct gve_priv *priv); #else /* CONFIG_PTP_1588_CLOCK */ -static inline int gve_clock_nic_ts_read(struct gve_priv *priv) -{ - return -EOPNOTSUPP; -} =20 static inline int gve_init_clock(struct gve_priv *priv) { diff --git a/drivers/net/ethernet/google/gve/gve_ethtool.c b/drivers/net/et= hernet/google/gve/gve_ethtool.c index dc2213b5ce24..4fd7e8a442c5 100644 --- a/drivers/net/ethernet/google/gve/gve_ethtool.c +++ b/drivers/net/ethernet/google/gve/gve_ethtool.c @@ -972,8 +972,7 @@ static int gve_get_ts_info(struct net_device *netdev, info->rx_filters |=3D BIT(HWTSTAMP_FILTER_NONE) | BIT(HWTSTAMP_FILTER_ALL); =20 - if (priv->ptp) - info->phc_index =3D ptp_clock_index(priv->ptp->clock); + info->phc_index =3D ptp_clock_index(priv->ptp->clock); } =20 return 0; diff --git a/drivers/net/ethernet/google/gve/gve_ptp.c b/drivers/net/ethern= et/google/gve/gve_ptp.c index 06b1cf4a5efc..ad15f1209a83 100644 --- a/drivers/net/ethernet/google/gve/gve_ptp.c +++ b/drivers/net/ethernet/google/gve/gve_ptp.c @@ -11,19 +11,20 @@ #define GVE_NIC_TS_SYNC_INTERVAL_MS 250 =20 /* Read the nic timestamp from hardware via the admin queue. */ -int gve_clock_nic_ts_read(struct gve_priv *priv) +static int gve_clock_nic_ts_read(struct gve_ptp *ptp, u64 *nic_raw) { - u64 nic_raw; int err; =20 - err =3D gve_adminq_report_nic_ts(priv, priv->nic_ts_report_bus); + mutex_lock(&ptp->nic_ts_read_lock); + err =3D gve_adminq_report_nic_ts(ptp->priv, ptp->nic_ts_report_bus); if (err) - return err; + goto out; =20 - nic_raw =3D be64_to_cpu(priv->nic_ts_report->nic_timestamp); - WRITE_ONCE(priv->last_sync_nic_counter, nic_raw); + *nic_raw =3D be64_to_cpu(ptp->nic_ts_report->nic_timestamp); =20 - return 0; +out: + mutex_unlock(&ptp->nic_ts_read_lock); + return err; } =20 static int gve_ptp_gettimex64(struct ptp_clock_info *info, @@ -41,17 +42,21 @@ static int gve_ptp_settime64(struct ptp_clock_info *inf= o, =20 static long gve_ptp_do_aux_work(struct ptp_clock_info *info) { - const struct gve_ptp *ptp =3D container_of(info, struct gve_ptp, info); + struct gve_ptp *ptp =3D container_of(info, struct gve_ptp, info); struct gve_priv *priv =3D ptp->priv; + u64 nic_raw; int err; =20 if (gve_get_reset_in_progress(priv) || !gve_get_admin_queue_ok(priv)) goto out; =20 - err =3D gve_clock_nic_ts_read(priv); - if (err && net_ratelimit()) - dev_err(&priv->pdev->dev, - "%s read err %d\n", __func__, err); + err =3D gve_clock_nic_ts_read(ptp, &nic_raw); + if (err) { + dev_err_ratelimited(&priv->pdev->dev, "%s read err %d\n", + __func__, err); + goto out; + } + WRITE_ONCE(priv->last_sync_nic_counter, nic_raw); =20 out: return msecs_to_jiffies(GVE_NIC_TS_SYNC_INTERVAL_MS); @@ -65,94 +70,71 @@ static const struct ptp_clock_info gve_ptp_caps =3D { .do_aux_work =3D gve_ptp_do_aux_work, }; =20 -static int gve_ptp_init(struct gve_priv *priv) +int gve_init_clock(struct gve_priv *priv) { struct gve_ptp *ptp; + u64 nic_raw; int err; =20 - priv->ptp =3D kzalloc_obj(*priv->ptp); - if (!priv->ptp) + ptp =3D kzalloc_obj(*priv->ptp); + if (!ptp) return -ENOMEM; =20 - ptp =3D priv->ptp; ptp->info =3D gve_ptp_caps; - ptp->clock =3D ptp_clock_register(&ptp->info, &priv->pdev->dev); - - if (IS_ERR(ptp->clock)) { - dev_err(&priv->pdev->dev, "PTP clock registration failed\n"); - err =3D PTR_ERR(ptp->clock); - goto free_ptp; - } - ptp->priv =3D priv; - return 0; - -free_ptp: - kfree(ptp); - priv->ptp =3D NULL; - return err; -} - -static void gve_ptp_release(struct gve_priv *priv) -{ - struct gve_ptp *ptp =3D priv->ptp; - - if (!ptp) - return; - - if (ptp->clock) - ptp_clock_unregister(ptp->clock); - - kfree(ptp); - priv->ptp =3D NULL; -} - -int gve_init_clock(struct gve_priv *priv) -{ - int err; - - err =3D gve_ptp_init(priv); - if (err) - return err; - - priv->nic_ts_report =3D + mutex_init(&ptp->nic_ts_read_lock); + ptp->nic_ts_report =3D dma_alloc_coherent(&priv->pdev->dev, sizeof(struct gve_nic_ts_report), - &priv->nic_ts_report_bus, - GFP_KERNEL); - if (!priv->nic_ts_report) { + &ptp->nic_ts_report_bus, GFP_KERNEL); + if (!ptp->nic_ts_report) { dev_err(&priv->pdev->dev, "%s dma alloc error\n", __func__); err =3D -ENOMEM; - goto release_ptp; + goto free_ptp; } - err =3D gve_clock_nic_ts_read(priv); + + err =3D gve_clock_nic_ts_read(ptp, &nic_raw); if (err) { dev_err(&priv->pdev->dev, "failed to read NIC clock %d\n", err); - goto release_nic_ts_report; + goto free_dma_mem; } - ptp_schedule_worker(priv->ptp->clock, + WRITE_ONCE(priv->last_sync_nic_counter, nic_raw); + + ptp->clock =3D ptp_clock_register(&ptp->info, &priv->pdev->dev); + if (IS_ERR(ptp->clock)) { + dev_err(&priv->pdev->dev, "PTP clock registration failed\n"); + err =3D PTR_ERR(ptp->clock); + goto free_dma_mem; + } + + priv->ptp =3D ptp; + ptp_schedule_worker(ptp->clock, msecs_to_jiffies(GVE_NIC_TS_SYNC_INTERVAL_MS)); =20 return 0; =20 -release_nic_ts_report: - dma_free_coherent(&priv->pdev->dev, - sizeof(struct gve_nic_ts_report), - priv->nic_ts_report, priv->nic_ts_report_bus); - priv->nic_ts_report =3D NULL; -release_ptp: - gve_ptp_release(priv); +free_dma_mem: + dma_free_coherent(&priv->pdev->dev, sizeof(struct gve_nic_ts_report), + ptp->nic_ts_report, ptp->nic_ts_report_bus); + ptp->nic_ts_report =3D NULL; +free_ptp: + mutex_destroy(&ptp->nic_ts_read_lock); + kfree(ptp); return err; } =20 void gve_teardown_clock(struct gve_priv *priv) { - gve_ptp_release(priv); + struct gve_ptp *ptp =3D priv->ptp; =20 - if (priv->nic_ts_report) { - dma_free_coherent(&priv->pdev->dev, - sizeof(struct gve_nic_ts_report), - priv->nic_ts_report, priv->nic_ts_report_bus); - priv->nic_ts_report =3D NULL; - } + if (!ptp) + return; + + priv->ptp =3D NULL; + ptp_clock_unregister(ptp->clock); + dma_free_coherent(&priv->pdev->dev, sizeof(struct gve_nic_ts_report), + ptp->nic_ts_report, ptp->nic_ts_report_bus); + ptp->nic_ts_report =3D NULL; + mutex_destroy(&ptp->nic_ts_read_lock); + kfree(ptp); } --=20 2.54.0.545.g6539524ca2-goog From nobody Wed Jun 17 01:50:47 2026 Received: from mail-pg1-f202.google.com (mail-pg1-f202.google.com [209.85.215.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 29FC424BBF0 for ; Wed, 29 Apr 2026 01:28:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777426110; cv=none; b=YbTw/COo/6h3ms0YTXicwNgEpqkKt6AMZJTnSNWKpDElIYyTC7sbJeUTK0gDo5es1iNvjs5Ka7RNlFy5MGTMGIlpvE56v+SFyUOludvnfY152Gy4Anrk/p3WJuPbqtEpZs6F9pa4LHTxA5IuR0pKgUP0PxGhnzDs5Vdw0z9/+KQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777426110; c=relaxed/simple; bh=tUdrUoemiv/vwhoaTQ7Gw2yBtZXQ32xzSCEGe9tkWzY=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=IqX4y2L8BLm9FMN4NPcNVSc5l3ZvYcyvmt8q7eJUuKZlY0WWmBA1TuODaCSAV4rmgXipRmwpmC1d9a1IlVty9asrfZCmmPOdE7F8GltFvPM9OJedXdK5DFq9zJ+d/kP90+GjX8Jpj1xzaBgL4ZeHY3fonKqyW5s9tMqZHSz5MW0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--hramamurthy.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=ppjxRX5y; arc=none smtp.client-ip=209.85.215.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--hramamurthy.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="ppjxRX5y" Received: by mail-pg1-f202.google.com with SMTP id 41be03b00d2f7-b630753cc38so16036081a12.1 for ; Tue, 28 Apr 2026 18:28:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1777426108; x=1778030908; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=k9kPEFniD/0ZHWqph0EJn/65IOZRh4FqwBrvpbKjUbI=; b=ppjxRX5yQ+pbVqPycP6eA/XoCTi0ttZGx65tZc1NJnNVUbAAqr/hKavvFRUD4T6wad 13Ahh6wRE8cPHMaLav1ZdpR8GKd2WA15HAVCbaG8qW7BN9Ia+Kh2cF133+viB+vlGTfo wlomGcIBT688jUXAZ2OFon401Ntk4mTcpz0d4I2YZShig/NpeBY7cGJ7TnDg8YD7FZJ1 plPAymJGo4URGxOqikroqLFkPDyMQVDoRCrvaDxrZqG97Gs59EYBTTMQcaTOsJRjFGOE QGrKDjDc+l7FRbhkTALyh1YoDJxDtLz6odAuQo0D9ZKfpp2dWjHQdfcTmyW8doq5X+NB gdTg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1777426108; x=1778030908; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=k9kPEFniD/0ZHWqph0EJn/65IOZRh4FqwBrvpbKjUbI=; b=FhDYcsNhwQyB5iSoOpyqX7VHJAUWeZaSB+btQF/+9X8phCEv6NOqeXKqgWPcm7AiG5 4VC+QxC2ELL0H+o/N6WtWmIB4X0azWh2UxnF98/QebO3wUfZxQW5l5kUldt6KStS73r6 WeTkiA1iW57ZybxI1YZDCRESaGIyq3r2lJiuSxIzCoL55KXGW53am+mfQCx1tjXWoAyY hYyl+h+K/FdKax4ZNXif5RIo9ECR8MiWML+rUVBNgj8diBEFlZokdgH2BWjx1eI8ko33 3rPmQEpOwAegjSzNjbNERZVvDp2KJGwwoVs7430EJCdNyMis2XkUBzR5GzooG2TpQ6UZ vWtA== X-Forwarded-Encrypted: i=1; AFNElJ8QVdkBiKk4o6c7Tyt5cACm+fcrJfP4OdoYjsT2HALt9MIb0QlPHwKycxlh4JUKDulPfn3t0TrAL75e0ag=@vger.kernel.org X-Gm-Message-State: AOJu0YxukMnNZueA8yGicnhEo//3F2BjTvsreRKdv3tKyBgkD9xq60ut X3ji5pW70VrGg9PMRTcuV8Mp4xAXmlZ2RCNNCFFeUVQ+R7IEOm63k08/qdvDAdV6QF8a8oalYGF qWtRpu07cl+or+042VD7NOoLoXA== X-Received: from pgl18.prod.google.com ([2002:a63:b12:0:b0:c76:651e:6d52]) (user=hramamurthy job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a20:1591:b0:398:d6a9:fc9 with SMTP id adf61e73a8af0-3a39c0fb3aemr6493516637.5.1777426108301; Tue, 28 Apr 2026 18:28:28 -0700 (PDT) Date: Wed, 29 Apr 2026 01:28:19 +0000 In-Reply-To: <20260429012819.3102675-1-hramamurthy@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260429012819.3102675-1-hramamurthy@google.com> X-Mailer: git-send-email 2.54.0.545.g6539524ca2-goog Message-ID: <20260429012819.3102675-4-hramamurthy@google.com> Subject: [PATCH net-next v5 3/3] gve: implement PTP gettimex64 From: Harshitha Ramamurthy To: netdev@vger.kernel.org Cc: joshwash@google.com, hramamurthy@google.com, andrew+netdev@lunn.ch, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, richardcochran@gmail.com, jstultz@google.com, tglx@kernel.org, sboyd@kernel.org, willemb@google.com, nktgrg@google.com, jfraker@google.com, ziweixiao@google.com, maolson@google.com, jordanrhee@google.com, thostet@google.com, alok.a.tiwari@oracle.com, pkaligineedi@google.com, horms@kernel.org, dwmw2@infradead.org, jacob.e.keller@intel.com, yyd@google.com, linux-kernel@vger.kernel.org, Naman Gulati Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Jordan Rhee Enable chrony and phc2sys to synchronize system clock to NIC clock. The system cycle counters are sampled by the device to minimize the uncertainty window. If the system times are sampled in the host, the delta between pre and post readings is 100us or more due to AQ command latency. The system times returned by the device have a delta of ~1us, which enables significantly more accurate clock synchronization. Reviewed-by: Willem de Bruijn Reviewed-by: Kevin Yang Reviewed-by: Naman Gulati Signed-off-by: Jordan Rhee Signed-off-by: Harshitha Ramamurthy --- Changes in v5: - Reformulate retry loop in terms of total timeout (Jakub Kicinski) Changes in v3: - Take system time snapshot inside the mutex - Return -EOPNOTSUPP if cross-timestamp is requested on an arch other than x86 or arm64 Changes in v2: - fix compilation warning on ARM by casting cycles_t to u64 --- drivers/net/ethernet/google/gve/gve_adminq.h | 4 +- drivers/net/ethernet/google/gve/gve_ptp.c | 196 ++++++++++++++++++- 2 files changed, 191 insertions(+), 9 deletions(-) diff --git a/drivers/net/ethernet/google/gve/gve_adminq.h b/drivers/net/eth= ernet/google/gve/gve_adminq.h index 22a74b6aa17e..e6dcf6da9091 100644 --- a/drivers/net/ethernet/google/gve/gve_adminq.h +++ b/drivers/net/ethernet/google/gve/gve_adminq.h @@ -411,8 +411,8 @@ static_assert(sizeof(struct gve_adminq_report_nic_ts) = =3D=3D 16); =20 struct gve_nic_ts_report { __be64 nic_timestamp; /* NIC clock in nanoseconds */ - __be64 reserved1; - __be64 reserved2; + __be64 pre_cycles; /* System cycle counter before NIC clock read */ + __be64 post_cycles; /* System cycle counter after NIC clock read */ __be64 reserved3; __be64 reserved4; }; diff --git a/drivers/net/ethernet/google/gve/gve_ptp.c b/drivers/net/ethern= et/google/gve/gve_ptp.c index ad15f1209a83..c6c98ef825aa 100644 --- a/drivers/net/ethernet/google/gve/gve_ptp.c +++ b/drivers/net/ethernet/google/gve/gve_ptp.c @@ -10,28 +10,210 @@ /* Interval to schedule a nic timestamp calibration, 250ms. */ #define GVE_NIC_TS_SYNC_INTERVAL_MS 250 =20 +/* + * Stores cycle counter samples in get_cycles() units from a + * sandwiched NIC clock read + */ +struct gve_sysclock_sample { + /* system time snapshot taken just before issuing AdminQ command */ + struct system_time_snapshot snapshot; + /* Cycle counter from NIC before clock read */ + u64 nic_pre_cycles; + /* Cycle counter from NIC after clock read */ + u64 nic_post_cycles; + /* Cycle counter from host before issuing AQ command */ + cycles_t host_pre_cycles; + /* Cycle counter from host after AQ command returns */ + cycles_t host_post_cycles; +}; + +/* + * Read NIC clock by issuing the AQ command. The command is subject to + * rate limiting and may need to be retried. Requires nic_ts_read_lock + * to be held. + */ +static int gve_ptp_read_timestamp(struct gve_ptp *ptp, cycles_t *pre_cycle= s, + cycles_t *post_cycles, + struct system_time_snapshot *snap) +{ + unsigned long deadline =3D jiffies + msecs_to_jiffies(100); + unsigned long delay_us =3D 1000; + int err; + + lockdep_assert_held(&ptp->nic_ts_read_lock); + + do { + if (snap) + ktime_get_snapshot(snap); + + *pre_cycles =3D get_cycles(); + err =3D gve_adminq_report_nic_ts(ptp->priv, + ptp->nic_ts_report_bus); + + /* Prevent get_cycles() from being speculatively executed + * before the AdminQ command + */ + rmb(); + *post_cycles =3D get_cycles(); + if (likely(err !=3D -EAGAIN)) + return err; + + fsleep(delay_us); + + /* Exponential backoff */ + delay_us *=3D 2; + } while (time_before(jiffies, deadline)); + + return -ETIMEDOUT; +} + /* Read the nic timestamp from hardware via the admin queue. */ -static int gve_clock_nic_ts_read(struct gve_ptp *ptp, u64 *nic_raw) +static int gve_clock_nic_ts_read(struct gve_ptp *ptp, u64 *nic_raw, + struct gve_sysclock_sample *sysclock) { + cycles_t host_pre_cycles, host_post_cycles; + struct gve_nic_ts_report *ts_report; int err; =20 mutex_lock(&ptp->nic_ts_read_lock); - err =3D gve_adminq_report_nic_ts(ptp->priv, ptp->nic_ts_report_bus); - if (err) + err =3D gve_ptp_read_timestamp(ptp, &host_pre_cycles, &host_post_cycles, + sysclock ? &sysclock->snapshot : NULL); + if (err) { + dev_err_ratelimited(&ptp->priv->pdev->dev, + "AdminQ timestamp read failed: %d\n", err); goto out; + } =20 - *nic_raw =3D be64_to_cpu(ptp->nic_ts_report->nic_timestamp); + ts_report =3D ptp->nic_ts_report; + *nic_raw =3D be64_to_cpu(ts_report->nic_timestamp); + + if (sysclock) { + sysclock->nic_pre_cycles =3D be64_to_cpu(ts_report->pre_cycles); + sysclock->nic_post_cycles =3D be64_to_cpu(ts_report->post_cycles); + sysclock->host_pre_cycles =3D host_pre_cycles; + sysclock->host_post_cycles =3D host_post_cycles; + } =20 out: mutex_unlock(&ptp->nic_ts_read_lock); return err; } =20 +struct gve_cycles_to_clock_callback_ctx { + u64 cycles; +}; + +static int gve_cycles_to_clock_fn(ktime_t *device_time, + struct system_counterval_t *system_counterval, + void *ctx) +{ + struct gve_cycles_to_clock_callback_ctx *context =3D ctx; + + *device_time =3D 0; + + system_counterval->cycles =3D context->cycles; + system_counterval->use_nsecs =3D false; + + if (IS_ENABLED(CONFIG_X86)) + system_counterval->cs_id =3D CSID_X86_TSC; + else if (IS_ENABLED(CONFIG_ARM64)) + system_counterval->cs_id =3D CSID_ARM_ARCH_COUNTER; + else + return -EOPNOTSUPP; + + return 0; +} + +/* + * Convert a raw cycle count (e.g. from get_cycles()) to the system clock + * type specified by clockid. The system_time_snapshot must be taken before + * the cycle counter is sampled. + */ +static int gve_cycles_to_timespec64(struct gve_priv *priv, clockid_t clock= id, + struct system_time_snapshot *snap, + u64 cycles, struct timespec64 *ts) +{ + struct gve_cycles_to_clock_callback_ctx ctx =3D {0}; + struct system_device_crosststamp xtstamp; + int err; + + ctx.cycles =3D cycles; + err =3D get_device_system_crosststamp(gve_cycles_to_clock_fn, &ctx, snap, + &xtstamp); + if (err) { + dev_err_ratelimited(&priv->pdev->dev, + "get_device_system_crosststamp() failed to convert %lld cycles to = system time: %d\n", + cycles, + err); + return err; + } + + switch (clockid) { + case CLOCK_REALTIME: + *ts =3D ktime_to_timespec64(xtstamp.sys_realtime); + break; + case CLOCK_MONOTONIC_RAW: + *ts =3D ktime_to_timespec64(xtstamp.sys_monoraw); + break; + default: + dev_err_ratelimited(&priv->pdev->dev, + "Cycle count conversion to clockid %d not supported\n", + clockid); + return -EOPNOTSUPP; + } + + return 0; +} + static int gve_ptp_gettimex64(struct ptp_clock_info *info, struct timespec64 *ts, struct ptp_system_timestamp *sts) { - return -EOPNOTSUPP; + struct gve_ptp *ptp =3D container_of(info, struct gve_ptp, info); + struct gve_sysclock_sample sysclock =3D {0}; + struct gve_priv *priv =3D ptp->priv; + u64 nic_ts; + int err; + + if (sts && !(IS_ENABLED(CONFIG_X86) || IS_ENABLED(CONFIG_ARM64))) + return -EOPNOTSUPP; + + err =3D gve_clock_nic_ts_read(ptp, &nic_ts, sts ? &sysclock : NULL); + if (err) + return err; + + if (sts) { + /* Reject samples with out of order system clock values */ + if (!(sysclock.host_pre_cycles <=3D sysclock.nic_pre_cycles && + sysclock.nic_pre_cycles <=3D sysclock.nic_post_cycles && + sysclock.nic_post_cycles <=3D sysclock.host_post_cycles)) { + dev_err_ratelimited(&priv->pdev->dev, + "AdminQ system clock cycle counts out of order. Expecting %llu <= =3D %llu <=3D %llu <=3D %llu\n", + (u64)sysclock.host_pre_cycles, + sysclock.nic_pre_cycles, + sysclock.nic_post_cycles, + (u64)sysclock.host_post_cycles); + return -EBADMSG; + } + + err =3D gve_cycles_to_timespec64(priv, sts->clockid, + &sysclock.snapshot, + sysclock.nic_pre_cycles, + &sts->pre_ts); + if (err) + return err; + + err =3D gve_cycles_to_timespec64(priv, sts->clockid, + &sysclock.snapshot, + sysclock.nic_post_cycles, + &sts->post_ts); + if (err) + return err; + } + + *ts =3D ns_to_timespec64(nic_ts); + + return 0; } =20 static int gve_ptp_settime64(struct ptp_clock_info *info, @@ -50,7 +232,7 @@ static long gve_ptp_do_aux_work(struct ptp_clock_info *i= nfo) if (gve_get_reset_in_progress(priv) || !gve_get_admin_queue_ok(priv)) goto out; =20 - err =3D gve_clock_nic_ts_read(ptp, &nic_raw); + err =3D gve_clock_nic_ts_read(ptp, &nic_raw, NULL); if (err) { dev_err_ratelimited(&priv->pdev->dev, "%s read err %d\n", __func__, err); @@ -93,7 +275,7 @@ int gve_init_clock(struct gve_priv *priv) goto free_ptp; } =20 - err =3D gve_clock_nic_ts_read(ptp, &nic_raw); + err =3D gve_clock_nic_ts_read(ptp, &nic_raw, NULL); if (err) { dev_err(&priv->pdev->dev, "failed to read NIC clock %d\n", err); goto free_dma_mem; --=20 2.54.0.545.g6539524ca2-goog