From nobody Tue Dec 16 07:12:35 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7260B2ED161 for ; Wed, 25 Jun 2025 16:43:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750869814; cv=none; b=FKxz7Xpsjd7a79hvVRtszZYdHAFXpEn94jqafEuGieY/0+axbEao0AYwuew7U0T+Hun3EpIq7lwBg9aNvagom5vI71biBuEGw2AkWwOMnT6/I9XdcU5kYXxJPwkn8epblmuXXMxrVmgNvii0DmGzNGS97OpljdxdO1BZNZ//m0Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750869814; c=relaxed/simple; bh=SXgvElHG+rgpKsJbAG0nTmqC5C4ZMYho2SybTvsVsWE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=YlTezjP9HOvEg0mg0T+2HjmyvrwicgeTISo1EFNM1QCU7ZaxIrfnIzndf7JWYv0kernKWitHFFPF92Xa5HUh3m+CXqMi1YW58Bd87pZvrSTt5ZPFjMdkuv/MHXEkfkjHHQUkhM4zKhvBHrQuEae0c4cPP0vnA7m4DWkvIzJh63w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=fLht+H10; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="fLht+H10" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1750869807; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VIsFGLeExHe5sqFNaVeivDuS43dfh1gqeqA6FrJfFrA=; b=fLht+H107SWqw56jGeACCEbzlmmkqfpHJY8o8y6FWm25q9mbw8P4/aq3O7pLiiu3HRnpR2 gLtUQ2nkfak7Qe75Zm9b8+t/9If78fBvCQm5C+QwIDAIdVxLifafmFEDPOJhZjW/cW5/Gm 4bZMcyN3KvlpmLCvbcR6qtiJBYLni88= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-562-FDhUDpVvOaG5pNTOQr4ZMA-1; Wed, 25 Jun 2025 12:43:20 -0400 X-MC-Unique: FDhUDpVvOaG5pNTOQr4ZMA-1 X-Mimecast-MFC-AGG-ID: FDhUDpVvOaG5pNTOQr4ZMA_1750869798 Received: from mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 829C2180028D; Wed, 25 Jun 2025 16:43:18 +0000 (UTC) Received: from warthog.procyon.org.com (unknown [10.42.28.81]) by mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id CECEA195608D; Wed, 25 Jun 2025 16:43:14 +0000 (UTC) From: David Howells To: Christian Brauner , Steve French Cc: David Howells , Paulo Alcantara , netfs@lists.linux.dev, linux-afs@lists.infradead.org, linux-cifs@vger.kernel.org, linux-nfs@vger.kernel.org, ceph-devel@vger.kernel.org, v9fs@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, Stefan Metzmacher , Tom Talpey , stable+noautosel@kernel.org Subject: [PATCH v2 11/16] smb: client: let smbd_post_send_iter() respect the peers max_send_size and transmit all data Date: Wed, 25 Jun 2025 17:42:06 +0100 Message-ID: <20250625164213.1408754-12-dhowells@redhat.com> In-Reply-To: <20250625164213.1408754-1-dhowells@redhat.com> References: <20250625164213.1408754-1-dhowells@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17 Content-Type: text/plain; charset="utf-8" From: Stefan Metzmacher We should not send smbdirect_data_transfer messages larger than the negotiated max_send_size, typically 1364 bytes, which means 24 bytes of the smbdirect_data_transfer header + 1340 payload bytes. This happened when doing an SMB2 write with more than 1340 bytes (which is done inline as it's below rdma_readwrite_threshold). It means the peer resets the connection. When testing between cifs.ko and ksmbd.ko something like this is logged: client: CIFS: VFS: RDMA transport re-established siw: got TERMINATE. layer 1, type 2, code 2 siw: got TERMINATE. layer 1, type 2, code 2 siw: got TERMINATE. layer 1, type 2, code 2 siw: got TERMINATE. layer 1, type 2, code 2 siw: got TERMINATE. layer 1, type 2, code 2 siw: got TERMINATE. layer 1, type 2, code 2 siw: got TERMINATE. layer 1, type 2, code 2 siw: got TERMINATE. layer 1, type 2, code 2 siw: got TERMINATE. layer 1, type 2, code 2 CIFS: VFS: \\carina Send error in SessSetup =3D -11 smb2_reconnect: 12 callbacks suppressed CIFS: VFS: reconnect tcon failed rc =3D -11 CIFS: VFS: reconnect tcon failed rc =3D -11 CIFS: VFS: reconnect tcon failed rc =3D -11 CIFS: VFS: SMB: Zero rsize calculated, using minimum value 65536 and: CIFS: VFS: RDMA transport re-established siw: got TERMINATE. layer 1, type 2, code 2 CIFS: VFS: smbd_recv:1894 disconnected siw: got TERMINATE. layer 1, type 2, code 2 The ksmbd dmesg is showing things like: smb_direct: Recv error. status=3D'local length error (1)' opcode=3D128 smb_direct: disconnected smb_direct: Recv error. status=3D'local length error (1)' opcode=3D128 ksmbd: smb_direct: disconnected ksmbd: sock_read failed: -107 As smbd_post_send_iter() limits the transmitted number of bytes we need loop over it in order to transmit the whole iter. Cc: Steve French Cc: Tom Talpey Cc: linux-cifs@vger.kernel.org Cc: # sp->max_send_size should be info->max_s= end_size in backports Fixes: 3d78fe73fa12 ("cifs: Build the RDMA SGE list directly from an iterat= or") Signed-off-by: Stefan Metzmacher Signed-off-by: David Howells --- fs/smb/client/smbdirect.c | 31 +++++++++++++++++++++++++++---- 1 file changed, 27 insertions(+), 4 deletions(-) diff --git a/fs/smb/client/smbdirect.c b/fs/smb/client/smbdirect.c index cbc85bca006f..a976bcf61226 100644 --- a/fs/smb/client/smbdirect.c +++ b/fs/smb/client/smbdirect.c @@ -907,8 +907,10 @@ static int smbd_post_send_iter(struct smbd_connection = *info, .local_dma_lkey =3D sc->ib.pd->local_dma_lkey, .direction =3D DMA_TO_DEVICE, }; + size_t payload_len =3D umin(*_remaining_data_length, + sp->max_send_size - sizeof(*packet)); =20 - rc =3D smb_extract_iter_to_rdma(iter, *_remaining_data_length, + rc =3D smb_extract_iter_to_rdma(iter, payload_len, &extract); if (rc < 0) goto err_dma; @@ -1013,6 +1015,27 @@ static int smbd_post_send_empty(struct smbd_connecti= on *info) return smbd_post_send_iter(info, NULL, &remaining_data_length); } =20 +static int smbd_post_send_full_iter(struct smbd_connection *info, + struct iov_iter *iter, + int *_remaining_data_length) +{ + int rc =3D 0; + + /* + * smbd_post_send_iter() respects the + * negotiated max_send_size, so we need to + * loop until the full iter is posted + */ + + while (iov_iter_count(iter) > 0) { + rc =3D smbd_post_send_iter(info, iter, _remaining_data_length); + if (rc < 0) + break; + } + + return rc; +} + /* * Post a receive request to the transport * The remote peer can only send data when a receive request is posted @@ -2032,14 +2055,14 @@ int smbd_send(struct TCP_Server_Info *server, klen +=3D rqst->rq_iov[i].iov_len; iov_iter_kvec(&iter, ITER_SOURCE, rqst->rq_iov, rqst->rq_nvec, klen); =20 - rc =3D smbd_post_send_iter(info, &iter, &remaining_data_length); + rc =3D smbd_post_send_full_iter(info, &iter, &remaining_data_length); if (rc < 0) break; =20 if (iov_iter_count(&rqst->rq_iter) > 0) { /* And then the data pages if there are any */ - rc =3D smbd_post_send_iter(info, &rqst->rq_iter, - &remaining_data_length); + rc =3D smbd_post_send_full_iter(info, &rqst->rq_iter, + &remaining_data_length); if (rc < 0) break; }