From nobody Tue Nov 26 11:39:19 2024 Received: from rcdn-iport-2.cisco.com (rcdn-iport-2.cisco.com [173.37.86.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C27CC202651; Fri, 18 Oct 2024 16:23:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=173.37.86.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729268641; cv=none; b=q1upzrswgmcxGzveDeFyGxXxynB5li48ZbDpz8QzqMuEdDp5317Hhc4XVUob9kqshn99zgNvbzmB6gbFDDVP89WEi4S6pZmKnbU3LfhzeUiTRPwik6oox+nz6zIrCkijRqEr+XuETNmyY4drYq/iRL6Bxjm0yABPP8dcwPeFsrM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729268641; c=relaxed/simple; bh=url/mgKgLjh//TfTBQQf3vfQHAxcnqlW6kjE7RyrXV0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=RfZWj/L87t6ajYd43hgAUHhd0gAhzTl9QDqEdLIm4UXI/zGlQENtFVW9wMIT9aG6ovUjSQH3h6U4y9EwiH3j4P7YEluX1X+4AY+GAmdyOTjxEt2fQKWFkF3pCQcZvUAcRsdleD4xbNCYzmdk5W7tJXMG8yGUXDin5uYZoBQ/Kmo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=cisco.com; spf=pass smtp.mailfrom=cisco.com; dkim=pass (1024-bit key) header.d=cisco.com header.i=@cisco.com header.b=LnTO0tcU; arc=none smtp.client-ip=173.37.86.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=cisco.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=cisco.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=cisco.com header.i=@cisco.com header.b="LnTO0tcU" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cisco.com; i=@cisco.com; l=11012; q=dns/txt; s=iport; t=1729268638; x=1730478238; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=coboEB8TpiopDbQlPKVss+wSga4L9ulKqRwxedFYWN4=; b=LnTO0tcU678PlMgmQmEMmPCmTBkbHtpnLEXz/St/miqFCB4cBgIf/PKc Ay71eNUfxEuJHe7tzly0HneLIb1R/c65lYh/RnAZoiBOWcwoh5N7WAYl2 uqZSc61iMiUkQot3elzQYOCV+TDoNeJtuzlPjoUmLIcDnXGss4DNvccK8 U=; X-CSE-ConnectionGUID: Iv0jocwSTzyRpe3HZpKvVg== X-CSE-MsgGUID: WigtPShXRCaX3oYl0wKF/A== X-IPAS-Result: =?us-ascii?q?A0AnAACvihJn/5D/Ja1aHAECAgEHARQBBAQBgXwHAQwBg?= =?us-ascii?q?hsvgU9DGS+McolRnhcUgREDVg8BAQEPRAQBAYUHAoojAiY0CQ4BAgQBAQEBA?= =?us-ascii?q?wIDAQEBAQEBAQEBDQEBBQEBAQIBBwWBDhOGCIZbAgEDJwsBRhBRVhmCKViCZ?= =?us-ascii?q?QOwBoF5M4EB3jOBbIFIAY1FcIR3JxUGgUlEgRWBO4E+b4FSgliGXQSGSjVph?= =?us-ascii?q?xCCfw6CDwEKg0eDUIE9ZhYlgTOICpFASIEhA1khAhEBVRMNCgsJBYk1gRCCF?= =?us-ascii?q?imBa4EIgwiFJYFnCWGIR4EHLYERgR86ggOBNkqFN0c/gk9qTjcCDQI3giSBA?= =?us-ascii?q?IJRhVI2QAMLGA1IESw1FBsGPm4HrFJGglwBexSBeEYBHgEKOpJYB0eRToE0n?= =?us-ascii?q?0qEJKE/GjOqTJh3pDqEZoFnPIFZMxoIGxU7gmdSGQ+OLRbLJCYyOwIHCwEBA?= =?us-ascii?q?wmOIwEB?= IronPort-Data: A9a23:/yM50K7SYvvosVTYD9C5XAxRtCnGchMFZxGqfqrLsTDasY5as4F+v jcXUW+DOPmPZ2f0fdAiPIjn9k4AuJeBy4BqQAY++y4xZn8b8sCt6fZ1gavT04J+CuWZESqLO u1HMoGowPgcFyKa+1H3dOC4/RGQ7InQLpLkEunIJyttcgFtTSYlmHpLlvUw6mJSqYDR7zil5 5Wr+qUzBHf/g2QoazpNu/rewP9SlK2aVA0w7wRWic9j5Dcyp1FNZLoDKKe4KWfPQ4U8NoaSW +bZwbilyXjS9hErB8nNuu6TnpoiG+O60aCm0xK6aoD66vRwjnVaPpUTaJLwXXxqZwChxLid/ jniWauYEm/FNoWU8AgUvoIx/ytWZcWq85efSZSzXFD6I0DuKxPRL/tS4E4eHJNA+OF8Hlpy5 +0iMD1WTiKbpPyIz+fuIgVsrpxLwMjDJogTvDRkiDreF/tjGc+FSKTR7tge1zA17ixMNa+BP IxCNnw1MUmGOkETUrsUIMpWcOOAhXDlbzxcoVG9rqss6G+Vxwt0uFToGICLI4HVHpwOzy50o Er5/mvlOioaBOa0wAre9UOo3vDss2TSDdd6+LqQs6QCbEeo7m4eChc+UVq9vOn/i0S7HdlYL iQ8/yM0sak0slSmUtTnRBC+iHmetxUYVpxbFOhSwAWMzLfEpgWUHG4JShZfZ9E88sw7Xzon0 hmOhdyBLThutqCFDGmW7ba8szy/I24WIHUEaCtCShEKi+QPu6kphR7JC9ImG6mvg5itRnf7w iuBq241gLB7YdM36phXNGvv21qEzqUlhCZujukLdgpJNj9EWbM= IronPort-HdrOrdr: A9a23:h4Vz762UBDb12c/KzgLffwqjBJ0kLtp133Aq2lEZdPWaSKClfq eV7ZAmPHDP5gr5NEtLpTnEAtjifZq+z+8R3WByB9aftWDd0QPCEGgh1/qB/9SKIULDH4BmuJ tIQuxXFMDwAV9mjczz/QW0V+o7zMLvytHOuQ6n9RdQpcUAUdAY0++/YTzrdHFLeA== X-Talos-CUID: 9a23:8V7kCm/mkm0sUq4j0liVv28KQsIieWLm8GXBGVe1UkZbUJ6Ld1DFrQ== X-Talos-MUID: 9a23:7BC+rAo4auZk9RuFAtsezw5jO9tQ2KSeMmsEoc4Bis3cKg9efB7I2Q== X-IronPort-Anti-Spam-Filtered: true X-IronPort-AV: E=Sophos;i="6.11,214,1725321600"; d="scan'208";a="261573139" Received: from rcdn-l-core-07.cisco.com ([173.37.255.144]) by rcdn-iport-2.cisco.com with ESMTP/TLS/TLS_AES_256_GCM_SHA384; 18 Oct 2024 16:23:57 +0000 Received: from fedora.cisco.com (unknown [10.24.40.136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: kartilak@cisco.com) by rcdn-l-core-07.cisco.com (Postfix) with ESMTPSA id 8157418000229; Fri, 18 Oct 2024 16:23:55 +0000 (GMT) From: Karan Tilak Kumar To: sebaddel@cisco.com Cc: arulponn@cisco.com, djhawar@cisco.com, gcboffa@cisco.com, mkai2@cisco.com, satishkh@cisco.com, aeasi@cisco.com, jejb@linux.ibm.com, martin.petersen@oracle.com, linux-scsi@vger.kernel.org, linux-kernel@vger.kernel.org, Karan Tilak Kumar Subject: [PATCH v5 13/14] scsi: fnic: Add support to handle port channel RSCN Date: Fri, 18 Oct 2024 09:14:08 -0700 Message-ID: <20241018161409.4442-14-kartilak@cisco.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241018161409.4442-1-kartilak@cisco.com> References: <20241018161409.4442-1-kartilak@cisco.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Authenticated-User: kartilak@cisco.com X-Outbound-SMTP-Client: 10.24.40.136, [10.24.40.136] X-Outbound-Node: rcdn-l-core-07.cisco.com Content-Type: text/plain; charset="utf-8" Add support to handle port channel RSCN. Port channel RSCN is a Cisco vendor specific RSCN event. It is applicable only to Cisco UCS fabrics. If there's a change in the port channel configuration, an RCSN is sent to fnic. This is used to serially reset the scsi initiator fnics so that there's no all paths down scenario. The affected fnics are added to a list that are reset with a small time gap between them. Reviewed-by: Sesidhar Baddela Reviewed-by: Arulprabhu Ponnusamy Reviewed-by: Gian Carlo Boffa Signed-off-by: Karan Tilak Kumar --- Changes between v2 and v3: Incorporate review comments from Hannes from other patches: Replace redundant definitions with standard definitions. Changes between v1 and v2: Replace pr_err with printk. Incorporate review comments from Hannes from other patches: Replace pr_err with dev_err. --- drivers/scsi/fnic/fdls_disc.c | 62 ++++++++++++++++++++++++++++------- drivers/scsi/fnic/fnic.h | 25 ++++++++++++++ drivers/scsi/fnic/fnic_fcs.c | 36 ++++++++++++++++++++ drivers/scsi/fnic/fnic_main.c | 30 +++++++++++++++++ 4 files changed, 141 insertions(+), 12 deletions(-) diff --git a/drivers/scsi/fnic/fdls_disc.c b/drivers/scsi/fnic/fdls_disc.c index 1d0b957c343b..dfa9406411a0 100644 --- a/drivers/scsi/fnic/fdls_disc.c +++ b/drivers/scsi/fnic/fdls_disc.c @@ -3905,6 +3905,9 @@ fdls_process_rscn(struct fnic_iport_s *iport, struct = fc_frame_header *fchdr) int newports =3D 0; struct fnic_fdls_fabric_s *fdls =3D &iport->fabric; struct fnic *fnic =3D iport->fnic; + int rscn_type =3D NOT_PC_RSCN; + uint32_t sid =3D ntoh24(fchdr->fh_s_id); + unsigned long reset_fnic_list_lock_flags =3D 0; uint16_t rscn_payload_len; =20 atomic64_inc(&iport->iport_stats.num_rscns); @@ -3927,16 +3930,24 @@ fdls_process_rscn(struct fnic_iport_s *iport, struc= t fc_frame_header *fchdr) || (rscn_payload_len > 1024) || (rscn->els.rscn_page_len !=3D 4)) { num_ports =3D 0; - FNIC_FCS_DBG(KERN_INFO, fnic->host, fnic->fnic_num, - "RSCN payload_len: 0x%x page_len: 0x%x", - rscn_payload_len, rscn->els.rscn_page_len); - /* if this happens then we need to send ADISC to all the tports. */ - list_for_each_entry_safe(tport, next, &iport->tport_list, links) { - if (tport->state =3D=3D FDLS_TGT_STATE_READY) - tport->flags |=3D FNIC_FDLS_TPORT_SEND_ADISC; + if ((rscn_payload_len =3D=3D 0xFFFF) + && (sid =3D=3D FC_FABRIC_CONTROLLER)) { + rscn_type =3D PC_RSCN; FNIC_FCS_DBG(KERN_INFO, fnic->host, fnic->fnic_num, - "RSCN for port id: 0x%x", tport->fcid); - } + "pcrscn: PCRSCN received. sid: 0x%x payload len: 0x%x", + sid, rscn_payload_len); + } else { + FNIC_FCS_DBG(KERN_INFO, fnic->host, fnic->fnic_num, + "RSCN payload_len: 0x%x page_len: 0x%x", + rscn_payload_len, rscn->els.rscn_page_len); + /* if this happens then we need to send ADISC to all the tports. */ + list_for_each_entry_safe(tport, next, &iport->tport_list, links) { + if (tport->state =3D=3D FDLS_TGT_STATE_READY) + tport->flags |=3D FNIC_FDLS_TPORT_SEND_ADISC; + FNIC_FCS_DBG(KERN_INFO, fnic->host, fnic->fnic_num, + "RSCN for port id: 0x%x", tport->fcid); + } + } /* end else */ } else { num_ports =3D (rscn_payload_len - 4) / rscn->els.rscn_page_len; rscn_port =3D (struct fc_els_rscn_page *)(rscn + 1); @@ -3983,10 +3994,37 @@ fdls_process_rscn(struct fnic_iport_s *iport, struc= t fc_frame_header *fchdr) tport->flags |=3D FNIC_FDLS_TPORT_SEND_ADISC; } =20 - FNIC_FCS_DBG(KERN_INFO, fnic->host, fnic->fnic_num, + if (pc_rscn_handling_feature_flag =3D=3D PC_RSCN_HANDLING_FEATURE_ON && + rscn_type =3D=3D PC_RSCN && fnic->role =3D=3D FNIC_ROLE_FCP_INITIATOR) { + + if (fnic->pc_rscn_handling_status =3D=3D PC_RSCN_HANDLING_IN_PROGRESS) { + FNIC_FCS_DBG(KERN_INFO, fnic->host, fnic->fnic_num, + "PCRSCN handling already in progress. Skip host reset: %d", + iport->fnic->fnic_num); + return; + } + + FNIC_FCS_DBG(KERN_INFO, fnic->host, fnic->fnic_num, + "Processing PCRSCN. Queuing fnic for host reset: %d", + iport->fnic->fnic_num); + fnic->pc_rscn_handling_status =3D PC_RSCN_HANDLING_IN_PROGRESS; + + spin_unlock_irqrestore(&fnic->fnic_lock, fnic->lock_flags); + + spin_lock_irqsave(&reset_fnic_list_lock, + reset_fnic_list_lock_flags); + list_add_tail(&fnic->links, &reset_fnic_list); + spin_unlock_irqrestore(&reset_fnic_list_lock, + reset_fnic_list_lock_flags); + + queue_work(reset_fnic_work_queue, &reset_fnic_work); + spin_lock_irqsave(&fnic->fnic_lock, fnic->lock_flags); + } else { + FNIC_FCS_DBG(KERN_INFO, fnic->host, fnic->fnic_num, "FDLS process RSCN sending GPN_FT: newports: %d", newports); - fdls_send_gpn_ft(iport, FDLS_STATE_RSCN_GPN_FT); - fdls_send_rscn_resp(iport, fchdr); + fdls_send_gpn_ft(iport, FDLS_STATE_RSCN_GPN_FT); + fdls_send_rscn_resp(iport, fchdr); + } } =20 void fnic_fdls_disc_start(struct fnic_iport_s *iport) diff --git a/drivers/scsi/fnic/fnic.h b/drivers/scsi/fnic/fnic.h index 5ad9d6df92a8..5fd506fcea35 100644 --- a/drivers/scsi/fnic/fnic.h +++ b/drivers/scsi/fnic/fnic.h @@ -211,11 +211,33 @@ enum reset_states { RESET_ERROR }; =20 +enum rscn_type { + NOT_PC_RSCN =3D 0, + PC_RSCN +}; + +enum pc_rscn_handling_status { + PC_RSCN_HANDLING_NOT_IN_PROGRESS =3D 0, + PC_RSCN_HANDLING_IN_PROGRESS +}; + +enum pc_rscn_handling_feature { + PC_RSCN_HANDLING_FEATURE_OFF =3D 0, + PC_RSCN_HANDLING_FEATURE_ON +}; + extern unsigned int fnic_fdmi_support; extern unsigned int fnic_log_level; extern unsigned int io_completions; extern struct workqueue_struct *fnic_event_queue; =20 +extern unsigned int pc_rscn_handling_feature_flag; +extern spinlock_t reset_fnic_list_lock; +extern struct list_head reset_fnic_list; +extern struct workqueue_struct *reset_fnic_work_queue; +extern struct work_struct reset_fnic_work; + + #define FNIC_MAIN_LOGGING 0x01 #define FNIC_FCS_LOGGING 0x02 #define FNIC_SCSI_LOGGING 0x04 @@ -396,6 +418,7 @@ struct fnic { int link_status; =20 struct list_head list; + struct list_head links; struct pci_dev *pdev; struct vnic_fc_config config; struct vnic_dev *vdev; @@ -422,6 +445,7 @@ struct fnic { =20 char subsys_desc[14]; int subsys_desc_len; + int pc_rscn_handling_status; =20 /*** FIP related data members -- start ***/ void (*set_vlan)(struct fnic *, u16 vlan); @@ -503,6 +527,7 @@ void fnic_stats_debugfs_remove(struct fnic *fnic); int fnic_is_abts_pending(struct fnic *, struct scsi_cmnd *); =20 void fnic_handle_fip_frame(struct work_struct *work); +void fnic_reset_work_handler(struct work_struct *work); void fnic_handle_fip_event(struct fnic *fnic); void fnic_fcoe_reset_vlans(struct fnic *fnic); extern void fnic_handle_fip_timer(struct timer_list *t); diff --git a/drivers/scsi/fnic/fnic_fcs.c b/drivers/scsi/fnic/fnic_fcs.c index 6b8edd368fe9..62f686e8a764 100644 --- a/drivers/scsi/fnic/fnic_fcs.c +++ b/drivers/scsi/fnic/fnic_fcs.c @@ -1108,3 +1108,39 @@ void fnic_flush_tport_event_list(struct fnic *fnic) } spin_unlock_irqrestore(&fnic->fnic_lock, flags); } + +void fnic_reset_work_handler(struct work_struct *work) +{ + struct fnic *cur_fnic, *next_fnic; + unsigned long reset_fnic_list_lock_flags; + int host_reset_ret_code; + + /* + * This is a single thread. It is per fnic module, not per fnic + * All the fnics that need to be reset + * have been serialized via the reset fnic list. + */ + spin_lock_irqsave(&reset_fnic_list_lock, reset_fnic_list_lock_flags); + list_for_each_entry_safe(cur_fnic, next_fnic, &reset_fnic_list, links) { + list_del(&cur_fnic->links); + spin_unlock_irqrestore(&reset_fnic_list_lock, + reset_fnic_list_lock_flags); + + dev_err(&cur_fnic->pdev->dev, "fnic: <%d>: issuing a host reset\n", + cur_fnic->fnic_num); + host_reset_ret_code =3D fnic_host_reset(cur_fnic->host); + dev_err(&cur_fnic->pdev->dev, + "fnic: <%d>: returned from host reset with status: %d\n", + cur_fnic->fnic_num, host_reset_ret_code); + + spin_lock_irqsave(&cur_fnic->fnic_lock, cur_fnic->lock_flags); + cur_fnic->pc_rscn_handling_status =3D + PC_RSCN_HANDLING_NOT_IN_PROGRESS; + spin_unlock_irqrestore(&cur_fnic->fnic_lock, cur_fnic->lock_flags); + + spin_lock_irqsave(&reset_fnic_list_lock, + reset_fnic_list_lock_flags); + } + spin_unlock_irqrestore(&reset_fnic_list_lock, + reset_fnic_list_lock_flags); +} diff --git a/drivers/scsi/fnic/fnic_main.c b/drivers/scsi/fnic/fnic_main.c index 6c4a06e1635d..af5588288882 100644 --- a/drivers/scsi/fnic/fnic_main.c +++ b/drivers/scsi/fnic/fnic_main.c @@ -44,6 +44,10 @@ static LIST_HEAD(fnic_list); static DEFINE_SPINLOCK(fnic_list_lock); static DEFINE_IDA(fnic_ida); =20 +struct work_struct reset_fnic_work; +LIST_HEAD(reset_fnic_list); +DEFINE_SPINLOCK(reset_fnic_list_lock); + /* Supported devices by fnic module */ static struct pci_device_id fnic_id_table[] =3D { { PCI_DEVICE(PCI_VENDOR_ID_CISCO, PCI_DEVICE_ID_CISCO_FNIC) }, @@ -88,6 +92,12 @@ static unsigned int fnic_max_qdepth =3D FNIC_DFLT_QUEUE_= DEPTH; module_param(fnic_max_qdepth, uint, S_IRUGO|S_IWUSR); MODULE_PARM_DESC(fnic_max_qdepth, "Queue depth to report for each LUN"); =20 +unsigned int pc_rscn_handling_feature_flag =3D PC_RSCN_HANDLING_FEATURE_ON; +module_param(pc_rscn_handling_feature_flag, uint, 0644); +MODULE_PARM_DESC(pc_rscn_handling_feature_flag, + "PCRSCN handling (0 for none. 1 to handle PCRSCN (default))"); + +struct workqueue_struct *reset_fnic_work_queue; struct workqueue_struct *fnic_fip_queue; =20 static int fnic_slave_alloc(struct scsi_device *sdev) @@ -1223,6 +1233,19 @@ static int __init fnic_init_module(void) goto err_create_fip_workq; } =20 + if (pc_rscn_handling_feature_flag =3D=3D PC_RSCN_HANDLING_FEATURE_ON) { + reset_fnic_work_queue =3D + create_singlethread_workqueue("reset_fnic_work_queue"); + if (!reset_fnic_work_queue) { + pr_err("reset fnic work queue create failed\n"); + err =3D -ENOMEM; + goto err_create_reset_fnic_workq; + } + spin_lock_init(&reset_fnic_list_lock); + INIT_LIST_HEAD(&reset_fnic_list); + INIT_WORK(&reset_fnic_work, fnic_reset_work_handler); + } + fnic_fc_transport =3D fc_attach_transport(&fnic_fc_functions); if (!fnic_fc_transport) { printk(KERN_ERR PFX "fc_attach_transport error\n"); @@ -1243,6 +1266,9 @@ static int __init fnic_init_module(void) err_fc_transport: destroy_workqueue(fnic_fip_queue); err_create_fip_workq: + if (pc_rscn_handling_feature_flag =3D=3D PC_RSCN_HANDLING_FEATURE_ON) + destroy_workqueue(reset_fnic_work_queue); +err_create_reset_fnic_workq: destroy_workqueue(fnic_event_queue); err_create_fnic_workq: kmem_cache_destroy(fnic_io_req_cache); @@ -1261,6 +1287,10 @@ static void __exit fnic_cleanup_module(void) { pci_unregister_driver(&fnic_driver); destroy_workqueue(fnic_event_queue); + + if (pc_rscn_handling_feature_flag =3D=3D PC_RSCN_HANDLING_FEATURE_ON) + destroy_workqueue(reset_fnic_work_queue); + if (fnic_fip_queue) { flush_workqueue(fnic_fip_queue); destroy_workqueue(fnic_fip_queue); --=20 2.31.1