From nobody Thu Nov 6 08:14:30 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=oracle.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1540243962162429.8104794953249; Mon, 22 Oct 2018 14:32:42 -0700 (PDT) Received: from localhost ([::1]:37165 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gEhoS-0004x9-HY for importer@patchew.org; Mon, 22 Oct 2018 17:32:36 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:54116) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gEhms-0004Mv-1T for qemu-devel@nongnu.org; Mon, 22 Oct 2018 17:30:59 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gEhmo-0002PU-NZ for qemu-devel@nongnu.org; Mon, 22 Oct 2018 17:30:58 -0400 Received: from userp2120.oracle.com ([156.151.31.85]:33484) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gEhmo-0002Lv-Bg for qemu-devel@nongnu.org; Mon, 22 Oct 2018 17:30:54 -0400 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.22/8.16.0.22) with SMTP id w9MLSfUp058835; Mon, 22 Oct 2018 21:30:51 GMT Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by userp2120.oracle.com with ESMTP id 2n7w0qh9nh-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 22 Oct 2018 21:30:51 +0000 Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id w9MLUjqH005216 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 22 Oct 2018 21:30:45 GMT Received: from abhmp0003.oracle.com (abhmp0003.oracle.com [141.146.116.9]) by aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id w9MLUje7014290; Mon, 22 Oct 2018 21:30:45 GMT Received: from dhcp-10-152-34-187.usdhcp.oraclecorp.com.com (/10.152.34.187) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 22 Oct 2018 14:30:45 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : subject : date : message-id; s=corp-2018-07-02; bh=As5OSlYICpWsIEJfbQ39gEbklrdDqGhJE/whkZZ6gAE=; b=PNGmRCLLnmfO/ZqjCg4uSzMAOvYv/xiNwE3gIPF6vqTsKGBHyupuFQQznVwSXOmvBkLr wn3UUBS2WARu0ZHJriS7fAPzfe+u6bWT3iohH49MMrB7+5S6aySoC2T2opiTlwX8rTMb r55tckQEp95POSvQDnXFIObyf+JsKfPvH3qcSaM38bEOP0vNzOAyyglVpmznvS1qYUfY y0cFnohys92xqWfpboHvMRKlSazCfI8r0fxwaHJj/KmzB//c65Vy96g6cORZf9a4bqLT 8TCaAg8XleeXKNXUNB6pmduhJKOUwvSdEP5tUcDPHw2/Z73peSUl6Dk/W3B3ID1aBk7T zA== From: George Kennedy To: pbonzini@redhat.com, famz@redhat.com, qemu-devel@nongnu.org, george.kennedy@oracle.com Date: Mon, 22 Oct 2018 17:28:24 -0400 Message-Id: <1540243704-17090-1-git-send-email-george.kennedy@oracle.com> X-Mailer: git-send-email 1.8.3.1 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9054 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=11 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1807170000 definitions=main-1810220180 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] X-Received-From: 156.151.31.85 Subject: [Qemu-devel] [PATCH v2] lsi: Reselection needed to remove pending commands from queue X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @oracle.com) X-ZohoMail: RDMRC_0 RDKM_0 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Under heavy IO (e.g. fio) the queue is not checked frequently enough for pending commands. As a result some pending commands are timed out by the linux sym53c8xx driver, which sends SCSI Abort messages for the timed out commands. The SCSI Abort messages result in linux errors, which show up in /var/log/messages. e.g. sd 0:0:3:0: [sdd] tag#33 ABORT operation started scsi target0:0:3: control msgout: 80 20 47 d sd 0:0:3:0: ABORT operation complete. scsi target0:0:4: message d sent on bad reselection Add a deadline along with the command when it is added to the queue. When the current command completes, check the queue for pending commands that have exceeded the deadline and if so, simulate a Wait Reselect to handle the pending commands on the queue. When a Wait Reselect is needed, intercept and save the current DMA Scripts Ptr (DSP) contents and load it instead with the pointer to the Reselection Scripts. When Reselection has completed, restore the original DSP contents. Signed-off-by: George Kennedy --- Thank you for reviewing, Paolo, As you suggested I moved the loading of "s->resel_dsp" down to the "Wait Re= select" case. The address of the Reselection Scripts, though, is contained in "s->d= sp - 8" and not in s->dnad. The reason the timeout is needed is that under heavy IO some pending comman= ds stay on the pending queue longer than the 30 second command timeout set by = the linux upper layer scsi driver (sym53c8xx). When command timeouts occur, the upper layer scsi driver sends SCSI Abort messages to remove the timed out commands. The command timeouts are caused by the fact that under heavy IO, lsi_reselect() in qemu "hw/scsi/lsi53c895a.c" is not being called before the upper layer scsi driver 30 second command timeout goes off. If lsi_reselect() were called more frequently, the command timeout problem = would probably not occur. There are a number of places where lsi_reselect() is su= pposed to get called (e.g. at the end of lsi_update_irq()), but the only place tha= t I have observed lsi_reselect() being called is from lsi_execute_script() when lsi_wait_reselect() is called because of a SCRIPT "Wait Select" IO Instruct= ion. The proposed patch adds a deadline timeout for each pending command added t= o the pending queue. The timeout is an arbitrary value (less than the upper layer command timeout) that gets checked after each command is completed when the pending queue is checked. If the deadline is exceeded, a flag is set indica= ting that a SCRIPT "Wait Select" IO Instruction is needed, which will result in lsi_wait_reselect() and lsi_reselect() being called to remove a command fro= m=20 the pending queue, reselect the target, continue and complete the command. hw/scsi/lsi53c895a.c | 54 ++++++++++++++++++++++++++++++++++++++++++++++++= +++- 1 file changed, 53 insertions(+), 1 deletion(-) diff --git a/hw/scsi/lsi53c895a.c b/hw/scsi/lsi53c895a.c index 996b406..8474399 100644 --- a/hw/scsi/lsi53c895a.c +++ b/hw/scsi/lsi53c895a.c @@ -198,6 +198,7 @@ typedef struct lsi_request { uint32_t dma_len; uint8_t *dma_buf; uint32_t pending; + uint64_t deadline; int out; QTAILQ_ENTRY(lsi_request) next; } lsi_request; @@ -232,6 +233,9 @@ typedef struct { int command_complete; QTAILQ_HEAD(, lsi_request) queue; lsi_request *current; + int want_resel; /* need resel to handle queued completed cmds */ + uint32_t resel_dsp; /* DMA Scripts Ptr (DSP) of reselection scsi scrip= ts */ + uint32_t next_dsp; /* if want_resel, will be loaded with above */ =20 uint32_t dsa; uint32_t temp; @@ -311,6 +315,20 @@ static inline int lsi_irq_on_rsl(LSIState *s) return (s->sien0 & LSI_SIST0_RSL) && (s->scid & LSI_SCID_RRE); } =20 +static int pending_past_deadline(LSIState *s) +{ + lsi_request *p; + + QTAILQ_FOREACH(p, &s->queue, next) { + if (p->pending) { + if (qemu_clock_get_ns(QEMU_CLOCK_REALTIME) > p->deadline) { + return 1; + } + } + } + return 0; +} + static void lsi_soft_reset(LSIState *s) { DPRINTF("Reset\n"); @@ -634,15 +652,22 @@ static void lsi_do_dma(LSIState *s, int out) } } =20 +/* Max time a completed command can be on the queue before Reselection nee= ded */ +#define LSI_DEADLINE 1000 =20 /* Add a command to the queue. */ static void lsi_queue_command(LSIState *s) { lsi_request *p =3D s->current; + uint64_t timeout_ms =3D LSI_DEADLINE; =20 DPRINTF("Queueing tag=3D0x%x\n", p->tag); assert(s->current !=3D NULL); assert(s->current->dma_len =3D=3D 0); + + p->deadline =3D qemu_clock_get_ns(QEMU_CLOCK_REALTIME) + + timeout_ms * 1000000ULL; + QTAILQ_INSERT_TAIL(&s->queue, s->current, next); s->current =3D NULL; =20 @@ -775,6 +800,9 @@ static void lsi_command_complete(SCSIRequest *req, uint= 32_t status, size_t resid lsi_request_free(s, s->current); scsi_req_unref(req); } + if (pending_past_deadline(s)) { + s->want_resel =3D 1; + } lsi_resume_script(s); } =20 @@ -987,7 +1015,7 @@ static void lsi_do_msgout(LSIState *s) s->select_tag |=3D lsi_get_msgbyte(s) | LSI_TAG_VALID; break; case 0x22: /* ORDERED queue */ - BADF("ORDERED queue not implemented\n"); + DPRINTF("ORDERED queue not implemented\n"); s->select_tag |=3D lsi_get_msgbyte(s) | LSI_TAG_VALID; break; case 0x0d: @@ -1078,6 +1106,9 @@ static void lsi_wait_reselect(LSIState *s) =20 DPRINTF("Wait Reselect\n"); =20 + if (s->current) + return; + QTAILQ_FOREACH(p, &s->queue, next) { if (p->pending) { lsi_reselect(s, p); @@ -1089,6 +1120,8 @@ static void lsi_wait_reselect(LSIState *s) } } =20 +#define SCRIPTS_LOAD_AND_STORE 0xe2340004 + static void lsi_execute_script(LSIState *s) { PCIDevice *pci_dev =3D PCI_DEVICE(s); @@ -1096,10 +1129,16 @@ static void lsi_execute_script(LSIState *s) uint32_t addr, addr_high; int opcode; int insn_processed =3D 0; + uint32_t save_dsp =3D 0; =20 s->istat1 |=3D LSI_ISTAT1_SRUN; again: insn_processed++; + if (s->next_dsp) { + save_dsp =3D s->dsp; + s->dsp =3D s->next_dsp; + DPRINTF("lsi_execute_script: setting up for wait_reselection...\n"= ); + } insn =3D read_dword(s, s->dsp); if (!insn) { /* If we receive an empty opcode increment the DSP by 4 bytes @@ -1107,6 +1146,12 @@ again: s->dsp +=3D 4; goto again; } + if (s->want_resel && s->resel_dsp && (insn =3D=3D SCRIPTS_LOAD_AND_STO= RE)) { + /* Reselection follows Load and Store */ + DPRINTF("lsi_execute_script: detects want_resel...\n"); + s->next_dsp =3D s->resel_dsp; + s->want_resel =3D 0; + } addr =3D read_dword(s, s->dsp + 4); addr_high =3D 0; DPRINTF("SCRIPTS dsp=3D%08x opcode %08x arg %08x\n", s->dsp, insn, add= r); @@ -1273,7 +1318,14 @@ again: s->scntl1 &=3D ~LSI_SCNTL1_CON; break; case 2: /* Wait Reselect */ + if (!s->resel_dsp) { + s->resel_dsp =3D s->dsp - 8; + } if (!lsi_irq_on_rsl(s)) { + if (save_dsp) { + s->dsp =3D save_dsp; + save_dsp =3D s->next_dsp =3D 0; + } lsi_wait_reselect(s); } break; --=20 1.8.3.1