From nobody Mon Sep 16 19:24:25 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1721470502; cv=none; d=zohomail.com; s=zohoarc; b=PCeOHIK+rT4kklDzKSz7CJ05l11LK++7PdK/NMhsT1bj3d0Aj9kPVI8KwtG5sB2Uvn51bL4zYAVtJBNx3XPeSoWDgY0v2rJNYfrTRNUXkrm6rQ32WoPRV6zTtaasORMPi1BfEFDaY2mDTwZR2ZPtjNYutCkj6OQO7YVYSHxkzlE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1721470502; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=PbJVDnv52Tg0eAb6ccUpzqtMp8g5kcnSvB8cItuXb20=; b=bruJXqxXfvxmzGz7mzrNgqTBuQgP15FePYtE8pNV/0N3MIvrqEAdzTIRg5DHO7W+hLVzJY8gd2U7dG/6sasCJZZ6yHesD01LmK9KOQCsifBl8Hdxym17Fsy9OcuFi7fTpjY+zp18bynNcrayBJivXzDMp3paJ7mUUhWbWX7p014= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1721470502968129.81195950067547; Sat, 20 Jul 2024 03:15:02 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sV769-0001QL-NU; Sat, 20 Jul 2024 06:13:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sV767-0001I0-Dc; Sat, 20 Jul 2024 06:13:51 -0400 Received: from mail-pj1-x1032.google.com ([2607:f8b0:4864:20::1032]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sV765-0007QX-W0; Sat, 20 Jul 2024 06:13:51 -0400 Received: by mail-pj1-x1032.google.com with SMTP id 98e67ed59e1d1-2cb5787b4a5so1503034a91.2; Sat, 20 Jul 2024 03:13:49 -0700 (PDT) Received: from amjad-pc.. ([202.185.212.207]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2cb76a5fc6csm4411208a91.0.2024.07.20.03.13.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 20 Jul 2024 03:13:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1721470427; x=1722075227; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=PbJVDnv52Tg0eAb6ccUpzqtMp8g5kcnSvB8cItuXb20=; b=iPSvMHGlhmEZ+sGqxLNt8UNzi7CVh2F5wdTKyeU9N25HIod4Etaygpyr4cOYuvtrhX wcTnYOhrwITVYabY4my+XgQLJQ4uVZIIOHtz3bpk5yyLxc4xb+mZTciQytSG1Kow/va/ CF7UkU962Lg8s4Lr4ev6eCxHwp8EUbUVbCTpxS15I/5qcdhzBV9bikKZscIUPbnSHNB8 jGg0Hlm1Kvf5p5v4CR4qcL7Hz2mtEnwJ5tXnJwhwlhqICSGqcDkfw/eZJ+S0QBgEmAce QBwmK6La+GeMuAK9GuvNZEWfscqaFlMMRvAES3NKrHMdFGzEZIdClSlXqKVjikYMdsmX xPQg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721470427; x=1722075227; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=PbJVDnv52Tg0eAb6ccUpzqtMp8g5kcnSvB8cItuXb20=; b=o4DhIIQcTD4q8stryYEk9Cr/Arv7uZzuThou0Kzfa+RguJ0yGqrwNKl+47QET7EGMM sHjq7zj2UwjA2eg5XywL8HqqDxoRzwrCPFiHYVwdqavkJOexegLasUsanOaPhjnteLle 3FeXohYrCcmaABhR4CQF+3yjn7UMyZXuYQLwaj2/aQBFwBOHMO8Qplrf3DnwA/QJ6yAJ dRd54cdIdb4vHflSNCGnJkaUvbF3lbqV04PFR1hA93TIBcSUt4//Laemp8uxEczth8ms b6fjjIwsQdf/rtro/YdVzQSTHxj42mwdo7hKKccj88oQ45WcAS9LWkyk3lPM5xIrm7kA PxvQ== X-Forwarded-Encrypted: i=1; AJvYcCXQ3cuDnFJpboPdKcR6bunXPQPDfnnaBwdMVzxGpdQtD6yrUBGW3Okl3MhIminRTo6Ype90P4hXwjQDVjgUXgYD9gjM7s0= X-Gm-Message-State: AOJu0YzyVM8AIXGqUIokt3OG1uURU4JcWRdveINnKg2XrQBMyr7vhsqG uFh1sOY0MdKywglWu0t4L62QH0n81qlaZ6Op3ocRWanGhqi7mwaTI/Qs0JQZDw+RGA== X-Google-Smtp-Source: AGHT+IH4aljVy5lBC3eS5I4gqELDgeo/tMtW2Izr+VoaYkFAXvCfM90oFf1gBVm9cQttTj9p89ESFw== X-Received: by 2002:a17:90b:1057:b0:2c9:65f5:5f61 with SMTP id 98e67ed59e1d1-2cd273d7a0emr468016a91.9.1721470426382; Sat, 20 Jul 2024 03:13:46 -0700 (PDT) From: Amjad Alsharafi To: qemu-devel@nongnu.org Cc: Hanna Reitz , Kevin Wolf , "open list:vvfat" , Amjad Alsharafi Subject: [PATCH v6 1/5] vvfat: Fix bug in writing to middle of file Date: Sat, 20 Jul 2024 18:13:30 +0800 Message-ID: X-Mailer: git-send-email 2.45.2 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::1032; envelope-from=amjadsharafi10@gmail.com; helo=mail-pj1-x1032.google.com X-Spam_score_int: -17 X-Spam_score: -1.8 X-Spam_bar: - X-Spam_report: (-1.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1721470503783116600 Content-Type: text/plain; charset="utf-8" Before this commit, the behavior when calling `commit_one_file` for example with `offset=3D0x2000` (second cluster), what will happen is that we won't fetch the next cluster from the fat, and instead use the first cluster for the read operation. This is due to off-by-one error here, where `i=3D0x2000 !< offset=3D0x2000`, thus not fetching the next cluster. Signed-off-by: Amjad Alsharafi Reviewed-by: Kevin Wolf Tested-by: Kevin Wolf --- block/vvfat.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/block/vvfat.c b/block/vvfat.c index 9d050ba3ae..19da009a5b 100644 --- a/block/vvfat.c +++ b/block/vvfat.c @@ -2525,8 +2525,9 @@ commit_one_file(BDRVVVFATState* s, int dir_index, uin= t32_t offset) return -1; } =20 - for (i =3D s->cluster_size; i < offset; i +=3D s->cluster_size) + for (i =3D 0; i < offset; i +=3D s->cluster_size) { c =3D modified_fat_get(s, c); + } =20 fd =3D qemu_open_old(mapping->path, O_RDWR | O_CREAT | O_BINARY, 0666); if (fd < 0) { --=20 2.45.2 From nobody Mon Sep 16 19:24:25 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1721470502; cv=none; d=zohomail.com; s=zohoarc; b=CK/JLGCcwpWHFlbukEJj5An/8wKtmpfT8vata09LbIa7ICgMtKjlDlyWpLuAmQWawDyUrYgt24LMBvua/RjojupGQlh+93iR3uTU/arkRwSefzaIqwSppUCDsVBwhxji2t03yirj5e7Pz0eVaLQ0hp0j4V1OLHOHmmFYOEwpsB0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1721470502; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=Z0TcqHDGmq+zPmeeTD6pafWi3xRQRQCgCCjNIOd26LE=; b=TIKCivuKLGhP+HgezXmcXy6MJFSKVYvAuiHV7R7WBE7dDcTTaUN0KZxA/oWXaP6B7DQHFtbr/jmcs7FVwhVlX7DwRzRmfU5nMXOGY6tcK3DhMuQypX97kltJlZco6qOzdVQF1SxK1uN/WIRcCzeZXQ+KvbfRW2uAPxwzv03QaxM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1721470502833474.9892914307518; Sat, 20 Jul 2024 03:15:02 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sV76A-0001Sg-A5; Sat, 20 Jul 2024 06:13:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sV768-0001Ld-Ip; Sat, 20 Jul 2024 06:13:52 -0400 Received: from mail-oo1-xc2a.google.com ([2607:f8b0:4864:20::c2a]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sV766-0007RH-Vt; Sat, 20 Jul 2024 06:13:52 -0400 Received: by mail-oo1-xc2a.google.com with SMTP id 006d021491bc7-5c6924f2383so1274980eaf.2; Sat, 20 Jul 2024 03:13:50 -0700 (PDT) Received: from amjad-pc.. ([202.185.212.207]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2cb76a5fc6csm4411208a91.0.2024.07.20.03.13.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 20 Jul 2024 03:13:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1721470429; x=1722075229; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Z0TcqHDGmq+zPmeeTD6pafWi3xRQRQCgCCjNIOd26LE=; b=hSsbivDy3fMpaiJsC9lWG/ovuShdG6cwIx6sNTnRhO3ECV5xT3Uuy/3S9j8SR3slRu HLCPPlV36MPEmcibVIOF03b+Ya5yvjtuRlXVC1wMDSA4GUwU3+7KywfxLDRa/2H2UqQ7 6xO1wL7/BNbkh4hqFEkXtBQTLUJz6E2+2X16T5MI0m6RUkYbwjOmaT9kR/5Vh6d0Qg1G m+UnjX36f8NTSucQeY4wtfNeDhOCJU6skH7ZYyqJDEBm4SAzpPi7oUt6aGKLpXDkOAr6 ntg2cb79MKkvSIc2rKHh5Oe/tRsHDZrqXYyF/UbN3jZHPsJ1iQLkuQwQQZ2Ip5e+omId Ca1g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721470429; x=1722075229; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Z0TcqHDGmq+zPmeeTD6pafWi3xRQRQCgCCjNIOd26LE=; b=ZNDzBY9gxwIpMcNjMk0bFYeOueBeVS7osIAXd42lbrGjl762TvDAhoT69WUU3IQrHj 2NRvGqovYEQttzmXfnzQLXlP/4Y/FfubaHY7Ab3IrgK0pjUDg5lTdIUa+xSbnifXEpwv chp5a88fySY3fzqyZoqFdoLqLXGzdCCl0mj/3blYG/7pYRYwh7oIccTwcAMSvseeytsI Sz5PI6RHueBTW0LW3QTsIXjk83XIo3PJrU2aDinmxkrRvwmxqGBwBxma3kmZwQLmDDhx gfMy75Tf1vUaZ0oUxkgZYlDApJMoEUwA6KlQGc+8Z8e9s25AmznZ0QXEZgsGf2VFguiO yA+A== X-Forwarded-Encrypted: i=1; AJvYcCXx8Sf2ObEMosuWlxlgH/Er1VRkbA0H1v0peMsquBCQhUtKj9xOGdu9lWo4ZOdirjiKZ1TMsXcFhB19UTIisZSHyEPW9TM= X-Gm-Message-State: AOJu0Ywy2f8U0MVx+bGKPh+fOeABj+FdBupCBh05K6N9weOeSWtN/U/g KT+h4c7Ich1deV6D3DEaA3MPKqiKXQ+SH3/kUNYPSqqoA8fO1TIeUL1X29Xem/EymQ== X-Google-Smtp-Source: AGHT+IEx3eZ2+QDRU3KSgCx+OV+pBNRFvlIeIxRWjJ8O0yjzM4DTtM3tHEIt/k5BZ6oeQUtPwypRXw== X-Received: by 2002:a05:6358:726:b0:1a5:f991:ae54 with SMTP id e5c5f4694b2df-1acc5c4b72cmr244477255d.20.1721470428817; Sat, 20 Jul 2024 03:13:48 -0700 (PDT) From: Amjad Alsharafi To: qemu-devel@nongnu.org Cc: Hanna Reitz , Kevin Wolf , "open list:vvfat" , Amjad Alsharafi Subject: [PATCH v6 2/5] vvfat: Fix usage of `info.file.offset` Date: Sat, 20 Jul 2024 18:13:31 +0800 Message-ID: <72f19a7903886dda1aa78bcae0e17702ee939262.1721470238.git.amjadsharafi10@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::c2a; envelope-from=amjadsharafi10@gmail.com; helo=mail-oo1-xc2a.google.com X-Spam_score_int: -17 X-Spam_score: -1.8 X-Spam_bar: - X-Spam_report: (-1.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1721470503786116600 Content-Type: text/plain; charset="utf-8" The field is marked as "the offset in the file (in clusters)", but it was being used like this `cluster_size*(nums)+mapping->info.file.offset`, which is incorrect. Signed-off-by: Amjad Alsharafi Reviewed-by: Kevin Wolf --- block/vvfat.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/block/vvfat.c b/block/vvfat.c index 19da009a5b..247b232608 100644 --- a/block/vvfat.c +++ b/block/vvfat.c @@ -1408,7 +1408,9 @@ read_cluster_directory: =20 assert(s->current_fd); =20 - offset=3Ds->cluster_size*(cluster_num-s->current_mapping->begin)+s= ->current_mapping->info.file.offset; + offset =3D s->cluster_size * + ((cluster_num - s->current_mapping->begin) + + s->current_mapping->info.file.offset); if(lseek(s->current_fd, offset, SEEK_SET)!=3Doffset) return -3; s->cluster=3Ds->cluster_buffer; @@ -1929,8 +1931,9 @@ get_cluster_count_for_direntry(BDRVVVFATState* s, dir= entry_t* direntry, const ch (mapping->mode & MODE_DIRECTORY) =3D=3D 0) { =20 /* was modified in qcow */ - if (offset !=3D mapping->info.file.offset + s->cluster= _size - * (cluster_num - mapping->begin)) { + if (offset !=3D s->cluster_size + * ((cluster_num - mapping->begin) + + mapping->info.file.offset)) { /* offset of this cluster in file chain has change= d */ abort(); copy_it =3D 1; @@ -2404,7 +2407,7 @@ static int commit_mappings(BDRVVVFATState* s, (mapping->end - mapping->begin); } else next_mapping->info.file.offset =3D mapping->info.file.offs= et + - mapping->end - mapping->begin; + (mapping->end - mapping->begin); =20 mapping =3D next_mapping; } --=20 2.45.2 From nobody Mon Sep 16 19:24:25 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1721470515; cv=none; d=zohomail.com; s=zohoarc; b=Q638HRs292RE8Hrcgec1rR9BT5cl6k6rgNjeilkN6gitRqSAd0K/zXe7iv89NdPqU4/WTIEr2v965Zf2fTsOyqFQxGF909x2vt2BXdhusSK+QFJ+udm3xRG+E2MU+FY7z2D9bdZ2iDCpBYCSXhCQkovowz0xwK/NNXNvWH06+bw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1721470515; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=VZ+98M5u4oIUEUTkUVwqPpKA8S8Llpagu9lbB1G2xCw=; b=RTy+Qakmail3NaCSqSzns0am1PUQvLPmVoXMSLMU+Cv8gZe42JM9lZu4JarZlJjidk8D/XKHu3julDjI6NlMKZ2Rbenn6f9xoUs2IihAZGZVAyMUwud0vgIPaOfoOIDRn1MSaqGSzAwxFEFCMWZlCKyiXY+pi6q1pzCs387w+fw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1721470515818742.6735253935634; Sat, 20 Jul 2024 03:15:15 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sV76C-0001d7-Ts; Sat, 20 Jul 2024 06:13:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sV76A-0001V1-Su; Sat, 20 Jul 2024 06:13:54 -0400 Received: from mail-pg1-x52a.google.com ([2607:f8b0:4864:20::52a]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sV769-0007SZ-8Y; Sat, 20 Jul 2024 06:13:54 -0400 Received: by mail-pg1-x52a.google.com with SMTP id 41be03b00d2f7-7a0e8b76813so63517a12.3; Sat, 20 Jul 2024 03:13:52 -0700 (PDT) Received: from amjad-pc.. ([202.185.212.207]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2cb76a5fc6csm4411208a91.0.2024.07.20.03.13.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 20 Jul 2024 03:13:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1721470431; x=1722075231; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=VZ+98M5u4oIUEUTkUVwqPpKA8S8Llpagu9lbB1G2xCw=; b=Ae02YyPpxFn6s23CBK5xfHmxXHP22QliAE7RcMw2QdsOgHBx+XfuUzzuIvWopGvhhg R1LBStR8B321BuePrGXZt7EYLBzjV+XiRUjO+xM8xaR58LZIQBFMlLyVDFxwVEd1m3Su R2Ly0LDvNeLGGd/PboUa32osUNoobESwU4u5hl2kYd5/lBrM4y2n2rn57PxL2Wx7SGNA IurbwjV8tGW91mgnbK4k542MhKc9cyk9xBHI97VyqxMjFnapqh9pf3siz315zIgnnt0a 1724LngC8LgWxKPJz64Z7PXJhAo1ZEa0aFR1IKTjW90oNzHHwwktGIYdp8PLwmp1UcKi Xp8Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721470431; x=1722075231; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=VZ+98M5u4oIUEUTkUVwqPpKA8S8Llpagu9lbB1G2xCw=; b=iIlrHYn2zyBl1YhTUxVi/ktm+PpRTlSCsyJQeZYzNjqElItKlwI8d3s8eAItXiuQy+ L3MHWr23oUTkCcSSE2163aU2Y7yV+hCNlO0s7xwDEZB+gK/oEqYxeSA1BzhkJMoAjgJh AKy3BFJOIioueW4a09ctcZp+S9dcx8SftJ+y1l+rMngVDGt3C1DZy8JEP5EU/gdidXh3 /ry8F1o1KHHVrnwBQP62DgWaLlHdEQxpHnoAGRPEYvXCApzwf/bfNy2hTz/FH15B5OM/ voGGG1Ar5natcpBfzbrmqSTK//l1fkNQEaFELnzT2B+tXKEsKHatyGs3pZlaj9clKYnZ C/0Q== X-Forwarded-Encrypted: i=1; AJvYcCWCMbLCEQ9ScaDYZeZ6UV0ZcWIBrCrYvi6lX4ctm7+NHM8GhTEic1wxtvao2Mwn8YLX/JTAWFuhy0//lnIeWmHH6caXN5M= X-Gm-Message-State: AOJu0Yyg5Y+yf+ASEoJXrEiVjHe5bWNCcaNpLTzgLE48nW3m5ax16e2s C7wQKO43piAA5pj/l/40ozvYcYmGsu5Jp4nwiwPr42RAKcQUqBUIlt80AfjQLDcLMg== X-Google-Smtp-Source: AGHT+IFzKucPvRWDzx9fRjpX16OIvXkNVcuveRHObIhj4XYIS3D9Y/Qei0m60aKVbStjks+pYadIbg== X-Received: by 2002:a05:6a20:7f91:b0:1c3:b61c:57cb with SMTP id adf61e73a8af0-1c42869aaa0mr1096511637.53.1721470430904; Sat, 20 Jul 2024 03:13:50 -0700 (PDT) From: Amjad Alsharafi To: qemu-devel@nongnu.org Cc: Hanna Reitz , Kevin Wolf , "open list:vvfat" , Amjad Alsharafi Subject: [PATCH v6 3/5] vvfat: Fix wrong checks for cluster mappings invariant Date: Sat, 20 Jul 2024 18:13:32 +0800 Message-ID: X-Mailer: git-send-email 2.45.2 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::52a; envelope-from=amjadsharafi10@gmail.com; helo=mail-pg1-x52a.google.com X-Spam_score_int: -17 X-Spam_score: -1.8 X-Spam_bar: - X-Spam_report: (-1.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1721470517338116600 Content-Type: text/plain; charset="utf-8" How this `abort` was intended to check for was: - if the `mapping->first_mapping_index` is not the same as `first_mapping_index`, which **should** happen only in one case, when we are handling the first mapping, in that case `mapping->first_mapping_index =3D=3D -1`, in all other cases, the other mappings after the first should have the condition `true`. - From above, we know that this is the first mapping, so if the offset is not `0`, then abort, since this is an invalid state. The issue was that `first_mapping_index` is not set if we are checking from the middle, the variable `first_mapping_index` is only set if we passed through the check `cluster_was_modified` with the first mapping, and in the same function call we checked the other mappings. One approach is to go into the loop even if `cluster_was_modified` is not true so that we will be able to set `first_mapping_index` for the first mapping, but since `first_mapping_index` is only used here, another approach is to just check manually for the `mapping->first_mapping_index !=3D -1` since we know that this is the value for the only entry where `offset =3D=3D 0` (i.e. first mapping). Signed-off-by: Amjad Alsharafi Reviewed-by: Kevin Wolf --- block/vvfat.c | 10 ++-------- 1 file changed, 2 insertions(+), 8 deletions(-) diff --git a/block/vvfat.c b/block/vvfat.c index 247b232608..b63ac5d045 100644 --- a/block/vvfat.c +++ b/block/vvfat.c @@ -1880,7 +1880,6 @@ get_cluster_count_for_direntry(BDRVVVFATState* s, dir= entry_t* direntry, const ch =20 uint32_t cluster_num =3D begin_of_direntry(direntry); uint32_t offset =3D 0; - int first_mapping_index =3D -1; mapping_t* mapping =3D NULL; const char* basename2 =3D NULL; =20 @@ -1942,14 +1941,9 @@ get_cluster_count_for_direntry(BDRVVVFATState* s, di= rentry_t* direntry, const ch =20 if (strcmp(basename, basename2)) copy_it =3D 1; - first_mapping_index =3D array_index(&(s->mapping),= mapping); - } - - if (mapping->first_mapping_index !=3D first_mapping_in= dex - && mapping->info.file.offset > 0) { - abort(); - copy_it =3D 1; } + assert(mapping->first_mapping_index =3D=3D -1 + || mapping->info.file.offset > 0); =20 /* need to write out? */ if (!was_modified && is_file(direntry)) { --=20 2.45.2 From nobody Mon Sep 16 19:24:25 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1721470511; cv=none; d=zohomail.com; s=zohoarc; b=LLYIKORysuS1MHP427EfO5QYf6W1cmdf0uzQZTAACBB/qjeIxm6xYtlPUodpm59YV4bZwpUj48ivVq3w472+ex1jjDyZS6a0vaJqjdz4t9bLom4s0PzseXQPTro3LC/qvb0IdRKeqmYfjfNykcJc8TaywFym6TF/htzwmohG5HM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1721470511; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=AsFMsus6VE/JFgo/xYfX/N41AZYUF7OSOz1IzwXOsx0=; b=Ld2H5woLagTdAhyiSoQZgZS/a2Zf3/BablujSe5Sxx8IDe+LOkDZH/Mp7RHOUJ1meZgZd/g9vdDqMAiEYpteLUszNTSkC2W/CQ1UqH2UGZPSGaDDZaPUdBmlOXaftrcmswlA+XWmMYjGwDzwtZRCeDf0skakX9HXxmeHuZOx3wE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1721470511763514.2118564476547; Sat, 20 Jul 2024 03:15:11 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sV76E-0001jE-EB; Sat, 20 Jul 2024 06:13:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sV76C-0001dH-U5; Sat, 20 Jul 2024 06:13:56 -0400 Received: from mail-pj1-x1029.google.com ([2607:f8b0:4864:20::1029]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sV76B-0007TH-9T; Sat, 20 Jul 2024 06:13:56 -0400 Received: by mail-pj1-x1029.google.com with SMTP id 98e67ed59e1d1-2cb53da06a9so1558622a91.0; Sat, 20 Jul 2024 03:13:54 -0700 (PDT) Received: from amjad-pc.. ([202.185.212.207]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2cb76a5fc6csm4411208a91.0.2024.07.20.03.13.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 20 Jul 2024 03:13:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1721470433; x=1722075233; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=AsFMsus6VE/JFgo/xYfX/N41AZYUF7OSOz1IzwXOsx0=; b=edT/JrnwhgwPKLLEDUPBQRf39BtiRi8p8TKBf7aTyPGcFCgQfi6ceJFV419dsLuBO+ cn5PgFqV35xVt8stkxfuaHNfjs4XqouUTFOWAEJfMQezG1sxAveHKY8clKz3vIXobORr HrThXZmhpl/9seDuMcaIZzW+c50k8F5I4Q3jgMAXwiCfGMUFGhqbf5/Z5OLAM4gnjtIJ l6ejHo14ZolYWVVElcBbvHu056HmYgzqSndzodu8oQq6fittLw1E0U7ZABAeprva+9hJ BQJTWZSCxkVzUZU5TR7GNjm+5WquW+y4IFs6nP7/6LTG6jVv1q6zzxN+2fUu4YTNWlng BQtQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721470433; x=1722075233; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=AsFMsus6VE/JFgo/xYfX/N41AZYUF7OSOz1IzwXOsx0=; b=q3t5Tz8elbjALAYCDUMsHcWABatH+mJVyfZmppNkAIJ5XMRPuDcoN7QsZwwcnUbaI1 12jjEYD7nhHffZEgBSqJNCwkDwmyE2FcQtdwtF7Vi37qtGD7h9qyYEK6GNA8FUs/yJke RQag/3vDS3nUOyH20mhxQ/jfrUJ5ml66Bf1sRRie7mMHq2dbjlXX+l9JYflGUk+rn1/S gAkM1awXgOWgdO1/ZyY7K/iNIwe17xyb4RUIjWV2IkCH22PnghkRpD5eFJjp8RLLkUz7 Z6qQosbbqUkkVlhx6P96X/K4iyeSHc+SY4IRL0RYRF9rh9REGrC5ihgnBa7hH2+a+o// uP4A== X-Forwarded-Encrypted: i=1; AJvYcCUgn80oG23Grw7iOxh3pWMDi/KwLVjBjbaArTeovwcMy1QN7Z4JtgrwNLG4P9Ms/MVper39KA1x+EeqBBlD3F5QQktSptA= X-Gm-Message-State: AOJu0YwLVWlDkhiErZxjSygi9TMMdma0VnmB+w8sKB6hvMQ0mecT71qC y2T1Cyk4ehs5N1m2QQ0ObkRm7xThZs3/bEvT1Ccyk1kb69Zlmby2n4xEmrbXBkeo5Q== X-Google-Smtp-Source: AGHT+IH1pm6kgzJXdS5U+Is4y7r6xBU4p3yaFKa29dHR6Ir458J8lnZsueWAN8/Rs9LA9E8aHuOBlw== X-Received: by 2002:a17:90b:2303:b0:2ab:8324:1b47 with SMTP id 98e67ed59e1d1-2cd170c92f3mr3716260a91.15.1721470432991; Sat, 20 Jul 2024 03:13:52 -0700 (PDT) From: Amjad Alsharafi To: qemu-devel@nongnu.org Cc: Hanna Reitz , Kevin Wolf , "open list:vvfat" , Amjad Alsharafi Subject: [PATCH v6 4/5] vvfat: Fix reading files with non-continuous clusters Date: Sat, 20 Jul 2024 18:13:33 +0800 Message-ID: <1f3ea115779abab62ba32c788073cdc99f9ad5dd.1721470238.git.amjadsharafi10@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::1029; envelope-from=amjadsharafi10@gmail.com; helo=mail-pj1-x1029.google.com X-Spam_score_int: -17 X-Spam_score: -1.8 X-Spam_bar: - X-Spam_report: (-1.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1721470513325116600 Content-Type: text/plain; charset="utf-8" When reading with `read_cluster` we get the `mapping` with `find_mapping_for_cluster` and then we call `open_file` for this mapping. The issue appear when its the same file, but a second cluster that is not immediately after it, imagine clusters `500 -> 503`, this will give us 2 mappings one has the range `500..501` and another `503..504`, both point to the same file, but different offsets. When we don't open the file since the path is the same, we won't assign `s->current_mapping` and thus accessing way out of bound of the file. From our example above, after `open_file` (that didn't open anything) we will get the offset into the file with `s->cluster_size*(cluster_num-s->current_mapping->begin)`, which will give us `0x2000 * (504-500)`, which is out of bound for this mapping and will produce some issues. Signed-off-by: Amjad Alsharafi --- block/vvfat.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/block/vvfat.c b/block/vvfat.c index b63ac5d045..d2b879705e 100644 --- a/block/vvfat.c +++ b/block/vvfat.c @@ -1360,15 +1360,17 @@ static int open_file(BDRVVVFATState* s,mapping_t* m= apping) { if(!mapping) return -1; - if(!s->current_mapping || - strcmp(s->current_mapping->path,mapping->path)) { + if (s->current_mapping !=3D mapping) { /* open file */ int fd =3D qemu_open_old(mapping->path, - O_RDONLY | O_BINARY | O_LARGEFILE); - if(fd<0) + O_RDONLY | O_BINARY | O_LARGEFILE); + if (fd < 0) { return -1; + } vvfat_close_current_file(s); + s->current_fd =3D fd; + assert(s->current_fd); s->current_mapping =3D mapping; } return 0; --=20 2.45.2 From nobody Mon Sep 16 19:24:25 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1721470503; cv=none; d=zohomail.com; s=zohoarc; b=AJtHxFNq8VBSFr+Te6tWDV4HroGOup41GkQE+G8PvGJY24H3bI2m8Va7NvTMxuDdRcCwiVzaZZ5OP7cnE3/mG5TUS1NXC4U90J4JpZhVY3zy0apsJDza6669KCyTOUwYkFUriXg4uKDnLztOnvIviXYVajeATgT7TeGYe3KFIb4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1721470503; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=C0BBV5nuncJnOTjO2WBAoROuWAORFmcild+shMcgOHg=; b=l62pQSOlmzGqik1a0iiPVvHyHnH+O5Zwr9/GV1RTbmj4upKxmr/NUeP1DCOlKuHL/E10Oc6rIT0zol6n4WFi/5rRsuXgknp/f9eWazCvvOZgyr1QKv5Ori44ICDgtBqPRQdTsLuywQ2gLIOXmT92j8iAeac0CnxBmCsjVGAE+tY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1721470503308670.2879310940818; Sat, 20 Jul 2024 03:15:03 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sV76K-0002Aj-A2; Sat, 20 Jul 2024 06:14:04 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sV76I-000249-LF; Sat, 20 Jul 2024 06:14:02 -0400 Received: from mail-pj1-x1031.google.com ([2607:f8b0:4864:20::1031]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sV76E-0007Ti-Gt; Sat, 20 Jul 2024 06:14:02 -0400 Received: by mail-pj1-x1031.google.com with SMTP id 98e67ed59e1d1-2c964f5a037so1517235a91.0; Sat, 20 Jul 2024 03:13:57 -0700 (PDT) Received: from amjad-pc.. ([202.185.212.207]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2cb76a5fc6csm4411208a91.0.2024.07.20.03.13.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 20 Jul 2024 03:13:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1721470436; x=1722075236; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=C0BBV5nuncJnOTjO2WBAoROuWAORFmcild+shMcgOHg=; b=Hp61c1OML+ZBmtjQt4UERUrhewA4kYBs4UqWW821s7ZtDavwmxHXTLc5JdjCLKBIm2 NmHvHO8yCYO6pfBLkSEty8e8CL4FRa5ZvhVI5xKNnrlZGlZHKE4sV2UlTKT2Fr1qsbQN OCBlILPj8aWS3e/UsSmFZCR4rtPqAZ2LrE+4Hp0Y8Hfq5b0z/0XX81obbyA5peTgDgpr a0Dm3iqAGYTuI6jlteyHPOWYZF2bKTrBbMII/wKW81sGyszVSOeHCjweJYVYPabsk14U 0IxSShgqLLws3b6uaPA+3ZOUkKyi3x5VZg/GJ9krVgXm7/ItzTnK0Hug3Q8wcD/QgLB6 n6Bw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721470436; x=1722075236; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=C0BBV5nuncJnOTjO2WBAoROuWAORFmcild+shMcgOHg=; b=Ugcx89ZosN6qg7GCTrFKveTluyWKzhNdjNeERCTiVZDQ3VTP/GW1TXAklCwh0r3dF+ euJXWvYqCByyBnkrCZaIb9FJ0Io8xVm2i/ABlwu69IseB0+bm+Qdeqn/rwciYCQ6Zx8Y rkA/ddWzUc2MpKgO6WNnJ2t/GLJlM5MRaoLSYGODucIG3ebgYKRghIwZZJPl4I9M5zOc zHz7adeaGk2mukD9xCTBDMIOMwKOnhIU5o9cPRz2s88Gwb2sWd/922K1XbIm1dRwh5aL 04MMEoE3bv32qs+X7CFGehyFyoVY9ekDj271jfyQFzTPBakrZRwf3Kvy1Q5V5yHMs95o JJpA== X-Forwarded-Encrypted: i=1; AJvYcCUAl19VlR9nQQ57zbxeNKaqhK3ok4FmcKuCGRMT4rWStn4ZPp5TOCxIIDMUaDAGY2zQUTk/AfXUhtM0cMT1frjb1y16D4U= X-Gm-Message-State: AOJu0YwmKN7QsGGRs9VS17QWxXmZh3wUTid9X4NCtRrzxas5UpnrjouT so5v2ogV1gkM0HY0cLQxTA4PhEj4CBMlrF2xf0Uzq7AMnM/xdgw+nTrRss6HAHExcA== X-Google-Smtp-Source: AGHT+IFV0bqn1aeULL6rEYh7Gyfw2XYTDiRBZhifKgHCOIQsU+vOv1zgn6sM6xh1vAIX0vjI9SlY+A== X-Received: by 2002:a05:6a21:670d:b0:1c2:8d2f:65f4 with SMTP id adf61e73a8af0-1c4229a42fbmr2131254637.44.1721470435191; Sat, 20 Jul 2024 03:13:55 -0700 (PDT) From: Amjad Alsharafi To: qemu-devel@nongnu.org Cc: Hanna Reitz , Kevin Wolf , "open list:vvfat" , Amjad Alsharafi Subject: [PATCH v6 5/5] iotests: Add `vvfat` tests Date: Sat, 20 Jul 2024 18:13:34 +0800 Message-ID: X-Mailer: git-send-email 2.45.2 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2607:f8b0:4864:20::1031; envelope-from=amjadsharafi10@gmail.com; helo=mail-pj1-x1031.google.com X-Spam_score_int: -17 X-Spam_score: -1.8 X-Spam_bar: - X-Spam_report: (-1.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1721470505357116600 Content-Type: text/plain; charset="utf-8" Added several tests to verify the implementation of the vvfat driver. We needed a way to interact with it, so created a basic `fat16.py` driver that handled writing correct sectors for us. Added `vvfat` to the non-generic formats, as its not a normal image format. Signed-off-by: Amjad Alsharafi Reviewed-by: Kevin Wolf Tested-by: Kevin Wolf --- tests/qemu-iotests/check | 2 +- tests/qemu-iotests/fat16.py | 675 +++++++++++++++++++++++++++++ tests/qemu-iotests/testenv.py | 2 +- tests/qemu-iotests/tests/vvfat | 458 +++++++++++++++++++ tests/qemu-iotests/tests/vvfat.out | 5 + 5 files changed, 1140 insertions(+), 2 deletions(-) create mode 100644 tests/qemu-iotests/fat16.py create mode 100755 tests/qemu-iotests/tests/vvfat create mode 100755 tests/qemu-iotests/tests/vvfat.out diff --git a/tests/qemu-iotests/check b/tests/qemu-iotests/check index 56d88ca423..545f9ec7bd 100755 --- a/tests/qemu-iotests/check +++ b/tests/qemu-iotests/check @@ -84,7 +84,7 @@ def make_argparser() -> argparse.ArgumentParser: p.set_defaults(imgfmt=3D'raw', imgproto=3D'file') =20 format_list =3D ['raw', 'bochs', 'cloop', 'parallels', 'qcow', 'qcow2', - 'qed', 'vdi', 'vpc', 'vhdx', 'vmdk', 'luks', 'dmg'] + 'qed', 'vdi', 'vpc', 'vhdx', 'vmdk', 'luks', 'dmg', 'vv= fat'] g_fmt =3D p.add_argument_group( ' image format options', 'The following options set the IMGFMT environment variable. ' diff --git a/tests/qemu-iotests/fat16.py b/tests/qemu-iotests/fat16.py new file mode 100644 index 0000000000..0acc987da6 --- /dev/null +++ b/tests/qemu-iotests/fat16.py @@ -0,0 +1,675 @@ +# A simple FAT16 driver that is used to test the `vvfat` driver in QEMU. +# +# Copyright (C) 2024 Amjad Alsharafi +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . + +from typing import List +import string + +SECTOR_SIZE =3D 512 +DIRENTRY_SIZE =3D 32 +ALLOWED_FILE_CHARS =3D set( + "!#$%&'()-@^_`{}~" + string.digits + string.ascii_uppercase +) + + +class MBR: + def __init__(self, data: bytes): + assert len(data) =3D=3D 512 + self.partition_table =3D [] + for i in range(4): + partition =3D data[446 + i * 16 : 446 + (i + 1) * 16] + self.partition_table.append( + { + "status": partition[0], + "start_head": partition[1], + "start_sector": partition[2] & 0x3F, + "start_cylinder": ((partition[2] & 0xC0) << 2) + | partition[3], + "type": partition[4], + "end_head": partition[5], + "end_sector": partition[6] & 0x3F, + "end_cylinder": ((partition[6] & 0xC0) << 2) | partiti= on[7], + "start_lba": int.from_bytes(partition[8:12], "little"), + "size": int.from_bytes(partition[12:16], "little"), + } + ) + + def __str__(self): + return "\n".join( + [ + f"{i}: {partition}" + for i, partition in enumerate(self.partition_table) + ] + ) + + +class FatBootSector: + def __init__(self, data: bytes): + assert len(data) =3D=3D 512 + self.bytes_per_sector =3D int.from_bytes(data[11:13], "little") + self.sectors_per_cluster =3D data[13] + self.reserved_sectors =3D int.from_bytes(data[14:16], "little") + self.fat_count =3D data[16] + self.root_entries =3D int.from_bytes(data[17:19], "little") + total_sectors_16 =3D int.from_bytes(data[19:21], "little") + self.media_descriptor =3D data[21] + self.sectors_per_fat =3D int.from_bytes(data[22:24], "little") + self.sectors_per_track =3D int.from_bytes(data[24:26], "little") + self.heads =3D int.from_bytes(data[26:28], "little") + self.hidden_sectors =3D int.from_bytes(data[28:32], "little") + total_sectors_32 =3D int.from_bytes(data[32:36], "little") + assert ( + total_sectors_16 =3D=3D 0 or total_sectors_32 =3D=3D 0 + ), "Both total sectors (16 and 32) fields are non-zero" + self.total_sectors =3D total_sectors_16 or total_sectors_32 + self.drive_number =3D data[36] + self.volume_id =3D int.from_bytes(data[39:43], "little") + self.volume_label =3D data[43:54].decode("ascii").strip() + self.fs_type =3D data[54:62].decode("ascii").strip() + + def root_dir_start(self): + """ + Calculate the start sector of the root directory. + """ + return self.reserved_sectors + self.fat_count * self.sectors_per_f= at + + def root_dir_size(self): + """ + Calculate the size of the root directory in sectors. + """ + return ( + self.root_entries * DIRENTRY_SIZE + self.bytes_per_sector - 1 + ) // self.bytes_per_sector + + def data_sector_start(self): + """ + Calculate the start sector of the data region. + """ + return self.root_dir_start() + self.root_dir_size() + + def first_sector_of_cluster(self, cluster: int): + """ + Calculate the first sector of the given cluster. + """ + return ( + self.data_sector_start() + (cluster - 2) * self.sectors_per_cl= uster + ) + + def cluster_bytes(self): + """ + Calculate the number of bytes in a cluster. + """ + return self.bytes_per_sector * self.sectors_per_cluster + + def __str__(self): + return ( + f"Bytes per sector: {self.bytes_per_sector}\n" + f"Sectors per cluster: {self.sectors_per_cluster}\n" + f"Reserved sectors: {self.reserved_sectors}\n" + f"FAT count: {self.fat_count}\n" + f"Root entries: {self.root_entries}\n" + f"Total sectors: {self.total_sectors}\n" + f"Media descriptor: {self.media_descriptor}\n" + f"Sectors per FAT: {self.sectors_per_fat}\n" + f"Sectors per track: {self.sectors_per_track}\n" + f"Heads: {self.heads}\n" + f"Hidden sectors: {self.hidden_sectors}\n" + f"Drive number: {self.drive_number}\n" + f"Volume ID: {self.volume_id}\n" + f"Volume label: {self.volume_label}\n" + f"FS type: {self.fs_type}\n" + ) + + +class FatDirectoryEntry: + def __init__(self, data: bytes, sector: int, offset: int): + self.name =3D data[0:8].decode("ascii").strip() + self.ext =3D data[8:11].decode("ascii").strip() + self.attributes =3D data[11] + self.reserved =3D data[12] + self.create_time_tenth =3D data[13] + self.create_time =3D int.from_bytes(data[14:16], "little") + self.create_date =3D int.from_bytes(data[16:18], "little") + self.last_access_date =3D int.from_bytes(data[18:20], "little") + high_cluster =3D int.from_bytes(data[20:22], "little") + self.last_mod_time =3D int.from_bytes(data[22:24], "little") + self.last_mod_date =3D int.from_bytes(data[24:26], "little") + low_cluster =3D int.from_bytes(data[26:28], "little") + self.cluster =3D (high_cluster << 16) | low_cluster + self.size_bytes =3D int.from_bytes(data[28:32], "little") + + # extra (to help write back to disk) + self.sector =3D sector + self.offset =3D offset + + def as_bytes(self) -> bytes: + return ( + self.name.ljust(8, " ").encode("ascii") + + self.ext.ljust(3, " ").encode("ascii") + + self.attributes.to_bytes(1, "little") + + self.reserved.to_bytes(1, "little") + + self.create_time_tenth.to_bytes(1, "little") + + self.create_time.to_bytes(2, "little") + + self.create_date.to_bytes(2, "little") + + self.last_access_date.to_bytes(2, "little") + + (self.cluster >> 16).to_bytes(2, "little") + + self.last_mod_time.to_bytes(2, "little") + + self.last_mod_date.to_bytes(2, "little") + + (self.cluster & 0xFFFF).to_bytes(2, "little") + + self.size_bytes.to_bytes(4, "little") + ) + + def whole_name(self): + if self.ext: + return f"{self.name}.{self.ext}" + else: + return self.name + + def __str__(self): + return ( + f"Name: {self.name}\n" + f"Ext: {self.ext}\n" + f"Attributes: {self.attributes}\n" + f"Reserved: {self.reserved}\n" + f"Create time tenth: {self.create_time_tenth}\n" + f"Create time: {self.create_time}\n" + f"Create date: {self.create_date}\n" + f"Last access date: {self.last_access_date}\n" + f"Last mod time: {self.last_mod_time}\n" + f"Last mod date: {self.last_mod_date}\n" + f"Cluster: {self.cluster}\n" + f"Size: {self.size_bytes}\n" + ) + + def __repr__(self): + # convert to dict + return str(vars(self)) + + +class Fat16: + def __init__( + self, + start_sector: int, + size: int, + sector_reader: callable, + sector_writer: callable, + ): + self.start_sector =3D start_sector + self.size_in_sectors =3D size + self.sector_reader =3D sector_reader + self.sector_writer =3D sector_writer + + self.boot_sector =3D FatBootSector(self.sector_reader(start_sector= )) + + fat_size_in_sectors =3D ( + self.boot_sector.sectors_per_fat * self.boot_sector.fat_count + ) + self.fats =3D self.read_sectors( + self.boot_sector.reserved_sectors, fat_size_in_sectors + ) + self.fats_dirty_sectors =3D set() + + def read_sectors(self, start_sector: int, num_sectors: int) -> bytes: + return self.sector_reader(start_sector + self.start_sector, num_se= ctors) + + def write_sectors(self, start_sector: int, data: bytes): + return self.sector_writer(start_sector + self.start_sector, data) + + def directory_from_bytes( + self, data: bytes, start_sector: int + ) -> List[FatDirectoryEntry]: + """ + Convert `bytes` into a list of `FatDirectoryEntry` objects. + Will ignore long file names. + Will stop when it encounters a 0x00 byte. + """ + + entries =3D [] + for i in range(0, len(data), DIRENTRY_SIZE): + entry =3D data[i : i + DIRENTRY_SIZE] + + current_sector =3D start_sector + (i // SECTOR_SIZE) + current_offset =3D i % SECTOR_SIZE + + if entry[0] =3D=3D 0: + break + elif entry[0] =3D=3D 0xE5: + # Deleted file + continue + + if entry[11] & 0xF =3D=3D 0xF: + # Long file name + continue + + entries.append( + FatDirectoryEntry(entry, current_sector, current_offset) + ) + return entries + + def read_root_directory(self) -> List[FatDirectoryEntry]: + root_dir =3D self.read_sectors( + self.boot_sector.root_dir_start(), self.boot_sector.root_dir_s= ize() + ) + return self.directory_from_bytes( + root_dir, self.boot_sector.root_dir_start() + ) + + def read_fat_entry(self, cluster: int) -> int: + """ + Read the FAT entry for the given cluster. + """ + fat_offset =3D cluster * 2 # FAT16 + return int.from_bytes(self.fats[fat_offset : fat_offset + 2], "lit= tle") + + def write_fat_entry(self, cluster: int, value: int): + """ + Write the FAT entry for the given cluster. + """ + fat_offset =3D cluster * 2 + self.fats =3D ( + self.fats[:fat_offset] + + value.to_bytes(2, "little") + + self.fats[fat_offset + 2 :] + ) + self.fats_dirty_sectors.add(fat_offset // SECTOR_SIZE) + + def flush_fats(self): + """ + Write the FATs back to the disk. + """ + for sector in self.fats_dirty_sectors: + data =3D self.fats[sector * SECTOR_SIZE : (sector + 1) * SECTO= R_SIZE] + sector =3D self.boot_sector.reserved_sectors + sector + self.write_sectors(sector, data) + self.fats_dirty_sectors =3D set() + + def next_cluster(self, cluster: int) -> int | None: + """ + Get the next cluster in the chain. + If its `None`, then its the last cluster. + The function will crash if the next cluster + is `FREE` (unexpected) or invalid entry. + """ + fat_entry =3D self.read_fat_entry(cluster) + if fat_entry =3D=3D 0: + raise Exception("Unexpected: FREE cluster") + elif fat_entry =3D=3D 1: + raise Exception("Unexpected: RESERVED cluster") + elif fat_entry >=3D 0xFFF8: + return None + elif fat_entry >=3D 0xFFF7: + raise Exception("Invalid FAT entry") + else: + return fat_entry + + def next_free_cluster(self) -> int: + """ + Find the next free cluster. + """ + # simple linear search + for i in range(2, 0xFFFF): + if self.read_fat_entry(i) =3D=3D 0: + return i + raise Exception("No free clusters") + + def next_free_cluster_non_continuous(self) -> int: + """ + Find the next free cluster, but makes sure + that the cluster before and after it are not allocated. + """ + # simple linear search + before =3D False + for i in range(2, 0xFFFF): + if self.read_fat_entry(i) =3D=3D 0: + if before and self.read_fat_entry(i + 1) =3D=3D 0: + return i + else: + before =3D True + else: + before =3D False + + raise Exception("No free clusters") + + def read_cluster(self, cluster: int) -> bytes: + """ + Read the cluster at the given cluster. + """ + return self.read_sectors( + self.boot_sector.first_sector_of_cluster(cluster), + self.boot_sector.sectors_per_cluster, + ) + + def write_cluster(self, cluster: int, data: bytes): + """ + Write the cluster at the given cluster. + """ + assert len(data) =3D=3D self.boot_sector.cluster_bytes() + return self.write_sectors( + self.boot_sector.first_sector_of_cluster(cluster), + data, + ) + + def read_directory(self, cluster: int) -> List[FatDirectoryEntry]: + """ + Read the directory at the given cluster. + """ + entries =3D [] + while cluster is not None: + data =3D self.read_cluster(cluster) + entries.extend( + self.directory_from_bytes( + data, self.boot_sector.first_sector_of_cluster(cluster) + ) + ) + cluster =3D self.next_cluster(cluster) + return entries + + def add_direntry( + self, cluster: int | None, name: str, ext: str, attributes: int + ): + """ + Add a new directory entry to the given cluster. + If the cluster is `None`, then it will be added to the root direct= ory. + """ + + def find_free_entry(data: bytes): + for i in range(0, len(data), DIRENTRY_SIZE): + entry =3D data[i : i + DIRENTRY_SIZE] + if entry[0] =3D=3D 0 or entry[0] =3D=3D 0xE5: + return i + return None + + assert len(name) <=3D 8, "Name must be 8 characters or less" + assert len(ext) <=3D 3, "Ext must be 3 characters or less" + assert attributes % 0x15 !=3D 0x15, "Invalid attributes" + + # initial dummy data + new_entry =3D FatDirectoryEntry(b"\0" * 32, 0, 0) + new_entry.name =3D name.ljust(8, " ") + new_entry.ext =3D ext.ljust(3, " ") + new_entry.attributes =3D attributes + new_entry.reserved =3D 0 + new_entry.create_time_tenth =3D 0 + new_entry.create_time =3D 0 + new_entry.create_date =3D 0 + new_entry.last_access_date =3D 0 + new_entry.last_mod_time =3D 0 + new_entry.last_mod_date =3D 0 + new_entry.cluster =3D self.next_free_cluster() + new_entry.size_bytes =3D 0 + + # mark as EOF + self.write_fat_entry(new_entry.cluster, 0xFFFF) + + if cluster is None: + for i in range(self.boot_sector.root_dir_size()): + sector_data =3D self.read_sectors( + self.boot_sector.root_dir_start() + i, 1 + ) + offset =3D find_free_entry(sector_data) + if offset is not None: + new_entry.sector =3D self.boot_sector.root_dir_start()= + i + new_entry.offset =3D offset + self.update_direntry(new_entry) + return new_entry + else: + while cluster is not None: + data =3D self.read_cluster(cluster) + offset =3D find_free_entry(data) + if offset is not None: + new_entry.sector =3D self.boot_sector.first_sector_of_= cluster( + cluster + ) + (offset // SECTOR_SIZE) + new_entry.offset =3D offset % SECTOR_SIZE + self.update_direntry(new_entry) + return new_entry + cluster =3D self.next_cluster(cluster) + + raise Exception("No free directory entries") + + def update_direntry(self, entry: FatDirectoryEntry): + """ + Write the directory entry back to the disk. + """ + sector =3D self.read_sectors(entry.sector, 1) + sector =3D ( + sector[: entry.offset] + + entry.as_bytes() + + sector[entry.offset + DIRENTRY_SIZE :] + ) + self.write_sectors(entry.sector, sector) + + def find_direntry(self, path: str) -> FatDirectoryEntry | None: + """ + Find the directory entry for the given path. + """ + assert path[0] =3D=3D "/", "Path must start with /" + + path =3D path[1:] # remove the leading / + parts =3D path.split("/") + directory =3D self.read_root_directory() + + current_entry =3D None + + for i, part in enumerate(parts): + is_last =3D i =3D=3D len(parts) - 1 + + for entry in directory: + if entry.whole_name() =3D=3D part: + current_entry =3D entry + break + if current_entry is None: + return None + + if is_last: + return current_entry + else: + if current_entry.attributes & 0x10 =3D=3D 0: + raise Exception( + f"{current_entry.whole_name()} is not a directory" + ) + else: + directory =3D self.read_directory(current_entry.cluste= r) + + def read_file(self, entry: FatDirectoryEntry | None) -> bytes | None: + """ + Read the content of the file at the given path. + """ + if entry is None: + return None + if entry.attributes & 0x10 !=3D 0: + raise Exception(f"{entry.whole_name()} is a directory") + + data =3D b"" + cluster =3D entry.cluster + while cluster is not None and len(data) <=3D entry.size_bytes: + data +=3D self.read_cluster(cluster) + cluster =3D self.next_cluster(cluster) + return data[: entry.size_bytes] + + def truncate_file( + self, + entry: FatDirectoryEntry, + new_size: int, + allocate_non_continuous: bool =3D False, + ): + """ + Truncate the file at the given path to the new size. + """ + if entry is None: + return Exception("entry is None") + if entry.attributes & 0x10 !=3D 0: + raise Exception(f"{entry.whole_name()} is a directory") + + def clusters_from_size(size: int): + return ( + size + self.boot_sector.cluster_bytes() - 1 + ) // self.boot_sector.cluster_bytes() + + # First, allocate new FATs if we need to + required_clusters =3D clusters_from_size(new_size) + current_clusters =3D clusters_from_size(entry.size_bytes) + + affected_clusters =3D set() + + # Keep at least one cluster, easier to manage this way + if required_clusters =3D=3D 0: + required_clusters =3D 1 + if current_clusters =3D=3D 0: + current_clusters =3D 1 + + if required_clusters > current_clusters: + # Allocate new clusters + cluster =3D entry.cluster + to_add =3D required_clusters + for _ in range(current_clusters - 1): + to_add -=3D 1 + assert cluster is not None, "Cluster is None" + affected_clusters.add(cluster) + cluster =3D self.next_cluster(cluster) + assert required_clusters > 0, "No new clusters to allocate" + assert cluster is not None, "Cluster is None" + assert ( + self.next_cluster(cluster) is None + ), "Cluster is not the last cluster" + + # Allocate new clusters + for _ in range(to_add - 1): + if allocate_non_continuous: + new_cluster =3D self.next_free_cluster_non_continuous() + else: + new_cluster =3D self.next_free_cluster() + self.write_fat_entry(cluster, new_cluster) + self.write_fat_entry(new_cluster, 0xFFFF) + cluster =3D new_cluster + + elif required_clusters < current_clusters: + # Truncate the file + cluster =3D entry.cluster + for _ in range(required_clusters - 1): + assert cluster is not None, "Cluster is None" + cluster =3D self.next_cluster(cluster) + assert cluster is not None, "Cluster is None" + + next_cluster =3D self.next_cluster(cluster) + # mark last as EOF + self.write_fat_entry(cluster, 0xFFFF) + # free the rest + while next_cluster is not None: + cluster =3D next_cluster + next_cluster =3D self.next_cluster(next_cluster) + self.write_fat_entry(cluster, 0) + + self.flush_fats() + + # verify number of clusters + cluster =3D entry.cluster + count =3D 0 + while cluster is not None: + count +=3D 1 + affected_clusters.add(cluster) + cluster =3D self.next_cluster(cluster) + assert ( + count =3D=3D required_clusters + ), f"Expected {required_clusters} clusters, got {count}" + + # update the size + entry.size_bytes =3D new_size + self.update_direntry(entry) + + # trigger every affected cluster + for cluster in affected_clusters: + first_sector =3D self.boot_sector.first_sector_of_cluster(clus= ter) + first_sector_data =3D self.read_sectors(first_sector, 1) + self.write_sectors(first_sector, first_sector_data) + + def write_file(self, entry: FatDirectoryEntry, data: bytes): + """ + Write the content of the file at the given path. + """ + if entry is None: + return Exception("entry is None") + if entry.attributes & 0x10 !=3D 0: + raise Exception(f"{entry.whole_name()} is a directory") + + data_len =3D len(data) + + self.truncate_file(entry, data_len) + + cluster =3D entry.cluster + while cluster is not None: + data_to_write =3D data[: self.boot_sector.cluster_bytes()] + if len(data_to_write) < self.boot_sector.cluster_bytes(): + old_data =3D self.read_cluster(cluster) + data_to_write +=3D old_data[len(data_to_write) :] + + self.write_cluster(cluster, data_to_write) + data =3D data[self.boot_sector.cluster_bytes() :] + if len(data) =3D=3D 0: + break + cluster =3D self.next_cluster(cluster) + + assert ( + len(data) =3D=3D 0 + ), "Data was not written completely, clusters missing" + + def create_file(self, path: str): + """ + Create a new file at the given path. + """ + assert path[0] =3D=3D "/", "Path must start with /" + + path =3D path[1:] # remove the leading / + + parts =3D path.split("/") + + directory_cluster =3D None + directory =3D self.read_root_directory() + + parts, filename =3D parts[:-1], parts[-1] + + for _, part in enumerate(parts): + current_entry =3D None + for entry in directory: + if entry.whole_name() =3D=3D part: + current_entry =3D entry + break + if current_entry is None: + return None + + if current_entry.attributes & 0x10 =3D=3D 0: + raise Exception( + f"{current_entry.whole_name()} is not a directory" + ) + else: + directory =3D self.read_directory(current_entry.cluster) + directory_cluster =3D current_entry.cluster + + # add new entry to the directory + + filename, ext =3D filename.split(".") + + if len(ext) > 3: + raise Exception("Ext must be 3 characters or less") + if len(filename) > 8: + raise Exception("Name must be 8 characters or less") + + for c in filename + ext: + + if c not in ALLOWED_FILE_CHARS: + raise Exception("Invalid character in filename") + + return self.add_direntry(directory_cluster, filename, ext, 0) diff --git a/tests/qemu-iotests/testenv.py b/tests/qemu-iotests/testenv.py index 588f30a4f1..4053d29de4 100644 --- a/tests/qemu-iotests/testenv.py +++ b/tests/qemu-iotests/testenv.py @@ -250,7 +250,7 @@ def __init__(self, source_dir: str, build_dir: str, self.qemu_img_options =3D os.getenv('QEMU_IMG_OPTIONS') self.qemu_nbd_options =3D os.getenv('QEMU_NBD_OPTIONS') =20 - is_generic =3D self.imgfmt not in ['bochs', 'cloop', 'dmg'] + is_generic =3D self.imgfmt not in ['bochs', 'cloop', 'dmg', 'vvfat= '] self.imgfmt_generic =3D 'true' if is_generic else 'false' =20 self.qemu_io_options =3D f'--cache {self.cachemode} --aio {self.ai= omode}' diff --git a/tests/qemu-iotests/tests/vvfat b/tests/qemu-iotests/tests/vvfat new file mode 100755 index 0000000000..a6c536b886 --- /dev/null +++ b/tests/qemu-iotests/tests/vvfat @@ -0,0 +1,458 @@ +#!/usr/bin/env python3 +# group: rw vvfat +# +# Test vvfat driver implementation +# Here, we use a simple FAT16 implementation and check the behavior of the= vvfat driver. +# +# Copyright (C) 2024 Amjad Alsharafi +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . + +import os, shutil +import iotests +from iotests import imgfmt, QMPTestCase +from fat16 import MBR, Fat16, DIRENTRY_SIZE + +filesystem =3D os.path.join(iotests.test_dir, "filesystem") + +nbd_sock =3D iotests.file_path("nbd.sock", base_dir=3Diotests.sock_dir) +nbd_uri =3D "nbd+unix:///disk?socket=3D" + nbd_sock + +SECTOR_SIZE =3D 512 + + +class TestVVFatDriver(QMPTestCase): + def setUp(self) -> None: + if os.path.exists(filesystem): + if os.path.isdir(filesystem): + shutil.rmtree(filesystem) + else: + print(f"Error: {filesystem} exists and is not a directory") + exit(1) + os.mkdir(filesystem) + + # Add some text files to the filesystem + for i in range(10): + with open(os.path.join(filesystem, f"file{i}.txt"), "w") as f: + f.write(f"Hello, world! {i}\n") + + # Add 2 large files, above the cluster size (8KB) + with open(os.path.join(filesystem, "large1.txt"), "wb") as f: + # write 'A' * 1KB, 'B' * 1KB, 'C' * 1KB, ... + for i in range(8 * 2): # two clusters + f.write(bytes([0x41 + i] * 1024)) + + with open(os.path.join(filesystem, "large2.txt"), "wb") as f: + # write 'A' * 1KB, 'B' * 1KB, 'C' * 1KB, ... + for i in range(8 * 3): # 3 clusters + f.write(bytes([0x41 + i] * 1024)) + + self.vm =3D iotests.VM() + + self.vm.add_blockdev( + self.vm.qmp_to_opts( + { + "driver": imgfmt, + "node-name": "disk", + "rw": "true", + "fat-type": "16", + "dir": filesystem, + } + ) + ) + + self.vm.launch() + + self.vm.qmp_log("block-dirty-bitmap-add", **{"node": "disk", "name= ": "bitmap0"}) + + # attach nbd server + self.vm.qmp_log( + "nbd-server-start", + **{"addr": {"type": "unix", "data": {"path": nbd_sock}}}, + filters=3D[], + ) + + self.vm.qmp_log( + "nbd-server-add", + **{"device": "disk", "writable": True, "bitmap": "bitmap0"}, + ) + + self.qio =3D iotests.QemuIoInteractive("-f", "raw", nbd_uri) + + def tearDown(self) -> None: + self.qio.close() + self.vm.shutdown() + # print(self.vm.get_log()) + shutil.rmtree(filesystem) + + def read_sectors(self, sector: int, num: int =3D 1) -> bytes: + """ + Read `num` sectors starting from `sector` from the `disk`. + This uses `QemuIoInteractive` to read the sectors into `stdout` an= d then parse the output. + """ + self.assertGreater(num, 0) + # The output contains the content of the sector in hex dump format + # We need to extract the content from it + output =3D self.qio.cmd(f"read -v {sector * SECTOR_SIZE} {num * SE= CTOR_SIZE}") + # Each row is 16 bytes long, and we are writing `num` sectors + rows =3D num * SECTOR_SIZE // 16 + output_rows =3D output.split("\n")[:rows] + + hex_content =3D "".join( + [(row.split(": ")[1]).split(" ")[0] for row in output_rows] + ) + bytes_content =3D bytes.fromhex(hex_content) + + self.assertEqual(len(bytes_content), num * SECTOR_SIZE) + + return bytes_content + + def write_sectors(self, sector: int, data: bytes): + """ + Write `data` to the `disk` starting from `sector`. + This uses `QemuIoInteractive` to write the data into the disk. + """ + + self.assertGreater(len(data), 0) + self.assertEqual(len(data) % SECTOR_SIZE, 0) + + temp_file =3D os.path.join(iotests.test_dir, "temp.bin") + with open(temp_file, "wb") as f: + f.write(data) + + self.qio.cmd(f"write -s {temp_file} {sector * SECTOR_SIZE} {len(da= ta)}") + + os.remove(temp_file) + + def init_fat16(self): + mbr =3D MBR(self.read_sectors(0)) + return Fat16( + mbr.partition_table[0]["start_lba"], + mbr.partition_table[0]["size"], + self.read_sectors, + self.write_sectors, + ) + + # Tests + + def test_fat_filesystem(self): + """ + Test that vvfat produce a valid FAT16 and MBR sectors + """ + mbr =3D MBR(self.read_sectors(0)) + + self.assertEqual(mbr.partition_table[0]["status"], 0x80) + self.assertEqual(mbr.partition_table[0]["type"], 6) + + fat16 =3D Fat16( + mbr.partition_table[0]["start_lba"], + mbr.partition_table[0]["size"], + self.read_sectors, + self.write_sectors, + ) + self.assertEqual(fat16.boot_sector.bytes_per_sector, 512) + self.assertEqual(fat16.boot_sector.volume_label, "QEMU VVFAT") + + def test_read_root_directory(self): + """ + Test the content of the root directory + """ + fat16 =3D self.init_fat16() + + root_dir =3D fat16.read_root_directory() + + self.assertEqual(len(root_dir), 13) # 12 + 1 special file + + files =3D { + "QEMU VVF.AT": 0, # special empty file + "FILE0.TXT": 16, + "FILE1.TXT": 16, + "FILE2.TXT": 16, + "FILE3.TXT": 16, + "FILE4.TXT": 16, + "FILE5.TXT": 16, + "FILE6.TXT": 16, + "FILE7.TXT": 16, + "FILE8.TXT": 16, + "FILE9.TXT": 16, + "LARGE1.TXT": 0x2000 * 2, + "LARGE2.TXT": 0x2000 * 3, + } + + for entry in root_dir: + self.assertIn(entry.whole_name(), files) + self.assertEqual(entry.size_bytes, files[entry.whole_name()]) + + def test_direntry_as_bytes(self): + """ + Test if we can convert Direntry back to bytes, so that we can writ= e it back to the disk safely. + """ + fat16 =3D self.init_fat16() + + root_dir =3D fat16.read_root_directory() + first_entry_bytes =3D fat16.read_sectors(fat16.boot_sector.root_di= r_start(), 1) + # The first entry won't be deleted, so we can compare it with the = first entry in the root directory + self.assertEqual(root_dir[0].as_bytes(), first_entry_bytes[:DIRENT= RY_SIZE]) + + def test_read_files(self): + """ + Test reading the content of the files + """ + fat16 =3D self.init_fat16() + + for i in range(10): + file =3D fat16.find_direntry(f"/FILE{i}.TXT") + self.assertIsNotNone(file) + self.assertEqual( + fat16.read_file(file), f"Hello, world! {i}\n".encode("asci= i") + ) + + # test large files + large1 =3D fat16.find_direntry("/LARGE1.TXT") + with open(os.path.join(filesystem, "large1.txt"), "rb") as f: + self.assertEqual(fat16.read_file(large1), f.read()) + + large2 =3D fat16.find_direntry("/LARGE2.TXT") + self.assertIsNotNone(large2) + with open(os.path.join(filesystem, "large2.txt"), "rb") as f: + self.assertEqual(fat16.read_file(large2), f.read()) + + def test_write_file_same_content_direct(self): + """ + Similar to `test_write_file_in_same_content`, but we write the fil= e directly clusters + and thus we don't go through the modification of direntry. + """ + fat16 =3D self.init_fat16() + + file =3D fat16.find_direntry("/FILE0.TXT") + self.assertIsNotNone(file) + + data =3D fat16.read_cluster(file.cluster) + fat16.write_cluster(file.cluster, data) + + with open(os.path.join(filesystem, "file0.txt"), "rb") as f: + self.assertEqual(fat16.read_file(file), f.read()) + + def test_write_file_in_same_content(self): + """ + Test writing the same content to the file back to it + """ + fat16 =3D self.init_fat16() + + file =3D fat16.find_direntry("/FILE0.TXT") + self.assertIsNotNone(file) + + self.assertEqual(fat16.read_file(file), b"Hello, world! 0\n") + + fat16.write_file(file, b"Hello, world! 0\n") + self.assertEqual(fat16.read_file(file), b"Hello, world! 0\n") + + with open(os.path.join(filesystem, "file0.txt"), "rb") as f: + self.assertEqual(f.read(), b"Hello, world! 0\n") + + def test_modify_content_same_clusters(self): + """ + Test modifying the content of the file without changing the number= of clusters + """ + fat16 =3D self.init_fat16() + + file =3D fat16.find_direntry("/FILE0.TXT") + self.assertIsNotNone(file) + + new_content =3D b"Hello, world! Modified\n" + self.assertEqual(fat16.read_file(file), b"Hello, world! 0\n") + + fat16.write_file(file, new_content) + self.assertEqual(fat16.read_file(file), new_content) + + with open(os.path.join(filesystem, "file0.txt"), "rb") as f: + self.assertEqual(f.read(), new_content) + + def test_truncate_file_same_clusters_less(self): + """ + Test truncating the file without changing number of clusters + Test decreasing the file size + """ + fat16 =3D self.init_fat16() + + file =3D fat16.find_direntry("/FILE0.TXT") + self.assertIsNotNone(file) + + self.assertEqual(fat16.read_file(file), b"Hello, world! 0\n") + + fat16.truncate_file(file, 5) + new_content =3D fat16.read_file(file) + self.assertEqual(new_content, b"Hello") + + with open(os.path.join(filesystem, "file0.txt"), "rb") as f: + self.assertEqual(f.read(), new_content) + + def test_truncate_file_same_clusters_more(self): + """ + Test truncating the file without changing number of clusters + Test increase the file size + """ + fat16 =3D self.init_fat16() + + file =3D fat16.find_direntry("/FILE0.TXT") + self.assertIsNotNone(file) + + self.assertEqual(fat16.read_file(file), b"Hello, world! 0\n") + + fat16.truncate_file(file, 20) + new_content =3D fat16.read_file(file) + self.assertIsNotNone(new_content) + + # random pattern will be appended to the file, and its not always = the same + self.assertEqual(new_content[:16], b"Hello, world! 0\n") + self.assertEqual(len(new_content), 20) + + with open(os.path.join(filesystem, "file0.txt"), "rb") as f: + self.assertEqual(f.read(), new_content) + + def test_write_large_file(self): + """ + Test writing a large file + """ + fat16 =3D self.init_fat16() + + file =3D fat16.find_direntry("/LARGE1.TXT") + self.assertIsNotNone(file) + + # The content of LARGE1 is A * 1KB, B * 1KB, C * 1KB, ..., P * 1KB + # Lets change it to be Z * 1KB, Y * 1KB, X * 1KB, ..., K * 1KB + # without changing the number of clusters or filesize + new_content =3D b"".join([bytes([0x5A - i] * 1024) for i in range(= 16)]) + fat16.write_file(file, new_content) + self.assertEqual(fat16.read_file(file), new_content) + + with open(os.path.join(filesystem, "large1.txt"), "rb") as f: + self.assertEqual(f.read(), new_content) + + def test_truncate_file_change_clusters_less(self): + """ + Test truncating a file by reducing the number of clusters + """ + fat16 =3D self.init_fat16() + + file =3D fat16.find_direntry("/LARGE1.TXT") + self.assertIsNotNone(file) + + fat16.truncate_file(file, 1) + self.assertEqual(fat16.read_file(file), b"A") + + with open(os.path.join(filesystem, "large1.txt"), "rb") as f: + self.assertEqual(f.read(), b"A") + + def test_write_file_change_clusters_less(self): + """ + Test truncating a file by reducing the number of clusters + """ + fat16 =3D self.init_fat16() + + file =3D fat16.find_direntry("/LARGE2.TXT") + self.assertIsNotNone(file) + + new_content =3D b"X" * 8 * 1024 + b"Y" * 8 * 1024 + fat16.write_file(file, new_content) + self.assertEqual(fat16.read_file(file), new_content) + + with open(os.path.join(filesystem, "large2.txt"), "rb") as f: + self.assertEqual(f.read(), new_content) + + def test_write_file_change_clusters_more(self): + """ + Test truncating a file by increasing the number of clusters + """ + fat16 =3D self.init_fat16() + + file =3D fat16.find_direntry("/LARGE2.TXT") + self.assertIsNotNone(file) + + # from 3 clusters to 4 clusters + new_content =3D ( + b"W" * 8 * 1024 + b"X" * 8 * 1024 + b"Y" * 8 * 1024 + b"Z" * 8= * 1024 + ) + fat16.write_file(file, new_content) + self.assertEqual(fat16.read_file(file), new_content) + + with open(os.path.join(filesystem, "large2.txt"), "rb") as f: + self.assertEqual(f.read(), new_content) + + def test_write_file_change_clusters_more_non_contiguous_2_mappings(sel= f): + """ + Test truncating a file by increasing the number of clusters + Here we allocate the new clusters in a way that makes them non-con= tiguous + so that we will get 2 cluster mappings for the file + """ + fat16 =3D self.init_fat16() + + file =3D fat16.find_direntry("/LARGE1.TXT") + self.assertIsNotNone(file) + + # from 2 clusters to 3 clusters with non-contiguous allocation + fat16.truncate_file(file, 3 * 0x2000, allocate_non_continuous=3DTr= ue) + new_content =3D b"X" * 8 * 1024 + b"Y" * 8 * 1024 + b"Z" * 8 * 1024 + fat16.write_file(file, new_content) + self.assertEqual(fat16.read_file(file), new_content) + + with open(os.path.join(filesystem, "large1.txt"), "rb") as f: + self.assertEqual(f.read(), new_content) + + def test_write_file_change_clusters_more_non_contiguous_3_mappings(sel= f): + """ + Test truncating a file by increasing the number of clusters + Here we allocate the new clusters in a way that makes them non-con= tiguous + so that we will get 3 cluster mappings for the file + """ + fat16 =3D self.init_fat16() + + file =3D fat16.find_direntry("/LARGE1.TXT") + self.assertIsNotNone(file) + + # from 2 clusters to 4 clusters with non-contiguous allocation + fat16.truncate_file(file, 4 * 0x2000, allocate_non_continuous=3DTr= ue) + new_content =3D ( + b"W" * 8 * 1024 + b"X" * 8 * 1024 + b"Y" * 8 * 1024 + b"Z" * 8= * 1024 + ) + fat16.write_file(file, new_content) + self.assertEqual(fat16.read_file(file), new_content) + + with open(os.path.join(filesystem, "large1.txt"), "rb") as f: + self.assertEqual(f.read(), new_content) + + def test_create_file(self): + """ + Test creating a new file + """ + fat16 =3D self.init_fat16() + + new_file =3D fat16.create_file("/NEWFILE.TXT") + + self.assertIsNotNone(new_file) + self.assertEqual(new_file.size_bytes, 0) + + new_content =3D b"Hello, world! New file\n" + fat16.write_file(new_file, new_content) + self.assertEqual(fat16.read_file(new_file), new_content) + + with open(os.path.join(filesystem, "newfile.txt"), "rb") as f: + self.assertEqual(f.read(), new_content) + + # TODO: support deleting files + + +if __name__ =3D=3D "__main__": + # This is a specific test for vvfat driver + iotests.main(supported_fmts=3D["vvfat"], supported_protocols=3D["file"= ]) diff --git a/tests/qemu-iotests/tests/vvfat.out b/tests/qemu-iotests/tests/= vvfat.out new file mode 100755 index 0000000000..b6f257674e --- /dev/null +++ b/tests/qemu-iotests/tests/vvfat.out @@ -0,0 +1,5 @@ +................ +---------------------------------------------------------------------- +Ran 16 tests + +OK --=20 2.45.2