From nobody Fri May 10 20:05:39 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=linaro.org ARC-Seal: i=1; a=rsa-sha256; t=1709910505; cv=none; d=zohomail.com; s=zohoarc; b=AikTm1G0Z+spdXwMq85AUfAsha94IsImU9gWZZCWe36zMOlGXNf4NVfWTBakdtN6Z42Nud9I+UHCAP8CoNSLmlZn7yEclRHSPCKNMorz+/TfN3Uf69QJabhX8nh/ZAZgqikR+vnaFydxcAtPR8bwJu9paRoie1DMMmjW+AWV/ao= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1709910505; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=DXkqpQ2d+nXcY4e+AOIltUbvXiT1OvsvmOZuFvpyCvA=; b=hr9U4aEcXTcBjmUGgupe6rFKUjyEiurnxEY9+ZUe9ZQsYbDqi+gQBH5aeU5euZiv5kdQTYO5MYE+UoGxzldxMqgvPG79VCXTn1boLDe/l1EBxUw1iVVu7NueCFUw450X5in1OwY7F8sVqOTFHjlWxqLeY1jeHW989y5IVAwQO/E= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1709910505007438.9431336412923; Fri, 8 Mar 2024 07:08:25 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ribpI-0000wu-1l; Fri, 08 Mar 2024 10:08:00 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ribpF-0000qu-Lq for qemu-devel@nongnu.org; Fri, 08 Mar 2024 10:07:57 -0500 Received: from mail-wm1-x332.google.com ([2a00:1450:4864:20::332]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ribpC-0000sR-QX for qemu-devel@nongnu.org; Fri, 08 Mar 2024 10:07:57 -0500 Received: by mail-wm1-x332.google.com with SMTP id 5b1f17b1804b1-412e6ba32easo6924045e9.0 for ; Fri, 08 Mar 2024 07:07:54 -0800 (PST) Received: from draig.lan ([85.9.250.243]) by smtp.gmail.com with ESMTPSA id bw1-20020a0560001f8100b0033d6bc17d0esm23686459wrb.74.2024.03.08.07.07.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 Mar 2024 07:07:52 -0800 (PST) Received: from draig.lan (localhost [IPv6:::1]) by draig.lan (Postfix) with ESMTP id 36BD15F753; Fri, 8 Mar 2024 15:07:52 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1709910473; x=1710515273; darn=nongnu.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=DXkqpQ2d+nXcY4e+AOIltUbvXiT1OvsvmOZuFvpyCvA=; b=NVkNYpdQcRWh++rJDaqXkRVvTptyPFVnZKrqgmsPCedfc88zaWYB9TyayBL0l6g93W TY8fdtksTk/tC1HIJv75UWOMbVxENidwx8Ava6gyL2TkxUzLnhyxh0UnjynmIaeM7XVt MxF6cRlNNakXESq8ojThM1uy+Te+aUsHAV/+cCQef+5qa5WhRy+XoOBl/5hhb4zLchAi Dqa9VQUm1nDrf7RRvXuKLe5n78LL/13CJGsHcvxoX7PqqCMmHq7T+Y6UEWgsVXo1sEfz 7aqi8KkcoaHDuBMPUpMELQvkxPHw18sMuaekW3/9TtJkHXSKjXF/zLAIopd5eY5Vx+do Ky1A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709910473; x=1710515273; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=DXkqpQ2d+nXcY4e+AOIltUbvXiT1OvsvmOZuFvpyCvA=; b=ZMh2RaC/UARQ2UGlyp3qt6DGmm6J7i9ETn/69p2DhCjTS+9GijcE67aqC8ESt6MLUb kQpezyfglrYODJ8NlhsAE6KDqJodc3ciq7ZPfwL4AyIz+q6raOV2U8cf/fQwdPlplQle tR1bTu42jvYLkt0HXbLx0P0Tdn8UQk9FnmCjUSWeFwyAa5WaUcAIYcblWlKptU02J4Y6 ATxzUTumWxppAGSDvrJrzxriqhGjTY+8TWtOcZlUkbTdgYBcVMGG7EATkvULwQqzwPj9 yLxgGx8cLlI1A1BHzsDbFl0SGeocdu70GfoULt0/oB9gcvQF+/A4vnzK1pdM9fF843tj 58hw== X-Gm-Message-State: AOJu0YwD3AZ+sRyfheGz8prwc9UVpLd0PrCQxUKzSiWpQp/SHj5eYyRP hlyoP9Fe/493ewVAnXZoG7xAUYz6l3hZsN7vZvnez3Kpc2QdFBu4eETt3CfxqZ4= X-Google-Smtp-Source: AGHT+IF5cbysXBuu65fq32gdCih+C/ghQUW3kuOOAdJ3TM0rwyuj3dIy+wELybNOyPK9fvXUDHc2CA== X-Received: by 2002:adf:a34a:0:b0:33e:7b09:8a66 with SMTP id d10-20020adfa34a000000b0033e7b098a66mr305155wrb.28.1709910473069; Fri, 08 Mar 2024 07:07:53 -0800 (PST) From: =?UTF-8?q?Alex=20Benn=C3=A9e?= To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Thomas Huth , Wainer dos Santos Moschetta , Beraldo Leal , Yonggang Luo Subject: [RFC PATCH] gitlab: aggressively avoid extra GIT data Date: Fri, 8 Mar 2024 15:07:48 +0000 Message-Id: <20240308150748.894699-1-alex.bennee@linaro.org> X-Mailer: git-send-email 2.39.2 MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::332; envelope-from=alex.bennee@linaro.org; helo=mail-wm1-x332.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @linaro.org) X-ZM-MESSAGEID: 1709910506618100003 This avoids fetching blobs and tree references for branches we are not going to worry about. Also skip tag references which are similarly not useful and keep the default --prune. This keeps the .git data to around 100M rather than the ~400M even a shallow clone takes. So we can check the savings we also run a quick du while setting up the build. We also have to have special settings of GIT_FETCH_EXTRA_FLAGS for the Windows build (git too old?) and the migration legacy test where we build an older QEMU alongside the main one. Signed-off-by: Alex Benn=C3=A9e --- .gitlab-ci.d/base.yml | 4 ++++ .gitlab-ci.d/buildtest-template.yml | 1 + .gitlab-ci.d/buildtest.yml | 2 ++ .gitlab-ci.d/windows.yml | 2 ++ 4 files changed, 9 insertions(+) diff --git a/.gitlab-ci.d/base.yml b/.gitlab-ci.d/base.yml index ef173a34e6..7cb800fc3b 100644 --- a/.gitlab-ci.d/base.yml +++ b/.gitlab-ci.d/base.yml @@ -24,6 +24,10 @@ variables: # Each script line from will be in a collapsible section in the job ou= tput # and show the duration of each line. FF_SCRIPT_SECTIONS: 1 + # The project has a fairly fat GIT repo so we try and avoid bringing i= n things + # we don't need. The --filter options avoid blobs and tree references = we aren't going to use + # and we also avoid fetching tags. + GIT_FETCH_EXTRA_FLAGS: --filter=3Dblob:none --filter=3Dtree:0 --no-tag= s --prune =20 interruptible: true =20 diff --git a/.gitlab-ci.d/buildtest-template.yml b/.gitlab-ci.d/buildtest-t= emplate.yml index 4fbfeb6667..22045add80 100644 --- a/.gitlab-ci.d/buildtest-template.yml +++ b/.gitlab-ci.d/buildtest-template.yml @@ -14,6 +14,7 @@ - export CCACHE_DIR=3D"$CCACHE_BASEDIR/ccache" - export CCACHE_MAXSIZE=3D"500M" - export PATH=3D"$CCACHE_WRAPPERSDIR:$PATH" + - du -sh .git - mkdir build - cd build - ccache --zero-stats diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml index a1c030337b..def469a9dc 100644 --- a/.gitlab-ci.d/buildtest.yml +++ b/.gitlab-ci.d/buildtest.yml @@ -187,6 +187,8 @@ build-previous-qemu: variables: IMAGE: opensuse-leap TARGETS: x86_64-softmmu aarch64-softmmu + # Override the default flags as we need more to grab the old version + GIT_FETCH_EXTRA_FLAGS: --prune before_script: - export QEMU_PREV_VERSION=3D"$(sed 's/\([0-9.]*\)\.[0-9]*/v\1.0/' VER= SION)" - git remote add upstream https://gitlab.com/qemu-project/qemu diff --git a/.gitlab-ci.d/windows.yml b/.gitlab-ci.d/windows.yml index f116b8012d..f9da745042 100644 --- a/.gitlab-ci.d/windows.yml +++ b/.gitlab-ci.d/windows.yml @@ -28,6 +28,8 @@ msys2-64bit: # qTests don't run successfully with "--without-default-devices", # so let's exclude the qtests from CI for now. TEST_ARGS: --no-suite qtest + # The Windows git is a bit older so override the default + GIT_FETCH_EXTRA_FLAGS: --no-tags --prune artifacts: name: "$CI_JOB_NAME-$CI_COMMIT_REF_SLUG" expire_in: 7 days --=20 2.39.2