From nobody Sat Nov 23 18:22:27 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1732153853; cv=none; d=zohomail.com; s=zohoarc; b=DBx7Mx7IfTs+2LtdBcg8mkoh8kl/D1HQ3XC//lBSmPSK0k+oLUSYj4icGXXpZayBlAH6cLYjyByFIUg7AD/9MxNwkmUHxKnMicGT1co3njdTHG/5KhvidD21O1ir7DeLxmZT1C0H7ceGyuB6wPnr47nvR134eo2ZU0JIT/kWAiw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1732153853; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Reply-To:Reply-To:References:Sender:Subject:Subject:To:To:Message-Id; bh=CNnUELic1Nay7/rcgzfoEkgqXL52+S53mtLCKeakdMg=; b=BZE9MuWVJd+pjKLjAqCm9p4l+IA8Xq+Ug+S5iLw3N7OiKzv/PQnGitlzDy7iBmRtj3F3oeUoPFbrexIfDFu1fcHuQQD2SKFvOThdVBG/9MQHGQVgoGYjd0zaecetUJIJr4wfUGN9CauD6LULkKVlTFQSnMxVC6Iz7emXwNRiKeg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 17321538539701006.1968031003778; Wed, 20 Nov 2024 17:50:53 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tDwJx-0006lJ-N1; Wed, 20 Nov 2024 20:49:25 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tDwJk-0006Pr-Ki for qemu-devel@nongnu.org; Wed, 20 Nov 2024 20:49:13 -0500 Received: from rev.ng ([94.130.142.21]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tDwJf-0004qR-Il for qemu-devel@nongnu.org; Wed, 20 Nov 2024 20:49:12 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=rev.ng; s=dkim; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive:List-Unsubscribe:List-Unsubscribe-Post: List-Help; bh=CNnUELic1Nay7/rcgzfoEkgqXL52+S53mtLCKeakdMg=; b=tAvx0ko0bfoqDEF E/dNsUFouD+MiVBEgIJk2SaM8q1C3NI5zZEPPGyI0XgMwcCv7Fn/40YPKNo/JukrGBLIJz1idm2m1 GgT7TweXFlZrkWjS5VTnUC8xcgKt+KTxrw8aO0n4RM3QsgSCYAoylt4S5dYFtQ9T3fAoS6Aa4/tNn OE=; To: qemu-devel@nongnu.org Cc: ale@rev.ng, ltaylorsimpson@gmail.com, bcain@quicinc.com, richard.henderson@linaro.org, philmd@linaro.org, alex.bennee@linaro.org Subject: [RFC PATCH v1 31/43] helper-to-tcg: Introduce TcgGenPass Date: Thu, 21 Nov 2024 02:49:35 +0100 Message-ID: <20241121014947.18666-32-anjo@rev.ng> In-Reply-To: <20241121014947.18666-1-anjo@rev.ng> References: <20241121014947.18666-1-anjo@rev.ng> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=94.130.142.21; envelope-from=anjo@rev.ng; helo=rev.ng X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-to: Anton Johansson From: Anton Johansson via Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1732153854920116600 Content-Type: text/plain; charset="utf-8" Adds a backend pass, taking previously optimized and canonicalized LLVM IR and for each function: 1. Runs the TcgV register allocator; 2. Iterates over instructions and calls appropriate functions in TcgEmit.h to emit TCG code. Signed-off-by: Anton Johansson --- .../helper-to-tcg/include/CmdLineOptions.h | 6 + subprojects/helper-to-tcg/meson.build | 1 + .../passes/backend/TcgGenPass.cpp | 1812 +++++++++++++++++ .../helper-to-tcg/passes/backend/TcgGenPass.h | 57 + .../helper-to-tcg/pipeline/Pipeline.cpp | 49 + 5 files changed, 1925 insertions(+) create mode 100644 subprojects/helper-to-tcg/passes/backend/TcgGenPass.cpp create mode 100644 subprojects/helper-to-tcg/passes/backend/TcgGenPass.h diff --git a/subprojects/helper-to-tcg/include/CmdLineOptions.h b/subprojec= ts/helper-to-tcg/include/CmdLineOptions.h index f59b700914..3787fbbaec 100644 --- a/subprojects/helper-to-tcg/include/CmdLineOptions.h +++ b/subprojects/helper-to-tcg/include/CmdLineOptions.h @@ -30,3 +30,9 @@ extern llvm::cl::opt GuestPtrSize; // Options for TcgEmit extern llvm::cl::opt MmuIndexFunction; extern llvm::cl::opt TempVectorBlock; +// Options for TcgGenPass +extern llvm::cl::opt OutputSourceFile; +extern llvm::cl::opt OutputHeaderFile; +extern llvm::cl::opt OutputEnabledFile; +extern llvm::cl::opt OutputLogFile; +extern llvm::cl::opt ErrorOnTranslationFailure; diff --git a/subprojects/helper-to-tcg/meson.build b/subprojects/helper-to-= tcg/meson.build index 55a177bd94..4f045eb1da 100644 --- a/subprojects/helper-to-tcg/meson.build +++ b/subprojects/helper-to-tcg/meson.build @@ -52,6 +52,7 @@ sources =3D [ 'passes/PrepareForTcgPass/IdentityMap.cpp', 'passes/backend/TcgTempAllocationPass.cpp', 'passes/backend/TcgEmit.cpp', + 'passes/backend/TcgGenPass.cpp', ] =20 clang =3D bindir / 'clang' diff --git a/subprojects/helper-to-tcg/passes/backend/TcgGenPass.cpp b/subp= rojects/helper-to-tcg/passes/backend/TcgGenPass.cpp new file mode 100644 index 0000000000..81adb42a5d --- /dev/null +++ b/subprojects/helper-to-tcg/passes/backend/TcgGenPass.cpp @@ -0,0 +1,1812 @@ +// +// Copyright(c) 2024 rev.ng Labs Srl. All Rights Reserved. +// +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, see . +// + +#include "TcgGenPass.h" +#include "CmdLineOptions.h" +#include "Error.h" +#include "FunctionAnnotation.h" +#include "PseudoInst.h" +#include "TcgEmit.h" +#include "TcgTempAllocationPass.h" +#include "TcgType.h" +#include "llvm-compat.h" + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +// For std::swap +#include + +using namespace llvm; + +// Wrapper class around a TcgV to cast it to/from 32-/64-bit +class TcgSizeAdapter +{ + raw_ostream &Out; + const TcgV Orig; + Optional Adapted; + + public: + TcgSizeAdapter(raw_ostream &Out, const TcgV Orig) : Out(Out), Orig(Ori= g) {} + + const TcgV get(uint32_t Size) + { + if (Orig.Kind =3D=3D IrImmediate or (Orig.TcgSize =3D=3D Size)) { + return Orig; + } else if (!Adapted.hasValue()) { + initAdapted(Size); + } + return *Adapted; + } + + private: + void initAdapted(uint32_t Size) + { + assert(!Adapted.hasValue()); + assert((Size =3D=3D 32 and Orig.TcgSize =3D=3D 64) or + (Size =3D=3D 64 and Orig.TcgSize =3D=3D 32)); + + Adapted =3D TcgV::makeTemp(Size, Orig.LlvmSize, Orig.Kind); + tcg::defineNewTemp(Out, *Adapted); + if (Size =3D=3D 32) { + tcg::genExtrlI64I32(Out, *Adapted, Orig); + } else { + tcg::genExtuI32I64(Out, *Adapted, Orig); + } + } +}; + +class Mapper +{ + raw_ostream &Out; + llvm::DenseMap Map; + llvm::DenseMap Labels; + + // Keep track of whether a TcgV has been defined already, or not + SmallBitVector HasBeenDefined; + + const TempAllocationData &TAD; + + public: + Mapper(raw_ostream &Out, const TcgGlobalMap &TcgGlobals, const Module = &M, + const TempAllocationData &TAD) + : Out(Out), TAD(TAD) + { + // Default to size of previously mapped TcgVs + HasBeenDefined.resize(TAD.Map.size()); + } + + Expected getMapped(const Value *V) + { + auto It =3D Map.find(V); + if (It !=3D Map.end()) { + return It->second; + } + return mkError("Value not mapped"); + } + + TcgV mapBbAndEmit(BasicBlock *BB) + { + auto Find =3D Labels.find(BB); + if (Find =3D=3D Labels.end()) { + TcgV Label =3D TcgV::makeLabel(); + tcg::defineNewTemp(Out, Label); + return Labels.try_emplace(BB, Label).first->second; + } + return Find->second; + } + + void mapExplicitly(Value *Val, const TcgV &TcgVal) + { + assert(Map.find(Val) =3D=3D Map.end()); + Map.try_emplace(Val, TcgVal); + } + + void mapClear(Value *Val) + { + auto It =3D Map.find(Val); + assert(It !=3D Map.end()); + Map.erase(It); + } + + Expected mapAndEmit(const Value *V) + { + auto Mapped =3D getMapped(V); + if (Mapped) { + return Mapped.get(); + } + + auto It =3D TAD.Map.find(V); + if (It =3D=3D TAD.Map.end()) { + return mkError("Unable to map value: ", V); + } + + const TcgV Tcg =3D It->second; + + bool IsArg =3D TAD.Args.ArgInfoMap.find(V) !=3D TAD.Args.ArgInfoMa= p.end(); + + if (Tcg.Id >=3D HasBeenDefined.size()) { + HasBeenDefined.resize(Tcg.Id + 1); + } + + if (!IsArg and !HasBeenDefined[Tcg.Id] and + (!TAD.ReturnValue.hasValue() or Tcg !=3D *TAD.ReturnValue) and + Tcg.Kind !=3D IrImmediate and Tcg.Kind !=3D IrConst) { + HasBeenDefined.set(Tcg.Id); + tcg::defineNewTemp(Out, Tcg); + } + + // Logic for emitted TCG corresponding to constant LLVM vectors, t= wo + // cases are handled, splatted values + // + // + // + // and vectors where elements differ + // + // + // + // For the latter case, attemt to emit it as a constant splatted + // vector with a larger size by combining adjacent elements. This + // is an optimization as initialzing a constant vector with differ= ent + // elements is expensive compared to splatting. + auto ConstV =3D dyn_cast(V); + if (ConstV and V->getType()->isVectorTy()) { + Constant *Splat =3D ConstV->getSplatValue(); + if (Splat) { + // Constant splatted vector + auto It =3D TAD.Map.find(Splat); + assert(It !=3D TAD.Map.end()); + auto Size =3D TcgV::makeImmediate( + Twine(vectorSizeInBytes(Tcg)).str(), 64, 64); + tcg::genVecMemset(Out, Tcg, It->second, Size); + } else { + // Constant non-splatted vector, attempt to combine elemen= ts + // to make it splattable. + SmallVector Ints; + + // Copy over elements to a vector + for (unsigned I =3D 0; I < Tcg.VectorElementCount; ++I) { + Constant *Element =3D ConstV->getAggregateElement(I); + uint64_t Value =3D Element->getUniqueInteger().getZExt= Value(); + Ints.push_back(Value); + } + + // When combining adjacent elements, the maximum size supp= orted + // by TCG is 64-bit. MaxNumElements is the maximum amount= of + // elements to attempt to merge + size_t PatternLen =3D 0; + unsigned MaxNumElements =3D 8 * sizeof(uint64_t) / Tcg.Llv= mSize; + for (unsigned N =3D MaxNumElements; N > 1; N /=3D 2) { + // Attempt to combine N elements by checking if the fi= rst + // N elements tile the vector. + bool Match =3D true; + for (unsigned J =3D 0; J < Tcg.VectorElementCount; ++J= ) { + if (Ints[J % N] !=3D Ints[J]) { + Match =3D false; + break; + } + } + // If tiling succeeded, break out + if (Match) { + PatternLen =3D N; + break; + } + } + + if (PatternLen > 0) { + // Managed to tile vector with splattable element, com= pute + // final splattable value + uint64_t Value =3D 0; + for (unsigned I =3D 0; I < PatternLen; ++I) { + Value |=3D Ints[I] << I * Tcg.LlvmSize; + } + auto Splat =3D + TcgV::makeImmediate(Twine(Value).str(), 64, 64); + auto Size =3D TcgV::makeImmediate( + Twine(vectorSizeInBytes(Tcg)).str(), 64, 64); + tcg::genVecMemset(Out, Tcg, Splat, Size); + } else { + // Tiling failed, fall back to emitting an array copy = from + // C to a gvec vector. + SmallVector Arr; + for (unsigned I =3D 0; I < Tcg.VectorElementCount; ++I= ) { + Constant *Element =3D ConstV->getAggregateElement(= I); + auto It =3D TAD.Map.find(Element); + assert(It !=3D TAD.Map.end()); + Arr.push_back(It->second); + } + tcg::genVecArrSplat(Out, Tcg, Arr); + } + } + } + + return Map.try_emplace(V, It->second).first->second; + } + + Expected mapCondAndEmit(Value *V, uint32_t TcgSize, uint32_t Llv= mSize) + { + auto Mapped =3D getMapped(V); + if (Mapped) { + assert(Mapped.get().LlvmSize =3D=3D 1); + return Mapped.get(); + } + + auto It =3D TAD.Map.find(const_cast(V)); + if (It =3D=3D TAD.Map.end()) { + return mkError("Unable to map cond: ", V); + } + + const TcgV Tcg =3D It->second; + if (Tcg.Id >=3D HasBeenDefined.size()) { + HasBeenDefined.resize(Tcg.Id + 1); + } + if (!HasBeenDefined[Tcg.Id] and + (!TAD.ReturnValue.hasValue() or Tcg !=3D *TAD.ReturnValue)) { + HasBeenDefined.set(Tcg.Id); + tcg::defineNewTemp(Out, Tcg); + } + return Map.try_emplace(V, It->second).first->second; + } +}; + +struct TranslatedFunction { + std::string Name; + std::string Decl; + std::string Code; + std::string DispatchCode; + bool IsHelper; +}; + +static void ensureSignBitIsSet(raw_ostream &Out, const TcgV &V) +{ + if (V.LlvmSize =3D=3D V.TcgSize or V.Kind !=3D IrValue) { + return; + } + tcg::genExtract(Out, true, V, V, + TcgV::makeImmediate("0", V.TcgSize, V.LlvmSize), + TcgV::makeImmediate(Twine((int)V.LlvmSize).str(), V.Tc= gSize, + V.LlvmSize)); +} + +static Expected mapCallReturnValue(Mapper &Mapper, CallInst *Call) +{ + // Only map return value if it has > 0 uses. Destination values of ca= ll + // instructions are the only ones which LLVM will not remove if unused. + if (Call->getType()->isVoidTy() or Call->getNumUses() =3D=3D 0) { + return mkError("Invalid return type", Call); + } + return Mapper.mapAndEmit(Call); +} + +static Instruction::BinaryOps mapPseudoInstToOpcode(PseudoInst Inst) +{ + switch (Inst) { + case VecAddScalar: + case VecAddStore: + case VecAddScalarStore: + return Instruction::Add; + case VecSubScalar: + case VecSubStore: + case VecSubScalarStore: + return Instruction::Sub; + case VecMulScalar: + case VecMulStore: + case VecMulScalarStore: + return Instruction::Mul; + case VecXorScalar: + case VecXorStore: + case VecXorScalarStore: + return Instruction::Xor; + case VecOrScalar: + case VecOrStore: + case VecOrScalarStore: + return Instruction::Or; + case VecAndScalar: + case VecAndStore: + case VecAndScalarStore: + return Instruction::And; + case VecShlScalar: + case VecShlStore: + case VecShlScalarStore: + return Instruction::Shl; + case VecLShrScalar: + case VecLShrStore: + case VecLShrScalarStore: + return Instruction::LShr; + case VecAShrScalar: + case VecAShrStore: + case VecAShrScalarStore: + return Instruction::AShr; + default: + abort(); + } +} + +static bool translatePseudoInstCall(raw_ostream &Out, CallInst *Call, + PseudoInst PInst, + const SmallVector &Args, + Mapper &Mapper, + const TcgGlobalMap &TcgGlobals) +{ + switch (PInst) { + case IdentityMap: { + Mapper.mapExplicitly(Call, Args[0]); + } break; + case PtrAdd: { + if (Args[0].Kind =3D=3D IrPtr or Args[0].Kind =3D=3D IrEnv) { + Expected MaybeRes =3D mapCallReturnValue(Mapper, Call); + if (!MaybeRes) { + return false; + } + tcg::genAddPtr(Out, *MaybeRes, Args[0], Args[1]); + } else if ((Args[0].Kind =3D=3D IrImmediate or Args[0].Kind =3D=3D= IrConst) and + (Args[1].Kind =3D=3D IrConst or Args[1].Kind =3D=3D IrI= mmediate)) { + Mapper.mapExplicitly(Call, c::ptrAdd(Args[0], Args[1])); + } else if (Args[0].Kind =3D=3D IrPtrToOffset and + (Args[1].Kind =3D=3D IrConst or Args[1].Kind =3D=3D IrI= mmediate)) { + Mapper.mapExplicitly(Call, c::ptrAdd(Args[0], Args[1])); + } else { + // ptradd on vector types requires immediate offset + return false; + } + } break; + case AccessGlobalArray: { + auto Offset =3D cast(Call->getArgOperand(0))->getZExt= Value(); + auto It =3D TcgGlobals.find(Offset); + assert(It !=3D TcgGlobals.end()); + TcgGlobal Global =3D It->second; + uint32_t LlvmSize =3D Global.Size; + uint32_t TcgSize =3D llvmToTcgSize(LlvmSize); + if (Args[1].Kind !=3D IrImmediate) { + // globalArray access with non-immediate index + return false; + } + auto Code =3D Global.Code.str() + "[" + tcg::getName(Args[1]) + "]= "; + auto Tcg =3D + TcgV::makeConstantExpression(Code, TcgSize, LlvmSize, IrValue); + Mapper.mapExplicitly(Call, Tcg); + } break; + case AccessGlobalValue: { + auto Offset =3D cast(Call->getArgOperand(0))->getZExt= Value(); + auto It =3D TcgGlobals.find(Offset); + assert(It !=3D TcgGlobals.end()); + TcgGlobal Global =3D It->second; + auto LlvmSize =3D Global.Size; + auto TcgSize =3D llvmToTcgSize(LlvmSize); + auto Tcg =3D TcgV::makeConstantExpression(Global.Code.str(), TcgSi= ze, + LlvmSize, IrValue); + Mapper.mapExplicitly(Call, Tcg); + } break; + case Brcond: { + auto LlvmPred =3D static_cast( + cast(Call->getOperand(0))->getZExtValue()); + tcg::genBrcond(Out, LlvmPred, Args[1], Args[2], Args[3]); + if (!Call->hasMetadata("fallthrough")) { + tcg::genBr(Out, Args[4]); + } + } break; + case Movcond: { + Expected MaybeRes =3D mapCallReturnValue(Mapper, Call); + if (!MaybeRes) { + return false; + } + auto LlvmPred =3D static_cast( + cast(Call->getOperand(0))->getZExtValue()); + if (CmpInst::isSigned(LlvmPred)) { + ensureSignBitIsSet(Out, Args[1]); + ensureSignBitIsSet(Out, Args[2]); + } + tcg::genMovcond(Out, LlvmPred, *MaybeRes, Args[1], Args[2], Args[3= ], + Args[4]); + } break; + case VecSplat: { + Expected MaybeRes =3D mapCallReturnValue(Mapper, Call); + if (!MaybeRes) { + return false; + } + tcg::genVecSplat(Out, *MaybeRes, Args[0]); + } break; + case VecNot: { + Expected MaybeRes =3D mapCallReturnValue(Mapper, Call); + if (!MaybeRes) { + return false; + } + tcg::genVecNot(Out, *MaybeRes, Args[0]); + } break; + case VecNotStore: { + tcg::genVecNot(Out, Args[0], Args[1]); + } break; + case VecAddScalar: + case VecSubScalar: + case VecMulScalar: + case VecXorScalar: + case VecOrScalar: + case VecAndScalar: + case VecShlScalar: + case VecLShrScalar: + case VecAShrScalar: { + Expected MaybeRes =3D mapCallReturnValue(Mapper, Call); + if (!MaybeRes) { + return false; + } + auto Opcode =3D mapPseudoInstToOpcode(PInst); + tcg::genVecBinOp(Out, Opcode, *MaybeRes, Args[0], Args[1]); + } break; + case VecAddStore: + case VecSubStore: + case VecMulStore: + case VecXorStore: + case VecOrStore: + case VecAndStore: + case VecShlStore: + case VecLShrStore: + case VecAShrStore: + case VecAddScalarStore: + case VecSubScalarStore: + case VecMulScalarStore: + case VecXorScalarStore: + case VecOrScalarStore: + case VecAndScalarStore: + case VecShlScalarStore: + case VecLShrScalarStore: + case VecAShrScalarStore: { + auto Opcode =3D mapPseudoInstToOpcode(PInst); + tcg::genVecBinOp(Out, Opcode, Args[0], Args[1], Args[2]); + } break; + case VecSignedSatAddStore: { + tcg::genVecSignedSatAdd(Out, Args[0], Args[1], Args[2]); + } break; + case VecSignedSatSubStore: { + tcg::genVecSignedSatSub(Out, Args[0], Args[1], Args[2]); + } break; + case VecSelectStore: { + tcg::genVecBitsel(Out, Args[0], Args[1], Args[2], Args[3]); + } break; + case VecAbsStore: { + tcg::genAbs(Out, Args[0], Args[1]); + } break; + case VecSignedMaxStore: { + tcg::genVecSignedMax(Out, Args[0], Args[1], Args[2]); + } break; + case VecUnsignedMaxStore: { + tcg::genVecUnsignedMax(Out, Args[0], Args[1], Args[2]); + } break; + case VecSignedMinStore: { + tcg::genVecSignedMin(Out, Args[0], Args[1], Args[2]); + } break; + case VecUnsignedMinStore: { + tcg::genVecUnsignedMin(Out, Args[0], Args[1], Args[2]); + } break; + case VecTruncStore: { + tcg::genVecTrunc(Out, Args[0], Args[1]); + } break; + case VecCompare: { + Expected MaybeRes =3D mapCallReturnValue(Mapper, Call); + if (!MaybeRes) { + return false; + } + auto LlvmPred =3D static_cast( + cast(Call->getOperand(0))->getZExtValue()); + tcg::genVecCmp(Out, MaybeRes.get(), LlvmPred, Args[1], Args[2]); + } break; + case VecWideCondBitsel: { + Expected MaybeRes =3D mapCallReturnValue(Mapper, Call); + if (!MaybeRes) { + return false; + } + tcg::genVecBitsel(Out, MaybeRes.get(), Args[0], Args[1], Args[2]); + break; + } break; + case VecWideCondBitselStore: { + tcg::genVecBitsel(Out, Args[0], Args[1], Args[2], Args[3]); + break; + } break; + case GuestLoad: { + Expected MaybeRes =3D mapCallReturnValue(Mapper, Call); + if (!MaybeRes) { + return false; + } + uint8_t Sign =3D cast(Call->getOperand(1))->getZExtVa= lue(); + uint8_t Size =3D cast(Call->getOperand(2))->getZExtVa= lue(); + uint8_t Endianness =3D + cast(Call->getOperand(3))->getZExtValue(); + std::string MemOpStr =3D "MO_"; + raw_string_ostream MemOpStream(MemOpStr); + switch (Endianness) { + case 0: + break; // do nothing + case 1: + MemOpStream << "LE"; + break; + case 2: + MemOpStream << "BE"; + break; + default: + abort(); + } + switch (Sign) { + case 0: + MemOpStream << "U"; + break; + case 1: + MemOpStream << "S"; + break; + default: + abort(); + } + switch (Size) { + case 1: + MemOpStream << "B"; + break; + case 2: + MemOpStream << "W"; + break; + case 4: + MemOpStream << "L"; + break; + case 8: + MemOpStream << "Q"; + break; + default: + abort(); + } + tcg::genQemuLoad(Out, *MaybeRes, Args[0], MemOpStream.str().c_str(= )); + } break; + case GuestStore: { + uint8_t Size =3D cast(Call->getOperand(2))->getZExtVa= lue(); + uint8_t Endianness =3D + cast(Call->getOperand(3))->getZExtValue(); + std::string MemOpStr =3D "MO_"; + raw_string_ostream MemOpStream(MemOpStr); + switch (Endianness) { + case 0: + break; // do nothing + case 1: + MemOpStream << "LE"; + break; + case 2: + MemOpStream << "BE"; + break; + default: + abort(); + } + // Always unsigned for stores + MemOpStream << "U"; + switch (Size) { + case 1: + MemOpStream << "B"; + break; + case 2: + MemOpStream << "W"; + break; + case 4: + MemOpStream << "L"; + break; + case 8: + MemOpStream << "Q"; + break; + default: + abort(); + } + tcg::genQemuStore(Out, Args[0], Args[1], MemOpStream.str().c_str()= ); + } break; + case Exception: { + // Map and adapt arguments to the call + SmallVector IArgs; + for (auto Arg : Args) { + IArgs.push_back(tcg::materialize(Arg)); + } + tcg::genCallHelper(Out, "helper_raise_exception", IArgs.begin(), + IArgs.end()); + } break; + default: + // unmapped pseudo inst + return false; + } + return true; +} + +static bool translateIntrinsicCall(raw_ostream &Out, CallInst *Call, + Function *F, + const SmallVector &Args, + Mapper &Mapper) +{ + switch (F->getIntrinsicID()) { +#if LLVM_VERSION_MAJOR > 11 + case Intrinsic::abs: { + Expected MaybeRes =3D mapCallReturnValue(Mapper, Call); + if (!MaybeRes) { + return false; + } + tcg::genAbs(Out, *MaybeRes, Args[0]); + } break; + case Intrinsic::smax: { + Expected MaybeRes =3D mapCallReturnValue(Mapper, Call); + if (!MaybeRes) { + return false; + } + tcg::genVecSignedMax(Out, *MaybeRes, Args[0], Args[1]); + } break; + case Intrinsic::smin: { + Expected MaybeRes =3D mapCallReturnValue(Mapper, Call); + if (!MaybeRes) { + return false; + } + tcg::genVecSignedMin(Out, *MaybeRes, Args[0], Args[1]); + } break; + case Intrinsic::umax: { + Expected MaybeRes =3D mapCallReturnValue(Mapper, Call); + if (!MaybeRes) { + return false; + } + tcg::genVecUnsignedMax(Out, *MaybeRes, Args[0], Args[1]); + } break; + case Intrinsic::umin: { + Expected MaybeRes =3D mapCallReturnValue(Mapper, Call); + if (!MaybeRes) { + return false; + } + tcg::genVecUnsignedMin(Out, *MaybeRes, Args[0], Args[1]); + } break; +#endif + case Intrinsic::sadd_sat: { + Expected MaybeRes =3D mapCallReturnValue(Mapper, Call); + if (!MaybeRes) { + return false; + } + tcg::genVecSignedSatAdd(Out, *MaybeRes, Args[0], Args[1]); + } break; + case Intrinsic::ssub_sat: { + Expected MaybeRes =3D mapCallReturnValue(Mapper, Call); + if (!MaybeRes) { + return false; + } + tcg::genVecSignedSatSub(Out, *MaybeRes, Args[0], Args[1]); + } break; + case Intrinsic::ctlz: { + Expected MaybeRes =3D mapCallReturnValue(Mapper, Call); + if (!MaybeRes) { + return false; + } + if (Args[0].Kind =3D=3D IrPtrToOffset) { + // no gvec equivalent to clzi + return false; + } + tcg::genCountLeadingZeros(Out, *MaybeRes, Args[0]); + } break; + case Intrinsic::cttz: { + Expected MaybeRes =3D mapCallReturnValue(Mapper, Call); + if (!MaybeRes) { + return false; + } + if (Args[0].Kind =3D=3D IrPtrToOffset) { + // no gvec equivalent to ctti + return false; + } + tcg::genCountTrailingZeros(Out, *MaybeRes, Args[0]); + } break; + case Intrinsic::ctpop: { + Expected MaybeRes =3D mapCallReturnValue(Mapper, Call); + if (!MaybeRes) { + return false; + } + if (Args[0].Kind =3D=3D IrPtrToOffset) { + // no gvec equivalent to ctpop + return false; + } + tcg::genCountOnes(Out, *MaybeRes, Args[0]); + } break; + case Intrinsic::bswap: { + Expected MaybeRes =3D mapCallReturnValue(Mapper, Call); + if (!MaybeRes) { + return false; + } + tcg::genByteswap(Out, *MaybeRes, Args[0]); + } break; + case Intrinsic::fshl: { + Expected MaybeRes =3D mapCallReturnValue(Mapper, Call); + if (!MaybeRes) { + return false; + } + tcg::genFunnelShl(Out, *MaybeRes, Args[0], Args[1], Args[2]); + } break; + case Intrinsic::bitreverse: { + Expected MaybeRes =3D mapCallReturnValue(Mapper, Call); + if (!MaybeRes) { + return false; + } + tcg::genBitreverse(Out, *MaybeRes, Args[0]); + } break; + case Intrinsic::memcpy: { + tcg::genVecMemcpy(Out, Args[0], Args[1], Args[2]); + } break; + case Intrinsic::memset: { + tcg::genVecMemset(Out, Args[0], Args[1], Args[2]); + } break; + default: + // Unhandled LLVM intrinsic + return false; + } + return true; +} + +static Expected +translateFunction(const Function *F, const TcgGlobalMap &TcgGlobals, + const AnnotationMapTy &Annotations, + const SmallPtrSet HasTranslatedFunction) +{ + TranslatedFunction TF =3D { + .Name =3D F->getName().str(), + }; + + // Run TcgV register allocation + Expected MaybeTAD =3D + allocateTemporaries(*F, Annotations); + if (!MaybeTAD) { + return MaybeTAD.takeError(); + } + const TempAllocationData TAD =3D MaybeTAD.get(); + + { + StringRef NameRef(TF.Name); + std::string DemangledFuncName =3D demangle(TF.Name); + if (TF.Name !=3D DemangledFuncName) { + // If the function name changed when trying to demangle the na= me, + // the name was mangled. The resulting demangled name might l= ook + // something like + // + // namespace::subnamespace::function(...) + // + // extract the function name, this assumes 0 name collisions in + // the output. + size_t Index =3D 0; + NameRef =3D DemangledFuncName; + // Remove namespaces + Index =3D NameRef.find_last_of(':'); + if (Index !=3D StringRef::npos) { + NameRef =3D NameRef.substr(Index + 1); + } + // Remove arguments + Index =3D NameRef.find_first_of('('); + if (Index !=3D StringRef::npos) { + NameRef =3D NameRef.substr(0, Index); + } + } + + // Remove prefix for helper functions to get cleaner emitted names + TF.IsHelper =3D NameRef.consume_front("helper_"); + TF.Name =3D NameRef.str(); + } + + raw_string_ostream Out(TF.Code); + raw_string_ostream HeaderWriter(TF.Decl); + + raw_string_ostream DispatchWriter(TF.DispatchCode); + std::string DispatchCall; + raw_string_ostream DispatchCallWriter(DispatchCall); + int dispatch_arg_count =3D 0; + bool IsVectorInst =3D false; + + // Functions that should be ignored are convereted + // to declarations, see FilterFunctionsPass. + if (F->isDeclaration()) { + return mkError("Function is not translated"); + } + + Mapper Mapper(Out, TcgGlobals, *F->getParent(), TAD); + Optional RetVal =3D None; + Out << "// " << *F->getReturnType() << ' ' << F->getName() << '\n'; + HeaderWriter << "void " << "emit_" << TF.Name << '('; + SmallVector CArgs; + + if (!F->getReturnType()->isVoidTy()) { + assert(TAD.ReturnValue.hasValue()); + IsVectorInst =3D (*TAD.ReturnValue).Kind =3D=3D IrPtrToOffset; + CArgs.push_back(*TAD.ReturnValue); + } + + for (const Value *Arg : TAD.Args.Args) { + Expected MaybeMapped =3D Mapper.mapAndEmit(Arg); + if (!MaybeMapped) { + return mkError("failed mapping arg"); + } + IsVectorInst |=3D (MaybeMapped.get().Kind =3D=3D IrPtrToOffset); + CArgs.push_back(MaybeMapped.get()); + } + + auto CArgIt =3D CArgs.begin(); + if (CArgIt !=3D CArgs.end()) { + HeaderWriter << tcg::getType(*CArgIt) << ' ' << tcg::getName(*CArg= It); + ++CArgIt; + } + while (CArgIt !=3D CArgs.end()) { + HeaderWriter << ", " << tcg::getType(*CArgIt) << ' ' + << tcg::getName(*CArgIt); + ++CArgIt; + } + + if (!IsVectorInst) { + DispatchCallWriter << "emit_" << TF.Name << "("; + auto CArgIt =3D CArgs.begin(); + if (CArgIt !=3D CArgs.end()) { + DispatchWriter << tcg::getType(*CArgIt) << ' ' + << tcg::getName(*CArgIt) << " =3D "; + if (TAD.ReturnValue and CArgIt->Id =3D=3D (*TAD.ReturnValue).I= d) { + assert(CArgIt->Kind =3D=3D IrValue); + DispatchWriter << "temp_tcgv_i" << CArgIt->TcgSize + << "(ret_temp);\n"; + } else { + switch (CArgIt->Kind) { + case IrPtr: + case IrEnv: + DispatchWriter << "temp_tcgv_ptr(args[" + << dispatch_arg_count++ << "]);\n"; + break; + case IrValue: + DispatchWriter << "temp_tcgv_i" << CArgIt->TcgSize + << "(args[" << dispatch_arg_count++ + << "]);\n"; + break; + case IrImmediate: + DispatchWriter << "args[" << dispatch_arg_count++ + << "]->val;\n"; + break; + case IrPtrToOffset: + DispatchWriter << "args[" << dispatch_arg_count++ + << "]->val;\n"; + break; + default: + abort(); + }; + } + DispatchCallWriter << tcg::getName(*CArgIt); + ++CArgIt; + } + while (CArgIt !=3D CArgs.end()) { + DispatchWriter << tcg::getType(*CArgIt) << ' ' + << tcg::getName(*CArgIt) << " =3D "; + switch (CArgIt->Kind) { + case IrPtr: + case IrEnv: + DispatchWriter << "temp_tcgv_ptr(args[" << dispatch_arg_co= unt++ + << "]);\n"; + break; + case IrValue: + DispatchWriter << "temp_tcgv_i" << CArgIt->TcgSize << "(ar= gs[" + << dispatch_arg_count++ << "]);\n"; + break; + case IrImmediate: + DispatchWriter << "args[" << dispatch_arg_count++ + << "]->val;\n"; + break; + case IrPtrToOffset: + DispatchWriter << "args[" << dispatch_arg_count++ + << "]->val;\n"; + break; + default: + abort(); + }; + DispatchCallWriter << ", " << tcg::getName(*CArgIt); + ++CArgIt; + } + DispatchCallWriter << ");\n"; + DispatchWriter << DispatchCallWriter.str(); + } + + // Copy over function declaration from header to source file + HeaderWriter << ')'; + Out << HeaderWriter.str(); + Out << " {\n"; + HeaderWriter << ';'; + + ReversePostOrderTraversal RPOT((Function *)F); + for (auto BBI =3D RPOT.begin(); BBI !=3D RPOT.end(); ++BBI) { + BasicBlock &BB =3D **BBI; + + // Set label if not first BB + if (&BB !=3D &F->getEntryBlock()) { + TcgV Label =3D Mapper.mapBbAndEmit(&BB); + tcg::genSetLabel(Out, Label); + } + + // Emit TCG generators for the current BB + for (Instruction &I : BB) { + switch (I.getOpcode()) { + case Instruction::Alloca: { + auto Alloca =3D cast(&I); + Expected Res =3D Mapper.mapAndEmit(Alloca); + if (!Res) { + return Res.takeError(); + } + } break; + case Instruction::Br: { + // We need to keep the BB of the true branch alive + // so that we can iterate over the CFG as usual + // using LLVM. Or custom "opcode" @brcond is not an + // actual branch, so LLVM does not understand that + // we can branch to the true branch. + // + // For this reason we emit an extra dead branch + // to the true branch, and tag it as dead using + // metadata. The backend can later check that if + // this metadata is present and ignore the branch. + if (I.hasMetadata("dead-branch")) { + break; + } + + auto Branch =3D cast(&I); + if (Branch->isConditional()) { + assert(Branch->getNumSuccessors() =3D=3D 2); + Expected Condition =3D + Mapper.mapCondAndEmit(Branch->getCondition(), 32, = 1); + if (!Condition) + return mkError("couldn't map brcond condition ", + Branch->getCondition()); + const TcgV CCondition =3D tcg::materialize(Condition.g= et()); + const TcgV True =3D + Mapper.mapBbAndEmit(Branch->getSuccessor(0)); + const TcgV False =3D + Mapper.mapBbAndEmit(Branch->getSuccessor(1)); + + // Jump if condition is !=3D 0 + auto Zero =3D TcgV::makeImmediate("0", CCondition.TcgS= ize, 1); + tcg::genBrcond(Out, CmpInst::Predicate::ICMP_NE, CCond= ition, + Zero, True); + tcg::genBr(Out, False); + } else { + const TcgV Label =3D + Mapper.mapBbAndEmit(Branch->getSuccessor(0)); + tcg::genBr(Out, Label); + } + } break; + case Instruction::SExt: { + auto SExt =3D cast(&I); + + Expected SrcVal =3D Mapper.mapAndEmit(SExt->getOpera= nd(0)); + if (!SrcVal) { + return mkError("Couldn't map value ", SExt->getOperand= (0)); + } + if (SrcVal.get().Kind =3D=3D IrImmediate) { + auto ResLlvmSize =3D SExt->getDestTy()->getIntegerBitW= idth(); + Mapper.mapExplicitly(&I, + c::sext(SrcVal.get(), ResLlvmSize, + llvmToTcgSize(ResLlvmSize= ))); + } else if (SrcVal.get().Kind =3D=3D IrPtrToOffset) { + Expected Res =3D Mapper.mapAndEmit(&I); + if (!Res) { + return Res.takeError(); + } + tcg::genVecSext(Out, Res.get(), SrcVal.get()); + } else { + Expected Res =3D Mapper.mapAndEmit(&I); + if (!Res) { + return Res.takeError(); + } + if (Res.get().LlvmSize < 32) { + return mkError("sext to unsupported size: ", &I); + } + if (SrcVal.get().Kind =3D=3D IrPtrToOffset) { + return mkError("sext on vector type not supported:= ", + &I); + } + if (SrcVal.get().LlvmSize > 1 and + SrcVal.get().LlvmSize < 32) { + // TODO: Here we are using the fact that we + // support (16,64), (8,64). Also, move to TcgEmit + auto FuncStr =3D + Twine("tcg_gen_ext") + .concat(std::to_string(SrcVal.get().LlvmSi= ze)) + .concat("s_i") + .concat(std::to_string(Res.get().TcgSize)) + .str(); + auto ASrcVal =3D TcgSizeAdapter(Out, SrcVal.get()); + tcg::emitCallTcg( + Out, FuncStr, + {Res.get(), ASrcVal.get(Res.get().TcgSize)}); + } else if (SrcVal.get().LlvmSize =3D=3D 1 and + Res.get().TcgSize =3D=3D 32) { + tcg::genMov(Out, Res.get(), SrcVal.get()); + } else { + tcg::genExtI32I64(Out, Res.get(), SrcVal.get()); + } + } + } break; + case Instruction::ZExt: { + auto ZExt =3D cast(&I); + + Expected SrcVal =3D Mapper.mapAndEmit(ZExt->getOpera= nd(0)); + if (!SrcVal) + return mkError("Couldn't map value ", ZExt->getOperand= (0)); + + if (SrcVal.get().Kind =3D=3D IrImmediate) { + auto ResLlvmSize =3D ZExt->getDestTy()->getIntegerBitW= idth(); + if (ResLlvmSize > 64) { + return mkError("128-bit integers not supported: ",= &I); + } + Mapper.mapExplicitly(&I, + c::zext(SrcVal.get(), ResLlvmSize, + llvmToTcgSize(ResLlvmSize= ))); + break; + } + + auto *DestTy =3D ZExt->getDestTy(); + if (DestTy->isIntegerTy()) { + const uint32_t ResLlvmSize =3D + cast(DestTy)->getIntegerBitWidth(); + const uint32_t ResTcgSize =3D llvmToTcgSize(ResLlvmSiz= e); + if (ResLlvmSize > 64) { + return mkError("Invalid size: ", &I); + } + const uint32_t SrcLlvmSize =3D SrcVal.get().LlvmSize; + const uint32_t SrcTcgSize =3D SrcVal.get().TcgSize; + + Expected Res =3D Mapper.mapAndEmit(&I); + if (!Res) { + return Res.takeError(); + } + if (SrcTcgSize =3D=3D ResTcgSize) { + tcg::genMov(Out, Res.get(), SrcVal.get()); + } else if (SrcTcgSize > Res.get().TcgSize and + SrcLlvmSize =3D=3D 1) { + // Paradoxically we may need to emit an extract + // instruction for when a zero extension is reques= ted. + // This is to account for the fact that "booleans"= in + // tcg can be both 64- and 32-bit. So for instance= zext + // i1 -> i32, here i1 may actually be 64-bit. + tcg::genExtrlI64I32(Out, Res.get(), SrcVal.get()); + } else { + tcg::genExtuI32I64(Out, Res.get(), SrcVal.get()); + } + } else if (DestTy->isVectorTy()) { + Expected Res =3D Mapper.mapAndEmit(&I); + if (!Res) { + return Res.takeError(); + } + tcg::genVecZext(Out, Res.get(), SrcVal.get()); + } else { + return mkError("Invalid TcgSize!"); + } + } break; + case Instruction::Trunc: { + auto Trunc =3D cast(&I); + + Expected SrcVal =3D Mapper.mapAndEmit(Trunc->getOper= and(0)); + if (!SrcVal) { + return mkError("Couldn't map value ", Trunc->getOperan= d(0)); + } + if (SrcVal.get().Kind =3D=3D IrImmediate) { + Mapper.mapExplicitly(&I, SrcVal.get()); + break; + } + + Expected Res =3D Mapper.mapAndEmit(&I); + if (!Res) { + return Res.takeError(); + } + if (Res.get().Kind =3D=3D IrValue) { + if (SrcVal.get().TcgSize =3D=3D 64) { + if (Res.get().LlvmSize =3D=3D 32) { + // 64 -> 32 + tcg::genExtrlI64I32(Out, Res.get(), SrcVal.get= ()); + } else { + // 64 -> 16,8,1 + TcgV MRes =3D Res.get(); + TcgV MSrc =3D SrcVal.get(); + auto Offset =3D TcgV::makeImmediate("0", MRes.= TcgSize, + MRes.LlvmSiz= e); + auto Size =3D TcgV::makeImmediate( + Twine((int)MRes.LlvmSize).str(), MRes.TcgS= ize, + MRes.LlvmSize); + auto Temp =3D TcgV::makeTemp(64, 64, IrValue); + tcg::defineNewTemp(Out, Temp); + tcg::genExtract(Out, false, Temp, MSrc, Offset, + Size); + tcg::genExtrlI64I32(Out, MRes, Temp); + } + } else if (SrcVal.get().TcgSize =3D=3D 32) { + // 32 -> 16,8,1 + // 16 -> 8,1 + // 8 -> 1 + TcgV MRes =3D Res.get(); + TcgV MSrc =3D SrcVal.get(); + auto Offset =3D TcgV::makeImmediate("0", MRes.TcgS= ize, + MRes.LlvmSize); + auto Size =3D + TcgV::makeImmediate(Twine((int)MRes.LlvmSize).= str(), + MRes.TcgSize, MRes.LlvmSiz= e); + tcg::genExtract(Out, false, MRes, MSrc, Offset, Si= ze); + } else { + return mkError("Invalid TcgSize!"); + } + } else if (Res.get().Kind =3D=3D IrPtrToOffset) { + tcg::genVecTrunc(Out, Res.get(), SrcVal.get()); + } else { + return mkError("Invalid TcgSize!"); + } + } break; + case Instruction::Add: + case Instruction::And: + case Instruction::AShr: + case Instruction::LShr: + case Instruction::Mul: + case Instruction::UDiv: + case Instruction::SDiv: + case Instruction::Or: + case Instruction::Shl: + case Instruction::Sub: + case Instruction::Xor: { + auto Bin =3D cast(&I); + // Check we are working on integers + Expected MaybeOp1 =3D Mapper.mapAndEmit(Bin->getOper= and(0)); + if (!MaybeOp1) { + return MaybeOp1.takeError(); + } + Expected MaybeOp2 =3D Mapper.mapAndEmit(Bin->getOper= and(1)); + if (!MaybeOp2) { + return MaybeOp2.takeError(); + } + TcgV Op1 =3D MaybeOp1.get(); + TcgV Op2 =3D MaybeOp2.get(); + + // Swap operands if the first op. is an immediate + // and the operator is commutative + if (Op1.Kind =3D=3D IrImmediate and Op2.Kind !=3D IrImmedi= ate and + Bin->isCommutative()) { + std::swap(Op1, Op2); + } + + if (isa(Bin->getType())) { + if (Op1.Kind =3D=3D IrImmediate and Op2.Kind =3D=3D Ir= Immediate) { + Mapper.mapExplicitly( + Bin, c::binop(Bin->getOpcode(), Op1, Op2)); + } else { + Expected Res =3D Mapper.mapAndEmit(Bin); + if (!Res) { + return mkError("couldn't map binary op res", &= I); + } + + // Adapt sizes to account for boolean values, with + // LlvmSize =3D=3D 1 and TcgSize =3D=3D 32 or 64. = Materialize + // first op. to deal with non-commutative ops. + TcgSizeAdapter AOp1(Out, tcg::materialize(Op1)); + TcgSizeAdapter AOp2(Out, Op2); + + const uint32_t ResSize =3D Res.get().TcgSize; + tcg::genBinOp(Out, Res.get(), Bin->getOpcode(), + AOp1.get(ResSize), AOp2.get(ResSize)= ); + } + } else if (isa(Bin->getType())) { + Expected Res =3D Mapper.mapAndEmit(Bin); + if (!Res) { + return Res.takeError(); + } + assert(Res.get().Kind =3D=3D IrPtrToOffset); + tcg::genVecBinOp(Out, Bin->getOpcode(), Res.get(), Op1, + Op2); + } + } break; + case Instruction::Call: { + auto Call =3D cast(&I); + Function *F =3D Call->getCalledFunction(); + if (!F) { + return mkError("Indirect function calls not handled: "= , &I); + } + assert(F->hasName()); + StringRef Name =3D F->getName(); + + // These are the calls we currently no-op/ignore + if (Name =3D=3D "__assert_fail" or + Name =3D=3D "g_assertion_message_expr" or + isa(I) or isa(I)) { + break; + } + + SmallVector Args; + for (uint32_t i =3D 0; i < Call->arg_size(); ++i) { + if (auto Bb =3D + dyn_cast(Call->getArgOperand(i))) { + Args.push_back(Mapper.mapBbAndEmit(Bb)); + } else { + Expected Mapped =3D + Mapper.mapAndEmit(Call->getArgOperand(i)); + if (!Mapped) { + return Mapped.takeError(); + } + Args.push_back(Mapped.get()); + } + } + + // Function names sometimes contain embedded type informat= ion to + // handle polymorphic arguments, for instance + // + // llvm.memcpy.p0i8.p0i8.i64 + // + // specifying the source and desination pointer types as i= 8* and + // the size argument as an i64. + // + // Find the index for the first '.' before the types are + // specified + // + // llvm.memcpy.p0i8.p0i8.i64 + // ^- index of this '.' + size_t IndexBeforeTypes =3D StringRef::npos; + for (size_t i =3D Name.size() - 1; i > 0; --i) { + const char c =3D Name[i]; + bool ValidType =3D (c >=3D '0' and c <=3D '9') or c = =3D=3D 'i' or + c =3D=3D 'p' or c =3D=3D 'a' or c =3D= =3D 'v' or + c =3D=3D 'x'; + if (c =3D=3D '.') { + IndexBeforeTypes =3D i; + } else if (!ValidType) { + break; + } + } + StringRef StrippedName =3D Name.substr(0, IndexBeforeTypes= ); + + PseudoInst PInst =3D getPseudoInstFromCall(Call); + + if (F->isIntrinsic()) { + if (!translateIntrinsicCall(Out, Call, F, Args, Mapper= )) { + return mkError("Unable to map intrinsic: ", Call); + } + } else if (PInst !=3D InvalidPseudoInst) { + if (!translatePseudoInstCall(Out, Call, PInst, Args, M= apper, + TcgGlobals)) { + return mkError("Unable to map pseudo inst: ", Call= ); + } + } else if (StrippedName =3D=3D "extract32") { + Expected MaybeRes =3D mapCallReturnValue(Mapper,= Call); + if (!MaybeRes) { + return MaybeRes.takeError(); + } + tcg::genExtract(Out, false, *MaybeRes, Args[0], Args[1= ], + Args[2]); + } else if (StrippedName =3D=3D "extract64") { + Expected MaybeRes =3D mapCallReturnValue(Mapper,= Call); + if (!MaybeRes) { + return MaybeRes.takeError(); + } + tcg::genExtract(Out, false, *MaybeRes, Args[0], Args[1= ], + Args[2]); + } else if (StrippedName =3D=3D "sextract32") { + Expected MaybeRes =3D mapCallReturnValue(Mapper,= Call); + if (!MaybeRes) { + return MaybeRes.takeError(); + } + tcg::genExtract(Out, true, *MaybeRes, Args[0], Args[1], + Args[2]); + } else if (StrippedName =3D=3D "sextract64") { + Expected MaybeRes =3D mapCallReturnValue(Mapper,= Call); + if (!MaybeRes) { + return MaybeRes.takeError(); + } + tcg::genExtract(Out, true, *MaybeRes, Args[0], Args[1], + Args[2]); + } else if (StrippedName =3D=3D "deposit32") { + Expected MaybeRes =3D mapCallReturnValue(Mapper,= Call); + if (!MaybeRes) { + return MaybeRes.takeError(); + } + tcg::genDeposit(Out, *MaybeRes, Args[0], Args[1], Args= [2], + Args[3]); + } else if (StrippedName =3D=3D "deposit64") { + Expected MaybeRes =3D mapCallReturnValue(Mapper,= Call); + if (!MaybeRes) { + return MaybeRes.takeError(); + } + tcg::genDeposit(Out, *MaybeRes, Args[0], Args[1], Args= [2], + Args[3]); + } else if (Name.startswith("helper")) { + // Map and adapt arguments to the call + SmallVector IArgs; + for (auto Arg : Args) { + IArgs.push_back(tcg::materialize(Arg)); + } + tcg::genCallHelper(Out, Name, IArgs.begin(), IArgs.end= ()); + } else { + if (F->isDeclaration()) { + return mkError("call to declaration: ", Call); + } + if (HasTranslatedFunction.find(F) =3D=3D + HasTranslatedFunction.end()) { + return mkError( + "call to function which failed to translate: ", + Call); + } + + // Map and adapt arguments to the call + + Expected MaybeRes =3D mapCallReturnValue(Mapper,= Call); + + StringRef Name =3D F->getName(); + Name.consume_front("helper_"); + Out << "emit_" << Name << "("; + + if (MaybeRes) { + Out << tcg::getName(MaybeRes.get()); + if (!Args.empty()) { + Out << ", "; + } + } + + for (unsigned i =3D 0; i < Args.size(); ++i) { + Out << tcg::getName(tcg::materialize(Args[i])); + if (i < Args.size() - 1) { + Out << ", "; + } + } + Out << ");\n"; + } + + } break; + case Instruction::ICmp: { + auto *ICmp =3D cast(&I); + Expected Op1 =3D Mapper.mapAndEmit(I.getOperand(0)); + if (!Op1) { + return mkError("Couldn't map first op: ", ICmp); + } + Expected Op2 =3D Mapper.mapAndEmit(I.getOperand(1)); + if (!Op2) { + return mkError("Couldn't map first op: ", ICmp); + } + // If both operands are immediates (constant expressions, = we can + // perform the operation as a constant expression. + if (Op1.get().Kind =3D=3D IrImmediate and + Op2.get().Kind =3D=3D IrImmediate) { + Mapper.mapExplicitly( + ICmp, + c::compare(ICmp->getPredicate(), Op1.get(), Op2.ge= t())); + break; + } + + ICmpInst::Predicate LlvmPred =3D ICmp->getPredicate(); + + if (Op1.get().Kind =3D=3D IrPtrToOffset) { + Expected Res =3D Mapper.mapCondAndEmit( + &I, Op1.get().TcgSize, Op1.get().LlvmSize); + if (!Res) { + return mkError("couldn't map icmp result", &I); + } + tcg::genVecCmp(Out, Res.get(), LlvmPred, Op1.get(), + Op2.get()); + } else { + Expected Res =3D + Mapper.mapCondAndEmit(&I, Op1.get().TcgSize, 1); + if (!Res) { + return mkError("couldn't map icmp result", &I); + } + auto IOp1 =3D tcg::materialize(Op1.get()); + if (ICmp->isSigned()) { + ensureSignBitIsSet(Out, IOp1); + ensureSignBitIsSet(Out, Op2.get()); + } + if (Op2.get().Kind =3D=3D IrImmediate) { + tcg::genSetcondI(Out, LlvmPred, Res.get(), IOp1, + Op2.get()); + } else { + tcg::genSetcond(Out, LlvmPred, Res.get(), IOp1, + Op2.get()); + } + } + + } break; + case Instruction::Select: { + auto Select =3D cast(&I); + Expected Res =3D Mapper.mapAndEmit(&I); + if (!Res) { + return mkError("Couldn't map select result", &I); + } + if (Res.get().Kind =3D=3D IrPtr) { + return mkError( + "Select statements for pointer types not supported= : ", + Select); + } + Expected Cond =3D Mapper.mapAndEmit(Select->getCondi= tion()); + if (!Cond) { + return mkError("Error mapping select cond"); + } + Expected True =3D Mapper.mapAndEmit(Select->getTrueV= alue()); + if (!True) { + return mkError("Couldn't map True for select instructi= on: ", + Select); + } + Expected False =3D + Mapper.mapAndEmit(Select->getFalseValue()); + if (!False) { + return mkError( + "Couldn't map False for select instruction: ", Sel= ect); + } + + if (Res.get().Kind =3D=3D IrPtrToOffset) { + tcg::genVecBitsel(Out, Res.get(), Cond.get(), True.get= (), + False.get()); + } else if (Cond.get().Kind =3D=3D IrImmediate) { + assert(Res.get().Kind !=3D IrImmediate); + const TcgV MTrue =3D tcg::materialize(True.get()); + const TcgV MFalse =3D tcg::materialize(False.get()); + tcg::genMov(Out, Res.get(), + c::ternary(Cond.get(), MTrue, MFalse)); + } else { + TcgV Zero =3D TcgV::makeImmediate("0", Res.get().TcgSi= ze, 1); + TcgSizeAdapter ACond(Out, Cond.get()); + TcgSizeAdapter ATrue(Out, True.get()); + TcgSizeAdapter AFalse(Out, False.get()); + if (True.get().Kind =3D=3D IrImmediate or + False.get().Kind =3D=3D IrImmediate) { + auto CTrue =3D + tcg::materialize(ATrue.get(Res.get().TcgSize)); + auto CFalse =3D + tcg::materialize(AFalse.get(Res.get().TcgSize)= ); + + tcg::genMovcond(Out, CmpInst::Predicate::ICMP_NE, + Res.get(), ACond.get(CTrue.TcgSize= ), + Zero, CTrue, CFalse); + } else { + tcg::genMovcond(Out, CmpInst::Predicate::ICMP_NE, + Res.get(), + ACond.get(True.get().TcgSize), Zer= o, + ATrue.get(Res.get().TcgSize), + AFalse.get(Res.get().TcgSize)); + } + } + } break; + case Instruction::Ret: { + auto Ret =3D cast(&I); + if (Ret->getNumOperands() =3D=3D 0) + break; + + assert(TAD.ReturnValue.hasValue()); + Expected Tcg =3D Mapper.mapAndEmit(Ret->getReturnVal= ue()); + if (!Tcg) { + return Tcg.takeError(); + } + if (Tcg.get().Kind =3D=3D IrImmediate) { + tcg::genMovI(Out, *TAD.ReturnValue, Tcg.get()); + } else if (!TAD.SkipReturnMov) { + tcg::genMov(Out, *TAD.ReturnValue, Tcg.get()); + } + } break; + case Instruction::BitCast: { + // We currently identity-map `BitCast`s + // + // If the bitcast has a larger lifetime than the source + // variable, we need to allocate a new variable so we + // don't accidentally free too soon. + auto Bitcast =3D cast(&I); + Expected SrcVal =3D + Mapper.mapAndEmit(Bitcast->getOperand(0)); + if (!SrcVal) { + return SrcVal.takeError(); + } + auto *DstTy =3D Bitcast->getType(); + if (SrcVal.get().Kind =3D=3D IrPtrToOffset) { + auto *PtrTy =3D cast(DstTy); + auto *VecTy =3D + dyn_cast(PtrTy->getPointerElementType(= )); + if (!VecTy) { + return mkError("bitcast to unsuppored type: ", Bit= cast); + } + auto *IntTy =3D cast(VecTy->getElementTyp= e()); + uint32_t LlvmSize =3D IntTy->getBitWidth(); + uint32_t VectorElements =3D + compat::getVectorElementCount(VecTy); + uint32_t VectorSize =3D LlvmSize * VectorElements; + TcgV Tcg =3D SrcVal.get(); + uint32_t TcgVectorSize =3D llvmToTcgSize(VectorSize); + Tcg.TcgSize =3D TcgVectorSize; + Tcg.LlvmSize =3D LlvmSize; + Tcg.VectorElementCount =3D VectorElements; + Tcg.Kind =3D IrPtrToOffset; + Mapper.mapExplicitly(Bitcast, Tcg); + } else if (DstTy->isPointerTy()) { + auto *ElmTy =3D DstTy->getPointerElementType(); + if (ElmTy->isIntegerTy()) { + auto *IntTy =3D cast(ElmTy); + const uint32_t TcgSize =3D + llvmToTcgSize(IntTy->getBitWidth()); + if (TcgSize =3D=3D SrcVal.get().TcgSize) { + Mapper.mapExplicitly(Bitcast, SrcVal.get()); + } else { + return mkError("Invalid bitcast changes tcg si= ze: ", + &I); + } + } else if (ElmTy->isArrayTy()) { + return mkError("Bitcast to unsupported type: ", &I= ); + } else { + Mapper.mapExplicitly(Bitcast, SrcVal.get()); + } + } else if (DstTy->isVectorTy()) { + auto *VecTy =3D cast(DstTy); + auto *IntTy =3D cast(VecTy->getElementTyp= e()); + uint32_t LlvmSize =3D IntTy->getBitWidth(); + uint32_t VectorElements =3D + compat::getVectorElementCount(VecTy); + uint32_t VectorSize =3D LlvmSize * VectorElements; + uint32_t TcgVectorSize =3D llvmToTcgSize(VectorSize); + TcgV Tcg =3D SrcVal.get(); + Tcg.TcgSize =3D TcgVectorSize; + Tcg.LlvmSize =3D LlvmSize; + Tcg.VectorElementCount =3D VectorElements; + Tcg.Kind =3D IrPtrToOffset; + Mapper.mapExplicitly(Bitcast, Tcg); + } else { + return mkError("Unhandled bitcast type: ", Bitcast); + } + } break; + case Instruction::Load: { + auto *Load =3D cast(&I); + auto *LlvmPtr =3D Load->getPointerOperand(); + + Expected Mapped =3D Mapper.mapAndEmit(LlvmPtr); + if (!Mapped) { + return Mapped.takeError(); + } + switch (Mapped.get().Kind) { + case IrPtr: { + Expected Res =3D Mapper.mapAndEmit(Load); + if (!Res) { + return Res.takeError(); + } + tcg::genLd(Out, Res.get(), Mapped.get(), 0); + } break; + case IrImmediate: { + Expected Res =3D Mapper.mapAndEmit(Load); + if (!Res) { + return Res.takeError(); + } + // Add pointer dereference to immediate address + tcg::genMovI(Out, Res.get(), + c::deref(Mapped.get(), Res.get().LlvmSize, + Res.get().TcgSize)); + } break; + case IrValue: { + Expected Res =3D Mapper.mapAndEmit(Load); + if (!Res) { + return Res.takeError(); + } + tcg::genMov(Out, Res.get(), Mapped.get()); + } break; + case IrPtrToOffset: { + // Loads from IrPtrToOffset are identity mapped, they = are an + // artifact of IrPtrToOffset arguments being pointers. + // Stores to results are instead taken care of by what= ever + // instruction generated the result. + if (isa(Load->getType())) { + Mapper.mapExplicitly(Load, Mapped.get()); + } + } break; + default: + return mkError("Load from unsupported TcgV type"); + }; + + } break; + case Instruction::Store: { + auto *Store =3D cast(&I); + Expected Val =3D + Mapper.mapAndEmit(Store->getValueOperand()); + if (!Val) { + return Val.takeError(); + } + auto *LlvmPtr =3D Store->getPointerOperand(); + Expected Mapped =3D Mapper.mapAndEmit(LlvmPtr); + if (!Mapped) { + return Mapped.takeError(); + } + if (Mapped.get().Kind =3D=3D IrValue) { + switch (Val.get().Kind) { + case IrImmediate: { + tcg::genMovI(Out, Mapped.get(), Val.get()); + } break; + case IrValue: { + tcg::genMov(Out, Mapped.get(), Val.get()); + } break; + default: + return mkError("Store from unsupported TcgV type"); + }; + } else if (Mapped.get().Kind =3D=3D IrPtr) { + tcg::genSt(Out, Mapped.get(), tcg::materialize(Val.get= ()), + 0); + } else if (Mapped.get().Kind =3D=3D IrPtrToOffset) { + // Stores to IrPtrToOffset are ignored, they are an ar= tifact + // of IrPtrToOffset arguments being pointers. Stores to + // results are instead taken care of by whatever instr= uction + // generated the result. + } else { + return mkError("Store to unsupported TcgV kind: ", Sto= re); + } + } break; + case Instruction::Unreachable: { + Out << "/* unreachable */\n"; + } break; + case Instruction::Switch: { + auto Switch =3D cast(&I); + // Operands to switch instructions alternate between + // case values and the corresponding label: + // Operands: { Cond, DefaultLabel, Case0, Label0, Case1, + // Label1, ... } + Expected Val =3D Mapper.mapAndEmit(Switch->getOperan= d(0)); + if (!Val) { + return Val.takeError(); + } + const TcgV DefaultLabel =3D Mapper.mapBbAndEmit( + cast(Switch->getOperand(1))); + for (uint32_t i =3D 2; i < Switch->getNumOperands(); i += =3D 2) { + Expected BranchVal =3D + Mapper.mapAndEmit(Switch->getOperand(i)); + if (!BranchVal) { + return BranchVal.takeError(); + } + const TcgV BranchLabel =3D Mapper.mapBbAndEmit( + cast(Switch->getOperand(i + 1))); + tcg::genBrcond(Out, CmpInst::Predicate::ICMP_EQ, Val.g= et(), + BranchVal.get(), BranchLabel); + } + tcg::genBr(Out, DefaultLabel); + } break; + case Instruction::Freeze: { + } break; + default: { + return mkError("Instruction not yet implemented", &I); + } + } + } + } + + Out << "}\n"; + + Out.flush(); + HeaderWriter.flush(); + DispatchWriter.flush(); + DispatchCallWriter.flush(); + + return TF; +} + +PreservedAnalyses TcgGenPass::run(Module &M, ModuleAnalysisManager &MAM) +{ + auto &CG =3D MAM.getResult(M); + + // Vector of translation results + SmallVector TranslatedFunctions; + // Two sets used for quickly looking up whether or not a function has + // already been translated, or the translation failed. + SmallPtrSet FailedToTranslateFunction; + SmallPtrSet HasTranslatedFunction; + for (Function &F : M) { + if (F.isDeclaration()) { + continue; + } + + // Depth first traversal of call graph. Needed to ensure called + // functions are translated before the current function. + CallGraphNode *Node =3D CG[&F]; + for (auto *N : make_range(po_begin(Node), po_end(Node))) { + Function *F =3D N->getFunction(); + + // If F in the call graph has already been translated and fail= ed, + // abort translation of the current function. (NOTE: use of .f= ind() + // over .contains() is to appease LLVM 10.) + bool FailedTranslation =3D FailedToTranslateFunction.find(F) != =3D + FailedToTranslateFunction.end(); + if (FailedTranslation) { + break; + } + + // Skip translation of invalid functions or functions that have + // already been translated. (NOTE: use of .find() over .contai= ns() + // is to appease LLVM 10.) + bool AlreadyTranslated =3D + HasTranslatedFunction.find(F) !=3D HasTranslatedFunction.e= nd(); + if (!F or F->isDeclaration() or AlreadyTranslated) { + continue; + } + + tcg::resetNameIndices(); + + auto Translated =3D translateFunction(F, TcgGlobals, Annotatio= ns, + HasTranslatedFunction); + if (!Translated) { + FailedToTranslateFunction.insert(F); + OutLog << F->getName() << ": " << Translated.takeError() + << "\n"; + if (ErrorOnTranslationFailure) { + return PreservedAnalyses::all(); + } else { + break; + } + } + + TranslatedFunctions.push_back(*Translated); + HasTranslatedFunction.insert(F); + OutLog << F->getName() << ": OK\n"; + } + } + + // Preamble + OutSource << "#include \"qemu/osdep.h\"\n"; + OutSource << "#include \"qemu/log.h\"\n"; + OutSource << "#include \"cpu.h\"\n"; + OutSource << "#include \"tcg/tcg-op.h\"\n"; + OutSource << "#include \"tcg/tcg-op-gvec.h\"\n"; + OutSource << "#include \"tcg/tcg.h\"\n"; + OutSource << "#include \"tcg/tcg-global-mappings.h\"\n"; + OutSource << "#include \"exec/exec-all.h\"\n"; + OutSource << "#include \"exec/helper-gen.h\"\n"; + OutSource << '\n'; + + OutSource << "#include \"" + << HeaderPath.substr(HeaderPath.find_last_of('/') + 1) << "\= "\n"; + OutSource << '\n'; + + // Emit extern definitions for all global TCGv_* that are mapped + // to the CPUState. + for (auto &P : TcgGlobals) { + const TcgGlobal &Global =3D P.second; + const uint32_t Size =3D llvmToTcgSize(Global.Size); + OutSource << "extern " << "TCGv_i" << Size << " " << Global.Code; + if (Global.NumElements > 1) { + OutSource << "[" << Global.NumElements << "]"; + } + OutSource << ";\n"; + } + + c::emitVectorPreamble(OutSource); + + // Emit translated functions + for (auto &TF : TranslatedFunctions) { + OutSource << TF.Code << '\n'; + OutHeader << TF.Decl << '\n'; + OutEnabled << TF.Name << '\n'; + } + + // Emit a dispatched to go from helper function address to our + // emitted code, if we succeeded. + OutHeader << "int helper_to_tcg_dispatcher(void *func, TCGTemp *ret_te= mp, " + "int nargs, TCGTemp **args);\n"; + + OutSource << "\n"; + OutSource << "#include \"exec/helper-proto.h\"\n"; + OutSource << "int helper_to_tcg_dispatcher(void *func, TCGTemp *ret_te= mp, " + "int nargs, TCGTemp **args) {\n"; + for (auto &TF : TranslatedFunctions) { + if (!TF.IsHelper or TF.DispatchCode.empty()) { + continue; + } + OutSource << " if ((uintptr_t) func =3D=3D (uintptr_t) helper_" + << TF.Name << ") {\n"; + OutSource << TF.DispatchCode; + OutSource << " return 1;\n"; + OutSource << " }\n"; + } + OutSource << " return 0;\n"; + OutSource << "}\n"; + + return PreservedAnalyses::all(); +} diff --git a/subprojects/helper-to-tcg/passes/backend/TcgGenPass.h b/subpro= jects/helper-to-tcg/passes/backend/TcgGenPass.h new file mode 100644 index 0000000000..0bbd4782e2 --- /dev/null +++ b/subprojects/helper-to-tcg/passes/backend/TcgGenPass.h @@ -0,0 +1,57 @@ +// +// Copyright(c) 2024 rev.ng Labs Srl. All Rights Reserved. +// +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, see . +// + +#pragma once + +#include "FunctionAnnotation.h" +#include "TcgGlobalMap.h" +#include + +// +// TcgGenPass +// +// Backend pass responsible for emitting the final TCG code. Ideally this= pass +// should be as simple as possible simply mapping one expression LLVM IR +// directly to another in TCG. +// +// However, we currently still rely on this pass to perform the mapping of +// constants. (mapping of values is handled by the TcgTempAllocationPass.) +// + +class TcgGenPass : public llvm::PassInfoMixin { + llvm::raw_ostream &OutSource; + llvm::raw_ostream &OutHeader; + llvm::raw_ostream &OutEnabled; + llvm::raw_ostream &OutLog; + llvm::StringRef HeaderPath; + const AnnotationMapTy &Annotations; + const TcgGlobalMap &TcgGlobals; + +public: + TcgGenPass(llvm::raw_ostream &OutSource, llvm::raw_ostream &OutHeader, + llvm::raw_ostream &OutEnabled, llvm::raw_ostream &OutLog, + llvm::StringRef HeaderPath, const AnnotationMapTy &Annotati= ons, + const TcgGlobalMap &TcgGlobals) + : OutSource(OutSource), OutHeader(OutHeader), OutEnabled(OutEnable= d), + OutLog(OutLog), HeaderPath(HeaderPath), Annotations(Annotations), + TcgGlobals(TcgGlobals) + { + } + + llvm::PreservedAnalyses run(llvm::Module &M, + llvm::ModuleAnalysisManager &MAM); +}; diff --git a/subprojects/helper-to-tcg/pipeline/Pipeline.cpp b/subprojects/= helper-to-tcg/pipeline/Pipeline.cpp index 004c16550a..3664603451 100644 --- a/subprojects/helper-to-tcg/pipeline/Pipeline.cpp +++ b/subprojects/helper-to-tcg/pipeline/Pipeline.cpp @@ -34,12 +34,14 @@ #include #include #include +#include #include #include #include =20 #include #include +#include #include =20 using namespace llvm; @@ -81,6 +83,30 @@ cl::opt "for allocating temporary gvec variables"), cl::init("tmp_vmem"), cl::cat(Cat)); =20 +// Options for TcgGenPass +cl::opt OutputSourceFile("output-source", + cl::desc("output .c file"), + cl::init("helper-to-tcg-emitted.c"), + cl::cat(Cat)); + +cl::opt OutputHeaderFile("output-header", + cl::desc("output .h file"), + cl::init("helper-to-tcg-emitted.h"), + cl::cat(Cat)); + +cl::opt + OutputEnabledFile("output-enabled", + cl::desc("output list of parsed functions"), + cl::init("helper-to-tcg-enabled"), cl::cat(Cat)); + +cl::opt OutputLogFile("output-log", cl::desc("output log file= "), + cl::init("helper-to-tcg-log"), cl::cat(= Cat)); + +cl::opt + ErrorOnTranslationFailure("error-on-translation-failure", + cl::desc("Abort translation on first failure= "), + cl::init(false), cl::cat(Cat)); + // Define a TargetTransformInfo (TTI) subclass, this allows for overriding // common per-llvm-target information expected by other LLVM passes, such // as the width of the largest scalar/vector registers. Needed for consis= tent @@ -244,5 +270,28 @@ int main(int argc, char **argv) MPM.addPass(createModuleToFunctionPassAdaptor(std::move(FPM))); } =20 + // + // Finally we run a backend pass that converts from LLVM IR to TCG, + // and emits the final code. + // + + std::error_code EC; + ToolOutputFile OutSource(OutputSourceFile, EC, compat::OpenFlags); + ToolOutputFile OutHeader(OutputHeaderFile, EC, compat::OpenFlags); + ToolOutputFile OutEnabled(OutputEnabledFile, EC, compat::OpenFlags); + ToolOutputFile OutLog(OutputLogFile, EC, compat::OpenFlags); + assert(!EC); + + MPM.addPass(TcgGenPass(OutSource.os(), OutHeader.os(), OutEnabled.os(), + OutLog.os(), OutputHeaderFile, Annotations, + TcgGlobals)); + + MPM.run(*M, MAM); + + OutSource.keep(); + OutHeader.keep(); + OutEnabled.keep(); + OutLog.keep(); + return 0; } --=20 2.45.2