Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[MLIR][LLVM] Make DISubprogramAttr cyclic #106571

Merged
merged 3 commits into from
Aug 31, 2024

Conversation

gysit
Copy link
Contributor

@gysit gysit commented Aug 29, 2024

This commit implements LLVM_DIRecursiveTypeAttrInterface for the DISubprogramAttr to ensure cyclic subprograms can be imported properly. In the process multiple shortcuts around the recently introduced DIImportedEntityAttr can be removed.

@gysit gysit force-pushed the make-subprogram-recursive branch 3 times, most recently from 9274e39 to fad0bda Compare August 29, 2024 16:02
@gysit gysit marked this pull request as ready for review August 29, 2024 16:05
@gysit gysit requested review from abidh and zyx-billy August 29, 2024 16:05
@llvmbot
Copy link
Collaborator

llvmbot commented Aug 29, 2024

@llvm/pr-subscribers-flang-fir-hlfir

@llvm/pr-subscribers-mlir-llvm

Author: Tobias Gysi (gysit)

Changes

This commit implements LLVM_DIRecursiveTypeAttrInterface for the DISubprogramAttr to ensure cyclic subprograms can be imported properly. In the process multiple shortcuts around the recently introduced DIImportedEntityAttr can be removed.


Patch is 35.66 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/106571.diff

12 Files Affected:

  • (modified) mlir/include/mlir-c/Dialect/LLVM.h (+14-8)
  • (modified) mlir/include/mlir/Dialect/LLVMIR/LLVMAttrDefs.td (+23-10)
  • (modified) mlir/lib/CAPI/Dialect/LLVM.cpp (+21-10)
  • (modified) mlir/lib/Dialect/LLVMIR/IR/LLVMAttrs.cpp (+16)
  • (modified) mlir/lib/Dialect/LLVMIR/IR/LLVMDialect.cpp (+3-3)
  • (modified) mlir/lib/Dialect/LLVMIR/Transforms/DIScopeForLLVMFuncOp.cpp (+2-2)
  • (modified) mlir/lib/Target/LLVMIR/DebugImporter.cpp (+15-8)
  • (modified) mlir/lib/Target/LLVMIR/DebugTranslation.cpp (+54-44)
  • (modified) mlir/lib/Target/LLVMIR/DebugTranslation.h (+9-9)
  • (modified) mlir/test/CAPI/llvm.c (+14-5)
  • (modified) mlir/test/Target/LLVMIR/Import/debug-info.ll (+9-11)
  • (modified) mlir/test/Target/LLVMIR/llvmir-debug.mlir (+20-15)
diff --git a/mlir/include/mlir-c/Dialect/LLVM.h b/mlir/include/mlir-c/Dialect/LLVM.h
index 5eb96a86e472d6..561d698f722afe 100644
--- a/mlir/include/mlir-c/Dialect/LLVM.h
+++ b/mlir/include/mlir-c/Dialect/LLVM.h
@@ -234,6 +234,9 @@ MLIR_CAPI_EXPORTED MlirAttribute mlirLLVMDIBasicTypeAttrGet(
     MlirContext ctx, unsigned int tag, MlirAttribute name, uint64_t sizeInBits,
     MlirLLVMTypeEncoding encoding);
 
+/// Creates a self-referencing LLVM DICompositeType attribute.
+MlirAttribute mlirLLVMDICompositeTypeAttrGetSelfRec(MlirAttribute recId);
+
 /// Creates a LLVM DICompositeType attribute.
 MLIR_CAPI_EXPORTED MlirAttribute mlirLLVMDICompositeTypeAttrGet(
     MlirContext ctx, unsigned int tag, MlirAttribute recId, MlirAttribute name,
@@ -311,13 +314,16 @@ MLIR_CAPI_EXPORTED MlirAttribute mlirLLVMDILocalVariableAttrGet(
     MlirAttribute diFile, unsigned int line, unsigned int arg,
     unsigned int alignInBits, MlirAttribute diType, int64_t flags);
 
+/// Creates a self-referencing LLVM DISubprogramAttr attribute.
+MlirAttribute mlirLLVMDISubprogramAttrGetSelfRec(MlirAttribute recId);
+
 /// Creates a LLVM DISubprogramAttr attribute.
 MLIR_CAPI_EXPORTED MlirAttribute mlirLLVMDISubprogramAttrGet(
-    MlirContext ctx, MlirAttribute id, MlirAttribute compileUnit,
-    MlirAttribute scope, MlirAttribute name, MlirAttribute linkageName,
-    MlirAttribute file, unsigned int line, unsigned int scopeLine,
-    uint64_t subprogramFlags, MlirAttribute type, intptr_t nRetainedNodes,
-    MlirAttribute const *retainedNodes);
+    MlirContext ctx, MlirAttribute id, MlirAttribute recId,
+    MlirAttribute compileUnit, MlirAttribute scope, MlirAttribute name,
+    MlirAttribute linkageName, MlirAttribute file, unsigned int line,
+    unsigned int scopeLine, uint64_t subprogramFlags, MlirAttribute type,
+    intptr_t nRetainedNodes, MlirAttribute const *retainedNodes);
 
 /// Gets the scope from this DISubprogramAttr.
 MLIR_CAPI_EXPORTED MlirAttribute
@@ -356,9 +362,9 @@ MLIR_CAPI_EXPORTED MlirAttribute mlirLLVMDIModuleAttrGet(
 
 /// Creates a LLVM DIImportedEntityAttr attribute.
 MLIR_CAPI_EXPORTED MlirAttribute mlirLLVMDIImportedEntityAttrGet(
-    MlirContext ctx, unsigned int tag, MlirAttribute entity, MlirAttribute file,
-    unsigned int line, MlirAttribute name, intptr_t nElements,
-    MlirAttribute const *elements);
+    MlirContext ctx, unsigned int tag, MlirAttribute scope,
+    MlirAttribute entity, MlirAttribute file, unsigned int line,
+    MlirAttribute name, intptr_t nElements, MlirAttribute const *elements);
 
 /// Gets the scope of this DIModuleAttr.
 MLIR_CAPI_EXPORTED MlirAttribute
diff --git a/mlir/include/mlir/Dialect/LLVMIR/LLVMAttrDefs.td b/mlir/include/mlir/Dialect/LLVMIR/LLVMAttrDefs.td
index e57be7f760d380..224e58f3216abf 100644
--- a/mlir/include/mlir/Dialect/LLVMIR/LLVMAttrDefs.td
+++ b/mlir/include/mlir/Dialect/LLVMIR/LLVMAttrDefs.td
@@ -554,14 +554,16 @@ def LLVM_DILocalVariableAttr : LLVM_Attr<"DILocalVariable", "di_local_variable",
 //===----------------------------------------------------------------------===//
 
 def LLVM_DISubprogramAttr : LLVM_Attr<"DISubprogram", "di_subprogram",
-                                      /*traits=*/[], "DIScopeAttr"> {
+                                      [LLVM_DIRecursiveTypeAttrInterface],
+                                      "DIScopeAttr"> {
   let parameters = (ins
     OptionalParameter<"DistinctAttr">:$id,
+    OptionalParameter<"DistinctAttr">:$recId,
     OptionalParameter<"DICompileUnitAttr">:$compileUnit,
-    "DIScopeAttr":$scope,
+    OptionalParameter<"DIScopeAttr">:$scope,
     OptionalParameter<"StringAttr">:$name,
     OptionalParameter<"StringAttr">:$linkageName,
-    "DIFileAttr":$file,
+    OptionalParameter<"DIFileAttr">:$file,
     OptionalParameter<"unsigned">:$line,
     OptionalParameter<"unsigned">:$scopeLine,
     OptionalParameter<"DISubprogramFlags">:$subprogramFlags,
@@ -577,13 +579,28 @@ def LLVM_DISubprogramAttr : LLVM_Attr<"DISubprogram", "di_subprogram",
       "ArrayRef<DINodeAttr>":$retainedNodes
     ), [{
       MLIRContext *ctx = file.getContext();
-      return $_get(ctx, id, compileUnit, scope, StringAttr::get(ctx, name),
+      return $_get(ctx, id, /*recId=*/nullptr, compileUnit, scope,
+                   StringAttr::get(ctx, name),
                    StringAttr::get(ctx, linkageName), file, line,
                    scopeLine, subprogramFlags, type, retainedNodes);
     }]>
   ];
-
   let assemblyFormat = "`<` struct(params) `>`";
+  let extraClassDeclaration = [{
+    /// Requirements of DIRecursiveTypeAttrInterface.
+    /// @{
+
+    /// Get whether this attr describes a recursive self reference.
+    bool isRecSelf() { return !getScope(); }
+
+    /// Get a copy of this type attr but with the recursive ID set to `recId`.
+    DIRecursiveTypeAttrInterface withRecId(DistinctAttr recId);
+
+    /// Build a rec-self instance using the provided `recId`.
+    static DIRecursiveTypeAttrInterface getRecSelf(DistinctAttr recId);
+
+    /// @}
+  }];
 }
 
 //===----------------------------------------------------------------------===//
@@ -627,13 +644,9 @@ def LLVM_DINamespaceAttr : LLVM_Attr<"DINamespace", "di_namespace",
 
 def LLVM_DIImportedEntityAttr : LLVM_Attr<"DIImportedEntity", "di_imported_entity",
                                            /*traits=*/[], "DINodeAttr"> {
-  /// TODO: DIImportedEntity has a 'scope' field which represents the scope where
-  /// this entity is imported. Currently, we are not adding a 'scope' field in
-  /// DIImportedEntityAttr to avoid cyclic dependency. As DIImportedEntityAttr
-  /// entries will be contained inside a scope entity (e.g. DISubprogramAttr),
-  /// the scope can easily be inferred.
   let parameters = (ins
     LLVM_DITagParameter:$tag,
+    "DIScopeAttr":$scope,
     "DINodeAttr":$entity,
     OptionalParameter<"DIFileAttr">:$file,
     OptionalParameter<"unsigned">:$line,
diff --git a/mlir/lib/CAPI/Dialect/LLVM.cpp b/mlir/lib/CAPI/Dialect/LLVM.cpp
index 13341f0c4de881..ed4d028e192f53 100644
--- a/mlir/lib/CAPI/Dialect/LLVM.cpp
+++ b/mlir/lib/CAPI/Dialect/LLVM.cpp
@@ -159,6 +159,11 @@ MlirAttribute mlirLLVMDIBasicTypeAttrGet(MlirContext ctx, unsigned int tag,
       unwrap(ctx), tag, cast<StringAttr>(unwrap(name)), sizeInBits, encoding));
 }
 
+MlirAttribute mlirLLVMDICompositeTypeAttrGetSelfRec(MlirAttribute recId) {
+  return wrap(
+      DICompositeTypeAttr::getRecSelf(cast<DistinctAttr>(unwrap(recId))));
+}
+
 MlirAttribute mlirLLVMDICompositeTypeAttrGet(
     MlirContext ctx, unsigned int tag, MlirAttribute recId, MlirAttribute name,
     MlirAttribute file, uint32_t line, MlirAttribute scope,
@@ -289,16 +294,21 @@ MlirAttribute mlirLLVMDISubroutineTypeAttrGet(MlirContext ctx,
                           [](Attribute a) { return cast<DITypeAttr>(a); })));
 }
 
+MlirAttribute mlirLLVMDISubprogramAttrGetSelfRec(MlirAttribute recId) {
+  return wrap(DISubprogramAttr::getRecSelf(cast<DistinctAttr>(unwrap(recId))));
+}
+
 MlirAttribute mlirLLVMDISubprogramAttrGet(
-    MlirContext ctx, MlirAttribute id, MlirAttribute compileUnit,
-    MlirAttribute scope, MlirAttribute name, MlirAttribute linkageName,
-    MlirAttribute file, unsigned int line, unsigned int scopeLine,
-    uint64_t subprogramFlags, MlirAttribute type, intptr_t nRetainedNodes,
-    MlirAttribute const *retainedNodes) {
+    MlirContext ctx, MlirAttribute id, MlirAttribute recId,
+    MlirAttribute compileUnit, MlirAttribute scope, MlirAttribute name,
+    MlirAttribute linkageName, MlirAttribute file, unsigned int line,
+    unsigned int scopeLine, uint64_t subprogramFlags, MlirAttribute type,
+    intptr_t nRetainedNodes, MlirAttribute const *retainedNodes) {
   SmallVector<Attribute> nodesStorage;
   nodesStorage.reserve(nRetainedNodes);
   return wrap(DISubprogramAttr::get(
       unwrap(ctx), cast<DistinctAttr>(unwrap(id)),
+      cast<DistinctAttr>(unwrap(recId)),
       cast<DICompileUnitAttr>(unwrap(compileUnit)),
       cast<DIScopeAttr>(unwrap(scope)), cast<StringAttr>(unwrap(name)),
       cast<StringAttr>(unwrap(linkageName)), cast<DIFileAttr>(unwrap(file)),
@@ -353,14 +363,15 @@ MlirAttribute mlirLLVMDIModuleAttrGetScope(MlirAttribute diModule) {
 }
 
 MlirAttribute mlirLLVMDIImportedEntityAttrGet(
-    MlirContext ctx, unsigned int tag, MlirAttribute entity, MlirAttribute file,
-    unsigned int line, MlirAttribute name, intptr_t nElements,
-    MlirAttribute const *elements) {
+    MlirContext ctx, unsigned int tag, MlirAttribute scope,
+    MlirAttribute entity, MlirAttribute file, unsigned int line,
+    MlirAttribute name, intptr_t nElements, MlirAttribute const *elements) {
   SmallVector<Attribute> elementsStorage;
   elementsStorage.reserve(nElements);
   return wrap(DIImportedEntityAttr::get(
-      unwrap(ctx), tag, cast<DINodeAttr>(unwrap(entity)),
-      cast<DIFileAttr>(unwrap(file)), line, cast<StringAttr>(unwrap(name)),
+      unwrap(ctx), tag, cast<DIScopeAttr>(unwrap(scope)),
+      cast<DINodeAttr>(unwrap(entity)), cast<DIFileAttr>(unwrap(file)), line,
+      cast<StringAttr>(unwrap(name)),
       llvm::map_to_vector(unwrapList(nElements, elements, elementsStorage),
                           [](Attribute a) { return cast<DINodeAttr>(a); })));
 }
diff --git a/mlir/lib/Dialect/LLVMIR/IR/LLVMAttrs.cpp b/mlir/lib/Dialect/LLVMIR/IR/LLVMAttrs.cpp
index 98a9659735e7e6..0ce719b9a750ca 100644
--- a/mlir/lib/Dialect/LLVMIR/IR/LLVMAttrs.cpp
+++ b/mlir/lib/Dialect/LLVMIR/IR/LLVMAttrs.cpp
@@ -215,6 +215,22 @@ DICompositeTypeAttr::getRecSelf(DistinctAttr recId) {
                                   {}, DIFlags(), 0, 0, {}, {}, {}, {}, {});
 }
 
+//===----------------------------------------------------------------------===//
+// DISubprogramAttr
+//===----------------------------------------------------------------------===//
+
+DIRecursiveTypeAttrInterface DISubprogramAttr::withRecId(DistinctAttr recId) {
+  return DISubprogramAttr::get(
+      getContext(), getId(), recId, getCompileUnit(), getScope(), getName(),
+      getLinkageName(), getFile(), getLine(), getScopeLine(),
+      getSubprogramFlags(), getType(), getRetainedNodes());
+}
+
+DIRecursiveTypeAttrInterface DISubprogramAttr::getRecSelf(DistinctAttr recId) {
+  return DISubprogramAttr::get(recId.getContext(), {}, recId, {}, {}, {}, {}, 0,
+                               0, {}, {}, {}, {});
+}
+
 //===----------------------------------------------------------------------===//
 // TargetFeaturesAttr
 //===----------------------------------------------------------------------===//
diff --git a/mlir/lib/Dialect/LLVMIR/IR/LLVMDialect.cpp b/mlir/lib/Dialect/LLVMIR/IR/LLVMDialect.cpp
index 3870aab52f199d..6e4a964f1fc93c 100644
--- a/mlir/lib/Dialect/LLVMIR/IR/LLVMDialect.cpp
+++ b/mlir/lib/Dialect/LLVMIR/IR/LLVMDialect.cpp
@@ -3155,9 +3155,9 @@ struct LLVMOpAsmDialectInterface : public OpAsmDialectInterface {
         .Case<AccessGroupAttr, AliasScopeAttr, AliasScopeDomainAttr,
               DIBasicTypeAttr, DICompileUnitAttr, DICompositeTypeAttr,
               DIDerivedTypeAttr, DIFileAttr, DIGlobalVariableAttr,
-              DIGlobalVariableExpressionAttr, DILabelAttr, DILexicalBlockAttr,
-              DILexicalBlockFileAttr, DILocalVariableAttr, DIModuleAttr,
-              DINamespaceAttr, DINullTypeAttr, DIStringTypeAttr,
+              DIGlobalVariableExpressionAttr, DIImportedEntityAttr, DILabelAttr,
+              DILexicalBlockAttr, DILexicalBlockFileAttr, DILocalVariableAttr,
+              DIModuleAttr, DINamespaceAttr, DINullTypeAttr, DIStringTypeAttr,
               DISubprogramAttr, DISubroutineTypeAttr, LoopAnnotationAttr,
               LoopVectorizeAttr, LoopInterleaveAttr, LoopUnrollAttr,
               LoopUnrollAndJamAttr, LoopLICMAttr, LoopDistributeAttr,
diff --git a/mlir/lib/Dialect/LLVMIR/Transforms/DIScopeForLLVMFuncOp.cpp b/mlir/lib/Dialect/LLVMIR/Transforms/DIScopeForLLVMFuncOp.cpp
index c75f44bf3976a9..ad9aa7ac80cafa 100644
--- a/mlir/lib/Dialect/LLVMIR/Transforms/DIScopeForLLVMFuncOp.cpp
+++ b/mlir/lib/Dialect/LLVMIR/Transforms/DIScopeForLLVMFuncOp.cpp
@@ -76,8 +76,8 @@ static void addScopeToFunction(LLVM::LLVMFuncOp llvmFunc,
     compileUnitAttr = {};
   }
   auto subprogramAttr = LLVM::DISubprogramAttr::get(
-      context, id, compileUnitAttr, fileAttr, funcNameAttr, funcNameAttr,
-      fileAttr,
+      context, id, /*recId=*/{}, compileUnitAttr, fileAttr, funcNameAttr,
+      funcNameAttr, fileAttr,
       /*line=*/line,
       /*scopeline=*/col, subprogramFlags, subroutineTypeAttr,
       /*retainedNodes=*/{});
diff --git a/mlir/lib/Target/LLVMIR/DebugImporter.cpp b/mlir/lib/Target/LLVMIR/DebugImporter.cpp
index ce3643f513d34a..509451057dd737 100644
--- a/mlir/lib/Target/LLVMIR/DebugImporter.cpp
+++ b/mlir/lib/Target/LLVMIR/DebugImporter.cpp
@@ -217,8 +217,8 @@ DebugImporter::translateImpl(llvm::DIImportedEntity *node) {
   }
 
   return DIImportedEntityAttr::get(
-      context, node->getTag(), translate(node->getEntity()),
-      translate(node->getFile()), node->getLine(),
+      context, node->getTag(), translate(node->getScope()),
+      translate(node->getEntity()), translate(node->getFile()), node->getLine(),
       getStringAttrOrNull(node->getRawName()), elements);
 }
 
@@ -227,6 +227,7 @@ DISubprogramAttr DebugImporter::translateImpl(llvm::DISubprogram *node) {
   mlir::DistinctAttr id;
   if (node->isDistinct())
     id = getOrCreateDistinctID(node);
+
   // Return nullptr if the scope or type is invalid.
   DIScopeAttr scope = translate(node->getScope());
   if (node->getScope() && !scope)
@@ -238,16 +239,19 @@ DISubprogramAttr DebugImporter::translateImpl(llvm::DISubprogram *node) {
   if (node->getType() && !type)
     return nullptr;
 
+  // Convert the retained nodes but drop all of them if one of them is invalid.
   SmallVector<DINodeAttr> retainedNodes;
   for (llvm::DINode *retainedNode : node->getRetainedNodes())
     retainedNodes.push_back(translate(retainedNode));
+  if (llvm::is_contained(retainedNodes, nullptr))
+    retainedNodes.clear();
 
-  return DISubprogramAttr::get(context, id, translate(node->getUnit()), scope,
-                               getStringAttrOrNull(node->getRawName()),
-                               getStringAttrOrNull(node->getRawLinkageName()),
-                               translate(node->getFile()), node->getLine(),
-                               node->getScopeLine(), *subprogramFlags, type,
-                               retainedNodes);
+  return DISubprogramAttr::get(
+      context, id, /*recId=*/{}, translate(node->getUnit()), scope,
+      getStringAttrOrNull(node->getRawName()),
+      getStringAttrOrNull(node->getRawLinkageName()),
+      translate(node->getFile()), node->getLine(), node->getScopeLine(),
+      *subprogramFlags, type, retainedNodes);
 }
 
 DISubrangeAttr DebugImporter::translateImpl(llvm::DISubrange *node) {
@@ -374,6 +378,9 @@ getRecSelfConstructor(llvm::DINode *node) {
       .Case([&](llvm::DICompositeType *) {
         return CtorType(DICompositeTypeAttr::getRecSelf);
       })
+      .Case([&](llvm::DISubprogram *) {
+        return CtorType(DISubprogramAttr::getRecSelf);
+      })
       .Default(CtorType());
 }
 
diff --git a/mlir/lib/Target/LLVMIR/DebugTranslation.cpp b/mlir/lib/Target/LLVMIR/DebugTranslation.cpp
index 042e015f107fea..55060659dd3560 100644
--- a/mlir/lib/Target/LLVMIR/DebugTranslation.cpp
+++ b/mlir/lib/Target/LLVMIR/DebugTranslation.cpp
@@ -96,6 +96,17 @@ llvm::MDString *DebugTranslation::getMDStringOrNull(StringAttr stringAttr) {
   return llvm::MDString::get(llvmCtx, stringAttr);
 }
 
+llvm::MDTuple *
+DebugTranslation::getMDTupleOrNull(ArrayRef<DINodeAttr> elements) {
+  if (elements.empty())
+    return nullptr;
+  SmallVector<llvm::Metadata *> llvmElements = llvm::to_vector(
+      llvm::map_range(elements, [&](DINodeAttr attr) -> llvm::Metadata * {
+        return translate(attr);
+      }));
+  return llvm::MDNode::get(llvmCtx, llvmElements);
+}
+
 llvm::DIBasicType *DebugTranslation::translateImpl(DIBasicTypeAttr attr) {
   return llvm::DIBasicType::get(
       llvmCtx, attr.getTag(), getMDStringOrNull(attr.getName()),
@@ -138,6 +149,17 @@ DebugTranslation::translateTemporaryImpl(DICompositeTypeAttr attr) {
       /*VTableHolder=*/nullptr);
 }
 
+llvm::TempDISubprogram
+DebugTranslation::translateTemporaryImpl(DISubprogramAttr attr) {
+  return llvm::DISubprogram::getTemporary(
+      llvmCtx, /*Scope=*/nullptr, /*Name=*/{}, /*LinkageName=*/{},
+      /*File=*/nullptr, attr.getLine(), /*Type=*/nullptr,
+      /*ScopeLine=*/0, /*ContainingType=*/nullptr, /*VirtualIndex=*/0,
+      /*ThisAdjustment=*/0, llvm::DINode::FlagZero,
+      static_cast<llvm::DISubprogram::DISPFlags>(attr.getSubprogramFlags()),
+      /*Unit=*/nullptr);
+}
+
 llvm::DICompositeType *
 DebugTranslation::translateImpl(DICompositeTypeAttr attr) {
   // TODO: Use distinct attributes to model this, once they have landed.
@@ -151,10 +173,6 @@ DebugTranslation::translateImpl(DICompositeTypeAttr attr) {
     isDistinct = true;
   }
 
-  SmallVector<llvm::Metadata *> elements;
-  for (DINodeAttr member : attr.getElements())
-    elements.push_back(translate(member));
-
   return getDistinctOrUnique<llvm::DICompositeType>(
       isDistinct, llvmCtx, attr.getTag(), getMDStringOrNull(attr.getName()),
       translate(attr.getFile()), attr.getLine(), translate(attr.getScope()),
@@ -162,7 +180,7 @@ DebugTranslation::translateImpl(DICompositeTypeAttr attr) {
       attr.getAlignInBits(),
       /*OffsetInBits=*/0,
       /*Flags=*/static_cast<llvm::DINode::DIFlags>(attr.getFlags()),
-      llvm::MDNode::get(llvmCtx, elements),
+      getMDTupleOrNull(attr.getElements()),
       /*RuntimeLang=*/0, /*VTableHolder=*/nullptr,
       /*TemplateParams=*/nullptr, /*Identifier=*/nullptr,
       /*Discriminator=*/nullptr,
@@ -242,22 +260,21 @@ DebugTranslation::translateImpl(DIGlobalVariableAttr attr) {
       attr.getIsDefined(), nullptr, nullptr, attr.getAlignInBits(), nullptr);
 }
 
-llvm::DIType *
+llvm::DINode *
 DebugTranslation::translateRecursive(DIRecursiveTypeAttrInterface attr) {
   DistinctAttr recursiveId = attr.getRecId();
-  if (auto *iter = recursiveTypeMap.find(recursiveId);
-      iter != recursiveTypeMap.end()) {
+  if (auto *iter = recursiveNodeMap.find(recursiveId);
+      iter != recursiveNodeMap.end()) {
     return iter->second;
-  } else {
-    assert(!attr.isRecSelf() && "unbound DI recursive self type");
   }
+  assert(!attr.isRecSelf() && "unbound DI recursive self reference");
 
-  auto setRecursivePlaceholder = [&](llvm::DIType *placeholder) {
-    recursiveTypeMap.try_emplace(recursiveId, placeholder);
+  auto setRecursivePlaceholder = [&](llvm::DINode *placeholder) {
+    recursiveNodeMap.try_emplace(recursiveId, placeholder);
   };
 
-  llvm::DIType *result =
-      TypeSwitch<DIRecursiveTypeAttrInterface, llvm::DIType *>(attr)
+  llvm::DINode *result =
+      TypeSwitch<DIRecursiveTypeAttrInterface, llvm::DINode *>(attr)
           .Case<DICompositeTypeAttr>([&](auto attr) {
             auto temporary = translateTemporaryImpl(attr);
             setRecursivePlaceholder(temporary.get());
@@ -266,11 +283,20 @@ DebugTranslation::translateRecursive(DIRecursiveTypeAttrInterface attr) {
             auto *concrete = translateImpl(attr);
             temporary->replaceAllUsesWith(concrete);
             return concrete;
+          })
+          .Case<DISubprogramAttr>([&](auto attr) {
+            auto temporary = translateTemporaryImpl(attr);
+            setRecursivePlaceholder(temporary.get());
+            // Must call `translateImpl` directly instead of `translate` to
+            // avoid handling the recursive interface again.
+            auto *concrete = translateImpl(attr);
+            temporary->replaceAllUsesWith(concrete);
+            return concrete;
           });
 
-  assert(recursiveTypeMap.back().first == recursiveId &&
+  assert(recursiveNodeMap.back().first == recursiveId &&
          "internal inconsistency: unexpected recursive translation stack");
-  recursiveTypeMap.pop_back();
+  recursiveNodeMap.pop_back();
 
   return result;
 }
@@ -297,6 +323,7 @@ llvm::DISubprogram *DebugTranslation::translateImpl(DISubprogramAttr attr) {
 
   bool isDefinition = static_cast<bool>(attr.getSubprogramFlags() &
                                         LLVM::DISubprogramFlags::Definition);
+
   llvm::DISubprogram *node = getDistinctOrUnique<llvm::DI...
[truncated]

@llvmbot
Copy link
Collaborator

llvmbot commented Aug 29, 2024

@llvm/pr-subscribers-mlir

Author: Tobias Gysi (gysit)

Changes

This commit implements LLVM_DIRecursiveTypeAttrInterface for the DISubprogramAttr to ensure cyclic subprograms can be imported properly. In the process multiple shortcuts around the recently introduced DIImportedEntityAttr can be removed.


Patch is 35.66 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/106571.diff

12 Files Affected:

  • (modified) mlir/include/mlir-c/Dialect/LLVM.h (+14-8)
  • (modified) mlir/include/mlir/Dialect/LLVMIR/LLVMAttrDefs.td (+23-10)
  • (modified) mlir/lib/CAPI/Dialect/LLVM.cpp (+21-10)
  • (modified) mlir/lib/Dialect/LLVMIR/IR/LLVMAttrs.cpp (+16)
  • (modified) mlir/lib/Dialect/LLVMIR/IR/LLVMDialect.cpp (+3-3)
  • (modified) mlir/lib/Dialect/LLVMIR/Transforms/DIScopeForLLVMFuncOp.cpp (+2-2)
  • (modified) mlir/lib/Target/LLVMIR/DebugImporter.cpp (+15-8)
  • (modified) mlir/lib/Target/LLVMIR/DebugTranslation.cpp (+54-44)
  • (modified) mlir/lib/Target/LLVMIR/DebugTranslation.h (+9-9)
  • (modified) mlir/test/CAPI/llvm.c (+14-5)
  • (modified) mlir/test/Target/LLVMIR/Import/debug-info.ll (+9-11)
  • (modified) mlir/test/Target/LLVMIR/llvmir-debug.mlir (+20-15)
diff --git a/mlir/include/mlir-c/Dialect/LLVM.h b/mlir/include/mlir-c/Dialect/LLVM.h
index 5eb96a86e472d6..561d698f722afe 100644
--- a/mlir/include/mlir-c/Dialect/LLVM.h
+++ b/mlir/include/mlir-c/Dialect/LLVM.h
@@ -234,6 +234,9 @@ MLIR_CAPI_EXPORTED MlirAttribute mlirLLVMDIBasicTypeAttrGet(
     MlirContext ctx, unsigned int tag, MlirAttribute name, uint64_t sizeInBits,
     MlirLLVMTypeEncoding encoding);
 
+/// Creates a self-referencing LLVM DICompositeType attribute.
+MlirAttribute mlirLLVMDICompositeTypeAttrGetSelfRec(MlirAttribute recId);
+
 /// Creates a LLVM DICompositeType attribute.
 MLIR_CAPI_EXPORTED MlirAttribute mlirLLVMDICompositeTypeAttrGet(
     MlirContext ctx, unsigned int tag, MlirAttribute recId, MlirAttribute name,
@@ -311,13 +314,16 @@ MLIR_CAPI_EXPORTED MlirAttribute mlirLLVMDILocalVariableAttrGet(
     MlirAttribute diFile, unsigned int line, unsigned int arg,
     unsigned int alignInBits, MlirAttribute diType, int64_t flags);
 
+/// Creates a self-referencing LLVM DISubprogramAttr attribute.
+MlirAttribute mlirLLVMDISubprogramAttrGetSelfRec(MlirAttribute recId);
+
 /// Creates a LLVM DISubprogramAttr attribute.
 MLIR_CAPI_EXPORTED MlirAttribute mlirLLVMDISubprogramAttrGet(
-    MlirContext ctx, MlirAttribute id, MlirAttribute compileUnit,
-    MlirAttribute scope, MlirAttribute name, MlirAttribute linkageName,
-    MlirAttribute file, unsigned int line, unsigned int scopeLine,
-    uint64_t subprogramFlags, MlirAttribute type, intptr_t nRetainedNodes,
-    MlirAttribute const *retainedNodes);
+    MlirContext ctx, MlirAttribute id, MlirAttribute recId,
+    MlirAttribute compileUnit, MlirAttribute scope, MlirAttribute name,
+    MlirAttribute linkageName, MlirAttribute file, unsigned int line,
+    unsigned int scopeLine, uint64_t subprogramFlags, MlirAttribute type,
+    intptr_t nRetainedNodes, MlirAttribute const *retainedNodes);
 
 /// Gets the scope from this DISubprogramAttr.
 MLIR_CAPI_EXPORTED MlirAttribute
@@ -356,9 +362,9 @@ MLIR_CAPI_EXPORTED MlirAttribute mlirLLVMDIModuleAttrGet(
 
 /// Creates a LLVM DIImportedEntityAttr attribute.
 MLIR_CAPI_EXPORTED MlirAttribute mlirLLVMDIImportedEntityAttrGet(
-    MlirContext ctx, unsigned int tag, MlirAttribute entity, MlirAttribute file,
-    unsigned int line, MlirAttribute name, intptr_t nElements,
-    MlirAttribute const *elements);
+    MlirContext ctx, unsigned int tag, MlirAttribute scope,
+    MlirAttribute entity, MlirAttribute file, unsigned int line,
+    MlirAttribute name, intptr_t nElements, MlirAttribute const *elements);
 
 /// Gets the scope of this DIModuleAttr.
 MLIR_CAPI_EXPORTED MlirAttribute
diff --git a/mlir/include/mlir/Dialect/LLVMIR/LLVMAttrDefs.td b/mlir/include/mlir/Dialect/LLVMIR/LLVMAttrDefs.td
index e57be7f760d380..224e58f3216abf 100644
--- a/mlir/include/mlir/Dialect/LLVMIR/LLVMAttrDefs.td
+++ b/mlir/include/mlir/Dialect/LLVMIR/LLVMAttrDefs.td
@@ -554,14 +554,16 @@ def LLVM_DILocalVariableAttr : LLVM_Attr<"DILocalVariable", "di_local_variable",
 //===----------------------------------------------------------------------===//
 
 def LLVM_DISubprogramAttr : LLVM_Attr<"DISubprogram", "di_subprogram",
-                                      /*traits=*/[], "DIScopeAttr"> {
+                                      [LLVM_DIRecursiveTypeAttrInterface],
+                                      "DIScopeAttr"> {
   let parameters = (ins
     OptionalParameter<"DistinctAttr">:$id,
+    OptionalParameter<"DistinctAttr">:$recId,
     OptionalParameter<"DICompileUnitAttr">:$compileUnit,
-    "DIScopeAttr":$scope,
+    OptionalParameter<"DIScopeAttr">:$scope,
     OptionalParameter<"StringAttr">:$name,
     OptionalParameter<"StringAttr">:$linkageName,
-    "DIFileAttr":$file,
+    OptionalParameter<"DIFileAttr">:$file,
     OptionalParameter<"unsigned">:$line,
     OptionalParameter<"unsigned">:$scopeLine,
     OptionalParameter<"DISubprogramFlags">:$subprogramFlags,
@@ -577,13 +579,28 @@ def LLVM_DISubprogramAttr : LLVM_Attr<"DISubprogram", "di_subprogram",
       "ArrayRef<DINodeAttr>":$retainedNodes
     ), [{
       MLIRContext *ctx = file.getContext();
-      return $_get(ctx, id, compileUnit, scope, StringAttr::get(ctx, name),
+      return $_get(ctx, id, /*recId=*/nullptr, compileUnit, scope,
+                   StringAttr::get(ctx, name),
                    StringAttr::get(ctx, linkageName), file, line,
                    scopeLine, subprogramFlags, type, retainedNodes);
     }]>
   ];
-
   let assemblyFormat = "`<` struct(params) `>`";
+  let extraClassDeclaration = [{
+    /// Requirements of DIRecursiveTypeAttrInterface.
+    /// @{
+
+    /// Get whether this attr describes a recursive self reference.
+    bool isRecSelf() { return !getScope(); }
+
+    /// Get a copy of this type attr but with the recursive ID set to `recId`.
+    DIRecursiveTypeAttrInterface withRecId(DistinctAttr recId);
+
+    /// Build a rec-self instance using the provided `recId`.
+    static DIRecursiveTypeAttrInterface getRecSelf(DistinctAttr recId);
+
+    /// @}
+  }];
 }
 
 //===----------------------------------------------------------------------===//
@@ -627,13 +644,9 @@ def LLVM_DINamespaceAttr : LLVM_Attr<"DINamespace", "di_namespace",
 
 def LLVM_DIImportedEntityAttr : LLVM_Attr<"DIImportedEntity", "di_imported_entity",
                                            /*traits=*/[], "DINodeAttr"> {
-  /// TODO: DIImportedEntity has a 'scope' field which represents the scope where
-  /// this entity is imported. Currently, we are not adding a 'scope' field in
-  /// DIImportedEntityAttr to avoid cyclic dependency. As DIImportedEntityAttr
-  /// entries will be contained inside a scope entity (e.g. DISubprogramAttr),
-  /// the scope can easily be inferred.
   let parameters = (ins
     LLVM_DITagParameter:$tag,
+    "DIScopeAttr":$scope,
     "DINodeAttr":$entity,
     OptionalParameter<"DIFileAttr">:$file,
     OptionalParameter<"unsigned">:$line,
diff --git a/mlir/lib/CAPI/Dialect/LLVM.cpp b/mlir/lib/CAPI/Dialect/LLVM.cpp
index 13341f0c4de881..ed4d028e192f53 100644
--- a/mlir/lib/CAPI/Dialect/LLVM.cpp
+++ b/mlir/lib/CAPI/Dialect/LLVM.cpp
@@ -159,6 +159,11 @@ MlirAttribute mlirLLVMDIBasicTypeAttrGet(MlirContext ctx, unsigned int tag,
       unwrap(ctx), tag, cast<StringAttr>(unwrap(name)), sizeInBits, encoding));
 }
 
+MlirAttribute mlirLLVMDICompositeTypeAttrGetSelfRec(MlirAttribute recId) {
+  return wrap(
+      DICompositeTypeAttr::getRecSelf(cast<DistinctAttr>(unwrap(recId))));
+}
+
 MlirAttribute mlirLLVMDICompositeTypeAttrGet(
     MlirContext ctx, unsigned int tag, MlirAttribute recId, MlirAttribute name,
     MlirAttribute file, uint32_t line, MlirAttribute scope,
@@ -289,16 +294,21 @@ MlirAttribute mlirLLVMDISubroutineTypeAttrGet(MlirContext ctx,
                           [](Attribute a) { return cast<DITypeAttr>(a); })));
 }
 
+MlirAttribute mlirLLVMDISubprogramAttrGetSelfRec(MlirAttribute recId) {
+  return wrap(DISubprogramAttr::getRecSelf(cast<DistinctAttr>(unwrap(recId))));
+}
+
 MlirAttribute mlirLLVMDISubprogramAttrGet(
-    MlirContext ctx, MlirAttribute id, MlirAttribute compileUnit,
-    MlirAttribute scope, MlirAttribute name, MlirAttribute linkageName,
-    MlirAttribute file, unsigned int line, unsigned int scopeLine,
-    uint64_t subprogramFlags, MlirAttribute type, intptr_t nRetainedNodes,
-    MlirAttribute const *retainedNodes) {
+    MlirContext ctx, MlirAttribute id, MlirAttribute recId,
+    MlirAttribute compileUnit, MlirAttribute scope, MlirAttribute name,
+    MlirAttribute linkageName, MlirAttribute file, unsigned int line,
+    unsigned int scopeLine, uint64_t subprogramFlags, MlirAttribute type,
+    intptr_t nRetainedNodes, MlirAttribute const *retainedNodes) {
   SmallVector<Attribute> nodesStorage;
   nodesStorage.reserve(nRetainedNodes);
   return wrap(DISubprogramAttr::get(
       unwrap(ctx), cast<DistinctAttr>(unwrap(id)),
+      cast<DistinctAttr>(unwrap(recId)),
       cast<DICompileUnitAttr>(unwrap(compileUnit)),
       cast<DIScopeAttr>(unwrap(scope)), cast<StringAttr>(unwrap(name)),
       cast<StringAttr>(unwrap(linkageName)), cast<DIFileAttr>(unwrap(file)),
@@ -353,14 +363,15 @@ MlirAttribute mlirLLVMDIModuleAttrGetScope(MlirAttribute diModule) {
 }
 
 MlirAttribute mlirLLVMDIImportedEntityAttrGet(
-    MlirContext ctx, unsigned int tag, MlirAttribute entity, MlirAttribute file,
-    unsigned int line, MlirAttribute name, intptr_t nElements,
-    MlirAttribute const *elements) {
+    MlirContext ctx, unsigned int tag, MlirAttribute scope,
+    MlirAttribute entity, MlirAttribute file, unsigned int line,
+    MlirAttribute name, intptr_t nElements, MlirAttribute const *elements) {
   SmallVector<Attribute> elementsStorage;
   elementsStorage.reserve(nElements);
   return wrap(DIImportedEntityAttr::get(
-      unwrap(ctx), tag, cast<DINodeAttr>(unwrap(entity)),
-      cast<DIFileAttr>(unwrap(file)), line, cast<StringAttr>(unwrap(name)),
+      unwrap(ctx), tag, cast<DIScopeAttr>(unwrap(scope)),
+      cast<DINodeAttr>(unwrap(entity)), cast<DIFileAttr>(unwrap(file)), line,
+      cast<StringAttr>(unwrap(name)),
       llvm::map_to_vector(unwrapList(nElements, elements, elementsStorage),
                           [](Attribute a) { return cast<DINodeAttr>(a); })));
 }
diff --git a/mlir/lib/Dialect/LLVMIR/IR/LLVMAttrs.cpp b/mlir/lib/Dialect/LLVMIR/IR/LLVMAttrs.cpp
index 98a9659735e7e6..0ce719b9a750ca 100644
--- a/mlir/lib/Dialect/LLVMIR/IR/LLVMAttrs.cpp
+++ b/mlir/lib/Dialect/LLVMIR/IR/LLVMAttrs.cpp
@@ -215,6 +215,22 @@ DICompositeTypeAttr::getRecSelf(DistinctAttr recId) {
                                   {}, DIFlags(), 0, 0, {}, {}, {}, {}, {});
 }
 
+//===----------------------------------------------------------------------===//
+// DISubprogramAttr
+//===----------------------------------------------------------------------===//
+
+DIRecursiveTypeAttrInterface DISubprogramAttr::withRecId(DistinctAttr recId) {
+  return DISubprogramAttr::get(
+      getContext(), getId(), recId, getCompileUnit(), getScope(), getName(),
+      getLinkageName(), getFile(), getLine(), getScopeLine(),
+      getSubprogramFlags(), getType(), getRetainedNodes());
+}
+
+DIRecursiveTypeAttrInterface DISubprogramAttr::getRecSelf(DistinctAttr recId) {
+  return DISubprogramAttr::get(recId.getContext(), {}, recId, {}, {}, {}, {}, 0,
+                               0, {}, {}, {}, {});
+}
+
 //===----------------------------------------------------------------------===//
 // TargetFeaturesAttr
 //===----------------------------------------------------------------------===//
diff --git a/mlir/lib/Dialect/LLVMIR/IR/LLVMDialect.cpp b/mlir/lib/Dialect/LLVMIR/IR/LLVMDialect.cpp
index 3870aab52f199d..6e4a964f1fc93c 100644
--- a/mlir/lib/Dialect/LLVMIR/IR/LLVMDialect.cpp
+++ b/mlir/lib/Dialect/LLVMIR/IR/LLVMDialect.cpp
@@ -3155,9 +3155,9 @@ struct LLVMOpAsmDialectInterface : public OpAsmDialectInterface {
         .Case<AccessGroupAttr, AliasScopeAttr, AliasScopeDomainAttr,
               DIBasicTypeAttr, DICompileUnitAttr, DICompositeTypeAttr,
               DIDerivedTypeAttr, DIFileAttr, DIGlobalVariableAttr,
-              DIGlobalVariableExpressionAttr, DILabelAttr, DILexicalBlockAttr,
-              DILexicalBlockFileAttr, DILocalVariableAttr, DIModuleAttr,
-              DINamespaceAttr, DINullTypeAttr, DIStringTypeAttr,
+              DIGlobalVariableExpressionAttr, DIImportedEntityAttr, DILabelAttr,
+              DILexicalBlockAttr, DILexicalBlockFileAttr, DILocalVariableAttr,
+              DIModuleAttr, DINamespaceAttr, DINullTypeAttr, DIStringTypeAttr,
               DISubprogramAttr, DISubroutineTypeAttr, LoopAnnotationAttr,
               LoopVectorizeAttr, LoopInterleaveAttr, LoopUnrollAttr,
               LoopUnrollAndJamAttr, LoopLICMAttr, LoopDistributeAttr,
diff --git a/mlir/lib/Dialect/LLVMIR/Transforms/DIScopeForLLVMFuncOp.cpp b/mlir/lib/Dialect/LLVMIR/Transforms/DIScopeForLLVMFuncOp.cpp
index c75f44bf3976a9..ad9aa7ac80cafa 100644
--- a/mlir/lib/Dialect/LLVMIR/Transforms/DIScopeForLLVMFuncOp.cpp
+++ b/mlir/lib/Dialect/LLVMIR/Transforms/DIScopeForLLVMFuncOp.cpp
@@ -76,8 +76,8 @@ static void addScopeToFunction(LLVM::LLVMFuncOp llvmFunc,
     compileUnitAttr = {};
   }
   auto subprogramAttr = LLVM::DISubprogramAttr::get(
-      context, id, compileUnitAttr, fileAttr, funcNameAttr, funcNameAttr,
-      fileAttr,
+      context, id, /*recId=*/{}, compileUnitAttr, fileAttr, funcNameAttr,
+      funcNameAttr, fileAttr,
       /*line=*/line,
       /*scopeline=*/col, subprogramFlags, subroutineTypeAttr,
       /*retainedNodes=*/{});
diff --git a/mlir/lib/Target/LLVMIR/DebugImporter.cpp b/mlir/lib/Target/LLVMIR/DebugImporter.cpp
index ce3643f513d34a..509451057dd737 100644
--- a/mlir/lib/Target/LLVMIR/DebugImporter.cpp
+++ b/mlir/lib/Target/LLVMIR/DebugImporter.cpp
@@ -217,8 +217,8 @@ DebugImporter::translateImpl(llvm::DIImportedEntity *node) {
   }
 
   return DIImportedEntityAttr::get(
-      context, node->getTag(), translate(node->getEntity()),
-      translate(node->getFile()), node->getLine(),
+      context, node->getTag(), translate(node->getScope()),
+      translate(node->getEntity()), translate(node->getFile()), node->getLine(),
       getStringAttrOrNull(node->getRawName()), elements);
 }
 
@@ -227,6 +227,7 @@ DISubprogramAttr DebugImporter::translateImpl(llvm::DISubprogram *node) {
   mlir::DistinctAttr id;
   if (node->isDistinct())
     id = getOrCreateDistinctID(node);
+
   // Return nullptr if the scope or type is invalid.
   DIScopeAttr scope = translate(node->getScope());
   if (node->getScope() && !scope)
@@ -238,16 +239,19 @@ DISubprogramAttr DebugImporter::translateImpl(llvm::DISubprogram *node) {
   if (node->getType() && !type)
     return nullptr;
 
+  // Convert the retained nodes but drop all of them if one of them is invalid.
   SmallVector<DINodeAttr> retainedNodes;
   for (llvm::DINode *retainedNode : node->getRetainedNodes())
     retainedNodes.push_back(translate(retainedNode));
+  if (llvm::is_contained(retainedNodes, nullptr))
+    retainedNodes.clear();
 
-  return DISubprogramAttr::get(context, id, translate(node->getUnit()), scope,
-                               getStringAttrOrNull(node->getRawName()),
-                               getStringAttrOrNull(node->getRawLinkageName()),
-                               translate(node->getFile()), node->getLine(),
-                               node->getScopeLine(), *subprogramFlags, type,
-                               retainedNodes);
+  return DISubprogramAttr::get(
+      context, id, /*recId=*/{}, translate(node->getUnit()), scope,
+      getStringAttrOrNull(node->getRawName()),
+      getStringAttrOrNull(node->getRawLinkageName()),
+      translate(node->getFile()), node->getLine(), node->getScopeLine(),
+      *subprogramFlags, type, retainedNodes);
 }
 
 DISubrangeAttr DebugImporter::translateImpl(llvm::DISubrange *node) {
@@ -374,6 +378,9 @@ getRecSelfConstructor(llvm::DINode *node) {
       .Case([&](llvm::DICompositeType *) {
         return CtorType(DICompositeTypeAttr::getRecSelf);
       })
+      .Case([&](llvm::DISubprogram *) {
+        return CtorType(DISubprogramAttr::getRecSelf);
+      })
       .Default(CtorType());
 }
 
diff --git a/mlir/lib/Target/LLVMIR/DebugTranslation.cpp b/mlir/lib/Target/LLVMIR/DebugTranslation.cpp
index 042e015f107fea..55060659dd3560 100644
--- a/mlir/lib/Target/LLVMIR/DebugTranslation.cpp
+++ b/mlir/lib/Target/LLVMIR/DebugTranslation.cpp
@@ -96,6 +96,17 @@ llvm::MDString *DebugTranslation::getMDStringOrNull(StringAttr stringAttr) {
   return llvm::MDString::get(llvmCtx, stringAttr);
 }
 
+llvm::MDTuple *
+DebugTranslation::getMDTupleOrNull(ArrayRef<DINodeAttr> elements) {
+  if (elements.empty())
+    return nullptr;
+  SmallVector<llvm::Metadata *> llvmElements = llvm::to_vector(
+      llvm::map_range(elements, [&](DINodeAttr attr) -> llvm::Metadata * {
+        return translate(attr);
+      }));
+  return llvm::MDNode::get(llvmCtx, llvmElements);
+}
+
 llvm::DIBasicType *DebugTranslation::translateImpl(DIBasicTypeAttr attr) {
   return llvm::DIBasicType::get(
       llvmCtx, attr.getTag(), getMDStringOrNull(attr.getName()),
@@ -138,6 +149,17 @@ DebugTranslation::translateTemporaryImpl(DICompositeTypeAttr attr) {
       /*VTableHolder=*/nullptr);
 }
 
+llvm::TempDISubprogram
+DebugTranslation::translateTemporaryImpl(DISubprogramAttr attr) {
+  return llvm::DISubprogram::getTemporary(
+      llvmCtx, /*Scope=*/nullptr, /*Name=*/{}, /*LinkageName=*/{},
+      /*File=*/nullptr, attr.getLine(), /*Type=*/nullptr,
+      /*ScopeLine=*/0, /*ContainingType=*/nullptr, /*VirtualIndex=*/0,
+      /*ThisAdjustment=*/0, llvm::DINode::FlagZero,
+      static_cast<llvm::DISubprogram::DISPFlags>(attr.getSubprogramFlags()),
+      /*Unit=*/nullptr);
+}
+
 llvm::DICompositeType *
 DebugTranslation::translateImpl(DICompositeTypeAttr attr) {
   // TODO: Use distinct attributes to model this, once they have landed.
@@ -151,10 +173,6 @@ DebugTranslation::translateImpl(DICompositeTypeAttr attr) {
     isDistinct = true;
   }
 
-  SmallVector<llvm::Metadata *> elements;
-  for (DINodeAttr member : attr.getElements())
-    elements.push_back(translate(member));
-
   return getDistinctOrUnique<llvm::DICompositeType>(
       isDistinct, llvmCtx, attr.getTag(), getMDStringOrNull(attr.getName()),
       translate(attr.getFile()), attr.getLine(), translate(attr.getScope()),
@@ -162,7 +180,7 @@ DebugTranslation::translateImpl(DICompositeTypeAttr attr) {
       attr.getAlignInBits(),
       /*OffsetInBits=*/0,
       /*Flags=*/static_cast<llvm::DINode::DIFlags>(attr.getFlags()),
-      llvm::MDNode::get(llvmCtx, elements),
+      getMDTupleOrNull(attr.getElements()),
       /*RuntimeLang=*/0, /*VTableHolder=*/nullptr,
       /*TemplateParams=*/nullptr, /*Identifier=*/nullptr,
       /*Discriminator=*/nullptr,
@@ -242,22 +260,21 @@ DebugTranslation::translateImpl(DIGlobalVariableAttr attr) {
       attr.getIsDefined(), nullptr, nullptr, attr.getAlignInBits(), nullptr);
 }
 
-llvm::DIType *
+llvm::DINode *
 DebugTranslation::translateRecursive(DIRecursiveTypeAttrInterface attr) {
   DistinctAttr recursiveId = attr.getRecId();
-  if (auto *iter = recursiveTypeMap.find(recursiveId);
-      iter != recursiveTypeMap.end()) {
+  if (auto *iter = recursiveNodeMap.find(recursiveId);
+      iter != recursiveNodeMap.end()) {
     return iter->second;
-  } else {
-    assert(!attr.isRecSelf() && "unbound DI recursive self type");
   }
+  assert(!attr.isRecSelf() && "unbound DI recursive self reference");
 
-  auto setRecursivePlaceholder = [&](llvm::DIType *placeholder) {
-    recursiveTypeMap.try_emplace(recursiveId, placeholder);
+  auto setRecursivePlaceholder = [&](llvm::DINode *placeholder) {
+    recursiveNodeMap.try_emplace(recursiveId, placeholder);
   };
 
-  llvm::DIType *result =
-      TypeSwitch<DIRecursiveTypeAttrInterface, llvm::DIType *>(attr)
+  llvm::DINode *result =
+      TypeSwitch<DIRecursiveTypeAttrInterface, llvm::DINode *>(attr)
           .Case<DICompositeTypeAttr>([&](auto attr) {
             auto temporary = translateTemporaryImpl(attr);
             setRecursivePlaceholder(temporary.get());
@@ -266,11 +283,20 @@ DebugTranslation::translateRecursive(DIRecursiveTypeAttrInterface attr) {
             auto *concrete = translateImpl(attr);
             temporary->replaceAllUsesWith(concrete);
             return concrete;
+          })
+          .Case<DISubprogramAttr>([&](auto attr) {
+            auto temporary = translateTemporaryImpl(attr);
+            setRecursivePlaceholder(temporary.get());
+            // Must call `translateImpl` directly instead of `translate` to
+            // avoid handling the recursive interface again.
+            auto *concrete = translateImpl(attr);
+            temporary->replaceAllUsesWith(concrete);
+            return concrete;
           });
 
-  assert(recursiveTypeMap.back().first == recursiveId &&
+  assert(recursiveNodeMap.back().first == recursiveId &&
          "internal inconsistency: unexpected recursive translation stack");
-  recursiveTypeMap.pop_back();
+  recursiveNodeMap.pop_back();
 
   return result;
 }
@@ -297,6 +323,7 @@ llvm::DISubprogram *DebugTranslation::translateImpl(DISubprogramAttr attr) {
 
   bool isDefinition = static_cast<bool>(attr.getSubprogramFlags() &
                                         LLVM::DISubprogramFlags::Definition);
+
   llvm::DISubprogram *node = getDistinctOrUnique<llvm::DI...
[truncated]

@gysit
Copy link
Contributor Author

gysit commented Aug 29, 2024

@abidh this tries to address the import issue that came up after #103055 by supporting cyclic subprograms. @zyx-billy would be nice if you can have a look since I may have missed something.

Copy link
Contributor

@zyx-billy zyx-billy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @gysit for taking this on! LGTM just one question with the RecSelf definition.

mlir/lib/CAPI/Dialect/LLVM.cpp Outdated Show resolved Hide resolved
; CHECK-DAG: #[[SP_OUTER:.+]] = #llvm.di_subprogram<id = [[SP_ID]], compileUnit = #{{.*}}, scope = #[[COMP]], name = "class_method", file = #{{.*}}, subprogramFlags = Definition, type = #[[SP_TYPE_OUTER]]>
; CHECK-DAG: #[[LOC]] = loc(fused<#[[SP_OUTER]]>
; CHECK-DAG: #[[SP:.+]] = #llvm.di_subprogram<id = [[SP_ID:.+]], recId = [[REC_ID]], compileUnit = #{{.*}}, scope = #[[COMP]], name = "class_method", file = #{{.*}}, subprogramFlags = Definition, type = #[[SP_TYPE]]>
; CHECK-DAG: #[[LOC]] = loc(fused<#[[SP]]>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh nice! I guess this should also result in smaller translated mlir sizes in general, especially if previously there were many cycles that began with subprograms?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I would expect this further reduces ir size!

/// @{

/// Get whether this attr describes a recursive self reference.
bool isRecSelf() { return !getScope(); }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My only slight concern is if it's possible to see a DISubprogram without a scope naturally (it looks like it's allowed to be null here but I'm not sure if it's illegal/impractical elsewhere...). If it's not supposed to happen, can we enforce during import that all llvm's DISubprograms have a scope?

The concern is if we categorize a node as RecSelf when it isn't, we won't be able to tell the outer real node from the inner recursive reference (which triggers an assertion during export).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alternatively we could maybe add a tag parameter to the enum similar to the one we have on DICompositeType? It is a bit redundant though since a subprogram is a subprogram. So maybe a separate boolean makes sense.

It is indeed true that using the scope feels a bit hacky.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah looking back it was pure luck we had a tag in DICompositeTypeAttr for this 😂 I guess adding an enum tag or a bool flag probably works the same. A tag certainly increases the textual IR a lot... Or, worst case we just make recId itself a dedicated attr with DistinctAttr + bool (may be handy if we need it for more nodes later).

@abidh
Copy link
Contributor

abidh commented Aug 30, 2024

@abidh this tries to address the import issue that came up after #103055 by supporting cyclic subprograms. @zyx-billy would be nice if you can have a look since I may have missed something.

Thanks for doing this. The DISubprogramAttr here will need to be updated to fix the flang build after this change.

This commit implements LLVM_DIRecursiveTypeAttrInterface for the
DISubprogramAttr to ensure cyclic subprograms can be imported properly.
In the process multiple shortcuts around the recently introduced
DIImportedEntityAttr can be removed.
@llvmbot llvmbot added flang Flang issues not falling into any other category flang:fir-hlfir labels Aug 30, 2024
@gysit
Copy link
Contributor Author

gysit commented Aug 30, 2024

I went for a boolean flag but put everything at the beginning of the recursive attributes.

I tried to fix flang but building it locally turns out to be difficult due to some dependencies. I may have to follow up next week...

@gysit gysit requested a review from zyx-billy August 30, 2024 12:02
Copy link
Contributor

@zyx-billy zyx-billy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks for making the composite type consistent too.

getContext(), getId(), recId, getCompileUnit(), getScope(), getName(),
getLinkageName(), getFile(), getLine(), getScopeLine(),
getSubprogramFlags(), getType(), getRetainedNodes());
getContext(), recId, /*isRecSelf=*/false, getId(), getCompileUnit(),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor nit, I think it's safer to also getIsRecSelf() here just in case some place called this on an existing rec-self instance to change the ID (technically they should just use getRecSelf but we don't check).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah right. Makes sense!

@@ -448,7 +448,7 @@ llvm.func @func_debug_directives() {
#di_compile_unit = #llvm.di_compile_unit<id = distinct[1]<>, sourceLanguage = DW_LANG_C, file = #di_file, isOptimized = false, emissionKind = None>

// Recursive type itself.
#di_struct_self = #llvm.di_composite_type<tag = DW_TAG_null, recId = distinct[0]<>>
#di_struct_self = #llvm.di_composite_type<recId = distinct[0]<>, isRecSelf = false>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you mean to set it to true instead? I guess we didn't really check in the rec-self case, but might be good to set it so it's not confusing.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That was a classic search replace thingy just saw there are more such instances. Will fix them.

@gysit gysit merged commit d884b77 into llvm:main Aug 31, 2024
8 checks passed
@llvm-ci
Copy link
Collaborator

llvm-ci commented Aug 31, 2024

LLVM Buildbot has detected a new failure on builder mlir-nvidia running on mlir-nvidia while building flang,mlir at step 5 "build-check-mlir-build-only".

Full details are available at: https://lab.llvm.org/buildbot/#/builders/138/builds/2992

Here is the relevant piece of the build log for the reference
Step 5 (build-check-mlir-build-only) failure: build (failure)
...
298.984 [50/16/4885] Linking C executable bin/mlir-capi-sparse-tensor-test
298.991 [49/16/4886] Creating library symlink lib/libMLIRDLTITestPasses.so
299.014 [48/16/4887] Linking C executable bin/mlir-capi-transform-interpreter-test
299.034 [47/16/4888] Creating library symlink lib/libMLIRTestPass.so
299.035 [46/16/4889] Linking C executable bin/mlir-capi-execution-engine-test
299.038 [45/16/4890] Linking CXX shared library lib/libMLIRFuncTestPasses.so.20.0git
299.046 [44/16/4891] Creating library symlink lib/libMLIRFuncTestPasses.so
299.079 [43/16/4892] Linking CXX shared library lib/libMLIRMemRefTestPasses.so.20.0git
299.108 [42/16/4893] Creating library symlink lib/libMLIRMemRefTestPasses.so
299.120 [41/16/4894] Linking C executable bin/mlir-capi-llvm-test
FAILED: bin/mlir-capi-llvm-test 
: && /usr/bin/clang -fPIC -fno-semantic-interposition -Werror=date-time -Werror=unguarded-availability-new -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -pedantic -Wno-long-long -Wc++98-compat-extra-semi -Wimplicit-fallthrough -Wcovered-switch-default -Wstring-conversion -Wmisleading-indentation -Wctad-maybe-unsupported -fdiagnostics-color -ffunction-sections -fdata-sections -Werror=implicit-function-declaration -Wundef -Werror=mismatched-tags -O3 -DNDEBUG -fuse-ld=lld -Wl,--color-diagnostics     -Wl,--gc-sections tools/mlir/test/CAPI/CMakeFiles/mlir-capi-llvm-test.dir/llvm.c.o -o bin/mlir-capi-llvm-test  -Wl,-rpath,"\$ORIGIN/../lib:/vol/worker/mlir-nvidia/mlir-nvidia/llvm.obj/lib"  lib/libMLIRCAPILLVM.so.20.0git  lib/libMLIRCAPIRegisterEverything.so.20.0git  lib/libMLIRCAPIIR.so.20.0git  lib/libMLIRBytecodeWriter.so.20.0git  lib/libMLIRBytecodeOpInterface.so.20.0git  lib/libMLIRAffineTransformOps.so.20.0git  lib/libMLIRAMDGPUTransforms.so.20.0git  lib/libMLIRArithValueBoundsOpInterfaceImpl.so.20.0git  lib/libMLIRAsyncTransforms.so.20.0git  lib/libMLIRBufferizationPipelines.so.20.0git  lib/libMLIRBufferizationTransformOps.so.20.0git  lib/libMLIRControlFlowTransforms.so.20.0git  lib/libMLIRDLTITransformOps.so.20.0git  lib/libMLIRFuncTransformOps.so.20.0git  lib/libMLIRGPUTransformOps.so.20.0git  lib/libMLIRGPUPipelines.so.20.0git  lib/libMLIRMathTransforms.so.20.0git  lib/libMLIRMemRefTransformOps.so.20.0git  lib/libMLIRMLProgramTransforms.so.20.0git  lib/libMLIRMPIDialect.so.20.0git  lib/libMLIRNVGPUTransformOps.so.20.0git  lib/libMLIRNVGPUTransforms.so.20.0git  lib/libMLIROpenACCTransforms.so.20.0git  lib/libMLIRPolynomialDialect.so.20.0git  lib/libMLIRPtrDialect.so.20.0git  lib/libMLIRSCFTransformOps.so.20.0git  lib/libMLIRShapeOpsTransforms.so.20.0git  lib/libMLIRSparseTensorPipelines.so.20.0git  lib/libMLIRSparseTensorTransformOps.so.20.0git  lib/libMLIRLinalgTransformOps.so.20.0git  lib/libMLIRSparseTensorTransforms.so.20.0git  lib/libMLIRSparseTensorUtils.so.20.0git  lib/libMLIRSPIRVModuleCombiner.so.20.0git  lib/libMLIRTensorInferTypeOpInterfaceImpl.so.20.0git  lib/libMLIRTensorTransformOps.so.20.0git  lib/libMLIRTransformDebugExtension.so.20.0git  lib/libMLIRTransformDialectIRDLExtension.so.20.0git  lib/libMLIRIRDL.so.20.0git  lib/libMLIRTransformLoopExtension.so.20.0git  lib/libMLIRTransformPDLExtension.so.20.0git  lib/libMLIRTransformDialectTransforms.so.20.0git  lib/libMLIRVectorTransformOps.so.20.0git  lib/libMLIRTransformDialect.so.20.0git  lib/libMLIRTransformDialectInterfaces.so.20.0git  lib/libMLIRTransformDialectUtils.so.20.0git  lib/libMLIRXeGPUTransforms.so.20.0git  lib/libMLIRXeGPUDialect.so.20.0git  lib/libMLIRTargetCpp.so.20.0git  lib/libMLIRSPIRVTranslateRegistration.so.20.0git  lib/libMLIRSPIRVDeserialization.so.20.0git  lib/libMLIRToLLVMIRTranslationRegistration.so.20.0git  lib/libMLIRArmNeonToLLVMIRTranslation.so.20.0git  lib/libMLIRArmSMEToLLVMIRTranslation.so.20.0git  lib/libMLIRArmSVEToLLVMIRTranslation.so.20.0git  lib/libMLIRAMXToLLVMIRTranslation.so.20.0git  lib/libMLIRBuiltinToLLVMIRTranslation.so.20.0git  lib/libMLIRGPUToLLVMIRTranslation.so.20.0git  lib/libMLIRLLVMToLLVMIRTranslation.so.20.0git  lib/libMLIROpenACCToLLVMIRTranslation.so.20.0git  lib/libMLIROpenMPToLLVMIRTranslation.so.20.0git  lib/libMLIRSPIRVToLLVMIRTranslation.so.20.0git  lib/libMLIRVCIXToLLVMIRTranslation.so.20.0git  lib/libMLIRVCIXDialect.so.20.0git  lib/libMLIRX86VectorToLLVMIRTranslation.so.20.0git  lib/libMLIRFromLLVMIRTranslationRegistration.so.20.0git  lib/libMLIRLLVMIRToLLVMTranslation.so.20.0git  lib/libMLIRLLVMIRToNVVMTranslation.so.20.0git  lib/libMLIRTargetLLVMIRImport.so.20.0git  lib/libMLIRArithToAMDGPU.so.20.0git  lib/libMLIRArithToArmSME.so.20.0git  lib/libMLIRArithToEmitC.so.20.0git  lib/libMLIREmitCTransforms.so.20.0git  lib/libMLIRArmNeon2dToIntr.so.20.0git  lib/libMLIRArmSMEToSCF.so.20.0git  lib/libMLIRArmSMEToLLVM.so.20.0git  lib/libMLIRBufferizationToMemRef.so.20.0git  lib/libMLIRComplexToLibm.so.20.0git  lib/libMLIRComplexToLLVM.so.20.0git  lib/libMLIRComplexToSPIRV.so.20.0git  lib/libMLIRComplexToStandard.so.20.0git  lib/libMLIRControlFlowToSCF.so.20.0git  lib/libMLIRControlFlowToSPIRV.so.20.0git  lib/libMLIRConvertToSPIRVPass.so.20.0git  lib/libMLIRSPIRVTransforms.so.20.0git  lib/libMLIRFuncToEmitC.so.20.0git  lib/libMLIRGPUToLLVMSPV.so.20.0git  lib/libMLIRGPUToNVVMTransforms.so.20.0git  lib/libMLIRGPUToROCDLTransforms.so.20.0git  lib/libMLIRAMDGPUToROCDL.so.20.0git  lib/libMLIRAMDGPUUtils.so.20.0git  lib/libMLIRAMDGPUDialect.so.20.0git  lib/libMLIRGPUToSPIRV.so.20.0git  lib/libMLIRGPUToVulkanTransforms.so.20.0git  lib/libMLIRIndexToLLVM.so.20.0git  lib/libMLIRLinalgToStandard.so.20.0git  lib/libMLIRMathToFuncs.so.20.0git  lib/libMLIRMathToLibm.so.20.0git  lib/libMLIRMathToLLVM.so.20.0git  lib/libMLIRMathToROCDL.so.20.0git  lib/libMLIRMathToSPIRV.so.20.0git  lib/libMLIRMemRefToEmitC.so.20.0git  lib/libMLIRNVGPUToNVVM.so.20.0git  lib/libMLIRGPUToGPURuntimeTransforms.so.20.0git  lib/libMLIRAsyncToLLVM.so.20.0git  lib/libMLIRConvertToLLVMPass.so.20.0git  lib/libMLIRConvertToLLVMInterface.so.20.0git  lib/libMLIRNVVMToLLVM.so.20.0git  lib/libMLIROpenACCToSCF.so.20.0git  lib/libMLIROpenACCDialect.so.20.0git  lib/libMLIROpenMPToLLVM.so.20.0git  lib/libMLIRReconcileUnrealizedCasts.so.20.0git  lib/libMLIRSCFToControlFlow.so.20.0git  lib/libMLIRSCFToEmitC.so.20.0git  lib/libMLIREmitCDialect.so.20.0git  lib/libMLIRSCFToGPU.so.20.0git  lib/libMLIRGPUTransforms.so.20.0git  lib/libMLIRAsyncDialect.so.20.0git  lib/libMLIRSPIRVTarget.so.20.0git  lib/libMLIRSPIRVSerialization.so.20.0git  lib/libMLIRSPIRVBinaryUtils.so.20.0git  lib/libMLIRNVVMTarget.so.20.0git  lib/libMLIRNVVMToLLVMIRTranslation.so.20.0git  lib/libLLVMNVPTXCodeGen.so.20.0git  lib/libLLVMNVPTXDesc.so.20.0git  lib/libLLVMNVPTXInfo.so.20.0git  lib/libMLIRROCDLTarget.so.20.0git  lib/libMLIRROCDLToLLVMIRTranslation.so.20.0git  lib/libMLIRROCDLDialect.so.20.0git  lib/libMLIRTargetLLVM.so.20.0git  lib/libMLIRExecutionEngineUtils.so.20.0git  lib/libLLVMPasses.so.20.0git  lib/libLLVMTarget.so.20.0git  lib/libLLVMipo.so.20.0git  lib/libLLVMLinker.so.20.0git  lib/libMLIRAffineToStandard.so.20.0git  lib/libMLIRSCFToOpenMP.so.20.0git  lib/libMLIROpenMPDialect.so.20.0git  lib/libMLIROpenACCMPCommon.so.20.0git  lib/libMLIRSCFToSPIRV.so.20.0git  lib/libMLIRIndexToSPIRV.so.20.0git  lib/libMLIRMemRefToSPIRV.so.20.0git  lib/libMLIRShapeToStandard.so.20.0git  lib/libMLIRShapeDialect.so.20.0git  lib/libMLIRSPIRVToLLVM.so.20.0git  lib/libMLIRSPIRVUtils.so.20.0git  lib/libMLIRFuncToLLVM.so.20.0git  lib/libMLIRArithToLLVM.so.20.0git  lib/libMLIRControlFlowToLLVM.so.20.0git  lib/libMLIRMemRefToLLVM.so.20.0git  lib/libMLIRSPIRVAttrToLLVMConversion.so.20.0git  lib/libMLIRTensorToLinalg.so.20.0git  lib/libMLIRLinalgTransforms.so.20.0git  lib/libMLIRMemRefTransforms.so.20.0git  lib/libMLIRArithTransforms.so.20.0git  lib/libMLIRMeshTransforms.so.20.0git  lib/libMLIRTosaShardingInterfaceImpl.so.20.0git  lib/libMLIRTensorTilingInterfaceImpl.so.20.0git  lib/libMLIRTensorToSPIRV.so.20.0git  lib/libMLIRArithToSPIRV.so.20.0git  lib/libMLIRFuncToSPIRV.so.20.0git  lib/libMLIRTosaToArith.so.20.0git  lib/libMLIRTosaToLinalg.so.20.0git  lib/libMLIRLinalgUtils.so.20.0git  lib/libMLIRTosaToMLProgram.so.20.0git  lib/libMLIRMLProgramDialect.so.20.0git  lib/libMLIRTosaToSCF.so.20.0git  lib/libMLIRTosaToTensor.so.20.0git  lib/libMLIRTosaTransforms.so.20.0git  lib/libMLIRTosaDialect.so.20.0git  lib/libMLIRQuantUtils.so.20.0git  lib/libMLIRQuantDialect.so.20.0git  lib/libMLIRUBToLLVM.so.20.0git  lib/libMLIRUBToSPIRV.so.20.0git  lib/libMLIRVectorToGPU.so.20.0git  lib/libMLIRNVGPUUtils.so.20.0git  lib/libMLIRNVGPUDialect.so.20.0git  lib/libMLIRVectorToLLVMPass.so.20.0git  lib/libMLIRAMXTransforms.so.20.0git  lib/libMLIRAMXDialect.so.20.0git  lib/libMLIRArmNeonTransforms.so.20.0git  lib/libMLIRArmNeonDialect.so.20.0git  lib/libMLIRArmSMETransforms.so.20.0git  lib/libMLIRFuncTransforms.so.20.0git  lib/libMLIRIndexDialect.so.20.0git  lib/libMLIRSCFTransforms.so.20.0git  lib/libMLIRTensorTransforms.so.20.0git  lib/libMLIRAffineTransforms.so.20.0git  lib/libMLIRSCFUtils.so.20.0git  lib/libMLIRTensorUtils.so.20.0git  lib/libMLIRArmSVETransforms.so.20.0git  lib/libMLIRX86VectorTransforms.so.20.0git  lib/libMLIRX86VectorDialect.so.20.0git  lib/libMLIRVectorToArmSME.so.20.0git  lib/libMLIRArmSMEDialect.so.20.0git  lib/libMLIRArmSVEDialect.so.20.0git  lib/libMLIRVectorToLLVM.so.20.0git  lib/libMLIRTargetLLVMIRExport.so.20.0git  lib/libMLIRLLVMIRTransforms.so.20.0git  lib/libMLIRNVVMDialect.so.20.0git  lib/libMLIRTranslateLib.so.20.0git  lib/libLLVMFrontendOpenMP.so.20.0git  lib/libLLVMFrontendOffloading.so.20.0git  lib/libLLVMTransformUtils.so.20.0git  lib/libMLIRArithAttrToLLVMConversion.so.20.0git  lib/libMLIRLLVMCommonConversion.so.20.0git  lib/libMLIRLLVMDialect.so.20.0git  lib/libLLVMBitWriter.so.20.0git  lib/libLLVMMCParser.so.20.0git  lib/libLLVMIRReader.so.20.0git  lib/libLLVMAsmParser.so.20.0git  lib/libLLVMBitReader.so.20.0git  lib/libLLVMMC.so.20.0git  lib/libLLVMCore.so.20.0git  lib/libLLVMBinaryFormat.so.20.0git  lib/libLLVMTargetParser.so.20.0git  lib/libMLIRVectorToSCF.so.20.0git  lib/libMLIRVectorToSPIRV.so.20.0git  lib/libMLIRSPIRVConversion.so.20.0git  lib/libMLIRSPIRVDialect.so.20.0git  lib/libMLIRVectorTransforms.so.20.0git  lib/libMLIRAffineUtils.so.20.0git  lib/libMLIRBufferizationTransforms.so.20.0git  lib/libMLIRGPUDialect.so.20.0git  lib/libMLIRDLTIDialect.so.20.0git  lib/libMLIRLinalgDialect.so.20.0git  lib/libMLIRParser.so.20.0git  lib/libMLIRBytecodeReader.so.20.0git  lib/libMLIRAsmParser.so.20.0git  lib/libMLIRBufferizationDialect.so.20.0git  lib/libMLIRMathDialect.so.20.0git  lib/libMLIRSparseTensorDialect.so.20.0git  lib/libMLIRTilingInterface.so.20.0git  lib/libMLIRMemRefUtils.so.20.0git  lib/libMLIRVectorUtils.so.20.0git  lib/libMLIRAffineAnalysis.so.20.0git  lib/libMLIRSCFDialect.so.20.0git  lib/libMLIRVectorDialect.so.20.0git  lib/libMLIRVectorInterfaces.so.20.0git  lib/libMLIRMaskableOpInterface.so.20.0git  lib/libMLIRMaskingOpInterface.so.20.0git  lib/libMLIRFuncAllExtensions.so.20.0git  lib/libMLIRFuncInlinerExtension.so.20.0git  lib/libMLIRControlFlowDialect.so.20.0git  lib/libMLIRFuncMeshShardingExtensions.so.20.0git  lib/libMLIRFuncDialect.so.20.0git  lib/libMLIRTensorAllExtensions.so.20.0git  lib/libMLIRTensorMeshShardingExtensions.so.20.0git  lib/libMLIRShardingInterface.so.20.0git  lib/libMLIRMeshDialect.so.20.0git  lib/libMLIRTensorDialect.so.20.0git  lib/libMLIRAffineDialect.so.20.0git  lib/libMLIRMemRefDialect.so.20.0git  lib/libMLIRArithUtils.so.20.0git  lib/libMLIRComplexDialect.so.20.0git  lib/libMLIRArithDialect.so.20.0git  lib/libMLIRUBDialect.so.20.0git  lib/libMLIRInferIntRangeCommon.so.20.0git  lib/libMLIRDialect.so.20.0git  lib/libMLIRShapedOpInterfaces.so.20.0git  lib/libMLIRCastInterfaces.so.20.0git  lib/libMLIRParallelCombiningOpInterface.so.20.0git  lib/libMLIRDialectUtils.so.20.0git  lib/libMLIRCAPITransforms.so.20.0git  lib/libMLIRTransforms.so.20.0git  lib/libMLIRMemorySlotInterfaces.so.20.0git  lib/libMLIRTransformUtils.so.20.0git  lib/libMLIRSubsetOpInterface.so.20.0git  lib/libMLIRValueBoundsOpInterface.so.20.0git  lib/libMLIRDestinationStyleOpInterface.so.20.0git  lib/libMLIRRewrite.so.20.0git  lib/libMLIRRewritePDL.so.20.0git  lib/libMLIRPDLToPDLInterp.so.20.0git  lib/libMLIRPass.so.20.0git  lib/libMLIRAnalysis.so.20.0git  lib/libMLIRCallInterfaces.so.20.0git  lib/libMLIRControlFlowInterfaces.so.20.0git  lib/libMLIRDataLayoutInterfaces.so.20.0git  lib/libMLIRPresburger.so.20.0git  lib/libMLIRLoopLikeInterface.so.20.0git  lib/libMLIRViewLikeInterface.so.20.0git  lib/libMLIRInferIntRangeInterface.so.20.0git  lib/libMLIRPDLInterpDialect.so.20.0git  lib/libMLIRPDLDialect.so.20.0git  lib/libMLIRFunctionInterfaces.so.20.0git  lib/libMLIRInferTypeOpInterface.so.20.0git  lib/libMLIRSideEffectInterfaces.so.20.0git  lib/libMLIRCopyOpInterface.so.20.0git  lib/libMLIRRuntimeVerifiableOpInterface.so.20.0git  lib/libMLIRIR.so.20.0git  lib/libMLIRSupport.so.20.0git  lib/libLLVMSupport.so.20.0git  -Wl,-rpath-link,/vol/worker/mlir-nvidia/mlir-nvidia/llvm.obj/lib && :
ld.lld: error: undefined symbol: mlirLLVMDISubprogramAttrGetRecSelf
>>> referenced by llvm.c
>>>               tools/mlir/test/CAPI/CMakeFiles/mlir-capi-llvm-test.dir/llvm.c.o:(main)

ld.lld: error: undefined symbol: mlirLLVMDICompositeTypeAttrGetRecSelf
>>> referenced by llvm.c
>>>               tools/mlir/test/CAPI/CMakeFiles/mlir-capi-llvm-test.dir/llvm.c.o:(main)
clang: error: linker command failed with exit code 1 (use -v to see invocation)
299.135 [41/15/4895] Linking CXX shared library lib/libMLIRTestFromLLVMIRTranslation.so.20.0git
299.157 [41/14/4896] Linking CXX shared library lib/libMLIRTestToLLVMIRTranslation.so.20.0git
299.183 [41/13/4897] Linking CXX shared library lib/libMLIRTestIR.so.20.0git
299.192 [41/12/4898] Linking CXX shared library lib/libMLIRTestPDLL.so.20.0git
299.202 [41/11/4899] Linking CXX shared library lib/libMLIRTestTransforms.so.20.0git
299.276 [41/10/4900] Linking CXX executable bin/mlir-query
299.327 [41/9/4901] Linking CXX executable bin/mlir-vulkan-runner
302.026 [41/8/4902] Building CXX object tools/mlir/examples/transform/Ch2/CMakeFiles/transform-opt-ch2.dir/transform-opt/transform-opt.cpp.o
302.063 [41/7/4903] Building CXX object tools/mlir/tools/mlir-reduce/CMakeFiles/mlir-reduce.dir/mlir-reduce.cpp.o
302.527 [41/6/4904] Building CXX object tools/mlir/examples/toy/Ch5/CMakeFiles/toyc-ch5.dir/toyc.cpp.o
304.001 [41/5/4905] Building CXX object tools/mlir/examples/transform/Ch4/CMakeFiles/transform-opt-ch4.dir/transform-opt/transform-opt.cpp.o
307.034 [41/4/4906] Building CXX object tools/mlir/examples/toy/Ch6/CMakeFiles/toyc-ch6.dir/toyc.cpp.o
310.306 [41/3/4907] Building CXX object tools/mlir/examples/transform/Ch3/CMakeFiles/transform-opt-ch3.dir/transform-opt/transform-opt.cpp.o
310.679 [41/2/4908] Building CXX object tools/mlir/examples/toy/Ch7/CMakeFiles/toyc-ch7.dir/toyc.cpp.o
311.935 [41/1/4909] Building CXX object tools/mlir/examples/transform-opt/CMakeFiles/mlir-transform-opt.dir/mlir-transform-opt.cpp.o
ninja: build stopped: subcommand failed.

gysit added a commit that referenced this pull request Aug 31, 2024
gysit added a commit that referenced this pull request Aug 31, 2024
Reverts #106571

This commit breaks the following build bot:
https://lab.llvm.org/buildbot/#/builders/138/builds/2992
It looks like there is a missing dependency in this particular setup.
@gysit
Copy link
Contributor Author

gysit commented Aug 31, 2024

The reason for the failure seems to be a missing MLIR_CAPI_EXPORTED prefix for the newly added functions.

gysit added a commit to gysit/llvm-project that referenced this pull request Sep 2, 2024
…h fixes

This reverts commit fa93be4, restoring
commit d884b77, with fixes that ensure the CAPI declarations are
exported properly.

This commit implements LLVM_DIRecursiveTypeAttrInterface for the
DISubprogramAttr to ensure cyclic subprograms can be imported properly.
In the process multiple shortcuts around the recently introduced
DIImportedEntityAttr can be removed.
gysit added a commit that referenced this pull request Sep 2, 2024
…xes (#106947)

This reverts commit fa93be4, restoring
commit d884b77, with fixes that ensure the CAPI declarations are
exported properly.

This commit implements LLVM_DIRecursiveTypeAttrInterface for the
DISubprogramAttr to ensure cyclic subprograms can be imported properly.
In the process multiple shortcuts around the recently introduced
DIImportedEntityAttr can be removed.
nirvedhmeshram added a commit to iree-org/iree that referenced this pull request Sep 12, 2024
Bumps llvm to commit:
iree-org/llvm-project@e268afb
in branch
https://github.com/iree-org/llvm-project/tree/shared/integrates_20240910

Following changes are made:
1. Fix formatv call to pass validation added by
llvm/llvm-project#105745
2. API changes in DICompositeTypeAttr::get introduced by
llvm/llvm-project#106571
  3. Fix API call from llvm/llvm-project#100361
  4. Fix chipset comparison in ROCMTarget.cpp

There are two cherry-picks from upstream main as they contain fixes we
need and one cheery-pick that is yet to land
1.
iree-org/llvm-project@0d5d355
2.
iree-org/llvm-project@650d852
3.
iree-org/llvm-project@e268afb
(the upstream PR for this one is
llvm/llvm-project#108302

And a revert due to an outstanding torch-mlir issue

iree-org/llvm-project@cf22797
josemonsalve2 pushed a commit to josemonsalve2/iree that referenced this pull request Sep 14, 2024
Bumps llvm to commit:
iree-org/llvm-project@e268afb
in branch
https://github.com/iree-org/llvm-project/tree/shared/integrates_20240910

Following changes are made:
1. Fix formatv call to pass validation added by
llvm/llvm-project#105745
2. API changes in DICompositeTypeAttr::get introduced by
llvm/llvm-project#106571
  3. Fix API call from llvm/llvm-project#100361
  4. Fix chipset comparison in ROCMTarget.cpp

There are two cherry-picks from upstream main as they contain fixes we
need and one cheery-pick that is yet to land
1.
iree-org/llvm-project@0d5d355
2.
iree-org/llvm-project@650d852
3.
iree-org/llvm-project@e268afb
(the upstream PR for this one is
llvm/llvm-project#108302

And a revert due to an outstanding torch-mlir issue

iree-org/llvm-project@cf22797
@abidh
Copy link
Contributor

abidh commented Sep 23, 2024

Hi @gysit, @zyx-billy
I noticed that DIRecursiveTypeAttrInterface only works for DICompositeTypeAttr if the cyclic reference it contains is of the same type. It does not work if the cyclic dependency is between 2 different type. Consider the following example, it is Fortran code but I guess similar example could be created in C code. The type t2 contains a reference to type t1 and vice versa.

 type t2
   type(t1), pointer :: p1
 end type
 type t1
   type(t2), pointer :: p2
 end type

If t1 is processed first, the flow looks something like this:

  1. We create a DICompositeTypeAttr for t1 with isRecSelf=true
  2. We process its members and create type for t2
  3. 't2' needs type for 't1` and we use the type created in step1.
  4. Both types are created.

The t1 reference contained inside t2 is still isRecSelf=true. It will work in the DebugTranslation.cpp if we process the t1 first. The t1 will already be in recursiveNodeMap when t2 is processed. The place holder reference will be correctly updated.

But if t2 is processed first then that reference is not resolved and we hit the assert(!attr.getIsRecSelf() && "unbound DI recursive self reference"); assertions.

I was wondering if this is a known limitation or is there a workaround this problem?

@gysit
Copy link
Contributor Author

gysit commented Sep 23, 2024

If you have an LLVM IR example, it could make to run it through the LLVM IR to LLVM dialect import and see what happens (using mlir-translate -import-llvm filename.ll). The import uses a CyclicReplacerCache to construct the self recursive nodes. I assume in the front-end you may need a similar trick to decide where to put the self recursive nodes. So maybe studying DebugImport.cpp may be helpful (sorry for the unspecific hints but it has been a while since I reviewed this and the exact mechanism have been paged out of my brain).

@zyx-billy may have something like that working in their front-end?

@zyx-billy
Copy link
Contributor

Hi @abidh, IIUC are you talking about the case of mutual recursion? The way it works is we never "expose" attributes that contain unbound self-references. In your case, the debug importer is supposed to create two results (consisting of 6 distinct attributes):

t1 -> t2' -> t1_self
t2 -> t1' -> t2_self

Where t1_self & t1 share the same rec_id, and t2_self & t2 share the same rec_id. Note that t1' & t2' by themselves are illegal if they're ever referred to without t1 or t2 in an outer context. From outside, we only expect t1 and t2 to be used. So in your example, when exporting, we should always be processing t1 or t2 first before their ' versions.

We have some test cases starting here that might help explain how things would look (including some mutual recursions), but like @gysit says, feel free to let us know if you're seeing something that doesn't work properly though! 🙏

@abidh
Copy link
Contributor

abidh commented Sep 24, 2024

Hi @zyx-billy

Thanks for you comment. Can you kindly explain what the relationship of t1' and t2' will be with the t1 and t2 respectively. Will they have same rec_id.

My frontend was generating something like

t1 -> t2 -> t1_self
t2 -> t1_self

It worked if the t1 was referenced as it will be in outer context and the t1_self will be resolved correctly. But t2 did not work on its own as there was no outer t1 to resolve the t1_self. So I am interested to understand the relationship of t1' and t2' with rest of the attributes.

@zyx-billy
Copy link
Contributor

Hi @abidh,

In my example, t1' & t2' are not recursive (they don't have rec_ids). t1' is the version of t1 that occurs only inside t2 (maybe it's clearer to call it t1_2). It doesn't refer to t2, but refers to t2_self. So if you look at t1' by itself, it contains an "unbound" self-reference.

When you say your frontend was generating those attributes, do you mean that's what they look like after importing into the llvm dialect? As long as your t2 is only ever referred to by t1, then it just serves the purpose of t2' in my example, and is ok.

@abidh
Copy link
Contributor

abidh commented Sep 24, 2024

When you say your frontend was generating those attributes, do you mean that's what they look like after importing into the llvm dialect?

The frontend here is flang. Yes, thats how the attributes look just before going through LLVMIR translation.

As long as your t2 is only ever referred to by t1, then it just serves the purpose of t2' in my example, and is ok.

I guess the problem is that t2 could be referred outside of t1 and that case is currently unhandled. In my case, it was a local variable of type t2 that triggered it.

@zyx-billy
Copy link
Contributor

Oh so the attributes aren't generated by importing, but they were created natively in MLIR? In that case I think your frontend will need to make sure to not "reuse" the t2 from inside t1, but to create a separate t2 that is standalone and use that elsewhere in the IR. The LLVM dialect could probably use a verifier for this too, but today if you see the LLVM translation die here, that's indicating this problem.

abidh added a commit to abidh/llvm-project that referenced this pull request Sep 25, 2024
This commit fixes an issue that can cause an assertion to fail in some
circumstances at
https://github.com/llvm/llvm-project/blob/26029d77a57cb4aaa1479064109e985a90d0edd8/mlir/lib/Target/LLVMIR/DebugTranslation.cpp#L270.
The issue relates to the handling of recursive debug type in mlir.
See the discussion at the end of follow PR for more details.
llvm#106571

Problem could be explained with the following example code:

type t2
  type(t1), pointer :: p1
end type

type t1
  type(t2), pointer :: p2
end type

In the description below, type_self means a temporary type that is
generated as a place holder while the members of that type are being
processed.

If we process t1 first then we will have the following structure after
it has been processed.
 t1 -> t2 -> t1_self

This is because when we started processing t2, we did not have the
complete t1 but its place holder t1_self. Now if some entity requires
t2, we will already have that in cache and will return it. But this t2
refers to t1_self and not to t1.

In mlir handling, only those types are allowed to have _self reference
which are wrapped by entity whose reference it contains. So
t1 -> t2 -> t1_self is ok because the t1_self reference can be resolved
by the outer t1. But standalone t2 is not because there will be no way
to resolve it. Please see DebugTranslation::translateRecursive for
details on how mlir handles recursive types.

The fix is not to cache the type that will fail if used standalone.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
flang:fir-hlfir flang Flang issues not falling into any other category mlir:llvm mlir
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants