[clang][ModulesDriver] Add support for Clang modules to -fmodules-driver - #187606
Conversation
|
@llvm/pr-subscribers-clang-driver @llvm/pr-subscribers-clang Author: Naveen Seth Hanig (naveen-seth) ChangesThis PR adds basic support for explicit module builds using Clang modules only, managed natively by the Clang driver. RFC for driver-managed module builds: Full diff: https://github.com/llvm/llvm-project/pull/187606.diff 3 Files Affected:
diff --git a/clang/lib/Driver/ModulesDriver.cpp b/clang/lib/Driver/ModulesDriver.cpp
index 8740f9615d304..166c7749245c6 100644
--- a/clang/lib/Driver/ModulesDriver.cpp
+++ b/clang/lib/Driver/ModulesDriver.cpp
@@ -25,6 +25,7 @@
#include "llvm/ADT/DenseSet.h"
#include "llvm/ADT/DepthFirstIterator.h"
#include "llvm/ADT/DirectedGraph.h"
+#include "llvm/ADT/PostOrderIterator.h"
#include "llvm/ADT/STLExtras.h"
#include "llvm/ADT/SmallVectorExtras.h"
#include "llvm/ADT/TypeSwitch.h"
@@ -1528,9 +1529,12 @@ void driver::modules::runModulesDriver(
// TODO: Install all updated command-lines produced by the dependency scan.
// TODO: Fix-up command-lines for named module imports.
- // TODO: Sort the graph topologically before adding jobs back to the
- // Compilation being built.
- for (auto *N : Graph)
- if (auto *JN = dyn_cast<JobNode>(N))
- C.addCommand(std::move(JN->Job));
+ llvm::ReversePostOrderTraversal<CompilationGraph *> TopologicallySortedNodes(
+ &Graph);
+ assert(isa<RootNode>(*TopologicallySortedNodes.begin()) &&
+ "First node in topological order must be the root!");
+ auto TopologicallySortedJobNodes = llvm::map_range(
+ llvm::drop_begin(TopologicallySortedNodes), llvm::CastTo<JobNode>);
+ for (auto *JN : TopologicallySortedJobNodes)
+ C.addCommand(std::move(JN->Job));
}
diff --git a/clang/test/Driver/modules-driver-clang-modules-only.cpp b/clang/test/Driver/modules-driver-clang-modules-only.cpp
new file mode 100644
index 0000000000000..e99ce7883afba
--- /dev/null
+++ b/clang/test/Driver/modules-driver-clang-modules-only.cpp
@@ -0,0 +1,75 @@
+// Checks that -fmodules-driver correctly handles compilations using Clang modules.
+
+// RUN: split-file %s %t
+// RUN: rm -rf %t/modules-cache
+// RUN: %clang -std=c++23 \
+// RUN: -fmodules-driver -Rmodules-driver \
+// RUN: -fmodules -Rmodule-import \
+// RUN: -fmodule-map-file=%t/module.modulemap \
+// RUN: -fmodules-cache-path=%t/modules-cache \
+// RUN: -fsyntax-only %t/main.cpp 2>&1 \
+// RUN: | sed 's:\\\\\?:/:g' \
+// RUN: | FileCheck -DPREFIX=%/t %s
+
+// The scan itself will also produce [-Rmodule-import] remarks.
+// Let's skip past them, we only care about the final -cc1 commands.
+// CHECK: clang: remark: printing module dependency graph [-Rmodules-driver]
+// CHECK-NEXT: digraph "Module Dependency Graph" {
+// CHECK: }
+
+// CHECK: [[PREFIX]]/main.cpp:1:2: remark: importing module 'root' from
+// CHECK: [[PREFIX]]/main.cpp:1:2: remark: importing module 'direct1' into 'root'
+// CHECK: [[PREFIX]]/main.cpp:1:2: remark: importing module 'transitive1' into 'direct1'
+// CHECK: [[PREFIX]]/main.cpp:1:2: remark: importing module 'transitive2' into 'direct1'
+// CHECK: [[PREFIX]]/main.cpp:1:2: remark: importing module 'direct2' into 'root'
+// CHECK: [[PREFIX]]/main.cpp:1:2: remark: importing module 'transitive2' into 'direct2'
+
+// (Because of missing include guards, this example would also run into
+// redefinition errors when compiling without modules.)
+
+/--- module.modulemap
+module root { header "root.h"}
+module transitive1 { header "transitive1.h" }
+module transitive2 { header "transitive2.h" }
+module direct1 { header "direct1.h" }
+module direct2 { header "direct2.h" }
+
+//--- root.h
+#include "direct1.h"
+#include "direct2.h"
+int fromRoot() {
+ return fromDirect1() + fromDirect2();
+}
+
+//--- direct1.h
+#include "transitive1.h"
+#include "transitive2.h"
+
+int fromDirect1() {
+ return fromTransitive1() + fromTransitive2();
+}
+
+//--- direct2.h
+#include "transitive2.h"
+
+int fromDirect2() {
+ return fromTransitive2() + 2;
+}
+
+//--- transitive1.h
+int fromTransitive1() {
+ return 20;
+}
+
+//--- transitive2.h
+
+int fromTransitive2() {
+ return 10;
+}
+
+//--- main.cpp
+#include "root.h"
+
+int main() {
+ fromRoot();
+}
diff --git a/clang/test/Driver/modules-driver-manifest-input-args.cpp b/clang/test/Driver/modules-driver-manifest-input-args.cpp
index 99765a1943faf..6e3c7ac4c3eff 100644
--- a/clang/test/Driver/modules-driver-manifest-input-args.cpp
+++ b/clang/test/Driver/modules-driver-manifest-input-args.cpp
@@ -25,18 +25,12 @@
// RUN: -fmodules-cache-path=%t/modules-cache \
// RUN: %t/main.cpp \
// RUN: -### 2>&1 \
-// RUN: | sed 's:\\\\\?:/:g' \
-// RUN: | FileCheck %s -check-prefix=MAIN-CC1 -check-prefix=STDLIB-MOD-CC1 -DPREFIX=%/t
+// RUN: | sed 's/\\/\//g' \
+// RUN: | FileCheck %s -DPREFIX=%/t
-// MAIN-CC1: "-cc1"
-// MAIN-CC1-SAME: "main.cpp"
-// MAIN-CC1-NOT: "-Wno-reserved-module-identifier"
-// MAIN-CC1-NOT: "-internal-isystem" "[[PREFIX]]/Inputs/usr/lib/x86_64-linux-gnu/../share/libc++/v1/"
-
-// STDLIB-MOD-CC1: "-cc1"
-// STDLIB-MOD-CC1-SAME: "[[PREFIX]]/Inputs/usr/lib/x86_64-linux-gnu/../share/libc++/v1/std.cppm"
-// STDLIB-MOD-CC1-SAME: "-Wno-reserved-module-identifier"
-// STDLIB-MOD-CC1-SAME: "-internal-isystem" "[[PREFIX]]/Inputs/usr/lib/x86_64-linux-gnu/../share/libc++/v1/"
+// CHECK: "-cc1" {{.*}} "[[PREFIX]]/Inputs/usr/lib/x86_64-linux-gnu/../share/libc++/v1/std.cppm"
+// CHECK-SAME: "-Wno-reserved-module-identifier"
+// CHECK-SAME: "-internal-isystem" "[[PREFIX]]/Inputs/usr/lib/x86_64-linux-gnu/../share/libc++/v1/"
//--- main.cpp
import std;
|
|
@llvm/pr-subscribers-clang-driver Author: Naveen Seth Hanig (naveen-seth) ChangesThis PR adds basic support for explicit module builds using Clang modules only, managed natively by the Clang driver. RFC for driver-managed module builds: Full diff: https://github.com/llvm/llvm-project/pull/187606.diff 3 Files Affected:
diff --git a/clang/lib/Driver/ModulesDriver.cpp b/clang/lib/Driver/ModulesDriver.cpp
index 8740f9615d304..166c7749245c6 100644
--- a/clang/lib/Driver/ModulesDriver.cpp
+++ b/clang/lib/Driver/ModulesDriver.cpp
@@ -25,6 +25,7 @@
#include "llvm/ADT/DenseSet.h"
#include "llvm/ADT/DepthFirstIterator.h"
#include "llvm/ADT/DirectedGraph.h"
+#include "llvm/ADT/PostOrderIterator.h"
#include "llvm/ADT/STLExtras.h"
#include "llvm/ADT/SmallVectorExtras.h"
#include "llvm/ADT/TypeSwitch.h"
@@ -1528,9 +1529,12 @@ void driver::modules::runModulesDriver(
// TODO: Install all updated command-lines produced by the dependency scan.
// TODO: Fix-up command-lines for named module imports.
- // TODO: Sort the graph topologically before adding jobs back to the
- // Compilation being built.
- for (auto *N : Graph)
- if (auto *JN = dyn_cast<JobNode>(N))
- C.addCommand(std::move(JN->Job));
+ llvm::ReversePostOrderTraversal<CompilationGraph *> TopologicallySortedNodes(
+ &Graph);
+ assert(isa<RootNode>(*TopologicallySortedNodes.begin()) &&
+ "First node in topological order must be the root!");
+ auto TopologicallySortedJobNodes = llvm::map_range(
+ llvm::drop_begin(TopologicallySortedNodes), llvm::CastTo<JobNode>);
+ for (auto *JN : TopologicallySortedJobNodes)
+ C.addCommand(std::move(JN->Job));
}
diff --git a/clang/test/Driver/modules-driver-clang-modules-only.cpp b/clang/test/Driver/modules-driver-clang-modules-only.cpp
new file mode 100644
index 0000000000000..e99ce7883afba
--- /dev/null
+++ b/clang/test/Driver/modules-driver-clang-modules-only.cpp
@@ -0,0 +1,75 @@
+// Checks that -fmodules-driver correctly handles compilations using Clang modules.
+
+// RUN: split-file %s %t
+// RUN: rm -rf %t/modules-cache
+// RUN: %clang -std=c++23 \
+// RUN: -fmodules-driver -Rmodules-driver \
+// RUN: -fmodules -Rmodule-import \
+// RUN: -fmodule-map-file=%t/module.modulemap \
+// RUN: -fmodules-cache-path=%t/modules-cache \
+// RUN: -fsyntax-only %t/main.cpp 2>&1 \
+// RUN: | sed 's:\\\\\?:/:g' \
+// RUN: | FileCheck -DPREFIX=%/t %s
+
+// The scan itself will also produce [-Rmodule-import] remarks.
+// Let's skip past them, we only care about the final -cc1 commands.
+// CHECK: clang: remark: printing module dependency graph [-Rmodules-driver]
+// CHECK-NEXT: digraph "Module Dependency Graph" {
+// CHECK: }
+
+// CHECK: [[PREFIX]]/main.cpp:1:2: remark: importing module 'root' from
+// CHECK: [[PREFIX]]/main.cpp:1:2: remark: importing module 'direct1' into 'root'
+// CHECK: [[PREFIX]]/main.cpp:1:2: remark: importing module 'transitive1' into 'direct1'
+// CHECK: [[PREFIX]]/main.cpp:1:2: remark: importing module 'transitive2' into 'direct1'
+// CHECK: [[PREFIX]]/main.cpp:1:2: remark: importing module 'direct2' into 'root'
+// CHECK: [[PREFIX]]/main.cpp:1:2: remark: importing module 'transitive2' into 'direct2'
+
+// (Because of missing include guards, this example would also run into
+// redefinition errors when compiling without modules.)
+
+/--- module.modulemap
+module root { header "root.h"}
+module transitive1 { header "transitive1.h" }
+module transitive2 { header "transitive2.h" }
+module direct1 { header "direct1.h" }
+module direct2 { header "direct2.h" }
+
+//--- root.h
+#include "direct1.h"
+#include "direct2.h"
+int fromRoot() {
+ return fromDirect1() + fromDirect2();
+}
+
+//--- direct1.h
+#include "transitive1.h"
+#include "transitive2.h"
+
+int fromDirect1() {
+ return fromTransitive1() + fromTransitive2();
+}
+
+//--- direct2.h
+#include "transitive2.h"
+
+int fromDirect2() {
+ return fromTransitive2() + 2;
+}
+
+//--- transitive1.h
+int fromTransitive1() {
+ return 20;
+}
+
+//--- transitive2.h
+
+int fromTransitive2() {
+ return 10;
+}
+
+//--- main.cpp
+#include "root.h"
+
+int main() {
+ fromRoot();
+}
diff --git a/clang/test/Driver/modules-driver-manifest-input-args.cpp b/clang/test/Driver/modules-driver-manifest-input-args.cpp
index 99765a1943faf..6e3c7ac4c3eff 100644
--- a/clang/test/Driver/modules-driver-manifest-input-args.cpp
+++ b/clang/test/Driver/modules-driver-manifest-input-args.cpp
@@ -25,18 +25,12 @@
// RUN: -fmodules-cache-path=%t/modules-cache \
// RUN: %t/main.cpp \
// RUN: -### 2>&1 \
-// RUN: | sed 's:\\\\\?:/:g' \
-// RUN: | FileCheck %s -check-prefix=MAIN-CC1 -check-prefix=STDLIB-MOD-CC1 -DPREFIX=%/t
+// RUN: | sed 's/\\/\//g' \
+// RUN: | FileCheck %s -DPREFIX=%/t
-// MAIN-CC1: "-cc1"
-// MAIN-CC1-SAME: "main.cpp"
-// MAIN-CC1-NOT: "-Wno-reserved-module-identifier"
-// MAIN-CC1-NOT: "-internal-isystem" "[[PREFIX]]/Inputs/usr/lib/x86_64-linux-gnu/../share/libc++/v1/"
-
-// STDLIB-MOD-CC1: "-cc1"
-// STDLIB-MOD-CC1-SAME: "[[PREFIX]]/Inputs/usr/lib/x86_64-linux-gnu/../share/libc++/v1/std.cppm"
-// STDLIB-MOD-CC1-SAME: "-Wno-reserved-module-identifier"
-// STDLIB-MOD-CC1-SAME: "-internal-isystem" "[[PREFIX]]/Inputs/usr/lib/x86_64-linux-gnu/../share/libc++/v1/"
+// CHECK: "-cc1" {{.*}} "[[PREFIX]]/Inputs/usr/lib/x86_64-linux-gnu/../share/libc++/v1/std.cppm"
+// CHECK-SAME: "-Wno-reserved-module-identifier"
+// CHECK-SAME: "-internal-isystem" "[[PREFIX]]/Inputs/usr/lib/x86_64-linux-gnu/../share/libc++/v1/"
//--- main.cpp
import std;
|
This PR adds basic support for Clang modules to the -fmodules-driver feature, enabling compilations with Clang modules managed natively by the Clang driver. Caching is not included in this PR. RFC for driver-managed module builds: https://discourse.llvm.org/t/rfc-modules-support-simple-c-20-modules-use-from-the-clang-driver-without-a-build-system
7aacab7 to
c48ed20
Compare
qiongsiwu
left a comment
There was a problem hiding this comment.
Thanks for the PR! The code change looks reasonable to me. I have a few questions about the tests. Our new tests still pass even without the source change and that's a bit confusing to me.
|
Gentle ping for review |
|
LLVM Buildbot has detected a new failure on builder Full details are available at: https://lab.llvm.org/buildbot/#/builders/55/builds/26476 Here is the relevant piece of the build log for the reference |
|
LLVM Buildbot has detected a new failure on builder Full details are available at: https://lab.llvm.org/buildbot/#/builders/24/builds/19293 Here is the relevant piece of the build log for the reference |
…ules-driver" due to memory leak (#191122) Reverts #187606 due to a memory leak. See #187606 (comment)
…es to -fmodules-driver" due to memory leak (#191122) Reverts llvm/llvm-project#187606 due to a memory leak. See llvm/llvm-project#187606 (comment)
…dules-driver" (#191258) This relands #187606 (reverted with #191122). In the initial PR, the Clang module precompile jobs were created as `CC1Command` objects instead of regular `Command` objects, which introduced a memory leak. (See discussion in https://reviews.llvm.org/D74447) This has been fixed in this reland.
…es to -fmodules-driver" due to memory leak (#191122) Reverts llvm/llvm-project#187606 due to a memory leak. See llvm/llvm-project#187606 (comment)
…es to -fmodules-driver" due to memory leak (#191122) Reverts llvm/llvm-project#187606 due to a memory leak. See llvm/llvm-project#187606 (comment)
…ver (llvm#187606) This PR adds basic support for explicit module builds using Clang modules only, managed natively by the Clang driver. (Caching of built modules is not included in this PR.) RFC for driver-managed module builds: https://discourse.llvm.org/t/rfc-modules-support-simple-c-20-modules-use-from-the-clang-driver-without-a-build-system
…ules-driver" due to memory leak (llvm#191122) Reverts llvm#187606 due to a memory leak. See llvm#187606 (comment)
… std` (#193312) This patch adds basic support for explicit C++ named module builds, managed natively by the Clang driver, including support for use of the Standard library modules. This follows #187606, which adds the same for Clang modules. Current limitations: - Standard library modules are still compiled to object files instead of using the provided shared library. (This will be addressed in a follow-up soon.) - Caching is not supported yet (but likely to be added during the upcoming GSoC cycle). - Importing C++ standard library modules into Clang modules is not supported (and not expected in the near term). RFC: https://discourse.llvm.org/t/rfc-modules-support-simple-c-20-modules-use-from-the-clang-driver-without-a-build-system
…d `import std`" (#193815) This reverts #193677 and relands #193312. This adds basic support for explicit C++ named module builds, managed natively by the Clang driver, including support for use of the Standard library modules. This follows #187606, which adds the same for Clang modules. Current limitations: - Standard library modules are still compiled to object files instead of using the provided shared library. (This will be addressed in a follow-up soon.) - Caching is not supported yet (but likely to be added during the upcoming GSoC cycle). - Importing C++ standard library modules into Clang modules is not supported (and not expected in the near term). RFC: https://discourse.llvm.org/t/rfc-modules-support-simple-c-20-modules-use-from-the-clang-driver-without-a-build-system
…es to -fmodules-driver" due to memory leak (#191122) Reverts llvm/llvm-project#187606 due to a memory leak. See llvm/llvm-project#187606 (comment)
…d `import std`" This reverts llvm#193677 and relands llvm#193312. This adds basic support for explicit C++ named module builds, managed natively by the Clang driver, including support for use of the Standard library modules. This follows llvm#187606, which adds the same for Clang modules. Current limitations: - Standard library modules are still compiled to object files instead of using the provided shared library. (This will be addressed in a follow-up soon.) - Caching is not supported yet (but likely to be added during the upcoming GSoC cycle). - Importing C++ standard library modules into Clang modules is not supported (and not expected in the near term). RFC: https://discourse.llvm.org/t/rfc-modules-support-simple-c-20-modules-use-from-the-clang-driver-without-a-build-system
… import std" (2nd attempt) (#194475) This reverts #193857 and relands #193312. This adds basic support for explicit C++ named module builds, managed natively by the Clang driver, including support for use of the Standard library modules. This follows #187606, which adds the same for Clang modules. Current limitations: - Standard library modules are still compiled to object files instead of using the provided shared library. (This will be addressed in a follow-up soon.) - Caching is not supported yet (but likely to be added during the upcoming GSoC cycle). - Importing C++ standard library modules into Clang modules is not supported (and not expected in the near term). RFC: https://discourse.llvm.org/t/rfc-modules-support-simple-c-20-modules-use-from-the-clang-driver-without-a-build-system
… std` (llvm#193312) This patch adds basic support for explicit C++ named module builds, managed natively by the Clang driver, including support for use of the Standard library modules. This follows llvm#187606, which adds the same for Clang modules. Current limitations: - Standard library modules are still compiled to object files instead of using the provided shared library. (This will be addressed in a follow-up soon.) - Caching is not supported yet (but likely to be added during the upcoming GSoC cycle). - Importing C++ standard library modules into Clang modules is not supported (and not expected in the near term). RFC: https://discourse.llvm.org/t/rfc-modules-support-simple-c-20-modules-use-from-the-clang-driver-without-a-build-system
…d `import std`" (llvm#193815) This reverts llvm#193677 and relands llvm#193312. This adds basic support for explicit C++ named module builds, managed natively by the Clang driver, including support for use of the Standard library modules. This follows llvm#187606, which adds the same for Clang modules. Current limitations: - Standard library modules are still compiled to object files instead of using the provided shared library. (This will be addressed in a follow-up soon.) - Caching is not supported yet (but likely to be added during the upcoming GSoC cycle). - Importing C++ standard library modules into Clang modules is not supported (and not expected in the near term). RFC: https://discourse.llvm.org/t/rfc-modules-support-simple-c-20-modules-use-from-the-clang-driver-without-a-build-system
… import std" (2nd attempt) (llvm#194475) This reverts llvm#193857 and relands llvm#193312. This adds basic support for explicit C++ named module builds, managed natively by the Clang driver, including support for use of the Standard library modules. This follows llvm#187606, which adds the same for Clang modules. Current limitations: - Standard library modules are still compiled to object files instead of using the provided shared library. (This will be addressed in a follow-up soon.) - Caching is not supported yet (but likely to be added during the upcoming GSoC cycle). - Importing C++ standard library modules into Clang modules is not supported (and not expected in the near term). RFC: https://discourse.llvm.org/t/rfc-modules-support-simple-c-20-modules-use-from-the-clang-driver-without-a-build-system
… std` (llvm#193312) This patch adds basic support for explicit C++ named module builds, managed natively by the Clang driver, including support for use of the Standard library modules. This follows llvm#187606, which adds the same for Clang modules. Current limitations: - Standard library modules are still compiled to object files instead of using the provided shared library. (This will be addressed in a follow-up soon.) - Caching is not supported yet (but likely to be added during the upcoming GSoC cycle). - Importing C++ standard library modules into Clang modules is not supported (and not expected in the near term). RFC: https://discourse.llvm.org/t/rfc-modules-support-simple-c-20-modules-use-from-the-clang-driver-without-a-build-system
…d `import std`" (llvm#193815) This reverts llvm#193677 and relands llvm#193312. This adds basic support for explicit C++ named module builds, managed natively by the Clang driver, including support for use of the Standard library modules. This follows llvm#187606, which adds the same for Clang modules. Current limitations: - Standard library modules are still compiled to object files instead of using the provided shared library. (This will be addressed in a follow-up soon.) - Caching is not supported yet (but likely to be added during the upcoming GSoC cycle). - Importing C++ standard library modules into Clang modules is not supported (and not expected in the near term). RFC: https://discourse.llvm.org/t/rfc-modules-support-simple-c-20-modules-use-from-the-clang-driver-without-a-build-system
… import std" (2nd attempt) (llvm#194475) This reverts llvm#193857 and relands llvm#193312. This adds basic support for explicit C++ named module builds, managed natively by the Clang driver, including support for use of the Standard library modules. This follows llvm#187606, which adds the same for Clang modules. Current limitations: - Standard library modules are still compiled to object files instead of using the provided shared library. (This will be addressed in a follow-up soon.) - Caching is not supported yet (but likely to be added during the upcoming GSoC cycle). - Importing C++ standard library modules into Clang modules is not supported (and not expected in the near term). RFC: https://discourse.llvm.org/t/rfc-modules-support-simple-c-20-modules-use-from-the-clang-driver-without-a-build-system
…ules-driver" due to memory leak (#191122) Reverts llvm/llvm-project#187606 due to a memory leak. See llvm/llvm-project#187606 (comment)
This PR adds basic support for explicit module builds using Clang modules only, managed natively by the Clang driver.
(Caching of built modules is not included in this PR.)
RFC for driver-managed module builds:
https://discourse.llvm.org/t/rfc-modules-support-simple-c-20-modules-use-from-the-clang-driver-without-a-build-system