[clang][modules] Remove Module::ASTFile#185994
Open
jansvoboda11 wants to merge 2 commits intollvm:mainfrom
Open
[clang][modules] Remove Module::ASTFile#185994jansvoboda11 wants to merge 2 commits intollvm:mainfrom
Module::ASTFile#185994jansvoboda11 wants to merge 2 commits intollvm:mainfrom
Conversation
llvmbot
added
clang-tools-extra
clangd
clang-tidy
clang:frontend
Member
|
@llvm/pr-subscribers-clang-tools-extra @llvm/pr-subscribers-clangd Author: Jan Svoboda (jansvoboda11) ChangesThis removes the assumption that a deserialized module is backed by a Patch is 99.88 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/185994.diff 44 Files Affected:
diff --git a/clang-tools-extra/clang-tidy/ExpandModularHeadersPPCallbacks.cpp b/clang-tools-extra/clang-tidy/ExpandModularHeadersPPCallbacks.cpp
index 8a865f1352599..c6460aa56bcf2 100644
--- a/clang-tools-extra/clang-tidy/ExpandModularHeadersPPCallbacks.cpp
+++ b/clang-tools-extra/clang-tidy/ExpandModularHeadersPPCallbacks.cpp
@@ -164,7 +164,7 @@ void ExpandModularHeadersPPCallbacks::InclusionDirective(
if (ModuleImported) {
serialization::ModuleFile *MF =
Compiler.getASTReader()->getModuleManager().lookup(
- *SuggestedModule->getASTFile());
+ *SuggestedModule->getASTFileKey());
handleModuleFile(MF);
}
parseToLocation(DirectiveLoc);
diff --git a/clang-tools-extra/clangd/ModulesBuilder.cpp b/clang-tools-extra/clangd/ModulesBuilder.cpp
index 524ec620c4076..b5a0127d48b7e 100644
--- a/clang-tools-extra/clangd/ModulesBuilder.cpp
+++ b/clang-tools-extra/clangd/ModulesBuilder.cpp
@@ -275,8 +275,8 @@ bool IsModuleFileUpToDate(PathRef ModuleFilePath,
// listener.
Reader.setListener(nullptr);
- if (Reader.ReadAST(ModuleFilePath, serialization::MK_MainFile,
- SourceLocation(),
+ if (Reader.ReadAST(ModuleFileName::make_explicit(ModuleFilePath),
+ serialization::MK_MainFile, SourceLocation(),
ASTReader::ARR_None) != ASTReader::Success)
return false;
diff --git a/clang/include/clang/Basic/Module.h b/clang/include/clang/Basic/Module.h
index 69a1de6f79b35..327377877ff0c 100644
--- a/clang/include/clang/Basic/Module.h
+++ b/clang/include/clang/Basic/Module.h
@@ -54,6 +54,113 @@ class TargetInfo;
/// Describes the name of a module.
using ModuleId = SmallVector<std::pair<std::string, SourceLocation>, 2>;
+namespace serialization {
+class ModuleManager;
+} // namespace serialization
+
+/// Deduplication key for a loaded module file in \c ModuleManager.
+///
+/// For implicitly-built modules, this is the \c DirectoryEntry of the module
+/// cache and the module file name with the (optional) context hash.
+/// This enables using \c FileManager's inode-based canonicalization of the
+/// user-provided module cache path without hitting issues on file systems that
+/// recycle inodes for recompiled module files.
+///
+/// For explicitly-built modules, this is \c FileEntry.
+/// This uses \c FileManager's inode-based canonicalization of the user-provided
+/// module file path. Because input explicitly-built modules do not change
+/// during the lifetime of the compiler, inode recycling is not of concern here.
+class ModuleFileKey {
+ /// The FileManager entity used for deduplication.
+ const void *Ptr;
+ /// The path relative to the module cache path for implicit module file, empty
+ /// for other kinds of module files.
+ std::string PathSuffix;
+
+ friend class serialization::ModuleManager;
+ friend class ModuleFileName;
+ friend llvm::DenseMapInfo<ModuleFileKey>;
+
+ ModuleFileKey(const void *Ptr) : Ptr(Ptr) {}
+
+ ModuleFileKey(const FileEntry *ModuleFile) : Ptr(ModuleFile) {}
+
+ ModuleFileKey(const DirectoryEntry *ModuleCacheDir, StringRef PathSuffix)
+ : Ptr(ModuleCacheDir), PathSuffix(PathSuffix) {}
+
+public:
+ bool operator==(const ModuleFileKey &Other) const {
+ return Ptr == Other.Ptr && PathSuffix == Other.PathSuffix;
+ }
+
+ bool operator!=(const ModuleFileKey &Other) const {
+ return !operator==(Other);
+ }
+
+ bool operator==(const ModuleFileKey &Other) const {
+ return Ptr == Other.Ptr && PathSuffix == Other.PathSuffix;
+ }
+
+ bool operator!=(const ModuleFileKey &Other) const {
+ return !operator==(Other);
+ }
+};
+
+/// Identifies a module file to be loaded.
+///
+/// For implicitly-built module files, the path is split into the module cache
+/// path and the module file name with the (optional) context hash. For all
+/// other types of module files, this is just the file system path.
+class ModuleFileName {
+ std::string Path;
+ std::optional<uint32_t> Separator;
+
+public:
+ /// Creates an empty module file name.
+ ModuleFileName() = default;
+
+ /// Creates a file name for an explicit module.
+ static ModuleFileName make_explicit(std::string Name) {
+ ModuleFileName File;
+ File.Path = std::move(Name);
+ return File;
+ }
+
+ /// Creates a file name for an explicit module.
+ static ModuleFileName make_explicit(StringRef Name) {
+ return make_explicit(Name.str());
+ }
+
+ /// Creates a file name for an implicit module.
+ static ModuleFileName make_implicit(std::string Name, uint32_t Separator) {
+ ModuleFileName File;
+ File.Path = std::move(Name);
+ File.Separator = Separator;
+ return File;
+ }
+
+ /// Creates a file name for an implicit module.
+ static ModuleFileName make_implicit(StringRef Name, uint32_t Separator) {
+ return make_implicit(Name.str(), Separator);
+ }
+
+ /// Returns the plain module file name.
+ StringRef str() const { return Path; }
+
+ /// Converts to StringRef representing the plain module file name.
+ operator StringRef() const { return Path; }
+
+ /// Checks whether the module file name is empty.
+ bool empty() const { return Path.empty(); }
+
+ bool isInModuleCache(StringRef ModuleCache) const {
+ return Separator && ModuleCache == StringRef(Path).substr(0, *Separator);
+ }
+
+ /// Creates the deduplication key for use in \c ModuleManager.
+ std::optional<ModuleFileKey> makeKey(FileManager &FileMgr) const;
+};
+
/// The signature of a module, which is a hash of the AST content.
struct ASTFileSignature : std::array<uint8_t, 20> {
using BaseT = std::array<uint8_t, 20>;
@@ -258,9 +365,10 @@ class alignas(8) Module {
/// \c SubModules vector at which that submodule resides.
mutable llvm::StringMap<unsigned> SubModuleIndex;
- /// The AST file if this is a top-level module which has a
+ /// The AST file name if this is a top-level module which has a
/// corresponding serialized AST file, or null otherwise.
- OptionalFileEntryRef ASTFile;
+ std::optional<ModuleFileName> ASTFileName;
+ std::optional<ModuleFileKey> ASTFileKey;
/// The top-level headers associated with this module.
llvm::SmallSetVector<FileEntryRef, 2> TopHeaders;
@@ -733,15 +841,25 @@ class alignas(8) Module {
return getTopLevelModule()->Name;
}
- /// The serialized AST file for this module, if one was created.
- OptionalFileEntryRef getASTFile() const {
- return getTopLevelModule()->ASTFile;
+ /// The serialized AST file name for this module, if one was created.
+ const ModuleFileName *getASTFileName() const {
+ const Module *TopLevel = getTopLevelModule();
+ return TopLevel->ASTFileName ? &*TopLevel->ASTFileName : nullptr;
}
- /// Set the serialized AST file for the top-level module of this module.
- void setASTFile(OptionalFileEntryRef File) {
- assert((!getASTFile() || getASTFile() == File) && "file path changed");
- getTopLevelModule()->ASTFile = File;
+ /// The serialized AST file key for this module, if one was created.
+ const ModuleFileKey *getASTFileKey() const {
+ const Module *TopLevel = getTopLevelModule();
+ return TopLevel->ASTFileKey ? &*TopLevel->ASTFileKey : nullptr;
+ }
+
+ /// Set the serialized module file for the top-level module of this module.
+ void setASTFileNameAndKey(ModuleFileName NewName, ModuleFileKey NewKey) {
+ assert((!getASTFileName() && !getASTFileKey()) ||
+ *getASTFileKey() == NewKey && "file path changed");
+ Module *TopLevel = getTopLevelModule();
+ TopLevel->ASTFileName = NewName;
+ TopLevel->ASTFileKey = NewKey;
}
/// Retrieve the umbrella directory as written.
@@ -926,4 +1044,23 @@ class VisibleModuleSet {
} // namespace clang
+template <> struct llvm::DenseMapInfo<clang::ModuleFileKey> {
+ static clang::ModuleFileKey getEmptyKey() {
+ return DenseMapInfo<const void *>::getEmptyKey();
+ }
+
+ static clang::ModuleFileKey getTombstoneKey() {
+ return DenseMapInfo<const void *>::getTombstoneKey();
+ }
+
+ static unsigned getHashValue(const clang::ModuleFileKey &Val) {
+ return hash_combine(Val.Ptr, Val.PathSuffix);
+ }
+
+ static bool isEqual(const clang::ModuleFileKey &LHS,
+ const clang::ModuleFileKey &RHS) {
+ return LHS == RHS;
+ }
+};
+
#endif // LLVM_CLANG_BASIC_MODULE_H
diff --git a/clang/include/clang/DependencyScanning/ModuleDepCollector.h b/clang/include/clang/DependencyScanning/ModuleDepCollector.h
index 52035dde4a757..bbb0f5b4d659b 100644
--- a/clang/include/clang/DependencyScanning/ModuleDepCollector.h
+++ b/clang/include/clang/DependencyScanning/ModuleDepCollector.h
@@ -42,7 +42,7 @@ struct PrebuiltModuleDep {
explicit PrebuiltModuleDep(const Module *M)
: ModuleName(M->getTopLevelModuleName()),
- PCMFile(M->getASTFile()->getName()),
+ PCMFile(M->getASTFileName()->str()),
ModuleMapFile(M->PresumedModuleMapFile) {}
};
diff --git a/clang/include/clang/Frontend/CompilerInstance.h b/clang/include/clang/Frontend/CompilerInstance.h
index 266e0826b38f4..0d684d5c7f9fe 100644
--- a/clang/include/clang/Frontend/CompilerInstance.h
+++ b/clang/include/clang/Frontend/CompilerInstance.h
@@ -17,6 +17,7 @@
#include "clang/Frontend/PCHContainerOperations.h"
#include "clang/Frontend/Utils.h"
#include "clang/Lex/DependencyDirectivesScanner.h"
+#include "clang/Lex/HeaderSearch.h"
#include "clang/Lex/HeaderSearchOptions.h"
#include "clang/Lex/ModuleLoader.h"
#include "llvm/ADT/ArrayRef.h"
@@ -746,11 +747,6 @@ class CompilerInstance : public ModuleLoader {
GetDependencyDirectives = std::move(Getter);
}
- std::string getSpecificModuleCachePath(StringRef ContextHash);
- std::string getSpecificModuleCachePath() {
- return getSpecificModuleCachePath(getInvocation().computeContextHash());
- }
-
/// Create the AST context.
void createASTContext();
@@ -872,7 +868,7 @@ class CompilerInstance : public ModuleLoader {
void createASTReader();
- bool loadModuleFile(StringRef FileName,
+ bool loadModuleFile(ModuleFileName FileName,
serialization::ModuleFile *&LoadedModuleFile);
/// Configuration object for making the result of \c cloneForModuleCompile()
diff --git a/clang/include/clang/Lex/HeaderSearch.h b/clang/include/clang/Lex/HeaderSearch.h
index d7000da682c6e..fd52a0ad5b007 100644
--- a/clang/include/clang/Lex/HeaderSearch.h
+++ b/clang/include/clang/Lex/HeaderSearch.h
@@ -282,6 +282,10 @@ class HeaderSearch {
/// The specific module cache path containing ContextHash (unless suppressed).
std::string SpecificModuleCachePath;
+ /// The length of the normalized module cache path at the start of \c
+ /// SpecificModuleCachePath.
+ size_t NormalizedModuleCachePathLen = 0;
+
/// All of the preprocessor-specific data about files that are
/// included, indexed by the FileEntry's UID.
mutable std::vector<HeaderFileInfo> FileInfo;
@@ -467,20 +471,20 @@ class HeaderSearch {
return {};
}
- /// Set the context hash to use for module cache paths.
- void setContextHash(StringRef Hash) { ContextHash = std::string(Hash); }
+ /// Initialize the module cache path.
+ void initializeModuleCachePath(std::string ContextHash);
- /// Set the module cache path with the context hash (unless suppressed).
- void setSpecificModuleCachePath(StringRef Path) {
- SpecificModuleCachePath = std::string(Path);
+ /// Retrieve the module cache path with the context hash (unless suppressed).
+ StringRef getSpecificModuleCachePath() const {
+ return SpecificModuleCachePath;
}
/// Retrieve the context hash.
StringRef getContextHash() const { return ContextHash; }
- /// Retrieve the module cache path with the context hash (unless suppressed).
- StringRef getSpecificModuleCachePath() const {
- return SpecificModuleCachePath;
+ /// Retrieve the normalized module cache path.
+ StringRef getNormalizedModuleCachePath() const {
+ return getSpecificModuleCachePath().substr(0, NormalizedModuleCachePathLen);
}
/// Forget everything we know about headers so far.
@@ -657,7 +661,7 @@ class HeaderSearch {
///
/// \returns The name of the module file that corresponds to this module,
/// or an empty string if this module does not correspond to any module file.
- std::string getCachedModuleFileName(Module *Module);
+ ModuleFileName getCachedModuleFileName(Module *Module);
/// Retrieve the name of the prebuilt module file that should be used
/// to load a module with the given name.
@@ -669,8 +673,8 @@ class HeaderSearch {
///
/// \returns The name of the module file that corresponds to this module,
/// or an empty string if this module does not correspond to any module file.
- std::string getPrebuiltModuleFileName(StringRef ModuleName,
- bool FileMapOnly = false);
+ ModuleFileName getPrebuiltModuleFileName(StringRef ModuleName,
+ bool FileMapOnly = false);
/// Retrieve the name of the prebuilt module file that should be used
/// to load the given module.
@@ -679,7 +683,7 @@ class HeaderSearch {
///
/// \returns The name of the module file that corresponds to this module,
/// or an empty string if this module does not correspond to any module file.
- std::string getPrebuiltImplicitModuleFileName(Module *Module);
+ ModuleFileName getPrebuiltImplicitModuleFileName(Module *Module);
/// Retrieve the name of the (to-be-)cached module file that should
/// be used to load a module with the given name.
@@ -691,8 +695,8 @@ class HeaderSearch {
///
/// \returns The name of the module file that corresponds to this module,
/// or an empty string if this module does not correspond to any module file.
- std::string getCachedModuleFileName(StringRef ModuleName,
- StringRef ModuleMapPath);
+ ModuleFileName getCachedModuleFileName(StringRef ModuleName,
+ StringRef ModuleMapPath);
/// Lookup a module Search for a module with the given name.
///
@@ -808,13 +812,13 @@ class HeaderSearch {
/// \param ModuleMapPath A path that when combined with \c ModuleName
/// uniquely identifies this module. See Module::ModuleMap.
///
- /// \param CachePath A path to the module cache.
+ /// \param NormalizedCachePath The normalized path to the module cache.
///
/// \returns The name of the module file that corresponds to this module,
/// or an empty string if this module does not correspond to any module file.
- std::string getCachedModuleFileNameImpl(StringRef ModuleName,
- StringRef ModuleMapPath,
- StringRef CachePath);
+ ModuleFileName getCachedModuleFileNameImpl(StringRef ModuleName,
+ StringRef ModuleMapPath,
+ StringRef NormalizedCachePath);
/// Retrieve a module with the given name, which may be part of the
/// given framework.
@@ -1042,6 +1046,11 @@ void ApplyHeaderSearchOptions(HeaderSearch &HS,
void normalizeModuleCachePath(FileManager &FileMgr, StringRef Path,
SmallVectorImpl<char> &NormalizedPath);
+std::string createSpecificModuleCachePath(FileManager &FileMgr,
+ StringRef ModuleCachePath,
+ bool DisableModuleHash,
+ std::string ContextHash);
+
} // namespace clang
#endif // LLVM_CLANG_LEX_HEADERSEARCH_H
diff --git a/clang/include/clang/Serialization/ASTBitCodes.h b/clang/include/clang/Serialization/ASTBitCodes.h
index 752e7fd288aa6..4e8fe1d32d42e 100644
--- a/clang/include/clang/Serialization/ASTBitCodes.h
+++ b/clang/include/clang/Serialization/ASTBitCodes.h
@@ -44,7 +44,7 @@ namespace serialization {
/// Version 4 of AST files also requires that the version control branch and
/// revision match exactly, since there is no backward compatibility of
/// AST files at this time.
-const unsigned VERSION_MAJOR = 35;
+const unsigned VERSION_MAJOR = 37;
/// AST file minor version number supported by this version of
/// Clang.
diff --git a/clang/include/clang/Serialization/ASTReader.h b/clang/include/clang/Serialization/ASTReader.h
index f254459ce933d..e9706d0ea2f2b 100644
--- a/clang/include/clang/Serialization/ASTReader.h
+++ b/clang/include/clang/Serialization/ASTReader.h
@@ -185,8 +185,7 @@ class ASTReaderListener {
/// otherwise.
virtual bool ReadHeaderSearchOptions(const HeaderSearchOptions &HSOpts,
StringRef ModuleFilename,
- StringRef SpecificModuleCachePath,
- bool Complain) {
+ StringRef ContextHash, bool Complain) {
return false;
}
@@ -304,8 +303,7 @@ class ChainedASTReaderListener : public ASTReaderListener {
bool Complain) override;
bool ReadHeaderSearchOptions(const HeaderSearchOptions &HSOpts,
- StringRef ModuleFilename,
- StringRef SpecificModuleCachePath,
+ StringRef ModuleFilename, StringRef ContextHash,
bool Complain) override;
bool ReadPreprocessorOptions(const PreprocessorOptions &PPOpts,
StringRef ModuleFilename, bool ReadMacros,
@@ -349,8 +347,7 @@ class PCHValidator : public ASTReaderListener {
bool Complain,
std::string &SuggestedPredefines) override;
bool ReadHeaderSearchOptions(const HeaderSearchOptions &HSOpts,
- StringRef ModuleFilename,
- StringRef SpecificModuleCachePath,
+ StringRef ModuleFilename, StringRef ContextHash,
bool Complain) override;
void ReadCounter(const serialization::ModuleFile &M, uint32_t Value) override;
};
@@ -1552,7 +1549,7 @@ class ASTReader
: Mod(Mod), ImportedBy(ImportedBy), ImportLoc(ImportLoc) {}
};
- ASTReadResult ReadASTCore(StringRef FileName, ModuleKind Type,
+ ASTReadResult ReadASTCore(ModuleFileName FileName, ModuleKind Type,
SourceLocation ImportLoc, ModuleFile *ImportedBy,
SmallVectorImpl<ImportedModule> &Loaded,
off_t ExpectedSize, time_t ExpectedModTime,
@@ -1888,7 +1885,7 @@ class ASTReader
/// NewLoadedModuleFile would refer to the address of the new loaded top level
/// module. The state of NewLoadedModuleFile is unspecified if the AST file
/// isn't loaded successfully.
- ASTReadResult ReadAST(StringRef FileName, ModuleKind Type,
+ ASTReadResult ReadAST(ModuleFileName FileName, ModuleKind Type,
SourceLocation ImportLoc,
unsigned ClientLoadCapabilities,
ModuleFile **NewLoadedModuleFile = nullptr);
diff --git a/clang/include/clang/Serialization/ModuleFile.h b/clang/include/clang/Serialization/ModuleFile.h
index 519a74d920129..c3e41c2a58e6a 100644
--- a/clang/include/clang/Serialization/ModuleFile.h
+++ b/clang/include/clang/Serialization/ModuleFile.h
@@ -144,8 +144,9 @@ enum class InputFilesValidation {
/// other modules.
class ModuleFile {
public:
- ModuleFile(ModuleKind Kind, FileEntryRef File, unsigned Generation)
- : Kind(Kind), File(File), Generation(Generation) {}
+ ModuleFile(ModuleKind Kind, ModuleFileKey FileKey, FileEntryRef File,
+ unsigned Generation)
+ : Kind(Kind), FileKey(FileKey), File(File), Generation(Generation) {}
~ModuleFile();
// === General information ===
@@ -157,7 +158,10 @@ class ModuleFile {
ModuleKind Kind;
/// The file name of the module file.
- std::string FileName;
+ ModuleFileName FileName;
+
+ /// The key ModuleManager used for the module file.
+ ModuleFileKey FileKey;
/// The name of the module.
std::string ModuleName;
diff --git a/clang/include/clang/Serialization/ModuleManager.h b/clang/include/clang/Serialization/ModuleManager.h
index 8ab70b6630f47..162856f2f14c0 100644
--- a/clang/include/clang/Serialization/ModuleManager.h
+++ b/clang/incl...
[truncated]
|
Member
|
@llvm/pr-subscribers-clang-tidy Author: Jan Svoboda (jansvoboda11) ChangesThis removes the assumption that a deserialized module is backed by a Patch is 99.88 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/185994.diff 44 Files Affected:
diff --git a/clang-tools-extra/clang-tidy/ExpandModularHeadersPPCallbacks.cpp b/clang-tools-extra/clang-tidy/ExpandModularHeadersPPCallbacks.cpp
index 8a865f1352599..c6460aa56bcf2 100644
--- a/clang-tools-extra/clang-tidy/ExpandModularHeadersPPCallbacks.cpp
+++ b/clang-tools-extra/clang-tidy/ExpandModularHeadersPPCallbacks.cpp
@@ -164,7 +164,7 @@ void ExpandModularHeadersPPCallbacks::InclusionDirective(
if (ModuleImported) {
serialization::ModuleFile *MF =
Compiler.getASTReader()->getModuleManager().lookup(
- *SuggestedModule->getASTFile());
+ *SuggestedModule->getASTFileKey());
handleModuleFile(MF);
}
parseToLocation(DirectiveLoc);
diff --git a/clang-tools-extra/clangd/ModulesBuilder.cpp b/clang-tools-extra/clangd/ModulesBuilder.cpp
index 524ec620c4076..b5a0127d48b7e 100644
--- a/clang-tools-extra/clangd/ModulesBuilder.cpp
+++ b/clang-tools-extra/clangd/ModulesBuilder.cpp
@@ -275,8 +275,8 @@ bool IsModuleFileUpToDate(PathRef ModuleFilePath,
// listener.
Reader.setListener(nullptr);
- if (Reader.ReadAST(ModuleFilePath, serialization::MK_MainFile,
- SourceLocation(),
+ if (Reader.ReadAST(ModuleFileName::make_explicit(ModuleFilePath),
+ serialization::MK_MainFile, SourceLocation(),
ASTReader::ARR_None) != ASTReader::Success)
return false;
diff --git a/clang/include/clang/Basic/Module.h b/clang/include/clang/Basic/Module.h
index 69a1de6f79b35..327377877ff0c 100644
--- a/clang/include/clang/Basic/Module.h
+++ b/clang/include/clang/Basic/Module.h
@@ -54,6 +54,113 @@ class TargetInfo;
/// Describes the name of a module.
using ModuleId = SmallVector<std::pair<std::string, SourceLocation>, 2>;
+namespace serialization {
+class ModuleManager;
+} // namespace serialization
+
+/// Deduplication key for a loaded module file in \c ModuleManager.
+///
+/// For implicitly-built modules, this is the \c DirectoryEntry of the module
+/// cache and the module file name with the (optional) context hash.
+/// This enables using \c FileManager's inode-based canonicalization of the
+/// user-provided module cache path without hitting issues on file systems that
+/// recycle inodes for recompiled module files.
+///
+/// For explicitly-built modules, this is \c FileEntry.
+/// This uses \c FileManager's inode-based canonicalization of the user-provided
+/// module file path. Because input explicitly-built modules do not change
+/// during the lifetime of the compiler, inode recycling is not of concern here.
+class ModuleFileKey {
+ /// The FileManager entity used for deduplication.
+ const void *Ptr;
+ /// The path relative to the module cache path for implicit module file, empty
+ /// for other kinds of module files.
+ std::string PathSuffix;
+
+ friend class serialization::ModuleManager;
+ friend class ModuleFileName;
+ friend llvm::DenseMapInfo<ModuleFileKey>;
+
+ ModuleFileKey(const void *Ptr) : Ptr(Ptr) {}
+
+ ModuleFileKey(const FileEntry *ModuleFile) : Ptr(ModuleFile) {}
+
+ ModuleFileKey(const DirectoryEntry *ModuleCacheDir, StringRef PathSuffix)
+ : Ptr(ModuleCacheDir), PathSuffix(PathSuffix) {}
+
+public:
+ bool operator==(const ModuleFileKey &Other) const {
+ return Ptr == Other.Ptr && PathSuffix == Other.PathSuffix;
+ }
+
+ bool operator!=(const ModuleFileKey &Other) const {
+ return !operator==(Other);
+ }
+
+ bool operator==(const ModuleFileKey &Other) const {
+ return Ptr == Other.Ptr && PathSuffix == Other.PathSuffix;
+ }
+
+ bool operator!=(const ModuleFileKey &Other) const {
+ return !operator==(Other);
+ }
+};
+
+/// Identifies a module file to be loaded.
+///
+/// For implicitly-built module files, the path is split into the module cache
+/// path and the module file name with the (optional) context hash. For all
+/// other types of module files, this is just the file system path.
+class ModuleFileName {
+ std::string Path;
+ std::optional<uint32_t> Separator;
+
+public:
+ /// Creates an empty module file name.
+ ModuleFileName() = default;
+
+ /// Creates a file name for an explicit module.
+ static ModuleFileName make_explicit(std::string Name) {
+ ModuleFileName File;
+ File.Path = std::move(Name);
+ return File;
+ }
+
+ /// Creates a file name for an explicit module.
+ static ModuleFileName make_explicit(StringRef Name) {
+ return make_explicit(Name.str());
+ }
+
+ /// Creates a file name for an implicit module.
+ static ModuleFileName make_implicit(std::string Name, uint32_t Separator) {
+ ModuleFileName File;
+ File.Path = std::move(Name);
+ File.Separator = Separator;
+ return File;
+ }
+
+ /// Creates a file name for an implicit module.
+ static ModuleFileName make_implicit(StringRef Name, uint32_t Separator) {
+ return make_implicit(Name.str(), Separator);
+ }
+
+ /// Returns the plain module file name.
+ StringRef str() const { return Path; }
+
+ /// Converts to StringRef representing the plain module file name.
+ operator StringRef() const { return Path; }
+
+ /// Checks whether the module file name is empty.
+ bool empty() const { return Path.empty(); }
+
+ bool isInModuleCache(StringRef ModuleCache) const {
+ return Separator && ModuleCache == StringRef(Path).substr(0, *Separator);
+ }
+
+ /// Creates the deduplication key for use in \c ModuleManager.
+ std::optional<ModuleFileKey> makeKey(FileManager &FileMgr) const;
+};
+
/// The signature of a module, which is a hash of the AST content.
struct ASTFileSignature : std::array<uint8_t, 20> {
using BaseT = std::array<uint8_t, 20>;
@@ -258,9 +365,10 @@ class alignas(8) Module {
/// \c SubModules vector at which that submodule resides.
mutable llvm::StringMap<unsigned> SubModuleIndex;
- /// The AST file if this is a top-level module which has a
+ /// The AST file name if this is a top-level module which has a
/// corresponding serialized AST file, or null otherwise.
- OptionalFileEntryRef ASTFile;
+ std::optional<ModuleFileName> ASTFileName;
+ std::optional<ModuleFileKey> ASTFileKey;
/// The top-level headers associated with this module.
llvm::SmallSetVector<FileEntryRef, 2> TopHeaders;
@@ -733,15 +841,25 @@ class alignas(8) Module {
return getTopLevelModule()->Name;
}
- /// The serialized AST file for this module, if one was created.
- OptionalFileEntryRef getASTFile() const {
- return getTopLevelModule()->ASTFile;
+ /// The serialized AST file name for this module, if one was created.
+ const ModuleFileName *getASTFileName() const {
+ const Module *TopLevel = getTopLevelModule();
+ return TopLevel->ASTFileName ? &*TopLevel->ASTFileName : nullptr;
}
- /// Set the serialized AST file for the top-level module of this module.
- void setASTFile(OptionalFileEntryRef File) {
- assert((!getASTFile() || getASTFile() == File) && "file path changed");
- getTopLevelModule()->ASTFile = File;
+ /// The serialized AST file key for this module, if one was created.
+ const ModuleFileKey *getASTFileKey() const {
+ const Module *TopLevel = getTopLevelModule();
+ return TopLevel->ASTFileKey ? &*TopLevel->ASTFileKey : nullptr;
+ }
+
+ /// Set the serialized module file for the top-level module of this module.
+ void setASTFileNameAndKey(ModuleFileName NewName, ModuleFileKey NewKey) {
+ assert((!getASTFileName() && !getASTFileKey()) ||
+ *getASTFileKey() == NewKey && "file path changed");
+ Module *TopLevel = getTopLevelModule();
+ TopLevel->ASTFileName = NewName;
+ TopLevel->ASTFileKey = NewKey;
}
/// Retrieve the umbrella directory as written.
@@ -926,4 +1044,23 @@ class VisibleModuleSet {
} // namespace clang
+template <> struct llvm::DenseMapInfo<clang::ModuleFileKey> {
+ static clang::ModuleFileKey getEmptyKey() {
+ return DenseMapInfo<const void *>::getEmptyKey();
+ }
+
+ static clang::ModuleFileKey getTombstoneKey() {
+ return DenseMapInfo<const void *>::getTombstoneKey();
+ }
+
+ static unsigned getHashValue(const clang::ModuleFileKey &Val) {
+ return hash_combine(Val.Ptr, Val.PathSuffix);
+ }
+
+ static bool isEqual(const clang::ModuleFileKey &LHS,
+ const clang::ModuleFileKey &RHS) {
+ return LHS == RHS;
+ }
+};
+
#endif // LLVM_CLANG_BASIC_MODULE_H
diff --git a/clang/include/clang/DependencyScanning/ModuleDepCollector.h b/clang/include/clang/DependencyScanning/ModuleDepCollector.h
index 52035dde4a757..bbb0f5b4d659b 100644
--- a/clang/include/clang/DependencyScanning/ModuleDepCollector.h
+++ b/clang/include/clang/DependencyScanning/ModuleDepCollector.h
@@ -42,7 +42,7 @@ struct PrebuiltModuleDep {
explicit PrebuiltModuleDep(const Module *M)
: ModuleName(M->getTopLevelModuleName()),
- PCMFile(M->getASTFile()->getName()),
+ PCMFile(M->getASTFileName()->str()),
ModuleMapFile(M->PresumedModuleMapFile) {}
};
diff --git a/clang/include/clang/Frontend/CompilerInstance.h b/clang/include/clang/Frontend/CompilerInstance.h
index 266e0826b38f4..0d684d5c7f9fe 100644
--- a/clang/include/clang/Frontend/CompilerInstance.h
+++ b/clang/include/clang/Frontend/CompilerInstance.h
@@ -17,6 +17,7 @@
#include "clang/Frontend/PCHContainerOperations.h"
#include "clang/Frontend/Utils.h"
#include "clang/Lex/DependencyDirectivesScanner.h"
+#include "clang/Lex/HeaderSearch.h"
#include "clang/Lex/HeaderSearchOptions.h"
#include "clang/Lex/ModuleLoader.h"
#include "llvm/ADT/ArrayRef.h"
@@ -746,11 +747,6 @@ class CompilerInstance : public ModuleLoader {
GetDependencyDirectives = std::move(Getter);
}
- std::string getSpecificModuleCachePath(StringRef ContextHash);
- std::string getSpecificModuleCachePath() {
- return getSpecificModuleCachePath(getInvocation().computeContextHash());
- }
-
/// Create the AST context.
void createASTContext();
@@ -872,7 +868,7 @@ class CompilerInstance : public ModuleLoader {
void createASTReader();
- bool loadModuleFile(StringRef FileName,
+ bool loadModuleFile(ModuleFileName FileName,
serialization::ModuleFile *&LoadedModuleFile);
/// Configuration object for making the result of \c cloneForModuleCompile()
diff --git a/clang/include/clang/Lex/HeaderSearch.h b/clang/include/clang/Lex/HeaderSearch.h
index d7000da682c6e..fd52a0ad5b007 100644
--- a/clang/include/clang/Lex/HeaderSearch.h
+++ b/clang/include/clang/Lex/HeaderSearch.h
@@ -282,6 +282,10 @@ class HeaderSearch {
/// The specific module cache path containing ContextHash (unless suppressed).
std::string SpecificModuleCachePath;
+ /// The length of the normalized module cache path at the start of \c
+ /// SpecificModuleCachePath.
+ size_t NormalizedModuleCachePathLen = 0;
+
/// All of the preprocessor-specific data about files that are
/// included, indexed by the FileEntry's UID.
mutable std::vector<HeaderFileInfo> FileInfo;
@@ -467,20 +471,20 @@ class HeaderSearch {
return {};
}
- /// Set the context hash to use for module cache paths.
- void setContextHash(StringRef Hash) { ContextHash = std::string(Hash); }
+ /// Initialize the module cache path.
+ void initializeModuleCachePath(std::string ContextHash);
- /// Set the module cache path with the context hash (unless suppressed).
- void setSpecificModuleCachePath(StringRef Path) {
- SpecificModuleCachePath = std::string(Path);
+ /// Retrieve the module cache path with the context hash (unless suppressed).
+ StringRef getSpecificModuleCachePath() const {
+ return SpecificModuleCachePath;
}
/// Retrieve the context hash.
StringRef getContextHash() const { return ContextHash; }
- /// Retrieve the module cache path with the context hash (unless suppressed).
- StringRef getSpecificModuleCachePath() const {
- return SpecificModuleCachePath;
+ /// Retrieve the normalized module cache path.
+ StringRef getNormalizedModuleCachePath() const {
+ return getSpecificModuleCachePath().substr(0, NormalizedModuleCachePathLen);
}
/// Forget everything we know about headers so far.
@@ -657,7 +661,7 @@ class HeaderSearch {
///
/// \returns The name of the module file that corresponds to this module,
/// or an empty string if this module does not correspond to any module file.
- std::string getCachedModuleFileName(Module *Module);
+ ModuleFileName getCachedModuleFileName(Module *Module);
/// Retrieve the name of the prebuilt module file that should be used
/// to load a module with the given name.
@@ -669,8 +673,8 @@ class HeaderSearch {
///
/// \returns The name of the module file that corresponds to this module,
/// or an empty string if this module does not correspond to any module file.
- std::string getPrebuiltModuleFileName(StringRef ModuleName,
- bool FileMapOnly = false);
+ ModuleFileName getPrebuiltModuleFileName(StringRef ModuleName,
+ bool FileMapOnly = false);
/// Retrieve the name of the prebuilt module file that should be used
/// to load the given module.
@@ -679,7 +683,7 @@ class HeaderSearch {
///
/// \returns The name of the module file that corresponds to this module,
/// or an empty string if this module does not correspond to any module file.
- std::string getPrebuiltImplicitModuleFileName(Module *Module);
+ ModuleFileName getPrebuiltImplicitModuleFileName(Module *Module);
/// Retrieve the name of the (to-be-)cached module file that should
/// be used to load a module with the given name.
@@ -691,8 +695,8 @@ class HeaderSearch {
///
/// \returns The name of the module file that corresponds to this module,
/// or an empty string if this module does not correspond to any module file.
- std::string getCachedModuleFileName(StringRef ModuleName,
- StringRef ModuleMapPath);
+ ModuleFileName getCachedModuleFileName(StringRef ModuleName,
+ StringRef ModuleMapPath);
/// Lookup a module Search for a module with the given name.
///
@@ -808,13 +812,13 @@ class HeaderSearch {
/// \param ModuleMapPath A path that when combined with \c ModuleName
/// uniquely identifies this module. See Module::ModuleMap.
///
- /// \param CachePath A path to the module cache.
+ /// \param NormalizedCachePath The normalized path to the module cache.
///
/// \returns The name of the module file that corresponds to this module,
/// or an empty string if this module does not correspond to any module file.
- std::string getCachedModuleFileNameImpl(StringRef ModuleName,
- StringRef ModuleMapPath,
- StringRef CachePath);
+ ModuleFileName getCachedModuleFileNameImpl(StringRef ModuleName,
+ StringRef ModuleMapPath,
+ StringRef NormalizedCachePath);
/// Retrieve a module with the given name, which may be part of the
/// given framework.
@@ -1042,6 +1046,11 @@ void ApplyHeaderSearchOptions(HeaderSearch &HS,
void normalizeModuleCachePath(FileManager &FileMgr, StringRef Path,
SmallVectorImpl<char> &NormalizedPath);
+std::string createSpecificModuleCachePath(FileManager &FileMgr,
+ StringRef ModuleCachePath,
+ bool DisableModuleHash,
+ std::string ContextHash);
+
} // namespace clang
#endif // LLVM_CLANG_LEX_HEADERSEARCH_H
diff --git a/clang/include/clang/Serialization/ASTBitCodes.h b/clang/include/clang/Serialization/ASTBitCodes.h
index 752e7fd288aa6..4e8fe1d32d42e 100644
--- a/clang/include/clang/Serialization/ASTBitCodes.h
+++ b/clang/include/clang/Serialization/ASTBitCodes.h
@@ -44,7 +44,7 @@ namespace serialization {
/// Version 4 of AST files also requires that the version control branch and
/// revision match exactly, since there is no backward compatibility of
/// AST files at this time.
-const unsigned VERSION_MAJOR = 35;
+const unsigned VERSION_MAJOR = 37;
/// AST file minor version number supported by this version of
/// Clang.
diff --git a/clang/include/clang/Serialization/ASTReader.h b/clang/include/clang/Serialization/ASTReader.h
index f254459ce933d..e9706d0ea2f2b 100644
--- a/clang/include/clang/Serialization/ASTReader.h
+++ b/clang/include/clang/Serialization/ASTReader.h
@@ -185,8 +185,7 @@ class ASTReaderListener {
/// otherwise.
virtual bool ReadHeaderSearchOptions(const HeaderSearchOptions &HSOpts,
StringRef ModuleFilename,
- StringRef SpecificModuleCachePath,
- bool Complain) {
+ StringRef ContextHash, bool Complain) {
return false;
}
@@ -304,8 +303,7 @@ class ChainedASTReaderListener : public ASTReaderListener {
bool Complain) override;
bool ReadHeaderSearchOptions(const HeaderSearchOptions &HSOpts,
- StringRef ModuleFilename,
- StringRef SpecificModuleCachePath,
+ StringRef ModuleFilename, StringRef ContextHash,
bool Complain) override;
bool ReadPreprocessorOptions(const PreprocessorOptions &PPOpts,
StringRef ModuleFilename, bool ReadMacros,
@@ -349,8 +347,7 @@ class PCHValidator : public ASTReaderListener {
bool Complain,
std::string &SuggestedPredefines) override;
bool ReadHeaderSearchOptions(const HeaderSearchOptions &HSOpts,
- StringRef ModuleFilename,
- StringRef SpecificModuleCachePath,
+ StringRef ModuleFilename, StringRef ContextHash,
bool Complain) override;
void ReadCounter(const serialization::ModuleFile &M, uint32_t Value) override;
};
@@ -1552,7 +1549,7 @@ class ASTReader
: Mod(Mod), ImportedBy(ImportedBy), ImportLoc(ImportLoc) {}
};
- ASTReadResult ReadASTCore(StringRef FileName, ModuleKind Type,
+ ASTReadResult ReadASTCore(ModuleFileName FileName, ModuleKind Type,
SourceLocation ImportLoc, ModuleFile *ImportedBy,
SmallVectorImpl<ImportedModule> &Loaded,
off_t ExpectedSize, time_t ExpectedModTime,
@@ -1888,7 +1885,7 @@ class ASTReader
/// NewLoadedModuleFile would refer to the address of the new loaded top level
/// module. The state of NewLoadedModuleFile is unspecified if the AST file
/// isn't loaded successfully.
- ASTReadResult ReadAST(StringRef FileName, ModuleKind Type,
+ ASTReadResult ReadAST(ModuleFileName FileName, ModuleKind Type,
SourceLocation ImportLoc,
unsigned ClientLoadCapabilities,
ModuleFile **NewLoadedModuleFile = nullptr);
diff --git a/clang/include/clang/Serialization/ModuleFile.h b/clang/include/clang/Serialization/ModuleFile.h
index 519a74d920129..c3e41c2a58e6a 100644
--- a/clang/include/clang/Serialization/ModuleFile.h
+++ b/clang/include/clang/Serialization/ModuleFile.h
@@ -144,8 +144,9 @@ enum class InputFilesValidation {
/// other modules.
class ModuleFile {
public:
- ModuleFile(ModuleKind Kind, FileEntryRef File, unsigned Generation)
- : Kind(Kind), File(File), Generation(Generation) {}
+ ModuleFile(ModuleKind Kind, ModuleFileKey FileKey, FileEntryRef File,
+ unsigned Generation)
+ : Kind(Kind), FileKey(FileKey), File(File), Generation(Generation) {}
~ModuleFile();
// === General information ===
@@ -157,7 +158,10 @@ class ModuleFile {
ModuleKind Kind;
/// The file name of the module file.
- std::string FileName;
+ ModuleFileName FileName;
+
+ /// The key ModuleManager used for the module file.
+ ModuleFileKey FileKey;
/// The name of the module.
std::string ModuleName;
diff --git a/clang/include/clang/Serialization/ModuleManager.h b/clang/include/clang/Serialization/ModuleManager.h
index 8ab70b6630f47..162856f2f14c0 100644
--- a/clang/include/clang/Serialization/ModuleManager.h
+++ b/clang/incl...
[truncated]
|
Member
|
@llvm/pr-subscribers-clang-modules Author: Jan Svoboda (jansvoboda11) ChangesThis removes the assumption that a deserialized module is backed by a Patch is 99.88 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/185994.diff 44 Files Affected:
diff --git a/clang-tools-extra/clang-tidy/ExpandModularHeadersPPCallbacks.cpp b/clang-tools-extra/clang-tidy/ExpandModularHeadersPPCallbacks.cpp
index 8a865f1352599..c6460aa56bcf2 100644
--- a/clang-tools-extra/clang-tidy/ExpandModularHeadersPPCallbacks.cpp
+++ b/clang-tools-extra/clang-tidy/ExpandModularHeadersPPCallbacks.cpp
@@ -164,7 +164,7 @@ void ExpandModularHeadersPPCallbacks::InclusionDirective(
if (ModuleImported) {
serialization::ModuleFile *MF =
Compiler.getASTReader()->getModuleManager().lookup(
- *SuggestedModule->getASTFile());
+ *SuggestedModule->getASTFileKey());
handleModuleFile(MF);
}
parseToLocation(DirectiveLoc);
diff --git a/clang-tools-extra/clangd/ModulesBuilder.cpp b/clang-tools-extra/clangd/ModulesBuilder.cpp
index 524ec620c4076..b5a0127d48b7e 100644
--- a/clang-tools-extra/clangd/ModulesBuilder.cpp
+++ b/clang-tools-extra/clangd/ModulesBuilder.cpp
@@ -275,8 +275,8 @@ bool IsModuleFileUpToDate(PathRef ModuleFilePath,
// listener.
Reader.setListener(nullptr);
- if (Reader.ReadAST(ModuleFilePath, serialization::MK_MainFile,
- SourceLocation(),
+ if (Reader.ReadAST(ModuleFileName::make_explicit(ModuleFilePath),
+ serialization::MK_MainFile, SourceLocation(),
ASTReader::ARR_None) != ASTReader::Success)
return false;
diff --git a/clang/include/clang/Basic/Module.h b/clang/include/clang/Basic/Module.h
index 69a1de6f79b35..327377877ff0c 100644
--- a/clang/include/clang/Basic/Module.h
+++ b/clang/include/clang/Basic/Module.h
@@ -54,6 +54,113 @@ class TargetInfo;
/// Describes the name of a module.
using ModuleId = SmallVector<std::pair<std::string, SourceLocation>, 2>;
+namespace serialization {
+class ModuleManager;
+} // namespace serialization
+
+/// Deduplication key for a loaded module file in \c ModuleManager.
+///
+/// For implicitly-built modules, this is the \c DirectoryEntry of the module
+/// cache and the module file name with the (optional) context hash.
+/// This enables using \c FileManager's inode-based canonicalization of the
+/// user-provided module cache path without hitting issues on file systems that
+/// recycle inodes for recompiled module files.
+///
+/// For explicitly-built modules, this is \c FileEntry.
+/// This uses \c FileManager's inode-based canonicalization of the user-provided
+/// module file path. Because input explicitly-built modules do not change
+/// during the lifetime of the compiler, inode recycling is not of concern here.
+class ModuleFileKey {
+ /// The FileManager entity used for deduplication.
+ const void *Ptr;
+ /// The path relative to the module cache path for implicit module file, empty
+ /// for other kinds of module files.
+ std::string PathSuffix;
+
+ friend class serialization::ModuleManager;
+ friend class ModuleFileName;
+ friend llvm::DenseMapInfo<ModuleFileKey>;
+
+ ModuleFileKey(const void *Ptr) : Ptr(Ptr) {}
+
+ ModuleFileKey(const FileEntry *ModuleFile) : Ptr(ModuleFile) {}
+
+ ModuleFileKey(const DirectoryEntry *ModuleCacheDir, StringRef PathSuffix)
+ : Ptr(ModuleCacheDir), PathSuffix(PathSuffix) {}
+
+public:
+ bool operator==(const ModuleFileKey &Other) const {
+ return Ptr == Other.Ptr && PathSuffix == Other.PathSuffix;
+ }
+
+ bool operator!=(const ModuleFileKey &Other) const {
+ return !operator==(Other);
+ }
+
+ bool operator==(const ModuleFileKey &Other) const {
+ return Ptr == Other.Ptr && PathSuffix == Other.PathSuffix;
+ }
+
+ bool operator!=(const ModuleFileKey &Other) const {
+ return !operator==(Other);
+ }
+};
+
+/// Identifies a module file to be loaded.
+///
+/// For implicitly-built module files, the path is split into the module cache
+/// path and the module file name with the (optional) context hash. For all
+/// other types of module files, this is just the file system path.
+class ModuleFileName {
+ std::string Path;
+ std::optional<uint32_t> Separator;
+
+public:
+ /// Creates an empty module file name.
+ ModuleFileName() = default;
+
+ /// Creates a file name for an explicit module.
+ static ModuleFileName make_explicit(std::string Name) {
+ ModuleFileName File;
+ File.Path = std::move(Name);
+ return File;
+ }
+
+ /// Creates a file name for an explicit module.
+ static ModuleFileName make_explicit(StringRef Name) {
+ return make_explicit(Name.str());
+ }
+
+ /// Creates a file name for an implicit module.
+ static ModuleFileName make_implicit(std::string Name, uint32_t Separator) {
+ ModuleFileName File;
+ File.Path = std::move(Name);
+ File.Separator = Separator;
+ return File;
+ }
+
+ /// Creates a file name for an implicit module.
+ static ModuleFileName make_implicit(StringRef Name, uint32_t Separator) {
+ return make_implicit(Name.str(), Separator);
+ }
+
+ /// Returns the plain module file name.
+ StringRef str() const { return Path; }
+
+ /// Converts to StringRef representing the plain module file name.
+ operator StringRef() const { return Path; }
+
+ /// Checks whether the module file name is empty.
+ bool empty() const { return Path.empty(); }
+
+ bool isInModuleCache(StringRef ModuleCache) const {
+ return Separator && ModuleCache == StringRef(Path).substr(0, *Separator);
+ }
+
+ /// Creates the deduplication key for use in \c ModuleManager.
+ std::optional<ModuleFileKey> makeKey(FileManager &FileMgr) const;
+};
+
/// The signature of a module, which is a hash of the AST content.
struct ASTFileSignature : std::array<uint8_t, 20> {
using BaseT = std::array<uint8_t, 20>;
@@ -258,9 +365,10 @@ class alignas(8) Module {
/// \c SubModules vector at which that submodule resides.
mutable llvm::StringMap<unsigned> SubModuleIndex;
- /// The AST file if this is a top-level module which has a
+ /// The AST file name if this is a top-level module which has a
/// corresponding serialized AST file, or null otherwise.
- OptionalFileEntryRef ASTFile;
+ std::optional<ModuleFileName> ASTFileName;
+ std::optional<ModuleFileKey> ASTFileKey;
/// The top-level headers associated with this module.
llvm::SmallSetVector<FileEntryRef, 2> TopHeaders;
@@ -733,15 +841,25 @@ class alignas(8) Module {
return getTopLevelModule()->Name;
}
- /// The serialized AST file for this module, if one was created.
- OptionalFileEntryRef getASTFile() const {
- return getTopLevelModule()->ASTFile;
+ /// The serialized AST file name for this module, if one was created.
+ const ModuleFileName *getASTFileName() const {
+ const Module *TopLevel = getTopLevelModule();
+ return TopLevel->ASTFileName ? &*TopLevel->ASTFileName : nullptr;
}
- /// Set the serialized AST file for the top-level module of this module.
- void setASTFile(OptionalFileEntryRef File) {
- assert((!getASTFile() || getASTFile() == File) && "file path changed");
- getTopLevelModule()->ASTFile = File;
+ /// The serialized AST file key for this module, if one was created.
+ const ModuleFileKey *getASTFileKey() const {
+ const Module *TopLevel = getTopLevelModule();
+ return TopLevel->ASTFileKey ? &*TopLevel->ASTFileKey : nullptr;
+ }
+
+ /// Set the serialized module file for the top-level module of this module.
+ void setASTFileNameAndKey(ModuleFileName NewName, ModuleFileKey NewKey) {
+ assert((!getASTFileName() && !getASTFileKey()) ||
+ *getASTFileKey() == NewKey && "file path changed");
+ Module *TopLevel = getTopLevelModule();
+ TopLevel->ASTFileName = NewName;
+ TopLevel->ASTFileKey = NewKey;
}
/// Retrieve the umbrella directory as written.
@@ -926,4 +1044,23 @@ class VisibleModuleSet {
} // namespace clang
+template <> struct llvm::DenseMapInfo<clang::ModuleFileKey> {
+ static clang::ModuleFileKey getEmptyKey() {
+ return DenseMapInfo<const void *>::getEmptyKey();
+ }
+
+ static clang::ModuleFileKey getTombstoneKey() {
+ return DenseMapInfo<const void *>::getTombstoneKey();
+ }
+
+ static unsigned getHashValue(const clang::ModuleFileKey &Val) {
+ return hash_combine(Val.Ptr, Val.PathSuffix);
+ }
+
+ static bool isEqual(const clang::ModuleFileKey &LHS,
+ const clang::ModuleFileKey &RHS) {
+ return LHS == RHS;
+ }
+};
+
#endif // LLVM_CLANG_BASIC_MODULE_H
diff --git a/clang/include/clang/DependencyScanning/ModuleDepCollector.h b/clang/include/clang/DependencyScanning/ModuleDepCollector.h
index 52035dde4a757..bbb0f5b4d659b 100644
--- a/clang/include/clang/DependencyScanning/ModuleDepCollector.h
+++ b/clang/include/clang/DependencyScanning/ModuleDepCollector.h
@@ -42,7 +42,7 @@ struct PrebuiltModuleDep {
explicit PrebuiltModuleDep(const Module *M)
: ModuleName(M->getTopLevelModuleName()),
- PCMFile(M->getASTFile()->getName()),
+ PCMFile(M->getASTFileName()->str()),
ModuleMapFile(M->PresumedModuleMapFile) {}
};
diff --git a/clang/include/clang/Frontend/CompilerInstance.h b/clang/include/clang/Frontend/CompilerInstance.h
index 266e0826b38f4..0d684d5c7f9fe 100644
--- a/clang/include/clang/Frontend/CompilerInstance.h
+++ b/clang/include/clang/Frontend/CompilerInstance.h
@@ -17,6 +17,7 @@
#include "clang/Frontend/PCHContainerOperations.h"
#include "clang/Frontend/Utils.h"
#include "clang/Lex/DependencyDirectivesScanner.h"
+#include "clang/Lex/HeaderSearch.h"
#include "clang/Lex/HeaderSearchOptions.h"
#include "clang/Lex/ModuleLoader.h"
#include "llvm/ADT/ArrayRef.h"
@@ -746,11 +747,6 @@ class CompilerInstance : public ModuleLoader {
GetDependencyDirectives = std::move(Getter);
}
- std::string getSpecificModuleCachePath(StringRef ContextHash);
- std::string getSpecificModuleCachePath() {
- return getSpecificModuleCachePath(getInvocation().computeContextHash());
- }
-
/// Create the AST context.
void createASTContext();
@@ -872,7 +868,7 @@ class CompilerInstance : public ModuleLoader {
void createASTReader();
- bool loadModuleFile(StringRef FileName,
+ bool loadModuleFile(ModuleFileName FileName,
serialization::ModuleFile *&LoadedModuleFile);
/// Configuration object for making the result of \c cloneForModuleCompile()
diff --git a/clang/include/clang/Lex/HeaderSearch.h b/clang/include/clang/Lex/HeaderSearch.h
index d7000da682c6e..fd52a0ad5b007 100644
--- a/clang/include/clang/Lex/HeaderSearch.h
+++ b/clang/include/clang/Lex/HeaderSearch.h
@@ -282,6 +282,10 @@ class HeaderSearch {
/// The specific module cache path containing ContextHash (unless suppressed).
std::string SpecificModuleCachePath;
+ /// The length of the normalized module cache path at the start of \c
+ /// SpecificModuleCachePath.
+ size_t NormalizedModuleCachePathLen = 0;
+
/// All of the preprocessor-specific data about files that are
/// included, indexed by the FileEntry's UID.
mutable std::vector<HeaderFileInfo> FileInfo;
@@ -467,20 +471,20 @@ class HeaderSearch {
return {};
}
- /// Set the context hash to use for module cache paths.
- void setContextHash(StringRef Hash) { ContextHash = std::string(Hash); }
+ /// Initialize the module cache path.
+ void initializeModuleCachePath(std::string ContextHash);
- /// Set the module cache path with the context hash (unless suppressed).
- void setSpecificModuleCachePath(StringRef Path) {
- SpecificModuleCachePath = std::string(Path);
+ /// Retrieve the module cache path with the context hash (unless suppressed).
+ StringRef getSpecificModuleCachePath() const {
+ return SpecificModuleCachePath;
}
/// Retrieve the context hash.
StringRef getContextHash() const { return ContextHash; }
- /// Retrieve the module cache path with the context hash (unless suppressed).
- StringRef getSpecificModuleCachePath() const {
- return SpecificModuleCachePath;
+ /// Retrieve the normalized module cache path.
+ StringRef getNormalizedModuleCachePath() const {
+ return getSpecificModuleCachePath().substr(0, NormalizedModuleCachePathLen);
}
/// Forget everything we know about headers so far.
@@ -657,7 +661,7 @@ class HeaderSearch {
///
/// \returns The name of the module file that corresponds to this module,
/// or an empty string if this module does not correspond to any module file.
- std::string getCachedModuleFileName(Module *Module);
+ ModuleFileName getCachedModuleFileName(Module *Module);
/// Retrieve the name of the prebuilt module file that should be used
/// to load a module with the given name.
@@ -669,8 +673,8 @@ class HeaderSearch {
///
/// \returns The name of the module file that corresponds to this module,
/// or an empty string if this module does not correspond to any module file.
- std::string getPrebuiltModuleFileName(StringRef ModuleName,
- bool FileMapOnly = false);
+ ModuleFileName getPrebuiltModuleFileName(StringRef ModuleName,
+ bool FileMapOnly = false);
/// Retrieve the name of the prebuilt module file that should be used
/// to load the given module.
@@ -679,7 +683,7 @@ class HeaderSearch {
///
/// \returns The name of the module file that corresponds to this module,
/// or an empty string if this module does not correspond to any module file.
- std::string getPrebuiltImplicitModuleFileName(Module *Module);
+ ModuleFileName getPrebuiltImplicitModuleFileName(Module *Module);
/// Retrieve the name of the (to-be-)cached module file that should
/// be used to load a module with the given name.
@@ -691,8 +695,8 @@ class HeaderSearch {
///
/// \returns The name of the module file that corresponds to this module,
/// or an empty string if this module does not correspond to any module file.
- std::string getCachedModuleFileName(StringRef ModuleName,
- StringRef ModuleMapPath);
+ ModuleFileName getCachedModuleFileName(StringRef ModuleName,
+ StringRef ModuleMapPath);
/// Lookup a module Search for a module with the given name.
///
@@ -808,13 +812,13 @@ class HeaderSearch {
/// \param ModuleMapPath A path that when combined with \c ModuleName
/// uniquely identifies this module. See Module::ModuleMap.
///
- /// \param CachePath A path to the module cache.
+ /// \param NormalizedCachePath The normalized path to the module cache.
///
/// \returns The name of the module file that corresponds to this module,
/// or an empty string if this module does not correspond to any module file.
- std::string getCachedModuleFileNameImpl(StringRef ModuleName,
- StringRef ModuleMapPath,
- StringRef CachePath);
+ ModuleFileName getCachedModuleFileNameImpl(StringRef ModuleName,
+ StringRef ModuleMapPath,
+ StringRef NormalizedCachePath);
/// Retrieve a module with the given name, which may be part of the
/// given framework.
@@ -1042,6 +1046,11 @@ void ApplyHeaderSearchOptions(HeaderSearch &HS,
void normalizeModuleCachePath(FileManager &FileMgr, StringRef Path,
SmallVectorImpl<char> &NormalizedPath);
+std::string createSpecificModuleCachePath(FileManager &FileMgr,
+ StringRef ModuleCachePath,
+ bool DisableModuleHash,
+ std::string ContextHash);
+
} // namespace clang
#endif // LLVM_CLANG_LEX_HEADERSEARCH_H
diff --git a/clang/include/clang/Serialization/ASTBitCodes.h b/clang/include/clang/Serialization/ASTBitCodes.h
index 752e7fd288aa6..4e8fe1d32d42e 100644
--- a/clang/include/clang/Serialization/ASTBitCodes.h
+++ b/clang/include/clang/Serialization/ASTBitCodes.h
@@ -44,7 +44,7 @@ namespace serialization {
/// Version 4 of AST files also requires that the version control branch and
/// revision match exactly, since there is no backward compatibility of
/// AST files at this time.
-const unsigned VERSION_MAJOR = 35;
+const unsigned VERSION_MAJOR = 37;
/// AST file minor version number supported by this version of
/// Clang.
diff --git a/clang/include/clang/Serialization/ASTReader.h b/clang/include/clang/Serialization/ASTReader.h
index f254459ce933d..e9706d0ea2f2b 100644
--- a/clang/include/clang/Serialization/ASTReader.h
+++ b/clang/include/clang/Serialization/ASTReader.h
@@ -185,8 +185,7 @@ class ASTReaderListener {
/// otherwise.
virtual bool ReadHeaderSearchOptions(const HeaderSearchOptions &HSOpts,
StringRef ModuleFilename,
- StringRef SpecificModuleCachePath,
- bool Complain) {
+ StringRef ContextHash, bool Complain) {
return false;
}
@@ -304,8 +303,7 @@ class ChainedASTReaderListener : public ASTReaderListener {
bool Complain) override;
bool ReadHeaderSearchOptions(const HeaderSearchOptions &HSOpts,
- StringRef ModuleFilename,
- StringRef SpecificModuleCachePath,
+ StringRef ModuleFilename, StringRef ContextHash,
bool Complain) override;
bool ReadPreprocessorOptions(const PreprocessorOptions &PPOpts,
StringRef ModuleFilename, bool ReadMacros,
@@ -349,8 +347,7 @@ class PCHValidator : public ASTReaderListener {
bool Complain,
std::string &SuggestedPredefines) override;
bool ReadHeaderSearchOptions(const HeaderSearchOptions &HSOpts,
- StringRef ModuleFilename,
- StringRef SpecificModuleCachePath,
+ StringRef ModuleFilename, StringRef ContextHash,
bool Complain) override;
void ReadCounter(const serialization::ModuleFile &M, uint32_t Value) override;
};
@@ -1552,7 +1549,7 @@ class ASTReader
: Mod(Mod), ImportedBy(ImportedBy), ImportLoc(ImportLoc) {}
};
- ASTReadResult ReadASTCore(StringRef FileName, ModuleKind Type,
+ ASTReadResult ReadASTCore(ModuleFileName FileName, ModuleKind Type,
SourceLocation ImportLoc, ModuleFile *ImportedBy,
SmallVectorImpl<ImportedModule> &Loaded,
off_t ExpectedSize, time_t ExpectedModTime,
@@ -1888,7 +1885,7 @@ class ASTReader
/// NewLoadedModuleFile would refer to the address of the new loaded top level
/// module. The state of NewLoadedModuleFile is unspecified if the AST file
/// isn't loaded successfully.
- ASTReadResult ReadAST(StringRef FileName, ModuleKind Type,
+ ASTReadResult ReadAST(ModuleFileName FileName, ModuleKind Type,
SourceLocation ImportLoc,
unsigned ClientLoadCapabilities,
ModuleFile **NewLoadedModuleFile = nullptr);
diff --git a/clang/include/clang/Serialization/ModuleFile.h b/clang/include/clang/Serialization/ModuleFile.h
index 519a74d920129..c3e41c2a58e6a 100644
--- a/clang/include/clang/Serialization/ModuleFile.h
+++ b/clang/include/clang/Serialization/ModuleFile.h
@@ -144,8 +144,9 @@ enum class InputFilesValidation {
/// other modules.
class ModuleFile {
public:
- ModuleFile(ModuleKind Kind, FileEntryRef File, unsigned Generation)
- : Kind(Kind), File(File), Generation(Generation) {}
+ ModuleFile(ModuleKind Kind, ModuleFileKey FileKey, FileEntryRef File,
+ unsigned Generation)
+ : Kind(Kind), FileKey(FileKey), File(File), Generation(Generation) {}
~ModuleFile();
// === General information ===
@@ -157,7 +158,10 @@ class ModuleFile {
ModuleKind Kind;
/// The file name of the module file.
- std::string FileName;
+ ModuleFileName FileName;
+
+ /// The key ModuleManager used for the module file.
+ ModuleFileKey FileKey;
/// The name of the module.
std::string ModuleName;
diff --git a/clang/include/clang/Serialization/ModuleManager.h b/clang/include/clang/Serialization/ModuleManager.h
index 8ab70b6630f47..162856f2f14c0 100644
--- a/clang/include/clang/Serialization/ModuleManager.h
+++ b/clang/incl...
[truncated]
|
jansvoboda11
requested review from
Bigcheese,
benlangmuir,
cyndyishida,
qiongsiwu and
vsapsai
jansvoboda11
force-pushed
the
no-file-entry-module-cache-2
branch
from
5d27a5f to
ec01947
Compare
|
✅ With the latest revision this PR passed the C/C++ code formatter. |
jansvoboda11
force-pushed
the
no-file-entry-module-cache-2
branch
from
ec01947 to
3b52b23
Compare
🐧 Linux x64 Test Results
✅ The build succeeded and all tests passed. |
🪟 Windows x64 Test Results
✅ The build succeeded and all tests passed. |
jansvoboda11
force-pushed
the
no-file-entry-module-cache-2
branch
from
3b52b23 to
b62d6d6
Compare
benlangmuir
reviewed
Mar 17, 2026
jansvoboda11
force-pushed
the
no-file-entry-module-cache-2
branch
from
b62d6d6 to
93b085d
Compare
benlangmuir
approved these changes
Mar 18, 2026
Collaborator
benlangmuir
left a comment
benlangmuir
left a comment
There was a problem hiding this comment.
LGTM. If we needed to keep supporting the libclang API one way would be to change CXModule into a wrapper object that has access to a FileManager, since that would be hidden behind the ABI boundary, but I agree with your reasoning that it's probably not necessary.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This removes the assumption that a deserialized module is backed by a
FileEntry. TheOptionalFileEntryRefmember is replaced withModuleFile{Name,Key}.