Skip to content

Commit

Permalink
feat: automatic generation of compile_commands.json
Browse files Browse the repository at this point in the history
  • Loading branch information
fpelliccioni authored Jan 17, 2024
1 parent 395f565 commit 70a9347
Show file tree
Hide file tree
Showing 5 changed files with 361 additions and 26 deletions.
14 changes: 2 additions & 12 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -317,17 +317,6 @@ jobs:
modules-exclude-paths: ''
trace-commands: true

- name: Configure Boost.URL
working-directory: boost/libs/url
run: |
set -x
if [ -d "__build__" ]; then
rm -rf __build__
fi
mkdir __build__
cd __build__
cmake -D BOOST_URL_BUILD_TESTS=OFF -D BOOST_URL_BUILD_EXAMPLES=OFF -D CMAKE_EXPORT_COMPILE_COMMANDS=ON -D CMAKE_CXX_STANDARD_INCLUDE_DIRECTORIES="$default_includes" -D CMAKE_CXX_COMPILER=${{ steps.setup-cpp.outputs.cxx }} -D CMAKE_C_COMPILER=${{ steps.setup-cpp.outputs.cc }} ..
- name: Generate demos
run: |
config_template=$(printf '%s\n' \
Expand All @@ -338,6 +327,7 @@ jobs:
"multipage: %s" \
"inaccessible-members: never" \
"inaccessible-bases: never" \
"cmake: -D BOOST_URL_BUILD_TESTS=OFF -D BOOST_URL_BUILD_EXAMPLES=OFF -D CMAKE_EXPORT_COMPILE_COMMANDS=ON -D CMAKE_CXX_STANDARD_INCLUDE_DIRECTORIES='$default_includes' -D CMAKE_CXX_COMPILER=${{ steps.setup-cpp.outputs.cxx }} -D CMAKE_C_COMPILER=${{ steps.setup-cpp.outputs.cc }}" \
"filters:" \
" symbols:" \
" exclude:" \
Expand All @@ -350,7 +340,7 @@ jobs:
[[ $variant = multi ]] && multiline="true" || multiline="false"
printf "$config_template\n" $format $multiline > $(pwd)/boost/libs/url/mrdocs.yml
mkdir -p "demos/boost-url/$variant/$format"
mrdocs --config="$(pwd)/boost/libs/url/mrdocs.yml" "$(pwd)/boost/libs/url/__build__/compile_commands.json" --output="$(pwd)/demos/boost-url/$variant/$format"
mrdocs --config="$(pwd)/boost/libs/url/mrdocs.yml" "$(pwd)/boost/libs/url/" --output="$(pwd)/demos/boost-url/$variant/$format"
done
asciidoctor -d book -R "$(pwd)/demos/boost-url/$variant/adoc" -D "$(pwd)/demos/boost-url/$variant/adoc-asciidoc" "$(pwd)/demos/boost-url/$variant/adoc/**/*.adoc"
done
Expand Down
37 changes: 24 additions & 13 deletions docs/modules/ROOT/pages/usage.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -16,32 +16,26 @@ Where `MRDOCS_ROOT` is the path of the mrdocs executable, and `MRDOCS_CONFIG` is
We also assume `PROJECT_SOURCE_DIR` is the path to the root of your project's source code, where its main `CMakeLists.txt` file is located, and `PROJECT_BUILD_DIR` is the path to the directory where you want to generate the documentation.
Feel free to change these variables to suit your needs.

The first step to generate your documentation is to generate the `compile_commands.json` file by running `cmake` with the `-DCMAKE_EXPORT_COMPILE_COMMANDS=ON` flag.
MrDocs simplifies the documentation generation process. Generating the `compile_commands.json` file by running `cmake` is optional. If the path to `compile_commands.json` is not provided when calling MrDocs, the tool will automatically run `cmake` for you, provided that you have CMake version >=3.13.5 installed. Parameters for cmake, such as `-D BOOST_URL_BUILD_TESTS=OFF`, should be specified in the `cmake:` key of the `mrdocs.yml` configuration file.

[source,bash]
----
cd $PROJECT_SOURCE_DIR
mkdir $PROJECT_BUILD_DIR
cd $PROJECT_BUILD_DIR
cmake .. -DCMAKE_EXPORT_COMPILE_COMMANDS=ON
----
It is still possible, but optional, to manually generate the `compile_commands.json` file. For instructions on manually generating this file, see <<manual-compile-commands, this section>>.

At this step, you can also add any other flags you want to pass to `cmake`, such as `-DCMAKE_BUILD_TYPE=Release` or `-DCMAKE_CXX_COMPILER=clang++`.
By running CMake with the `CMAKE_EXPORT_COMPILE_COMMANDS` flag, you will generate a `compile_commands.json` file in your build directory.
This file contains all the information mrdocs needs to generate the documentation.

Now let's generate the reference files.
The following command will generate the documentation with the most common options:

[source,bash]
----
cd $PROJECT_BUILD_DIR
MRDOCS_OUTPUT=$PROJECT_BUILD_DIR/docs/reference/adoc
$MRDOCS_ROOT/mrdocs $PROJECT_BUILD_DIR/compile_commands.json --config=$MRDOCS_CONFIG --addons=$MRDOCS_ROOT/addons --output=$MRDOCS_OUTPUT
$MRDOCS_ROOT/mrdocs <ProjectPath> --config=$MRDOCS_CONFIG --addons=$MRDOCS_ROOT/addons --output=$MRDOCS_OUTPUT
----

Here's a description of these options:

* `<ProjectPath>`: the path to the project to document. This can be a path to a `compile_commands.json` file, a directory, or a `CMakeLists.txt` file.
If a path to a `compile_commands.json` file is provided, MrDocs will use this file and will not call CMake. It is assumed the user has already manually run CMake.
If a directory (not a file) is provided, it is assumed that this directory is the `ProjectPath` and contains a `CMakeLists.txt` describing the project. MrDocs will automatically run CMake to generate the `compile_commands.json`, using the parameters specified in the `mrdocs.yml` configuration file.
If a `CMakeLists.txt` file is provided, it is assumed that the directory containing this file is the `ProjectPath`. MrDocs will automatically run CMake to generate the `compile_commands.json`, using the parameters specified in the `mrdocs.yml` configuration file.
* `--config=$MRDOCS_CONFIG`: the path to the `mrdocs.yml` configuration file.
This file configures which generator is used, which directory to process,
and what symbols should be extracted.
Expand All @@ -54,6 +48,23 @@ This is where the generated documentation will be placed.

MrDocs ignores non-c++ source files, so nothing more needs to be done to generate the documentation for your project.

[[manual-compile-commands]]
=== Generating the compile_commands.json Manually

To generate the `compile_commands.json` file by running `cmake` with the `-DCMAKE_EXPORT_COMPILE_COMMANDS=ON` flag.

[source,bash]
----
cd $PROJECT_SOURCE_DIR
mkdir $PROJECT_BUILD_DIR
cd $PROJECT_BUILD_DIR
cmake .. -DCMAKE_EXPORT_COMPILE_COMMANDS=ON
----

At this step, you can also add any other flags you want to pass to `cmake`, such as `-DCMAKE_BUILD_TYPE=Release` or `-DCMAKE_CXX_COMPILER=clang++`.
By running CMake with the `CMAKE_EXPORT_COMPILE_COMMANDS` flag, you will generate a `compile_commands.json` file in your build directory.
This file contains all the information mrdocs needs to generate the documentation.

== Demos

A few examples of reference documentation generated with MrDocs are available in https://mrdocs.com/demos/.
Expand Down
236 changes: 236 additions & 0 deletions src/lib/Lib/CMakeExecution.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,236 @@
//
// Licensed under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
// Copyright (c) 2024 Fernando Pelliccioni ([email protected])
//
// Official repository: https://github.com/cppalliance/mrdocs
//

#include "lib/Lib/CMakeExecution.hpp"

#include <llvm/Support/FileSystem.h>
#include <llvm/Support/MemoryBuffer.h>
#include <llvm/Support/Path.h>
#include <llvm/Support/Program.h>

namespace clang {
namespace mrdocs {

namespace {

Expected<std::string>
getCmakePath() {
auto const path = llvm::sys::findProgramByName("cmake");
MRDOCS_CHECK(path, "CMake executable not found");
std::optional<llvm::StringRef> const redirects[] = {llvm::StringRef(), llvm::StringRef(), llvm::StringRef()};
std::vector<llvm::StringRef> const args = {*path, "--version"};
int const result = llvm::sys::ExecuteAndWait(*path, args, std::nullopt, redirects);
MRDOCS_CHECK(result == 0, "CMake execution failed when checking version");
return *path;
}

Expected<std::string>
executeCmakeHelp(llvm::StringRef cmakePath)
{
llvm::SmallString<128> outputPath;
MRDOCS_CHECK(!llvm::sys::fs::createTemporaryFile("cmake-help", "txt", outputPath),
"Failed to create temporary file");
std::optional<llvm::StringRef> const redirects[] = {llvm::StringRef(), outputPath.str(), llvm::StringRef()};
std::vector<llvm::StringRef> const args = {cmakePath, "--help"};
llvm::ArrayRef<llvm::StringRef> emptyEnv;
int const result = llvm::sys::ExecuteAndWait(cmakePath, args, emptyEnv, redirects);
MRDOCS_CHECK(result == 0, "CMake execution failed when trying to get help");

auto const bufferOrError = llvm::MemoryBuffer::getFile(outputPath);
MRDOCS_CHECK(bufferOrError, "Failed to read CMake help output");

return bufferOrError.get()->getBuffer().str();
}

Expected<std::string>
getCmakeDefaultGenerator(llvm::StringRef cmakePath)
{
MRDOCS_TRY(auto const cmakeHelp, executeCmakeHelp(cmakePath));

std::istringstream stream(cmakeHelp);
std::string line;
std::string defaultGenerator;

while (std::getline(stream, line)) {
if (line[0] == '*' && line[1] == ' ') {
size_t const start = 2;
size_t const end = line.find("=", start);
if (end == std::string::npos) {
continue;
}
return line.substr(start, end - start);
}
}
return Unexpected(Error("Default CMake generator not found"));
}

Expected<bool>
cmakeDefaultGeneratorIsVisualStudio(llvm::StringRef cmakePath)
{
MRDOCS_TRY(auto const defaultGenerator, getCmakeDefaultGenerator(cmakePath));
return defaultGenerator.starts_with("Visual Studio");
}

std::vector<std::string>
parseCmakeArgs(std::string const& cmakeArgsStr) {
std::vector<std::string> args;
std::string currentArg;
char quoteChar = '\0';
bool escapeNextChar = false;

for (char ch : cmakeArgsStr)
{
if (escapeNextChar)
{
currentArg += ch;
escapeNextChar = false;
}
else if (ch == '\\')
{
escapeNextChar = true;
}
else if ((ch == '"' || ch == '\''))
{
if (quoteChar == '\0')
{
quoteChar = ch;
}
else if (ch == quoteChar)
{
quoteChar = '\0';
}
else
{
currentArg.push_back(ch);
}
} else if (std::isspace(ch))
{
if (quoteChar != '\0')
{
currentArg.push_back(ch);
}
else
{
if ( ! currentArg.empty())
{
args.push_back(currentArg);
currentArg.clear();
}
}
} else
{
currentArg += ch;
}
}

if ( ! currentArg.empty())
{
args.push_back(currentArg);
}

return args;
}

} // anonymous namespace

Expected<std::string>
executeCmakeExportCompileCommands(llvm::StringRef projectPath, llvm::StringRef cmakeArgs)
{
MRDOCS_CHECK(llvm::sys::fs::exists(projectPath), "Project path does not exist");
MRDOCS_TRY(auto const cmakePath, getCmakePath());

llvm::SmallString<128> tempDir;
MRDOCS_CHECK(!llvm::sys::fs::createUniqueDirectory("compile_commands", tempDir), "Failed to create temporary directory");

llvm::SmallString<128> errorPath;
MRDOCS_CHECK(!llvm::sys::fs::createTemporaryFile("cmake-error", "txt", errorPath),
"Failed to create temporary file");

std::optional<llvm::StringRef> const redirects[] = {llvm::StringRef(), llvm::StringRef(), errorPath.str()};
std::vector<llvm::StringRef> args = {cmakePath, "-S", projectPath, "-B", tempDir.str(), "-DCMAKE_EXPORT_COMPILE_COMMANDS=ON"};

auto const additionalArgs = parseCmakeArgs(cmakeArgs.str());

bool visualStudioFound = false;
for (size_t i = 0; i < additionalArgs.size(); ++i)
{
auto const& arg = additionalArgs[i];
if (arg.starts_with("-G"))
{
if (arg.size() == 2)
{
if (i + 1 < additionalArgs.size())
{
auto const& generatorName = additionalArgs[i + 1];
if (generatorName.starts_with("Visual Studio"))
{
args.push_back("-GNinja");
visualStudioFound = true;
++i;
continue;
}
}
} else {
if (arg.find("Visual Studio", 2) != std::string::npos)
{
args.push_back("-GNinja");
visualStudioFound = true;
continue;
}
}
}

if (arg.starts_with("-D"))
{
if (arg.size() == 2)
{
if (i + 1 < additionalArgs.size())
{
auto const& optionName = additionalArgs[i + 1];
if (optionName.starts_with("CMAKE_EXPORT_COMPILE_COMMANDS"))
{
++i;
continue;
}
}
} else {
if (arg.find("CMAKE_EXPORT_COMPILE_COMMANDS", 2) != std::string::npos)
{
continue;
}
}
}
args.push_back(arg);
}

if ( ! visualStudioFound)
{
MRDOCS_TRY(auto const cmakeDefaultGeneratorIsVisualStudio, cmakeDefaultGeneratorIsVisualStudio(cmakePath));
if (cmakeDefaultGeneratorIsVisualStudio)
{
args.push_back("-GNinja");
}
}

int const result = llvm::sys::ExecuteAndWait(cmakePath, args, std::nullopt, redirects);
if (result != 0) {
auto bufferOrError = llvm::MemoryBuffer::getFile(errorPath);
MRDOCS_CHECK(bufferOrError, "CMake execution failed (no error output available)");
return Unexpected(Error("CMake execution failed: \n" + bufferOrError.get()->getBuffer().str()));
}

llvm::SmallString<128> compileCommandsPath(tempDir);
llvm::sys::path::append(compileCommandsPath, "compile_commands.json");

return compileCommandsPath.str().str();
}

} // mrdocs
} // clang
40 changes: 40 additions & 0 deletions src/lib/Lib/CMakeExecution.hpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
//
// Licensed under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
// Copyright (c) 2024 Fernando Pelliccioni ([email protected])
//
// Official repository: https://github.com/cppalliance/mrdocs
//

#ifndef MRDOCS_LIB_TOOL_CMAKE_EXECUTION_HPP
#define MRDOCS_LIB_TOOL_CMAKE_EXECUTION_HPP

#include <string>

#include <llvm/ADT/StringRef.h>
#include <mrdocs/Support/Error.hpp>

namespace clang {
namespace mrdocs {

/**
* Executes CMake to generate the `compile_commands.json` file for a project.
*
* This function runs CMake in a temporary directory for the given project path
* to create a `compile_commands.json` file.
*
* @param projectPath The path to the project directory.
* @param cmakeArgs The arguments to pass to CMake when generating the compilation database.
* @return An `Expected` object containing the path to the generated `compile_commands.json` file if successful.
* Returns `Unexpected` if the project path is not found or if CMake execution fails.
*/
Expected<std::string>
executeCmakeExportCompileCommands(llvm::StringRef projectPath, llvm::StringRef cmakeArgs);

} // mrdocs
} // clang

#endif // MRDOCS_LIB_TOOL_CMAKE_EXECUTION_HPP

Loading

0 comments on commit 70a9347

Please sign in to comment.