Skip to content

Apply formatting to markdown code blocks #22470

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
amyreese merged 45 commits into from
Jan 28, 2026
+664 −149
Merged

Apply formatting to markdown code blocks #22470

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
df5c04b
Create new source types for markdown files
amyreese Jan 9, 2026
296d101
Update source kind from path mapping
amyreese Jan 9, 2026
dc50d77
Minimal prototype using regex
amyreese Jan 13, 2026
b6ab7fe
Map pyi code block to pyi source type
amyreese Jan 17, 2026
cd94b2d
Skip linting .md files for now
amyreese Jan 20, 2026
d23610c
Use static lazy lock for markdown regex
amyreese Jan 20, 2026
9ea195a
Gate formatting markdown on preview mode
amyreese Jan 20, 2026
1521834
todos for markdown code block regex/parsing
amyreese Jan 20, 2026
0e787cb
Add markdown test fixture
amyreese Jan 20, 2026
a2c7b4f
Simple CLI tests
amyreese Jan 20, 2026
04d6636
Fix markdown formatting on stdin, add stdin test
amyreese Jan 20, 2026
79b23a9
test with relative paths
amyreese Jan 20, 2026
64bfbba
Skip walking markdown files in ty
amyreese Jan 21, 2026
f130cdc
Refactor source type/kind usage
amyreese Jan 22, 2026
c019e1f
Return better error types for unsupported range formatting and markdown
amyreese Jan 22, 2026
4c5d430
Use Self
amyreese Jan 22, 2026
68b5f95
Updated snapshots
amyreese Jan 22, 2026
c456eed
Add fixture path helper and default CRATE_ROOT filter
amyreese Jan 22, 2026
a417da8
Move markdown bits into new ruff_markdown crate
amyreese Jan 23, 2026
6be5947
Add some tests, fix clippy
amyreese Jan 23, 2026
2c61937
Better wording for 'experimental' message
amyreese Jan 23, 2026
bdd7047
Shortened fixture test cases
amyreese Jan 23, 2026
1b69970
Fix cargo files
amyreese Jan 23, 2026
341cfb3
Drop source kind toml
amyreese Jan 23, 2026
364325d
Include PySourceType in SourceKind::Python
amyreese Jan 23, 2026
8ec6ccc
Update error messages
amyreese Jan 23, 2026
7173d78
Update fixture and snapshots
amyreese Jan 23, 2026
def3641
Remove trivia dep
amyreese Jan 23, 2026
47f252d
snapshots
amyreese Jan 23, 2026
a3a2c56
Loop over captures and use range replacement instead of replace_all
amyreese Jan 24, 2026
2ba6eb5
Support unlabeled blocks, more tests
amyreese Jan 24, 2026
48fc2f5
Only wrap a bool type for source kind python
amyreese Jan 26, 2026
cde3adb
Update fuzz test
amyreese Jan 26, 2026
a61ab53
Get code range straight from regex
amyreese Jan 26, 2026
80911e2
clippy
amyreese Jan 26, 2026
7074b1a
static lifetime
amyreese Jan 27, 2026
cb1e001
Call it PreviewFeature
amyreese Jan 27, 2026
56e9eb9
explicit match arms
amyreese Jan 27, 2026
6f340cd
zero ver
amyreese Jan 27, 2026
d5f0a30
Build formatted string iteratively
amyreese Jan 27, 2026
3fd00de
Docs update
amyreese Jan 27, 2026
ce978f8
Refactor match arms
amyreese Jan 27, 2026
2060a2b
use nested match statements
amyreese Jan 27, 2026
d143fb1
Make SourceKind::Python a struct rather than tuple
amyreese Jan 27, 2026
0e5bbd3
Fuzz
amyreese Jan 27, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions Cargo.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ ruff_graph = { path = "crates/ruff_graph" }
ruff_index = { path = "crates/ruff_index" }
ruff_linter = { path = "crates/ruff_linter" }
ruff_macros = { path = "crates/ruff_macros" }
ruff_markdown = { path = "crates/ruff_markdown" }
ruff_memory_usage = { path = "crates/ruff_memory_usage" }
ruff_notebook = { path = "crates/ruff_notebook" }
ruff_options_metadata = { path = "crates/ruff_options_metadata" }
Expand Down
1 change: 1 addition & 0 deletions crates/ruff/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ ruff_diagnostics = { workspace = true }
ruff_graph = { workspace = true, features = ["serde", "clap"] }
ruff_linter = { workspace = true, features = ["clap"] }
ruff_macros = { workspace = true }
ruff_markdown = { workspace = true }
ruff_notebook = { workspace = true }
ruff_options_metadata = { workspace = true, features = ["serde"] }
ruff_python_ast = { workspace = true }
Expand Down
11 changes: 11 additions & 0 deletions crates/ruff/resources/test/fixtures/unformatted.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
This is a markdown document with two fenced code blocks:

```py
print( "hello" )
def foo(): pass
```

```pyi
print( "hello" )
def foo(): pass
```
3 changes: 2 additions & 1 deletion crates/ruff/src/cache.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -486,6 +486,7 @@ mod tests {
use anyhow::Result;
use filetime::{FileTime, set_file_mtime};
use itertools::Itertools;
use ruff_python_ast::SourceType;
use test_case::test_case;

use ruff_cache::CACHE_DIR_NAME;
Expand Down Expand Up @@ -1024,7 +1025,7 @@ mod tests {
format_path(
&file_path,
&self.settings.formatter,
PySourceType::Python,
SourceType::Python(PySourceType::Python),
FormatMode::Write,
None,
Some(cache),
Expand Down
2 changes: 1 addition & 1 deletion crates/ruff/src/commands/add_noqa.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ pub(crate) fn add_noqa(
{
return None;
}
let source_kind = match SourceKind::from_path(path, source_type) {
let source_kind = match SourceKind::from_path(path, SourceType::Python(source_type)) {
Ok(Some(source_kind)) => source_kind,
Ok(None) => return None,
Err(e) => {
Expand Down
9 changes: 8 additions & 1 deletion crates/ruff/src/commands/analyze_graph.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -144,6 +144,10 @@ pub(crate) fn analyze_graph(
debug!("Ignoring TOML file: {}", path.display());
continue;
}
SourceType::Markdown => {
debug!("Ignoring Markdown file: {}", path.display());
continue;
}
},
Some(language) => PySourceType::from(language),
};
Expand All @@ -164,7 +168,10 @@ pub(crate) fn analyze_graph(
let result = inner_result.clone();
scope.spawn(move |_| {
// Extract source code (handles both .py and .ipynb files)
let source_kind = match SourceKind::from_path(path.as_std_path(), source_type) {
let source_kind = match SourceKind::from_path(
path.as_std_path(),
SourceType::Python(source_type),
) {
Ok(Some(source_kind)) => source_kind,
Ok(None) => {
debug!("Skipping non-Python notebook: {path}");
Expand Down
118 changes: 88 additions & 30 deletions crates/ruff/src/commands/format.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ use ruff_db::diagnostic::{
};
use ruff_linter::message::{EmitterContext, create_panic_diagnostic, render_diagnostics};
use ruff_linter::settings::types::OutputFormat;
use ruff_markdown::{MarkdownResult, format_code_blocks};
use ruff_notebook::NotebookIndex;
use ruff_python_parser::ParseError;
use rustc_hash::{FxHashMap, FxHashSet};
Expand Down Expand Up @@ -123,15 +124,13 @@ pub(crate) fn format(
let settings = resolver.resolve(path);

let source_type = match settings.formatter.extension.get(path) {
None => match SourceType::from(path) {
SourceType::Python(source_type) => source_type,
SourceType::Toml(_) => {
// Ignore any non-Python files.
return None;
}
},
Some(language) => PySourceType::from(language),
None => SourceType::from(path),
Some(language) => SourceType::Python(PySourceType::from(language)),
};
if source_type.is_toml() {
// Ignore TOML files.
return None;
}

// Ignore files that are excluded from formatting
if (settings.file_resolver.force_exclude || !resolved_file.is_root())
Expand Down Expand Up @@ -261,7 +260,7 @@ pub(crate) fn format(
pub(crate) fn format_path(
path: &Path,
settings: &FormatterSettings,
source_type: PySourceType,
source_type: SourceType,
mode: FormatMode,
range: Option<FormatRange>,
cache: Option<&Cache>,
Expand Down Expand Up @@ -292,8 +291,7 @@ pub(crate) fn format_path(
let cache = cache.filter(|_| range.is_none());

// Format the source.
let format_result = match format_source(&unformatted, source_type, Some(path), settings, range)?
{
let format_result = match format_source(&unformatted, Some(path), settings, range)? {
FormattedSource::Formatted(formatted) => match mode {
FormatMode::Write => {
let mut writer = File::create(path).map_err(|err| {
Expand Down Expand Up @@ -357,14 +355,17 @@ impl From<FormattedSource> for FormatResult {
/// unchanged.
pub(crate) fn format_source(
source_kind: &SourceKind,
source_type: PySourceType,
path: Option<&Path>,
settings: &FormatterSettings,
range: Option<FormatRange>,
) -> Result<FormattedSource, FormatCommandError> {
match &source_kind {
SourceKind::Python(unformatted) => {
let options = settings.to_format_options(source_type, unformatted, path);
SourceKind::Python {
code: unformatted,
is_stub,
} => {
let py_source_type = source_kind.py_source_type();
let options = settings.to_format_options(py_source_type, unformatted, path);

let formatted = if let Some(range) = range {
let line_index = LineIndex::from_source_text(unformatted);
Expand Down Expand Up @@ -400,7 +401,10 @@ pub(crate) fn format_source(
if formatted.len() == unformatted.len() && formatted == *unformatted {
Ok(FormattedSource::Unchanged)
} else {
Ok(FormattedSource::Formatted(SourceKind::Python(formatted)))
Ok(FormattedSource::Formatted(SourceKind::Python {
code: formatted,
is_stub: *is_stub,
}))
}
}
SourceKind::IpyNotebook(notebook) => {
Expand All @@ -409,12 +413,13 @@ pub(crate) fn format_source(
}

if range.is_some() {
return Err(FormatCommandError::RangeFormatNotebook(
return Err(FormatCommandError::RangeFormatNotSupported(
path.map(Path::to_path_buf),
));
}

let options = settings.to_format_options(source_type, notebook.source_code(), path);
let options =
settings.to_format_options(PySourceType::Ipynb, notebook.source_code(), path);

let mut output: Option<String> = None;
let mut last: Option<TextSize> = None;
Expand Down Expand Up @@ -489,6 +494,26 @@ pub(crate) fn format_source(
formatted,
)))
}
SourceKind::Markdown(unformatted_document) => {
if !settings.preview.is_enabled() {
return Err(FormatCommandError::MarkdownExperimental(
path.map(Path::to_path_buf),
));
}

if range.is_some() {
return Err(FormatCommandError::RangeFormatNotSupported(
path.map(Path::to_path_buf),
));
}

match format_code_blocks(unformatted_document, path, settings) {
MarkdownResult::Formatted(formatted) => {
Ok(FormattedSource::Formatted(SourceKind::Markdown(formatted)))
}
MarkdownResult::Unchanged => Ok(FormattedSource::Unchanged),
}
}
}
}

Expand Down Expand Up @@ -830,7 +855,8 @@ pub(crate) enum FormatCommandError {
Read(Option<PathBuf>, SourceError),
Format(Option<PathBuf>, FormatModuleError),
Write(Option<PathBuf>, SourceError),
RangeFormatNotebook(Option<PathBuf>),
RangeFormatNotSupported(Option<PathBuf>),
MarkdownExperimental(Option<PathBuf>),
}

impl FormatCommandError {
Expand All @@ -848,7 +874,8 @@ impl FormatCommandError {
| Self::Read(path, _)
| Self::Format(path, _)
| Self::Write(path, _)
| Self::RangeFormatNotebook(path) => path.as_deref(),
| Self::RangeFormatNotSupported(path)
| Self::MarkdownExperimental(path) => path.as_deref(),
}
}
}
Expand Down Expand Up @@ -880,10 +907,15 @@ impl From<&FormatCommandError> for Diagnostic {
Diagnostic::new(DiagnosticId::Io, Severity::Error, source_error)
}
FormatCommandError::Format(_, format_module_error) => format_module_error.into(),
FormatCommandError::RangeFormatNotebook(_) => Diagnostic::new(
FormatCommandError::RangeFormatNotSupported(_) => Diagnostic::new(
DiagnosticId::InvalidCliOption,
Severity::Error,
"Range formatting isn't supported for notebooks.",
"Range formatting is only supported for Python files.",
),
FormatCommandError::MarkdownExperimental(_) => Diagnostic::new(
DiagnosticId::PreviewFeature,
Severity::Warning,
"Markdown formatting is experimental, enable preview mode.",
),
};

Expand Down Expand Up @@ -962,19 +994,36 @@ impl Display for FormatCommandError {
write!(f, "{header} {err}", header = "Failed to format:".bold())
}
}
Self::RangeFormatNotebook(path) => {
Self::RangeFormatNotSupported(path) => {
if let Some(path) = path {
write!(
f,
"{header}{path}{colon} Range formatting isn't supported for notebooks.",
"{header}{path}{colon} Range formatting is only supported for Python files.",
header = "Failed to format ".bold(),
path = fs::relativize_path(path).bold(),
colon = ":".bold()
)
} else {
write!(
f,
"{header} Range formatting isn't supported for notebooks",
"{header} Range formatting is only supported for Python files",
header = "Failed to format:".bold()
)
}
}
Self::MarkdownExperimental(path) => {
if let Some(path) = path {
write!(
f,
"{header}{path}{colon} Markdown formatting is experimental, enable preview mode.",
header = "Failed to format ".bold(),
path = fs::relativize_path(path).bold(),
colon = ":".bold()
)
} else {
write!(
f,
"{header} Markdown formatting is experimental, enable preview mode",
header = "Failed to format:".bold()
)
}
Expand Down Expand Up @@ -1248,7 +1297,10 @@ mod tests {
#[test]
fn error_diagnostics() -> anyhow::Result<()> {
let path = PathBuf::from("test.py");
let source_kind = SourceKind::Python("1".to_string());
let source_kind = SourceKind::Python {
code: "1".to_string(),
is_stub: false,
};

let panic_error = catch_unwind(|| {
panic!("Test panic for FormatCommandError");
Expand Down Expand Up @@ -1290,7 +1342,7 @@ mod tests {
"Cannot write to file",
)),
),
FormatCommandError::RangeFormatNotebook(Some(path)),
FormatCommandError::RangeFormatNotSupported(Some(path)),
];

let results = FormatResults::new(&[], FormatMode::Check);
Expand All @@ -1305,7 +1357,7 @@ mod tests {
settings.add_filter(r"(Panicked at) [^:]+:\d+:\d+", "$1 <location>");
let _s = settings.bind_to_scope();

assert_snapshot!(str::from_utf8(&buf)?, @"
assert_snapshot!(str::from_utf8(&buf)?, @r"
io: test.py: Permission denied
--> test.py:1:1

Expand All @@ -1321,7 +1373,7 @@ mod tests {
io: Cannot write to file
--> test.py:1:1

invalid-cli-option: Range formatting isn't supported for notebooks.
invalid-cli-option: Range formatting is only supported for Python files.
--> test.py:1:1

panic: Panicked at <location> when checking `test.py`: `Test panic for FormatCommandError`
Expand All @@ -1347,8 +1399,14 @@ mod tests {
expect_formatted: Range<u32>,
) {
let mr = ModifiedRange::new(
&SourceKind::Python(unformatted.to_string()),
&SourceKind::Python(formatted.to_string()),
&SourceKind::Python {
code: unformatted.to_string(),
is_stub: false,
},
&SourceKind::Python {
code: formatted.to_string(),
is_stub: false,
},
);
assert_eq!(
mr.unformatted,
Expand Down
8 changes: 4 additions & 4 deletions crates/ruff/src/commands/format_stdin.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,15 +53,15 @@ pub(crate) fn format_stdin(

let source_type = match path.and_then(|path| settings.extension.get(path)) {
None => match path.map(SourceType::from).unwrap_or_default() {
SourceType::Python(source_type) => source_type,
source_type @ (SourceType::Python(_) | SourceType::Markdown) => source_type,
SourceType::Toml(_) => {
if mode.is_write() {
parrot_stdin()?;
}
return Ok(ExitStatus::Success);
}
},
Some(language) => PySourceType::from(language),
Some(language) => SourceType::Python(PySourceType::from(language)),
};

// Format the file.
Expand All @@ -88,7 +88,7 @@ fn format_source_code(
path: Option<&Path>,
range: Option<FormatRange>,
settings: &FormatterSettings,
source_type: PySourceType,
source_type: SourceType,
mode: FormatMode,
) -> Result<FormatResult, FormatCommandError> {
// Read the source from stdin.
Expand All @@ -104,7 +104,7 @@ fn format_source_code(
};

// Format the source.
let formatted = format_source(&source_kind, source_type, path, settings, range)?;
let formatted = format_source(&source_kind, path, settings, range)?;

match &formatted {
FormattedSource::Formatted(formatted) => match mode {
Expand Down
Loading
Loading