Document force-ignore syntax

codegen/src/domains.rs

@@ -98,7 +98,7 @@ impl DocDomains {
                     writeln!(buffer, "## {name}")?;
                     writeln!(
                         buffer,
-                        "This domain contains rules that perform project-level analysis. This includes our module graph for dependency resolution, as well as type information. When enabling rules that belong to this domain, Biome will scan the entire project. The scanning phase will have a performance impact on the linting process. See the documentation on our [scanner](/internals/architecture/#scanner) to learn more about how you can influence the scanner behaviour."
+                        "This domain contains rules that perform project-level analysis. This includes our module graph for dependency resolution, as well as type information. When enabling rules that belong to this domain, Biome will scan the entire project. The scanning phase will have a performance impact on the linting process. See the documentation on our [scanner](/internals/architecture/#scanner) to learn more about the scanner."
                     )?;
                 }
                 RuleDomain::Vue => {

src/content/docs/internals/architecture.mdx

@@ -13,8 +13,8 @@ the scanner is used:
 
 - To discover nested `biome.json`/`biome.jsonc` files in monorepos.
 - To discover nested `.gitignore` files if the [`vcs.useIgnoreFile`](/reference/configuration/#vcsuseignorefile) setting is enabled.
-- To index all JavaScript/TypeScript files in a project if any rules from the
-  [project domain](/linter/domains/#project) are enabled.
+- To index `package.json` manifests as well as source files in a project if any
+  rules from the [project domain](/linter/domains/#project) are enabled.
 
 ### Scanner targeting
 
@@ -29,8 +29,7 @@ the following folders get scanned for nested configuration files and/or nested i
 - The `packages/` folder.
 - The `packages/foo/` folder.
 - Any folders that exist under `packages/foo/`, except `node_modules/` or
-  those that are excluded by your configuration (see
-  [below](#configuringthescanner)).
+  those that are excluded by your configuration.
 
 Other folders that may be adjacent to either `packages/` or `packages/foo/` will
 be automatically skipped.
@@ -42,21 +41,8 @@ If project rules are enabled, these optimisations don't apply.
 
 ### Configuring the scanner
 
-The scanner respects the
-[`files.includes`](/reference/configuration/#filesincludes) setting as well as
-any `.gitignore` files, but there is one major exception to be aware of:
-
-If the project domain or one of its rules is enabled, **dependencies of your
-included files are scanned as well**. This means that `.d.ts` files and
-`package.json` manifests inside `node_modules` may still get indexed, provided
-there is a source file that references them. Similarly, generated files that are
-normally ignored, may still be indexed if you import them. This is done in order
-to extract type information from such files.
-
-For the time being, if you really insist on preventing the scanner from
-accessing a given file or folder, you can use the
-[`files.experimentalScannerIgnores`](/reference/configuration/#filesexperimentalscannerignores)
-setting for that.
+The scanner can be configured through the
+[`files.includes`](/reference/configuration/#interactionwiththescanner) setting.
 
 ## Parser and CST
 

src/content/docs/reference/configuration.mdx

@@ -109,15 +109,46 @@ This means that:
 * `src/special.test.js` **is** processed.
 * `test/special.test.js` **is not** processed.
 
-#### Note on Biome's scanner
+Note that files inside `node_modules/` are ignored regardless of the
+`files.includes` setting.
 
-Biome has a scanner that is responsible for discovering nested `.gitignore`
-files as well as indexing projects if any of the rules from the project domain
-are enabled.
+#### Interaction with the scanner
 
-The scanner respects the `files.includes` setting, but there is some fineprint.
-See the [scanner documentation](/internals/architecture/#scanner) for more
-information.
+Biome has a [scanner](/internals/architecture/#scanner) that is responsible for
+discovering nested configuration files as well as `.gitignore` files. It can
+also index source files if one or more rules from the
+[project domain](/linter/domains/#project) are enabled.
+
+The scanner respects both `files.includes` and the ignored patterns from
+`.gitignore` files, but there is one exception to be aware of: If any rule from
+the project domain is enabled, the scanner will index source files _including
+their dependencies_. This means that files that are ignored as part of
+`files.includes` may still get indexed by the scanner, as long as there is
+another included file that that imports those files. And this also means that
+`.d.ts` files and `package.json` manifests inside `node_modules/` may still get
+indexed too.
+
+If you want to explicitly force some files to be ignored by the scanner, you can
+do so using a so-called _force-ignore pattern_. A force-ignore pattern looks
+like a regular negated pattern, but starts with a double exclamation mark
+(`!!`).
+
+For example, you can tell Biome to never look inside any `dist/` folder using
+the following configuration:
+
+```json title="biome.json"
+{
+  "files": {
+    "includes": ["**", "!!**/dist"]
+  }
+}
+```
+
+We recommend using the force-ignore syntax for any folders that contain _output_
+files, such as `build/` and `dist/`. For such folders, it is highly unlikely
+that indexing has any useful benefits. For folders containing generated files,
+we advise using regular ignore patterns so that type information can still be
+extracted from the files.
 
 ### `files.ignoreUnknown`
 
@@ -143,30 +174,17 @@ this limit will be ignored for performance reasons.
 
 ### `files.experimentalScannerIgnores`
 
-An array of literal paths that the scanner should ignore during the crawling. The ignored files won't be indexed, which means that these files won't be part of the module graph, and types won't be inferred from them.
-
-In the following example, the folders `lodash` and `dist` and the file `RedisCommander.d.ts` will be ignored:
-
-```json title="biome.json"
-{
-  "files" : {
-    "experimentalScannerIgnores": [
-      "lodash",
-      "dist",
-      "RedisCommander.d.ts"
-    ]
-  }
-}
-```
-
-You should use this option only as a last resort in cases Biome takes a lot of time to lint/check your project. (Glob) paths aren't supported, and only basenames are matched.
-
-See the [scanner documentation](/internals/architecture/#scanner) for more information.
-
 :::caution
-As an experimental option, its usage is subject to change. The goal is to make Biome as fast as possible and eventually remove the option.
+This option has been **deprecated** and may be removed in a future release.
+Please use the
+[_force-ignore syntax_](/reference/configuration/#interactionwiththescanner)
+with `files.includes` instead.
 :::
 
+An array of literal path segments that the scanner should ignore during the
+crawling. The ignored files won't be indexed, which means that these files won't
+be part of the module graph, and types won't be inferred from them.
+
 ## `vcs`
 
 Set of properties to integrate Biome with a VCS (Version Control Software).