Extremely noisy defaults (recommended rules)

[harjotgill]

We recently rolled out Biome to our large customer base at CodeRabbit. We were under the assumption that, just like Ruff, Biome would have recommended rules that everyone agreed upon. However, the roll-out has been a disaster, and we ended up with a lot of cancellations and complaints on all our channels. Users were pulling their hair and screaming at the AI bot to spare them from noisy suggestions. In particular, useImportType, useExhaustiveDependencies and noForEach decimated the user experience.

For a linter, the defaults shouldn't be more than 10% of the total rules. In case of Biome, 90% rules are enabled as defaults - something is clearly off here.

I suggest that the maintainers take a deep look at the rule list and disable any rule that comes across as an opinion or has caveats. Defaults should only be those rules that everyone can agree upon. Ruff has done a great job with defaults, and that is one of the reasons it has good traction.

[ematipico]

Thank you for your feedback! It's a pity that the rollout didn't go as expected.

We recently rolled out Biome to our large customer base at CodeRabbit. We were under the assumption that, just like Ruff, Biome would have recommended rules that everyone agreed upon. However, the roll-out has been a disaster, and we ended up with a lot of cancellations and complaints on all our channels. Users were pulling their hair and screaming at the AI bot to spare them from noisy suggestions. In particular, useImportType, useExhaustiveDependencies and noForEach decimated the user experience.

I admit that maybe a few rules don't need to be recommended. However, in this particular example, I would argue that useExhaustiveDependencies should be recommended because the React core team recommends it.

For a linter, the defaults shouldn't be more than 10% of the total rules.

Why it should be 10%? Are there articles or talks that you can share that explain the reason behind this?

In case of Biome, 90% rules are enabled as defaults - something is clearly off here.

I admit that we have a lot of recommended rules, however this information needs some important context. Biome lints three languages: JavaScript, JSX and TypeScript. These languages come with their quirks, patterns and side-effects. Plus, we recently rolled out CSS support, which is a very relaxed language, which requires more rules in order to build correct CSS. At the moment we have around 15 nursery rules for CSS, if we had to follow your 10% suggestion, we would need to recommend 2 rules, which doesn't work.

Plus, considering that Biome is very recent, and we already have tons of recommended rules in the ecosystem (eslint + plugins + other linters), have you considered that maybe we decided to write first those rules that are generally used by the majority of projects out there (and recommended)? A11y rules are a clear example. They are many, and I don't see any strong reason to not recommend them.

Defaults should only be those rules that everyone can agree upon.

Who is "everyone" in this case? :) Biome a young project with a small team and a small community 😄 so if the team and the small number of contributors that follow the project feel good about some defaults, we ship it. We have a relaxed process at the moment. Do you have any particular suggestions? Any particular idea in mind?

[ematipico]

How would you define the "most serious ones"? What kind of metrics would you use? It would be great to try to craft a framework that would help us to point us in the right direction

[lostpebble]

Probably everything under style, a11y, complexity and correctness could be warnings instead of errors. But I agree with you, its hard to define what is a grievous error and what could be considered a strong recommendation (still should be a warning).

[Conaclos]

I previously proposed for Biome 2.0 to associate a de fault severity level at the group level: I proposed error for correctness and warn for all other groups.

[Conaclos]

Personally, I am in favour of having a stronger default than eslint. However, we can discuss - as we are doing here - about how to reduce possible noise: for example by reducing false positives, deprecating or removing from recommended the rules with low value, ...

[harjotgill]

If it helps, this is what we are setting as defaults. Note that CodeRabbit has 2 AI profiles - "Chill" and "Assertive". We are setting the defaults based on the profile.

	const biomeConfig = {
		$schema: "https://biomejs.dev/schemas/1.7.3/schema.json",
		organizeImports: {
			enabled: false,
		},
		formatter: {
			enabled: false,
		},
		linter: {
			enabled: true,
			rules: {
				all: false,
				recommended: true,
				correctness: {
					useExhaustiveDependencies: "off",
				},
				style: {
					recommended: isAssertive,
					noArguments: "error",
					noCommaOperator: "error",
					noUselessElse: "error",
					useAsConstAssertion: "off",
					useBlockStatements: "off",
					useConsistentArrayType: "off",
					useDefaultParameterLast: "error",
					useEnumInitializers: "off",
					useExponentiationOperator: "error",
					useExportType: "off",
					useFragmentSyntax: "off",
					useImportType: "off",
					useLiteralEnumMembers: "error",
					useShorthandArrayType: "off",
					noUnusedTemplateLiteral: "off",
				},
				complexity: {
					noForEach: "off",
					noExcessiveCognitiveComplexity: isAssertive ? "error" : "off",
					useLiteralKeys: "off",
				},
				suspicious: {
					noArrayIndexKey: isAssertive ? "error" : "off",
					noExplicitAny: isAssertive ? "error" : "off",
					noImplicitAnyLet: isAssertive ? "error" : "off",
					noDoubleEquals: isAssertive ? "error" : "off",
				},
				nursery: {
					all: false,
				},
			},
		},
		css: {
			linter: {
				enabled: true,
			},
		},
	}