[Kgupta62]

What?

Closes #40023

cleanForSlug keeps the combining marks that scripts like Devanagari, Tamil and Bengali need, while Latin accents are still removed.

Why?

The final filter only kept letters and numbers. The vowel signs these scripts use are combining marks, so they were deleted and words got mangled. For example, typing अलग-अलग प्रकार के प्लंबिंग ट्रैप्स into the slug field saves अलग-अलग-परकर-क-पलबग-टरपस.

How?

The filter now also allows \p{M}. A new step strips the Latin combining range (U+0300 to U+036F) first, so decomposed input like Crème still becomes creme.

Testing Instructions

1. Open a post and go to the URL panel.
2. Type अलग-अलग प्रकार के प्लंबिंग ट्रैप्स into the slug field and click away. The slug should keep every character.
3. Check that Crème Brûlée still becomes creme-brulee.
4. Unit tests: npm run test:unit -- packages/url

Testing Instructions for Keyboard

Type the slug and Tab out of the field; the result should be the same.

Use of AI Tools

Claude Code was used to help investigate the issue and draft the patch and its test. I reviewed the change, ran the tests locally, and take responsibility for the result.

[Mamaduka]

Suggested change

	- `cleanForSlug`: keep combining marks so slugs in scripts like Devanagari, Tamil and Bengali are no longer mangled ([#40023](https://github.com/WordPress/gutenberg/issues/40023)).
	- `cleanForSlug`: keep combining marks so slugs in scripts like Devanagari, Tamil and Bengali are no longer mangled ([#83525](https://github.com/WordPress/gutenberg/issues/83525)).

Nit: changelog should reference the PR number.

[Kgupta62]

Good catch, fixed. The entry links this PR now.

[jorgefilipecosta]

I think we have an issue with emojis something like "I ❤️ WordPress" now leaves an invisible U+FE0F in the slug.

[ekamran]

I checked this in Playground with this PR (WordPress 7.1.3).

The URL field saves cleanForSlug() of what you type, and on this branch हिंदी व्याकरण becomes हिंदी-व्याकरण and अलग-अलग प्रकार के प्लंबिंग ट्रैप्स keeps every matra and halant. That fixes the Hindi case from #40023.

I can also confirm Jorge's concern. I ❤️ WordPress gives i-️-wordpress with an invisible U+FE0F, and ✔️, ©️ and the keycap 1️⃣ (U+FE0F plus U+20E3) do the same. Stripping variation selectors and enclosing marks first, for example .replace( /[\p{Variation_Selector}\p{Me}]/gu, '' ), and then allowing the remaining marks keeps the Hindi fix and gives i-wordpress again.

The branch also needs a rebase, since GitHub shows conflicts.

[github-actions]

🤖 PR meta 🤖

🎉 Props

Unlinked Accounts

The following contributors have not linked their GitHub and WordPress.org accounts: @pratikgandhi14.

Contributors, please read how to link your accounts to ensure your work is properly credited in WordPress releases.

If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message.

Unlinked contributors: pratikgandhi14.

Co-authored-by: Kgupta62 <kuber8821@git.wordpress.org>
Co-authored-by: Mamaduka <mamaduka@git.wordpress.org>
Co-authored-by: jorgefilipecosta <jorgefilipecosta@git.wordpress.org>
Co-authored-by: ekamran <ekamran@git.wordpress.org>
Co-authored-by: ndiego <ndiego@git.wordpress.org>
Co-authored-by: skorasaurus <skorasaurus@git.wordpress.org>
Co-authored-by: aristath <aristath@git.wordpress.org>

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

Updated as activity occurs, without notifying anyone named here. Add the props-bot label to refresh.

[Kgupta62]

Thanks @jorgefilipecosta for catching this, and @ekamran for testing all those cases.

Variation selectors and enclosing marks are now stripped before the mark check, so I ❤️ WordPress gives i-wordpress again, and 1️⃣, ✔️ and ©️ no longer leave anything invisible behind. I compared trunk against this branch on 11 emoji inputs, including skin tones, ZWJ families, flags, keycaps and emoji at the start and end of a string, and the output is identical in every case, with no stray or doubled separators.

The Devanagari and Tamil cases still keep their matras and halant, so the original fix is intact. Tests now cover the emoji and keycap cases and fail without the change. Changelog links this PR, and the branch is rebased on trunk.

[youknowriad]

Thanks @Kgupta62, and @ekamran and @jorgefilipecosta for testing. #84896 ports core’s sanitize_title() and closes #40023. The Devanagari vowel signs are kept, and emoji are kept whole like core does, so the variation-selector problem goes away too. Closing as superseded.