Transform languages by DmitrySharabin · Pull Request #3948 · PrismJS/prism

DmitrySharabin · 2025-05-28T13:41:44Z

I’m gonna add transformed languages here on the go.

github-actions · 2025-05-28T13:42:13Z

No JS Changes

Generated by 🚫 dangerJS against 3c8263d

DmitrySharabin · 2025-05-28T16:12:27Z

There are four languages (cshtml, jsx, tsx, typescript) left that use extend(), but I don't know how to transform them. Probably, they illustrate cases we haven't covered yet? 🤔

For example, in tsx, we have:

prism/src/languages/tsx.ts

Lines 9 to 10 in ccbc10b

    
           const typescript = extend('typescript', {}); 
        
           const tsx = extend('jsx', typescript);

Other languages have something similar (i.e., they extend more than one language via extend()).

In typescript, we add properties from TS to an object we later use to define the grammar:

prism/src/languages/typescript.ts

Line 11 in ccbc10b

const typeInside: Grammar = {};

prism/src/languages/typescript.ts

Line 41 in ccbc10b

Object.assign(typeInside, typescript);

prism/src/languages/typescript.ts

Line 65 in ccbc10b

inside: typeInside,

@LeaVerou, HEEEEELP! 😅

LeaVerou

This is too large to review, do you want to flag any parts that were not straightforward so I can focus on those?

LeaVerou · 2025-05-29T19:02:32Z

There are four languages (cshtml, jsx, tsx, typescript) left that use extend(), but I don't know how to transform them. Probably, they illustrate cases we haven't covered yet? 🤔

Possibly. Let’s focus on getting the other ones merged, and I’ll take a look after.

DmitrySharabin · 2025-05-31T17:56:31Z

src/languages/flow.ts

 import { toArray } from '../util/iterables';
 import javascript from './javascript';
-import type { GrammarToken, LanguageProto } from '../types';
+import type { Grammar, LanguageProto } from '../types';


Let me point you to the languages I'd love you to review.

The first in this one—Flow. The reason: I used base and $merge here.

DmitrySharabin · 2025-05-31T18:00:36Z

src/languages/hlsl.ts

 			// https://docs.microsoft.com/en-us/windows/win32/direct3dhlsl/dx-graphics-hlsl-appendix-reserved-words
 			'class-name': [
-				...toArray(c['class-name']),
+				...toArray(base!['class-name']),


Also this part

DmitrySharabin · 2025-05-31T18:02:02Z

src/languages/javadoc.ts

-	grammar ({ extend, getLanguage }) {
-		const java = getLanguage('java');
-		const { tag, entity } = getLanguage('markup');
+	base: javadoclike,


This language: javadoc

DmitrySharabin · 2025-05-31T18:03:27Z

src/languages/latte.ts

+					},
+				},
+			},
+			$tokenize: embeddedIn('markup'),


This part (the latte language).

DmitrySharabin · 2025-05-31T18:04:19Z

src/languages/mongodb.ts

-				pattern:
-					/\b(?:(?:[01]?\d\d?|2[0-4]\d|25[0-5])\.){3}(?:[01]?\d\d?|2[0-4]\d|25[0-5])\b/,
-				greedy: true,
+			$merge: {


This language (mongodb): because of $merge.

DmitrySharabin · 2025-05-31T18:05:43Z

src/languages/parser.ts

+							'keyword': base!.keyword,
+							'variable': base!.variable,
+							'function': base!.function,
+							'boolean': /\b(?:false|true)\b/,


The parser language: I used base inside the language definition.

DmitrySharabin · 2025-05-31T18:08:41Z

src/languages/sass.ts

+							'punctuation': /:/,
+							'variable': variable,
+							'operator': operator,
+							'important': base!.important,


This part (the sass language)

DmitrySharabin · 2025-05-31T18:09:41Z

src/languages/sqf.ts

+								pattern: /#[a-z]+\b/i,
+								alias: 'keyword',
+							},
+							'comment': base!.comment,


This part (the sqf language)

DmitrySharabin · 2025-05-31T18:10:25Z

src/languages/textile.ts

-			/<\/?(?!\d)[a-z0-9]+(?:\s+[^\s>\/=]+(?:=(?:("|')(?:\\[\s\S]|(?!\1)[^\\])*\1|[^\s'">=]+))?)*\s*\/?>/i;
-
-		return textile;
+			$merge: {


The textile language (because of $merge)

DmitrySharabin · 2025-05-31T18:11:16Z

src/languages/velocity.ts

-				inside: {
-					'punctuation': /^#\[\[|\]\]#$/,
+		return {
+			$merge: {


The velocity language (because of $merge)

DmitrySharabin · 2025-05-31T18:11:52Z

src/languages/wiki.ts

-		const tag = markup['tag'] as GrammarToken;
+	base: markup,
+	grammar ({ base }) {
+		const tag = base!['tag'] as GrammarToken;


This part (the wiki language)

DmitrySharabin · 2025-05-31T18:12:43Z

src/languages/xquery.ts

+	base: markup,
+	grammar () {
+		return {
+			$merge: {


The xquery language (because of $merge)

LeaVerou · 2025-05-31T18:31:51Z

src/languages/wiki.ts

+				'tag': {
+					// Prevent highlighting inside <nowiki>, <source> and <pre> tags
+					'nowiki': {
+						pattern: /<(nowiki|pre|source)\b[^>]*>[\s\S]*?<\/\1>/i,


When you're only inserting one token before another, $before: 'tag' in the nowiki token could help reduce the levels of indentation

Yes, this is much simpler. Thank you!

LeaVerou

I didn't review in a ton of depth, but at a glance everything LGTM!

DmitrySharabin force-pushed the transform-languages branch 2 times, most recently from 0d3f303 to 4cd6fc1 Compare May 28, 2025 16:01

DmitrySharabin requested a review from LeaVerou May 28, 2025 16:13

DmitrySharabin force-pushed the transform-languages branch from 38cf2f6 to 43492da Compare May 28, 2025 16:33

LeaVerou reviewed May 29, 2025

View reviewed changes

DmitrySharabin added 22 commits May 31, 2025 19:49

Vala

2276d38

Firestore security rules

96a1366

Xeora

6ffb7dc

Flow

117848b

Wiki

b4d3186

V

b9268b0

Velocity

763e2a1

glsl

fbee503

gml

7111d97

Go

3563429

gradle

9887b42

groovy

58b3f9a

haxe

f93c836

kotlin

ca7729a

javadoc

ae16a15

squirrel

cf50ee3

hlsl

3e76dc8

idris

6e83d98

json5

ed773b5

jsonp

abf2d62

less

9a87359

n4js

42ba110

DmitrySharabin added 11 commits May 31, 2025 19:49

latte

e062955

parser

c995387

textile

9f5ae15

qsharp

7900d65

ruby

b53c26d

sass

9f99a67

scss

f4e27b3

xquery

bdb73f0

http

47abee3

haml

c0d0421

rescript

fdf22de

DmitrySharabin commented May 31, 2025

View reviewed changes

src/languages/latte.ts

},

},

},

$tokenize: embeddedIn('markup'),

Copy link

Member Author

DmitrySharabin May 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This part (the latte language).

DmitrySharabin commented May 31, 2025

View reviewed changes

src/languages/xquery.ts

base: markup,

grammar () {

return {

$merge: {

Copy link

Member Author

DmitrySharabin May 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The xquery language (because of $merge)

LeaVerou reviewed May 31, 2025

View reviewed changes

LeaVerou approved these changes May 31, 2025

View reviewed changes

Address feedback: use $insert–$before syntactic sugar

3c8263d

DmitrySharabin force-pushed the transform-languages branch from 43492da to 3c8263d Compare May 31, 2025 20:54

DmitrySharabin merged commit 5660d05 into simplify May 31, 2025
2 checks passed

DmitrySharabin deleted the transform-languages branch May 31, 2025 20:55

DmitrySharabin mentioned this pull request Jun 12, 2025

Things blocking the simplify branch from being merged #3960

Closed

Uh oh!

Conversation

DmitrySharabin commented May 28, 2025

Uh oh!

github-actions bot commented May 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

No JS Changes

Uh oh!

DmitrySharabin commented May 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

LeaVerou left a comment

Choose a reason for hiding this comment

Uh oh!

LeaVerou commented May 29, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

LeaVerou left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

github-actions bot commented May 28, 2025 •

edited

Loading

DmitrySharabin commented May 28, 2025 •

edited

Loading