Some fixes to tokenize

[DmitrySharabin]

Summary

• Don’t choke on patterns that are not instances of RegExp
• In resolve() return resolvedGrammar and delete $rest after it is properly resolved

[github-actions]

No JS Changes

Generated by 🚫 dangerJS against 41093b8

[DmitrySharabin]

This is still not fully resolving cases like:

{
  id: "foo",
  require: bar
  grammar: {
    token: {
      ...
      inside: "bar"
    }
  }
}

The bar language is imported and can be accessed via the languages array associated with the foo language. However, languages["bar"] is evaluated lazily. Until it's accessed for the first time, the bar language won't be added to the languageRegistry, and resolve(prism, "bar") will always return undefined.

Honestly, I don't know how to solve it without providing some extra info on the grammar we resolve. 🤷‍♂️

[LeaVerou]

I think we'll need to traverse the grammar for resolvedGrammar and resolve anything that can be resolved without context. That will help with many things, e.g. self as well. It won't help with optional languages though. But we can replace it with a getter that overwrites itself.

[DmitrySharabin]

Sounds like a plan 👍