refactor: remove idToPkgMap

[sapphi-red]

Description

IIUC findNearestPackageData now has a similar cache mechanism with idToPkgMap cache which is used when resolving browser field mapping.
This PR removes idToPkgMap to make the logic simpler.

I tested this PR with #6030 and the performance wasn't degraded.

Additional context

---

What is the purpose of this pull request?

• Bug fix
• New Feature
• Documentation update
• Other

Before submitting the PR, please make sure you do the following

• Read the Contributing Guidelines.
• Read the Pull Request Guidelines and follow the PR Title Convention.
• Check that there isn't already a PR that solves the problem the same way to avoid creating a duplicate.
• Provide a description in this PR that addresses what the PR is solving, or reference the issue that it solves (e.g. fixes #123).
• Ideally, include relevant tests that fail without this PR but pass with it.

[stackblitz]

Run & review this pull request in StackBlitz Codeflow.

[patak-dev]

Awesome!

One thing we should review later is how these packages are used. We were discussing with @bluwy that the pkg.hasSideEffects call in the resolve plugin looks off. They are using the package of the importer and I think they need to be using the package of the resolved id.