fix: Restrict static & runtime top-level imports

[dangotbanned]

Fixes: #3478 (comment)
Supersedes: #2927

Description

This PR is primarily fixing pylance autocompletion that has regressed since #3431

However, this also required some runtime changes, which have been spread across commits to more clearly document what is/isn't accessible and where.

• The main fix comes from 2864c07
• I'd recommend reviewing the commits sequentially, as a lot of what comes after is organizing/re-writing imports.

---

Autocomplete

Before

After

• No _T aliases
• No* stdlib (e.g. alt.annotations)
  • Any will be removed following feat: Reimplement expr as a class that is understood by IDEs #3466
  • All others no longer appear for me in pylance

__all__ changes

Every instance of from module_name import * has either:

1. module_name declares an __all__, which is what gets imported
2. Every import * can be followed until reaching a defined __all__

2024-07-17.10-45-47.-.Wildcard.follow.mp4

---

Extra

An added bonus seems to be @deprecated is being recognised in tests/**
Note: deprecated functions still appear, but the visual cue & warning will be displayed

[binste]

Thanks for taking care of this! I've added one question as a comment. Also, in #3454 (comment) you say that this PR is an opportunity to discuss the boundaries of what we want to consider as breaking. Are you aware of anything in this PR that could be considered a breaking change? Probably the removal of some entries from the top-level Altair namespace? I'm wondering if we want to re-add them and only fix the pylance related autocompletion in this PR. Cleaning up the top-level namespace could be a bigger initiative to tackle for v6. thoughts?

[dangotbanned]

Thanks for taking care of this! I've added one question as a comment.

Thank you @binste for reviewing!

Also, in #3454 (comment) you say that this PR is an opportunity to discuss the boundaries of what we want to consider as breaking. Are you aware of anything in this PR that could be considered a breaking change? Probably the removal of some entries from the top-level Altair namespace? I'm wondering if we want to re-add them and only fix the pylance related autocompletion in this PR. Cleaning up the top-level namespace could be a bigger initiative to tackle for v6. thoughts?

"Breaking" Changes (alt._ only)

The following are not accessible at the top-level:

• Mixin classes
  • channels.DatumChannelMixin
  • channels.FieldChannelMixin
  • channels.ValueChannelMixin
• Module whose members are already accessible at top-level
  • schema.mixins
• Internal type alias
  • api.DataType
• Internal decorator
  • schemapi.with_property_setters
• Internal constant (used in private function)
  • api.TOPLEVEL_ONLY_KEYS

---

To elaborate on #3454 (comment), this is an effort to define what we consider public API.

I was hoping that by making some opinionated changes, that it would be easier for any maintainer to weigh in with either:

• a correction (identification of public API)
• an approval (identification of private API)

So for something like 973dfc7 (#3482) I hoped to open a discussion on

• Do we expect users to rely on these directly?
• What is the benefit of exposing such symbols at the top-level?
• Were these exposed intentionally?

---

Does that help paint a clearer picture of what I was going for?

[binste]

This helps, thanks! May I propose that this PR is changed to only fix the regression in the pylance autocompletion without removing elements from the top-level namespace. I think it's great that you push the discussion forward of what we want to consider the "public" part of the API and what guarantees we offer users. I see a lot of value in defining this.

I'd prefer if we look at defining the public API in one go holistically by going through all elements in a systematic way. By doing it in one go, we can hopefully reduce frictions for end users and provide clear indication of the changes, migration guides, and a guarantee of what is public going forward. It will involve tradeoffs between making Altair better for future users while not breaking too many things for existing codebases and users. Discussing this topic with an overview of all the parts of the package hopefully allows us to get this right.

Thanks again for continue to push Altair in a better direction! Much appreciated :)

[dangotbanned]

Thanks again for continue to push Altair in a better direction! Much appreciated :)

Thank you @binste

This helps, thanks! May I propose that this PR is changed to only fix the regression in the pylance autocompletion without removing elements from the top-level namespace.

That is absolutely fine with me!
I'll work on reverting those changes and simply link to my suggestions in the Motivation section of #3454 for future reference

[dangotbanned]

@binste altair.__all__ is now identical to main 👍

[binste]

That is absolutely fine with me! I'll work on reverting those changes and simply link to my suggestions in the Motivation section of #3454 for future reference

That's perfect, thank you! :)