Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: encoding/gob: RegisteredTypes method #71602

Open
laskoviymishka opened this issue Feb 7, 2025 · 8 comments · May be fixed by #71605
Open

proposal: encoding/gob: RegisteredTypes method #71602

laskoviymishka opened this issue Feb 7, 2025 · 8 comments · May be fixed by #71605
Labels
LibraryProposal Issues describing a requested change to the Go standard library or x/ libraries, but not to a tool Proposal
Milestone

Comments

@laskoviymishka
Copy link

laskoviymishka commented Feb 7, 2025

Proposal Details

Description:

Starting from Go 1.23, direct access to encoding/gob.nameToConcreteType is no longer possible due to the removal of internal symbol linking. While this change improves encapsulation, it also removes the ability to retrieve a list of all registered types in the gob package.

I propose adding a new function:

func RegisteredTypes() iter.Seq2[string, reflect.Type]

Use Case

Many applications rely on reflection or introspection to determine which types have been registered with gob.Register. Without access to this list, debugging serialization issues or dynamically inspecting registered types becomes more difficult.

For example here is how we track that gob.Registry is stable (since we rely on it to communicate between worker and agent). Agent and worker may be a separate binaries, so if agent has some missed type in registry - it will fail to start.

var nameToConcreteType sync.Map // map[string]reflect.Type

func RegisteredTypes() iter.Seq2[string, reflect.Type] {
	return func(yield func(string, reflect.Type) bool) {
		nameToConcreteType.Range(func(k, v any) bool {
			return yield(k.(string), v.(reflect.Type))
		})
	}
}

Instead of this, there would be a dedicated gob.RegisteredTypes with all binary known types.

Proposed Implementation

Internally, gob.Register already maintains a map of registered types. The new RegisteredTypes function would simply iterate over this map and return a list of registered types with their names.

Alternative Workarounds

Currently, developers need to maintain their own registry manually, which adds extra level of abstraction on top of gob.Register with unnecessary complexity and risks inconsistency.

Would the Go team consider adding this function to restore this functionality in an officially supported way?

@gopherbot gopherbot added this to the Proposal milestone Feb 7, 2025
@gabyhelp
Copy link

gabyhelp commented Feb 7, 2025

Related Documentation

(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)

@gabyhelp gabyhelp added the LibraryProposal Issues describing a requested change to the Go standard library or x/ libraries, but not to a tool label Feb 7, 2025
@laskoviymishka laskoviymishka changed the title proposal: encoding/gob: RegisteredTypes method proposal: encoding/gob: RegisteredNames method Feb 7, 2025
laskoviymishka added a commit to laskoviymishka/go that referenced this issue Feb 7, 2025
Starting from Go 1.23, direct access to `encoding/gob.nameToConcreteType` is no longer possible due to the removal of internal symbol linking. While this change improves encapsulation, it also removes the ability to retrieve a list of all registered types in the gob package.

Relates: golang#67401

Closes: golang#71602
@laskoviymishka laskoviymishka linked a pull request Feb 7, 2025 that will close this issue
@gopherbot
Copy link
Contributor

Change https://go.dev/cl/647536 mentions this issue: encoding/gob: add RegisteredNames()

@mateusz834
Copy link
Member

Maybe it should return an iter.Seq instead?

@laskoviymishka
Copy link
Author

Sounds good, I think it would be even better with iter.Seq

@seankhliao
Copy link
Member

can you point to existing open source code that would make use of this function instead of the alternate approaches (wrapping register, or debug builds using linkname)

laskoviymishka added a commit to laskoviymishka/go that referenced this issue Feb 7, 2025
Starting from Go 1.23, direct access to `encoding/gob.nameToConcreteType` is no longer possible due to the removal of internal symbol linking. While this change improves encapsulation, it also removes the ability to retrieve a list of all registered types in the gob package.

Relates: golang#67401

Closes: golang#71602

[]string -> iter.Seq2[string, reflect.Type]
@laskoviymishka
Copy link
Author

laskoviymishka commented Feb 7, 2025

alternate approaches

Per what I understand - you can now only use a wrapped registry for this case.

can you point to existing open-source code

We use this inside our private repo-s to ensure consistency between worker and agent nodes, this is exposed as test and as cli-command for hand-on debugging on a machine. I don't know any open-source project that use this.

@laskoviymishka laskoviymishka changed the title proposal: encoding/gob: RegisteredNames method proposal: encoding/gob: RegisteredTypes method Feb 7, 2025
laskoviymishka added a commit to laskoviymishka/go that referenced this issue Feb 7, 2025
…ype` is no longer possible due to the removal of internal symbol linking. While this change improves encapsulation, it also removes the ability to retrieve a list of all registered types in the gob package.

Relates: golang#67401

Closes: golang#71602
@ianlancetaylor ianlancetaylor moved this to Incoming in Proposals Feb 8, 2025
@mvdan
Copy link
Member

mvdan commented Feb 9, 2025

Please update this proposal with the latest API that you implemented in the patch; the proposal here is for func RegisteredNames() []string, but the latest patch is for func RegisteredTypes() iter.Seq2[string, reflect.Type].

This seems reasonable enough to me; I expect use cases to be relatively rare, but I also don't see the danger in exposing such an API, and I agree that it's rather difficult to do this yourself - particularly as other packages or existing code may be registering types with gob directly.

@laskoviymishka
Copy link
Author

@mvdan done, re-worded proposal to func RegisteredTypes() iter.Seq2[string, reflect.Type]

laskoviymishka added a commit to laskoviymishka/go that referenced this issue Feb 11, 2025
Starting from Go 1.23, direct access to
`encoding/gob.nameToConcreteType` is no longer possible
due to the removal of internal symbol linking.
While this change improves encapsulation, it also
removes the ability to retrieve a list of all registered types
in the gob package.

Relates: golang#67401

Closes: golang#71602
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
LibraryProposal Issues describing a requested change to the Go standard library or x/ libraries, but not to a tool Proposal
Projects
Status: Incoming
Development

Successfully merging a pull request may close this issue.

6 participants