plugin: Add support for closing plugins #20461

zacps · 2017-05-22T22:54:14Z

Adding support for closing plugins would give a simple way to do hot code reloading in long running applications like web servers.

This would add another method to src/plugin/plugin.go and a new file src/plugin/dl_close.go that would handle closing the dlhandle and destroying the go symbol map.
It would also add the handle as an un-exported unsafe pointer to the go plugin struct.

The main issue is how to signal that a plugin has been closed without breaking API compatibility. Currently the plugin has a channel called loaded which is closed when the plugin is loaded. The simplest solution is to add another channel which is closed when the plugin is closed, however this doesn't seem very 'neat' to me.

The text was updated successfully, but these errors were encountered:

davecheney · 2017-05-22T22:59:21Z

I don't see how this could be implemented safely. Other Go code could have references to functions and symbols loaded from the plugin-would unloading the plugin make those dangling references, or delay the unloading of a plugin until all the references have been reclaimed? Neither sound workable.

zacps · 2017-05-22T23:11:53Z

Would it be possible to make close fail until the reference count of all symbols in the plugin reached 1, so that the plugin itself was the only structure that held a reference?

Alternatively you could have a different function which checked whether references to the symbols existed and if there were none close the dlhandle, leaving the destruction of the plugin object and any other references up to the program. While this is far from ideal I think with adequate caution it could be a workable solution.

davecheney · 2017-05-22T23:14:12Z

Would it be possible to make close fail until the reference count of all symbols in the plugin reached 1, so that the plugin itself was the only structure that held a reference?

This is spinny and racy.

Alternatively you could have a different function which checked whether references to the symbols existed and if there were none close the dlhandle, leaving the destruction of the plugin object and any other references up to the program. While this is far from ideal I think with adequate caution it could be a workable solution.

This sounds like a finaliser, with the associated problems of finaliser non determinism.

zacps · 2017-05-22T23:19:09Z

Could you be a bit more specific about the second point?

davecheney · 2017-05-22T23:24:33Z

Your proposal is to add a way to free resources associated with a plugin. If this is delegated to a finaliser it becomes non deterministic when this freeing occurs as finalisers are not guaranteed to run promptly, or indeed, at all.

zacps · 2017-05-22T23:34:19Z

I'm not suggesting that it's run automatically by the GC, or in a defer (unfortunately).

I'm suggesting that if the needs to be unloaded that the developer will keep track of references to it's symbols then when they no longer need them destroy the references then explicitly call close.

In that situation they will either forget to call close, leaving the handle open, close successfully or attempt to close when there are still references which will fail. It could possibly fail with a panic rather than a plain error as attempting the close a resource that still has a reference is a pretty big bug.

I don't see what issues this approach has beyond reliance on the developer to not make mistakes. It does introduce a need to keep track of references which is far from perfect and not something I'd like to introduce to go. However I think that it's worth having this for those that want to use it and understand the additional complexity it introduces.

ianlancetaylor · 2017-05-22T23:48:17Z

What if the plugin started a goroutine?

Go as a language and a runtime environment tends to avoid providing facilities that are easy to get wrong when getting them wrong causes the program to crash or behave unpredictably. I don't see how to make this at all reliable, so to me it doesn't seem to be a good fit for Go.

zacps · 2017-05-22T23:54:44Z

goroutines would have to be handled in the same way, requiring the developer to close them.

I can understand not wanting to introduce functionality that's easy to misuse. I just think that it's better to have the ability to do it if you know what you're doing. I guess it does go against the philosophy of go though.

davecheney · 2017-05-23T01:27:32Z

How can you close a goroutine? The runtime doesn't give you any way to get a handle or an identifier for a goroutine?

…

On Tue, 23 May 2017, 09:54 Zac Pullar-Strecker ***@***.***> wrote: goroutines would have to be handled in the same way, requiring the developer to close them. I can understand not wanting to introduce functionality that's easy to misuse I just think that it's better to have the ability to do it if you know what you're doing. I guess it does go against the philosophy of go though. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#20461 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAAcA7TSJ5h7YYZU1PZDXlrNZfjSaaifks5r8iBGgaJpZM4Ni-0b> .

zacps · 2017-05-23T02:16:34Z

I took a quick look at runtime but couldn't find where exactly the goroutine is created. Are goroutines not closed when they return?

davecheney · 2017-05-23T02:20:41Z

Yup, a goroutine dies/quits/goes away when the function that was passed to the go statement returns. So that leaves the problem of ensuring every goroutine that may have access to a symbol from a plugin must exit before the plugin can be unloaded. This means either finalisers, which won't give you a timely response, or manual bookkeeping, which as Ian said is unreliable.

…

On Tue, May 23, 2017 at 12:16 PM, Zac Pullar-Strecker < ***@***.***> wrote: I took a quick look at runtime but couldn't find where exactly the goroutine is created. Are goroutines not closed when they return? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#20461 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAAcA-7gIyQn1lrDQJPC5XHyF1Oe4u_Mks5r8kGDgaJpZM4Ni-0b> .

zacps · 2017-05-23T02:34:15Z

Ok, that's what I thought.

I personally think that manual bookkeeping wouldn't be too bad. If you're writing something designed to be loaded/unloaded you're probably going to make sure you keep track of references and goroutines as is.

I'm trying to think whether there's a better way to do this. I guess it's not something other languages have had problems with.

zacps · 2017-05-24T22:28:28Z

I have another idea for how this could be solved, it's not simple though and I'm not sure whether it would make sense.

Basically I think it would be possible to implement a compile-time check that ensured there was no possibility for a dangling reference when a program was closed.

The compiler could check to see if close was called on any plugin object in the program. If it was we'd find all of the lookup calls associated with that plugin and determine whether there were any paths that could leave a live reference when or after closed was called.

I have no idea how you'd implement this or whether it'd be fast enough to be viable. Just an idea.

DemiMarie · 2017-06-04T01:45:08Z

What about having the GC close any plugins that are no longer referenced?

That's what the JVM does. And forcing a full GC whenever Go code must be unloaded is not hard.

davecheney · 2017-06-04T01:47:27Z

That's a finaliser.

…

On Sun, 4 Jun 2017, 11:45 Demi Marie Obenour ***@***.***> wrote: What about having the GC close any plugins that are no longer referenced? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#20461 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAAcAyloFURN5wCFLtkaTGCb3LPug7v7ks5sAgwmgaJpZM4Ni-0b> .

zacps · 2017-06-04T03:48:30Z

@davecheney Any comment on making it a compile-time check?

davecheney · 2017-06-04T03:59:03Z

It feels like somewhere between rusts ownership system and a finaliser.

…

On Sun, 4 Jun 2017, 13:48 Zac Pullar-Strecker ***@***.***> wrote: @davecheney <https://github.com/davecheney> Any comment on making it a compile-time check? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#20461 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAAcA5tJxnFTW952qctOd7bRsZVmylDHks5sAikQgaJpZM4Ni-0b> .

zacps · 2017-06-04T04:09:01Z

I don't see how it's like a finaliser, at least in the aspect of non-determinism.

It is pretty similar to rust's general approach; I don't think that's a bad thing though.

tarndt · 2017-06-05T22:56:38Z

If I load a plugin foobar-v1.so that has foobar() I could wrap it in my own functionfoobarWapper() that increments/decrements a waitgroup on every call. Things that want to call foobar do so accessing foobarWraper atomically and calling it. Later on I load foobar-v2.so, wrap it in a new wrapper, and atomically swap it out for out for the old foobarWrapper. Now I can use the old foobarWraper's wait group to make sure no one is active and when that is the case call dlclose using cgo.

This all would depend on the user not creating goroutines, but that could be checked for at the source level if you are building plugins from user submitted code.

@zacps Maybe something like the above would work for you.

zacps · 2017-06-09T04:39:30Z

Wouldn't you have to either use code generation or lose type safety though?

tarndt · 2017-06-10T16:16:03Z

Wouldn't you have to either use code generation or lose type safety though?

@zacps I'm not sure how type safety is related to this...

This all would depend on the user not creating goroutines, but that could be checked for at the source level if you are building plugins from user submitted code.

So using my approach you would write a program to vet the user code before building the plugin. Probably by having a white-list of importable packages, and searching the code to get for invocations of "go X" and any functions in white listed packages you want to also blacklist.

DemiMarie · 2017-06-10T19:34:49Z

I know it's a finalizer. It is also what the JVM does with JNI code, and the only safe solution I know of without Rust-esque lifetime tracking, generating proxies for each plugin function, or requiring reflection for every invocation.

…

On Sat, Jun 3, 2017, 9:47 PM Dave Cheney ***@***.***> wrote: That's a finaliser. On Sun, 4 Jun 2017, 11:45 Demi Marie Obenour ***@***.***> wrote: > What about having the GC close any plugins that are no longer referenced? > > — > You are receiving this because you commented. > > > Reply to this email directly, view it on GitHub > <#20461 (comment)>, or mute > the thread > < https://github.com/notifications/unsubscribe-auth/AAAcAyloFURN5wCFLtkaTGCb3LPug7v7ks5sAgwmgaJpZM4Ni-0b > > . > — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#20461 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AGGWBxUSgkhkS3oeftTUu49MXgnWoeFfks5sAgzNgaJpZM4Ni-0b> .

jaekwon · 2017-12-16T21:32:37Z

I second starting with an unsafe way to load and unload plugins, with the expectation that the user will sanitize code before compiling it. It's OK if attempting to unload a plugin panics when there exist dangling pointers/references, and it's also OK if the panic happens later nondeterministically when the dangling pointers/references are accessed.

Azareal · 2017-12-16T21:54:10Z

I currently use code generation for templates. I have a template transpiler which transpiles text/template templates to Pure Go (it's a little coupled with my application right now, but I could always split it off and it currently misses a few features).

It would be nice, if I could reload these templates on the fly without restarting the entire instance, something impossible without the ability to close plugins or rolling my own JIT compiler.

DemiMarie · 2017-12-18T01:52:49Z

Here is a thought: what about having plugins export their API as a first-class object, such that the plugin gets unloaded when the API object is garbage collected?

davecheney · 2017-12-18T01:58:15Z

@DemiMarie this approach is problematic, please see my comments about finalisers earlier in this thread.

pjebs · 2020-03-09T21:52:08Z

There are pure go wasm runtimes

samplesizeofone · 2020-03-09T22:09:12Z

Seems unfortunate not to have a nice embedded no third party stuff solution. Possibly a minor nitpick.

…

On Mon, Mar 9, 2020 at 2:52 PM pj ***@***.***> wrote: There are pure go wasm runtimes — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#20461?email_source=notifications&email_token=AADYSWNEKLTQLKWMKJNZRHTRGVQIZA5CNFSM4DML5UN2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEOJGO4Y#issuecomment-596797299>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AADYSWOIZBSRBFUHXBYDWI3RGVQIZANCNFSM4DML5UNQ> .

porfirion · 2020-05-14T13:33:20Z

Oberon's implementation BlackBox has ability to unload modules and able to hotswap them. When you ask runtime to unload module - it marks it as unreachable and when there are no pointers to it's memory - it is removed. If someone asks for types from unloaded module - runtime tries lo load it again, but it can be a newer version. So it is possible to have multiple instances of module at the same time. When all links to old one are removed - there will be only one!
I think it would be killer feature for web development or long running applications.

wizardishungry · 2021-05-04T21:28:03Z

In case anyone else had the brilliant idea to do something quick and dirty to support loading new versions of plugins, I tried:

Building the .so file with a different package path for each reload.
Copying the .so file to a random file on /tmp (tmpfs/memory filesystem on my machine)
Wrapping plugin.Open with some cgo to cache the handle returned by dlopen
Implementing a Close function that calls dlclose with the cached value of the shared library handle.
Removing the scratch .so file in /tmp

It didn't work for me. ~~The disk usage on the /tmp filesystem seemed to continue growing with each reload operation.~~ It panics 🤷‍♂️. I may be doing something wrong, of course.

// +build linux,cgo darwin,cgo freebsd,cgo

package unsafeplugin

/*
#cgo linux LDFLAGS: -ldl
#include <dlfcn.h>
#include <limits.h>
#include <stdlib.h>
#include <stdint.h>

#include <stdio.h>

static uintptr_t pluginOpen(const char* path, char** err) {
	void* h = dlopen(path, RTLD_NOW|RTLD_GLOBAL);
	if (h == NULL) {
		*err = (char*)dlerror();
	}
	return (uintptr_t)h;
}

static int pluginClose(uintptr_t h, char** err) {
	int i = dlclose((void*)h);
	if (i != 0) {
		*err = (char*)dlerror();
	}
}

static void* pluginLookup(uintptr_t h, const char* name, char** err) {
	void* r = dlsym((void*)h, name);
	if (r == NULL) {
		*err = (char*)dlerror();
	}
	return r;
}
*/
import "C"

import (
	"errors"
	"fmt"
	"path/filepath"
	"plugin"
	"sync"
	"unsafe"
)

var (
	mutex              sync.Mutex
	pathToDlopenHandle = make(map[*plugin.Plugin]uint64)
)

func Open(path string) (*plugin.Plugin, error) {
	mutex.Lock()
	defer mutex.Unlock()

	realPath, err := filepath.Abs(path)
	if err != nil {
		return nil, err
	}

	p, err := plugin.Open(realPath)
	if err != nil {
		return nil, err
	}

	cPath := C.CString(realPath)
	defer C.free(unsafe.Pointer(cPath))

	var cErr *C.char
	h := C.pluginOpen(cPath, &cErr)
	if h == 0 {
		return nil, errors.New(`unsafeplugin.Open("` + path + `"): ` + C.GoString(cErr))
	}

	fmt.Println("path", realPath, "h", h)

	pathToDlopenHandle[p] = uint64(h)

	return p, nil
}

func Close(p *plugin.Plugin) error {
	mutex.Lock()
	defer mutex.Unlock()
	h, ok := pathToDlopenHandle[p]
	if !ok {
		return errors.New(`unsafeplugin.Close()`)
	}

	var cErr *C.char

	// man dlclose:
	// > The dynamic linker maintains reference counts for  object  handles,
	// > so a dynamically loaded shared object is not deallocated until dlclose() has been
	// > called on it as many times as dlopen() has succeeded on  it.
	r := C.pluginClose((C.ulong)(h), &cErr)

	if r != 0 {
		return errors.New(`unsafeplugin.Close(): ` + C.GoString(cErr))
	}

	r = C.pluginClose((C.ulong)(h), &cErr)

	if r != 0 {
		return errors.New(`unsafeplugin.Close(): ` + C.GoString(cErr))
	}

	return nil
}

edwingeng · 2021-07-10T16:32:36Z

give a simple way to do hot code reloading...

I developed a solution for that. It is now open source. https://github.com/edwingeng/hotswap

Hotswap provides a solution for reloading your go code without restarting your server, interrupting or blocking any ongoing procedure. Hotswap is built upon the plugin mechanism.

Major Features:

Reload your code like a breeze
Run different versions of a plugin in complete isolation
Invoke an in-plugin function from its host program with Plugin.InvokeFunc()
Expose in-plugin data and functions with PluginManager.Vault.DataBag and/or PluginManager.Vault.Extension
Handle asynchronous jobs using the latest code with live function, live type, and live data
Link plugins statically for easy debugging
Expose functions to other plugins with Export()
Depend on other plugins with Import()

ChenJhua · 2021-12-04T03:03:09Z

When do you have plans to optimize the plugin? For example, the close function, the plugin so size

kevin-matthew · 2022-02-19T03:32:21Z

This thread is just very valid point after another. There should be no reason why plugins can be closed to hot-swap files. Especially comments #20461 (comment) and #20461 (comment).

In fact, I didn't even know golang lacked this functionality because I incorrectly assumed how fundamentally easy it would be to implement.

I just spent an entire day implementing a hot-swap mechanic for a web server that manages about 90 different plugins from hundreds of different developers. All plugins follow a shared spec that I previously helped write, ie "your plugin must close all goroutines when symbol XYZ is called" was included so I can implement this hot swap feature at a later date. Come to find out, at the end of the day that plugin.Open will cache results based on the file and per this thread, circumventing that results in a panic. So out of the 18 hours that I've just completely wasted, and an awkward phone call I have to make to the people I promised this feature too, please consider putting this function in.

With that being said, give me at least an unsafe way to do it. Not having this feature for the sake of "it'll be misused by those who don't know what they're doing" is a very-mega-big downer for those who do know what they're doing.

AlbinoGeek · 2022-03-23T11:26:20Z

I just spent an entire day implementing a hot-swap mechanic for a web server that manages about 90 different plugins from hundreds of different developers. All plugins follow a shared spec that I previously helped write, ie "your plugin must close all goroutines when symbol XYZ is called" was included so I can implement this hot swap feature at a later date. Come to find out, at the end of the day that plugin.Open will cache results based on the file and per this thread, circumventing that results in a panic. So out of the 18 hours that I've just completely wasted, and an awkward phone call I have to make to the people I promised this feature too, please consider putting this function in.

Workaround: Swap to new filenames (Yup....)

Sure memory usage climbs, but there's no other option... and it's been 5 years.

paralin · 2022-03-23T12:46:03Z

Adding ControllerBus to the discussion of plugin mechanisms / structures / proof-of-concept:

https://github.com/aperturerobotics/controllerbus/tree/master/plugin

This would greatly benefit from the plugin unloading improvements mentioned above; however, I've started working on strategies using IPC messaging to cross the Process barrier rather than sharing memory in the same process. This is a workaround for the lack of strong Plugin features.

helloLiRong · 2023-04-17T09:19:08Z

it's been a long time. is there any solution for it now?

zacps · 2023-04-17T09:37:41Z

In the time since I opened this issue I finished high school, completed a Bachelors degree in computer science & physics, and published a couple of scientific articles.

I also learnt Rust (thanks for the recommendation #20461 (comment)).

ianlancetaylor · 2023-04-18T00:28:16Z

There is no change here.

kolinfluence · 2023-06-28T01:48:47Z

looking at this too. any updates?
this is not working for golang 1.21 but for reference
https://github.com/pkujhd/godynamic

anyone taking on this issue? unloading plugin and removing memory used will be great

ianlancetaylor · 2023-06-28T02:07:41Z

Nobody is working on this issue and we have no idea how to implement it. Sorry.

DemiMarie · 2023-06-29T17:14:06Z

@ianlancetaylor From my PoV there are four options:

Use GC and finalizers to release a plugin when there are no more references to it. This is safe, but not deterministic.
Declare using a closed plugin to be undefined behavior (in the C sense): if you use a plugin that has been unloaded, anything at all can happen, including executing malicious code. This is deterministic, but not safe.
Wrap each plugin function in a trampoline that checks if the plugin is still alive and panics if not. This is memory safe and deterministic, but still risks crashing the program.
Implement something higher-level (such as hot code reloading) directly in the runtime. This would use one of the above options internally but not expose it.

Furthermore, I believe these are the only possible options for a language like Go that cannot encode lifetimes in the type system and does not have destructors. Is this accurate?

ianlancetaylor · 2023-06-30T02:11:35Z

@DemiMarie Thanks. However, I'm not sure that gets at the reasons that people care about this. What do people actually want to achieve by closing a plugin? If all that they want is for future calls to the plugin to fail, then I agree that option 3 suffices. But I suspect that they want something more: I suspect that they want to release any resources that the plugin is using. And that seems much harder.

mehrvarz · 2023-06-30T06:43:23Z

I'm not sure that gets at the reasons that people care about this. What do people actually want to achieve by closing a plugin?

They want to be able to load a new/modified revision of the (same) plugin.

Niko-Kk · 2023-06-30T06:50:46Z

I'm not sure that gets at the reasons that people care about this. What do people actually want to achieve by closing a plugin?

They want to be able to load a new/modified revision of the (same) plugin.

This has been exactly what I wanted for a long time, as well as releasing resources, but over time I realized that it wasn't necessary.

For what it's worth, I've managed to solve this in a slightly hacky method. Go is capable of running other executables. In this way, plugins are now run on a sort of plugin service, for which can be spooled up or down by the primary go executable. It's not great for intense data transfer, because I opted in for all plugin calls to be callable through a REST API, but it solved it in my case. Hopefully it can for some other developers wanting a similar scenario.

paralin · 2023-06-30T07:18:33Z

@Niko-Kk it's necessary as you said for performance.

Lots of projects use RPC with subprocesses for plugins, like hashicorp go-plugin. I wrote a RPC library for this purpose: https://github.com/aperturerobotics/starpc which also supports typescript. The approach works fine but having the plugins in-process definitely would be better for performance.

DemiMarie · 2023-07-01T01:40:16Z

@DemiMarie Thanks. However, I'm not sure that gets at the reasons that people care about this. What do people actually want to achieve by closing a plugin? If all that they want is for future calls to the plugin to fail, then I agree that option 3 suffices. But I suspect that they want something more: I suspect that they want to release any resources that the plugin is using. And that seems much harder.

It doesn’t seem much harder to me, provided that the plugin code is written correctly. First, the runtime will check if there are any goroutines executing the plugin’s code. If there are, the plugin can’t be closed. Otherwise, all of the plugin’s entry points will be made to panic, and a Close() function in the plugin will be called to release non-GC’d resources. Once that function returns, that the plugin code and data will never be used again, the runtime can safely unregister the plugin’s GC roots and unmap the plugin’s code and data, after which the plugin’s global variables are free to be garbage collected.

If this is too complex, a simpler approach would to for the application to drop all references to the plugin, after which the plugin will be unreachable and can be garbage collected. This is the approach used by Java, which will garbage collect classes and class loaders that are no longer reachable. To me, this is the solution that fits the Go langauge best, provided that there is a way (runtime logs?) to debug situations where something is keeping a plugin alive.

ianlancetaylor · 2023-07-01T02:09:14Z

First, the runtime will check if there are any goroutines executing the plugin’s code.

I'm not sure that's enough. There might be some variable out there somewhere that is an interface value that has methods that point to the plugin's code. That means that the plugin code might be invoked at some point in the future.

The garbage collection approach is more plausible, but it's not a simple change. Normally we can treat any global variable as a garbage collection root, but a global variable in a plugin would not be a root. We would have to develop a new kind of reference, by which a reference to plugin code would keep plugin global variables alive. That said, one nice feature of the garbage collection approach is that we wouldn't need to have a way to close plugins.

sh0umik · 2023-10-12T08:14:02Z

From what I've read, the desire to close plugins comes primarily from developers building a fast code reloading system. For example, I have a Go program that...

Compiles file_x.go into a plugin, loads it, and calls a function loaded from the plugin.

Watches file_x.go for changes

When the file changes, rebuild the plugin, reload it, and call the function again.

This has two issues:

Loading a plugin consumes memory which cannot be freed, so overall memory usage increases with each change to file_x.go

Plugins have identity, and it's not possible to load a plugin twice. To workaround this, it's possible to give each version of the plugin a unique identity. Instead of reloading a plugin, you load a new version with a unique identity. This feels hacky.

Regarding issue 1: perhaps the garbage collector, not the program, should be responsible for releasing the memory associated with a plugin. #28783 seems related.

It's worth calling out that if the garbage collector is responsible for freeing plugin resources, then there's no way for a program to know when those resources have been freed, and that's a good thing because it removes complexity and matches the overall design of memory management in Go.

Regarding issue 2: perhaps there's a better way to manage the identity of plugins that have many, rapid iterations. Or, maybe the existing hack is enough. Seems like we should define this problem clearly and explore solutions before we decide that implementing Plugin.Close is the best solution for that particular issue.

I'd like to note: I think this is an important issue. I don't consider fast code reloading to be an obscure edge case. There are many projects trying to build Go interpreters and REPLs and test runners and other interesting ways to iterate quickly on Go code. "Building a better Go linker" alludes to an interesting future where Go can compile, link, and load code directly into memory; that would open some interesting doors. Many projects would benefit significantly by having a way to collect unused plugin resources.

Example reloading code: https://github.com/buchanae/go-code-reload

I am having a similar problem while reloading the plugin using some solution like "issue 2" but still reloads the previous states. I guess we have to wait then ...

DemiMarie · 2023-10-14T20:17:35Z

First, the runtime will check if there are any goroutines executing the plugin’s code.

I'm not sure that's enough. There might be some variable out there somewhere that is an interface value that has methods that point to the plugin's code. That means that the plugin code might be invoked at some point in the future.

The garbage collection approach is more plausible, but it's not a simple change. Normally we can treat any global variable as a garbage collection root, but a global variable in a plugin would not be a root. We would have to develop a new kind of reference, by which a reference to plugin code would keep plugin global variables alive. That said, one nice feature of the garbage collection approach is that we wouldn't need to have a way to close plugins.

I suspect that in practice, anyone using this approach will write a helper function that drops all references to the plugin (by setting various pointers to nil) and then forces a GC.

sh0umik · 2023-10-17T10:38:55Z

Azareal

Did you found any solution since ?

Whole day of my life wasted: golang/go#20461. Fanfuckingtastic. Fuck you go.

bradfitz added this to the Unplanned milestone May 22, 2017

crawshaw mentioned this issue Sep 15, 2017

proposal: add a domain concept to make plugins unloadable. #21893

Closed

jaekwon mentioned this issue Dec 16, 2017

Proposal: Make Go2 better implement the Object-capability model for security. #23157

Closed

AlexRouSg mentioned this issue Apr 16, 2018

Feature request: plugin close #24880

Closed

pikeszfish mentioned this issue Jun 5, 2020

Go 语言的动态库和插件系统 | Go 语言设计与实现 · /golang-plugin draveness/blog-comments#196

Closed

FeldrinH mentioned this issue May 14, 2022

Add Windows Support livebud/bud#7

Closed

gopherbot added the compiler/runtime Issues related to the Go compiler and/or runtime. label Jul 7, 2022

randall77 mentioned this issue Nov 8, 2023

unique: new package with unique.Handle #62483

Closed

kevin-matthew added a commit to kevin-matthew/vorlage that referenced this issue Jan 29, 2024

Tried to implement go-auto-reloading.

60aff86

Whole day of my life wasted: golang/go#20461. Fanfuckingtastic. Fuck you go.

seankhliao added the FeatureRequest label Jul 13, 2024

plugin: Add support for closing plugins #20461

plugin: Add support for closing plugins #20461

Comments

zacps commented May 22, 2017

davecheney commented May 22, 2017

zacps commented May 22, 2017 • edited Loading

davecheney commented May 22, 2017

zacps commented May 22, 2017

davecheney commented May 22, 2017 • edited Loading

zacps commented May 22, 2017

ianlancetaylor commented May 22, 2017

zacps commented May 22, 2017 • edited Loading

davecheney commented May 23, 2017 via email

zacps commented May 23, 2017

davecheney commented May 23, 2017 via email

zacps commented May 23, 2017

zacps commented May 24, 2017 • edited Loading

DemiMarie commented Jun 4, 2017 • edited Loading

davecheney commented Jun 4, 2017 via email

zacps commented Jun 4, 2017

davecheney commented Jun 4, 2017 via email

zacps commented Jun 4, 2017

tarndt commented Jun 5, 2017

zacps commented Jun 9, 2017

tarndt commented Jun 10, 2017 • edited Loading

DemiMarie commented Jun 10, 2017 via email

jaekwon commented Dec 16, 2017 • edited Loading

Azareal commented Dec 16, 2017 • edited Loading

DemiMarie commented Dec 18, 2017

davecheney commented Dec 18, 2017

pjebs commented Mar 9, 2020

samplesizeofone commented Mar 9, 2020 via email

porfirion commented May 14, 2020

wizardishungry commented May 4, 2021 • edited Loading

edwingeng commented Jul 10, 2021 • edited Loading

ChenJhua commented Dec 4, 2021

kevin-matthew commented Feb 19, 2022

AlbinoGeek commented Mar 23, 2022 • edited Loading

paralin commented Mar 23, 2022

helloLiRong commented Apr 17, 2023

zacps commented Apr 17, 2023

ianlancetaylor commented Apr 18, 2023

kolinfluence commented Jun 28, 2023

ianlancetaylor commented Jun 28, 2023

DemiMarie commented Jun 29, 2023

ianlancetaylor commented Jun 30, 2023

mehrvarz commented Jun 30, 2023

Niko-Kk commented Jun 30, 2023

paralin commented Jun 30, 2023

DemiMarie commented Jul 1, 2023

ianlancetaylor commented Jul 1, 2023

sh0umik commented Oct 12, 2023

DemiMarie commented Oct 14, 2023

sh0umik commented Oct 17, 2023

zacps commented May 22, 2017 •

edited

Loading

davecheney commented May 22, 2017 •

edited

Loading

zacps commented May 22, 2017 •

edited

Loading

zacps commented May 24, 2017 •

edited

Loading

DemiMarie commented Jun 4, 2017 •

edited

Loading

tarndt commented Jun 10, 2017 •

edited

Loading

jaekwon commented Dec 16, 2017 •

edited

Loading

Azareal commented Dec 16, 2017 •

edited

Loading

wizardishungry commented May 4, 2021 •

edited

Loading

edwingeng commented Jul 10, 2021 •

edited

Loading

AlbinoGeek commented Mar 23, 2022 •

edited

Loading