proposal: all: add bare metal ARM support #46802
Comments
|
Hello, In the discussion of the previous proposal, there was some minimal talk about builders for GOOS=none, e.g. #37503 (comment) (which basically boils down to "Will there be a builder? Yes, there will be a builder."). I think it would be useful to have a bit more concrete/detailed conversation about builder support. To wit: suppose the proposal is accepted, and a GOOS=none builder is set up. Will this builder be capable of running "all.bash"? Will the builder provide full support for "gomote" operations? Suppose that someone from the Go team checks in a change to the compiler or linker, and a test fails on the GOOS=none builder. Who will look at that failure and how? Thanks. |
|
all.bash currently runs on all tamago-go releases, I think overall builder and linker tests can happen identically to other architectures. The challenge would only be for runtime tests that are written in a manner that depends on an OS being present, I am not sure how this is currently handled for platforms like plan9 or js. What I can say is that this can run under qemu. |
|
For JS, iOS, and Android, tests are run by having a separate host for builds and invoking the binaries using Presumably the TamaGo builder would need a similar script. |
Plan 9 can run the actual Go toolchain, albeit slowly. For JS, Android, and iOS, Test binaries that contain a mix of the two can use |
|
Understood, I see no problem in having a similar approach for TamaGo. We could provide an USB armory as separate host to receive the tests or maybe (easier) just have a script that spawns the test within qemu ? |
|
This may be of interest to @tinygo-org |
|
Why is it difficult to maintain this tree as a GitHub fork of golang/go? |
|
It is currently not that difficult, but its inclusion upstream would help ensuring that a GOOS=none support is minded in relation to potentially breaking changes, that its quality is consistent with the rest of the Go distribution code and that it would remain available over time. Secondarily it would also help promoting, and further develop, this use case among Go user base. Allowing easier hooks to generic "GOOS=none" within Go can certainly spawn up more interesting uses beyond the specific TamaGo goals and implementation, so I think in general it would help the Go ecosystem. |
|
What you are saying is that it would move the maintenance burden for this code off of your team and onto the Go contributors more generally. Saying "its inclusion upstream would help ensuring that a GOOS=none support is minded in relation to potentially breaking changes" is another way of saying "it would be yet another special case, little-used port that slows down maintenance and development of the primary ports", or perhaps of saying (without realizing it) "we don't intend to keep up with your changes and want you to do it". Contribution of new ports are "free like a puppy", to use the old Sun line: they are a gift that will require significant time and effort to keep alive. We already have a bunch of those, and inevitably the contributors move on, leaving us responsible for either continued upkeep (which takes away time we could be working on changes that benefit far more users) or removing the port (which makes us the bad guys). I would much rather figure out what is making it difficult to keep this port out of tree and fix those underlying issues. That would let you maintain the port as you see fit and decouple our development and yours. It would make clear who is responsible for ongoing maintenance and fixing issues. It would scale far better - we can't put every port into the main tree. And it would provide a model for the next niche port. |
|
I don't have a strong opinion either way, but: how does GOOS=none compare to, say, GOARCH=loong64? If the point is to have someone responsible for its maintenance in-tree, perhaps Andrea would be up for that. |
|
My team (more than once) expressed the firm intention to maintain this. Please stop saying that we want upstream inclusion to offload maintenance because this is simply not true. This proposal specifically would ease integration of TamaGo like ports in Go, so the way I see it it specifically addresses the current challenges in doing so by acting as a generic ARM template and possibly in the future for other architectures. This effort is to promote a generic framework and hooks to be accepted upstream with our contribution to maintain it. |
|
Let me be more clear and re-state my thinking. I think our patch specifically helps addressing the challenges in creating new ports of this kind and this is why we think it should be upstreamed (with our firm intention to maintain it at least for ARM). It is not the intention of our proposal to just "offload" this for our own convenience. We are excited about using Go on the bare metal, promote its use and help make it easy for all while preserving this possibility in the process. |
This is a good question. Empirically, GOARCHes require far less work to maintain than GOOSes. From Go's point of view, architectures vary far less than operating systems do: GOARCH variations essentially never get in the way of new work, while GOOS variations often do. (The variation of "no operating system" would be particularly different. In fact, the current patch assumes a single-CPU system - there is no threading or interrupts or note implementation. This is going to limit its applicability. And of course scaling up to a proper multiprocessor implementation would be significantly more work and more code to maintain.)
I appreciate the present intention to maintain the code. I don't doubt at all what you are saying about your current intentions. But suppose five years from now your priorities shift. In that case, we would be left with another port that we must either maintain ourselves or remove from the tree (as I laid out before). This is why we must be willing to take on maintenance of the port in order for it to land in the tree. The current variety of ports (again, especially GOOSes) already creates a real drag on being able to make important changes to, for example, the garbage collector and the scheduler. For ports that we cannot commit to long-term maintenance of, keeping them out-of-tree would make the responsibility much clearer. For as long as you maintain them, they would work and keep up with Go. And if at some point priorities shift and that project was no longer maintained, it would be clear what was no longer maintained. Either way, again, the responsibilities are much clearer if the port is out-of-tree. I realize that we don't have great support for out-of-tree ports right now. But again I think it is worth figuring out what we could do to make that better rather than default to continuing to add every possible new port to the main tree. Good support for out-of-tree ports would also provide a better path for ports that do eventually need to be removed from the main tree. |
I would like to see that future happen. The closest thing we have right now are first-class ports, but it's not quite the same as it still encourages all possible ports to live in a single place. |
|
On contributors leaving or losing interest...well that can be used as an argument against pretty much any contribution can't it? If any maintainers lose interest on the code they maintain the problem would be identical. If there are others that can pick the work up fine, otherwise the feature would be dropped I guess. Certainly I can see upstream inclusion of a GOOS giving it better chance of getting more manpower than keeping it external. Not sure how anyone is supposed to counteract this argument to be honest... On multiprocessor support, as the idea is for this GOOS to be a general template I am happy to adapt for easier integration of SMP CPUs, in fact this could be a great exercise on making it closer to what you ask in the first place (easy addition if external architectures in GOOS=none). On GOOS being harder than a new GOARCH, I am not sure I agree 100%. Some architectures are fairly broad in their scope while this is very a very specific and unique GOOS which actually has no dependency on a specific OS. So I think this effort, if generalized correctly, should not need too much maintenance in the long term or anyway its maintenance would be quite similar to what plan9/nacl/js are doing...in fact most of the complex bits are 100% cloned from these GOOSes. If there is interest I am happy to ponder an abstraction that allows SMP to take place as that is a valid concern. |
Not really. It can only be used as an argument against a change that the core Go team is not able to maintain. Relatively few changes fall into that category.
This isn't hypothetical, it's based on tracking the areas that take the most time for Go's compiler and runtime work over the last couple of years. I personally don't feel very strongly about this particular proposal. But I hope we can all agree that it would be better if it were easier to maintain Go ports out of the main repository. That state is clearly better than where we are today in every respect, and it also addresses this proposal. So let's try to understand how much work that would be, and whether it is a feasible goal. Thanks. |
|
I think we are confusing layers a little bit (including myself). Our proposal is a generic GOOS=none which would create a generic layer to extend the same GOOS to any bare metal SoC/architecture with external packages (like we are doing with tamago). Bare metal (a.k.a. supervisor/system mode) is different than a generic GOOS in user mode (e.g. linux, macOS) which typically relies on system calls. While I do think that our work could be somewhat be adapted to be so generic that it would also allow arbitrary user mode GOOS architectures, I probably think it would not be the best or more efficient approach due to this fundamental differences in execution mode. What I care about is promoting (and preserving) the fact that Go can be a prime language on OS-less embedded systems, and while I'd be pleased if our work could also help Go in achieving cleaner abstraction for arbitrary GOOSes, fundamentally I am not sure this patch would be the right beginning for it, as GOOS=none is likely very different than GOOS=<some_other_os_running_go_in_user_mode>. |
|
So essentially your proposal is to just add a GOOS=none, which is actually a plugin mechanism that would let people implement support for a specific architecture (like ARM) using an external package? And then if you set GOOS=none and provide an external package like the one you linked above for ARM, you would be able to enable support for none/arm as a GOOS/GOARCH couple? Because this is not very clear by reading this proposal. A few questions:
|
|
TinyGo is both an LLVM-based compiler for Go, and a partial set of replacement stdlibs. So not really a full port as I understand it. If I understand correctly, if a given stdlib, such as
It would be nice if there was a well defined API boundary, but something is better than nothing. Out-of-tree ports currently have no guarantee about stdlib internals, so Go toolchain developers not working about breaking out-of-tree
Could be a stub implementation with just enough filled in to run tests? |
|
Time will tell, but I think it may be unfair to characterize this as a niche. and now ARM deployments outnumber x86 deployments by at least 10x. But in the interim this attitude led to a full fork and eglibc. Is tamago a niche use? Go on a new platform is a niche use by definition. But should tamago become useful in, e.g., silicon root of trust (SROT) applications such as the usbarmory, then tamago might well end up as widely deployed as all other Go platforms combined, since SROT is becoming ubiquitous (SROT is used in all new chromebooks, and will be used in all new servers). Future platforms will include at least one SROT, and some have more. Further, I'm starting to like tamago as a potential stack for Baseboard Management Controllers (BMC), as opposed to the gigantic, and vulnerable, Linux-based stack that is used today. This goes beyond SROT. I understand your argument for forking golang. The Go team can only do so much. But you may be playing with fire here, because one way this can go is not "fork per unaccepted port", but "one community fork for all unaccepted ports", i.e., a new organization supporting the second-class ports. If enough momentum forms behind a "community Go fork" that's more friendly to new platforms, a new Go team outside Google might form. It would be friendlier to new ports as, e.g, Rust is, and the momentum and interest could move to community go. This is not a new story. Recall that the most commonly used BSD variant is not the original one. I understand this is a tough situation, but you're in the enviable situation of having succeeded well enough that people want to use Go. While telling people to "just fork" is an option, forks have a way of diverging -- ref eglibc. Parallel organizations are created that can eventually outshine the original one. I think you also need to define the rules by which a sufficiently successful/widely used platform can be brought into the upstream tree from community go. It can't look hopeless. That said, in anticipation that tamago will still not be accepted, I've created this: https://github.com/community-go/go The intent is this be a common place to support unaccepted/deprecated ports, or ports not quite ready for upstream but that might make the jump, with the current example being tamago. From this, we can try to learn how to make fork maintenance easy. Having maintained an out-of-tree port for several years for Harvey-OS, I can tell you it's anything but easy today; with luck we can improve the situation. |
|
@rminnich if I might make a suggestion - best to not name the org "community-go", as that can confuse those stumbling upon it. Go as a projecy already has a community contributing to the golang org, after all. Perhaps something more specific like go-extra-ports? |
(taking the liberty to s/tamago/none/ while linking at tamago examples to show how this would look like) Correct, the external package could then contribute:
Without the external packages there are no drivers, so it's impossible to do any meaningful I/O or use anything other then the CPU.
It's not completely independent, the following files would be GOARCH=arm dependent:
So I think I could make this completely independent from ARM if we really want to, otherwise even if we keep this scheme the patches for *_none_something else would be relatively simple and small to create and maintain.
I think the two most important pieces would be lock_none.go and mem_none.go (copied almost verbatim from js/wasm and plan9 respectively), however these are fairly self contained, with a clear goal and in my understanding they would entail a substantial change in the way Go works to be broken. The residual risk would be for the runtime, or some other vital standard package, to start relying on OS specific (e.g. system calls) aspects to perform basic operations. What allows Go to be so suited to the bare metal is its almost non-existent reliance on system calls for most of its core operations, which is what allows us to bring in almost every package without worries. Unless a testsuite for GOOS=none, with a sample external architecture such as tamago+arm, is included in Go standard build process this is a risk that will always remain. I am not suggesting this should also be addressed, I am just noting it for the sake of discussion. |
|
@rminnich I agree with @mvdan that naming a new project community-go is unhelpful. It's also premature. Anyhow, I see two issues here. The first is whether we can provide a reasonably stable runtime API to support arbitrary ports. The second is whether we can provide a reasonably stable runtime API to support |
|
I removed community-go. If we do decide to go with a fork, it will likely then be go-extra-ports or something similar. Also, apologies if the use of the ulrich drepper quote was offensive to anyone, I mainly used it to try and illustrate just how badly things can go wrong. I've been on any number of open source projects which forked when people could not come to agreement, and it's always sad to see. All things considered, I'd rather not see that happen. |
What about multithreading, or preemption? I see nothing in what you described as the latest patch that addresses either one. Those are core OS functionalities that Go does depend on. To the extent that the tamago port does not provide those, it is providing a sub-standard Go experience. In particular, not having the ability to run on multiple CPUs seems like a complete show-stopper to me. That's not going to age well at all. (Also, maybe I missed it but I don't even see how the code puts the CPU into idle mode to save power when there are no goroutines running.) I understand that you can get something kind of working easily. That was the case when I did GOOS=tiny a long time ago too. But I also understand, from other projects, how much work it is to create and maintain a real operating system capable of actually using bare hardware well. That's a huge amount of added complexity for the Go team to accept ultimate responsibility for maintaining. And it is ultimately why, although it was a fun demo, we also removed GOOS=tiny a long time ago too. As an administrative note, we are starting two Go team quiet weeks, which means I won't be replying on this issue again until July 12, to avoid having a discussion that others might want to take part in. |
|
There are existing architectures in upstream Go without preemption or multithreading, in fact those architectures provided the very same code that we use right now (NaCl, js/wasm) in parts of our patch. Regardless of this I already expressed intention in this issue to be explore how to make multithreading hooks possible. Concerning preemption this is possible for external implementation as external packages can create timed interrupts and supervisors, we plan to demonstrate this in our GoTEE project. This is not just a fun experiment, we have thousand of units in productions running secure elements using TamaGo. Lowering frequency is handled externally by the application through functions we expose in our SoC driver, however I am not sure why this would be a concern for this discussion as SoC support and drivers would be an outside concern. We have created full support for NXP i.MX6UL series and received external contributions for Raspberry Pi, to me this proves the effort in making this usable is not inconceivable. The topics you are raising fall in what the external packages would require to implement and are not meant to be addressed or cared by the GOOS=none support (intentionally and for the reasons yourself raised initially). So I guess now you are moving the discussion towards whether the whole Go in bare metal philosophy has merit or not rather then specific GOOS=none patch details? The very initial concerns to this entire effort was that it would "clutter" upstream Go too much with hardware specific patterns that would be difficult to maintain and now that we moved everything out the concern is (despite one proved example of solid driver/SoC/CPU implementation) that those are external packages would be difficult or a bad experience? I welcome all questions and clarifications on our proposal as I think this is due process, but I think we are going a little bit all over the place here and the feeling is that the mind is set on rejecting our proposal no matter what. If the projects we are creating and supporting (listed earlier: GoKey, our bootloader, GoTEE, Armory Drive) created with TamaGo are not sufficient to prove that Go helps tremendously on the bare metal I am honestly not sure what would be. |
|
On "how much work it is to create and maintain a real operating system capable of actually using bare hardware well.". I think our patch shows how this would not be a concern of the Go team given that all actual hardware use is taken care of externally from the patch. In fact that's the whole point of it. |
|
I am also interested in such applications. But wouldn't it be better if the entire runtime package as a whole can be replaceable?
A stable (or at least versioned), documented API between the compiler and the runtime and a command-line option of the toolchain to alter the runtime package? |
|
OK, I think everyone is back, and I'd like to get this to some conclusion. I don't detect acceptance for bare metal go, upstream. Instead, we've been encouraged to fork Go. There are other ports that have hit a wall over the years, such as Harvey-OS. Rather than a fork-per-unaccepted-port, we're thinking it would be nice to have a common place to collect unaccepted ports. An unaccepted port is one which has never been accepted, or is no longer accepted, upstream, i.e. it has been demoted. The name go-ports has been suggested by several folks. The rules would look something like this:
The remaining question is where go-ports would live: is it an organization, e.g. github.com/go-ports/... or is it under golang, github.com/golang/go-ports? |
|
For what it's worth, those rules sound good to me. My present area of interest is a not-quite-bare-metal port of Go above seL4. Seeing worked out projects like TamaGo helps me dodge problems that others have seen and coped with. github.com/go-ports/ would be preferable to avoid any misunderstanding about who provides support. If the repository also helps the Go team monitor unexpected ways that Go is being used, that's a nice extra. |
|
OK, it's done: Please let me know if I've messed anything up. I will try from here on out to do Tamago work on this repo. |
|
Thanks for setting up go-ports/go. That seems like a great place to coordinate. As I've noted before, if there are general things we can do to make out-of-tree ports easier, I'd be more than happy to see those proposed. |
|
This proposal has been added to the active column of the proposals project |
|
As i said before on the previous issue, the current code of the Go compiler and runtime is often not structured modularly but relies an conditional compilation. Support for out-of-tree ports would be a lot easier if a GOARCH or a GOOS could be provided by a (set of) package(s), perhaps even a go module, in stead of conditional compilation all over the code. Yes, that means more abstraction, but it would make the go compiler and runtime easier to read and easier to work on. That alone would be a significant benefit. |
What does this mean exactly? |
|
Maybe an example explains it better? For example: If we take a look at https://github.com/golang/go/tree/master/src/os, then we see that there are many files such as as dirent_aix.go, dirent_dragonfly.go, dirent_freebsd.go, dirent_js.go, dirent_linux.go, dirent_netbsd.go, dirent_openbsd.go, dirent_solaris.go, etc. This repeated for many functions, and selected though conditional compilation. If in stead, the platforms could be separate directories, and thus, Go packages, then this could be refactored into, say, https://github.com/golang/go/tree/master/src/platform/aix/os, https://github.com/golang/go/tree/master/src/platform/dragonfly/os, etc. where all platform related functionality is grouped. The platforms packages could then even become go modules in separate repositories. Then, adding a new platform could be as easy as making a new go module, and adding a bit of code in src/os, etc. to "hook it up" so to speak. |
|
Based on the discussion above, this proposal seems like a likely decline. |
|
No change in consensus, so declined. |
If I might extend this conversation just a bit, because this is definitely a topic of interest to me: it seems like we're talking about making things more modular such that different GOOS/GOARCH backends can be used, a la GCC and LLVM (which are also not exactly fully pluggable). I think that's a tremendously useful, if perhaps ambitious, concept. It would be decidedly easier to maintain out-of-tree port if the architecture/OS backends didn't need to drag the entirety of the compiler machinery with them. But that would make it a lot easier for maintaining alternate ports out of tree. For example, I know there is already a I say this without a deep knowledge of the current compiler backend, of course, but I assume that this would be a pretty substantial change and something to be planned for several versions down the line if it were to happen. There would need to be, as someone mentioned upthread, a well-defined and versioned API between frontend and backends, which would presumably involve dicing up the current architecture rather more than it is now. I do have some provisional interest in this, but I'm relatively green at Go contributions. Is there a place I should start if I want to propose an approach after doing further research? |
|
The place to start is to look at the runtime package. Specifically look at the code that is built for a particular GOARCH, the code that is built for a particular GOOS, and the code that is built for a combination of a particular GOARCH/GOOS. That will describe a complex set of APIs that need to be implemented for any new target. Then try to figure out how those APIs could be supplied out of tree. Actually a seperate GOARCH would be a big job requiring new assembler, compiler, and linker support. Probably best to focus on just GOOS to start. |
|
Indeed, TamaGo also only targets GOOS and never touches GOARCH. |
|
Since this is a closed issue, I'd be happy to take the discussion to the golang-dev mailing list if that's a better place. |
Hello,
I'd like to request reconsideration of our previous proposal (declined) as there have been new simplifications in our patch that should solve some, if not all, concerns previously raised.
Small recap: our TamaGo project adds support for vanilla Go runtime and packages execution on ARMv7 bare metal. Please see the original issue which remains valid in its initial description.
The main changes of our most recent integration are as follows:
we completely removed any ARM exception handling or low-level assembly, with the exception of rt0 entry point support (here and here, which is required and included in Go for all supported architectures and a
CallOnG0function to facilitate external integration of exception handlers (which is very similar to existing approaches).all remaining code is simple "glue" code adapter from other supported architectures and to extend support of a new
GOOSkeywordruntime support in os_tamago_arm.go is much reduced compared to the first proposed version and now consists only of stubs and simple "glue" code.
Overall our latest patch has ~4200 insertions over 133 files, however the overwhelming majority of changes are one-liners two add GOOS=tamago support, almost verbatim clones of existing code (lock_js.go > lock_tamago.go,, mem_plan9.go > mem_tamago.go) and os_tamago_arm.go remains the largest newly added file.
We think these changes should address the main concerns that resulted the rejection of our previous proposal.
Our new patch should not raise any longer concerns on chip erratas as it is as agnostic as possible in the low level management of SoC specific, or even ARM, architecture. It allows external packages to provide ARM support.
We demonstrated useful implementation of all Go packages, including os and net, for instance we successfully enable networking with gVisor in our Ethernet over USB driver.
Since our last issue we implemented a full Trusted Execution Environment (GoTEE) with this framework as well as a full OpenPGP/FIDO U2F smartcard (GoKey).
Others have expressed interest for running Go on the bare metal and we also received contributions which extend support to other platforms such as Raspberry PIs using the same original Go distribution patch.
Given the recent refactoring that takes chip errata concerns out of the way, provides a more minimal patch, and the recent demonstrations of what can be developed with this, I kindy ask re-consideration of our proposal.
Thanks!
The text was updated successfully, but these errors were encountered: