proposal: all: add bare metal ARM support #37503
Comments
|
From Requirements for a new port:
How is the builder story? |
|
I'm confused. You say you needed to modify the compiler. But when I look at the diff between f-secure-foundry/tamago-go and go tip, there don't appear to be any changes to cmd/compile. |
|
Related to #35956. I'd be very interested in the amd64 port of GOOS=none/noos/tamago, even if it only supports virtual machines. |
You are correct, my bad. We don't modify cmd/compile at all. We modify mainly the runtime, stdlib packages (mostly for stubs and build flags) and the dist and link commands to add the additional GOOS. |
We have an internal working amd64 port which runs under qemu, but it's not our focus and quite invasive. For this reason we propose only an ARM port as our target is ARM based embedded systems. Having said that this might pave the future, if accepted, for future similar efforts on other architectures. |
Thanks, I'll look into it. |
Interesting. Are you able to share the implementation? I'd also like to know what makes the amd64 port more invasive than the arm port. |
Unfortunately we are not confident in sharing this implementation, our focus is just the ARM version for now. The amd64 port is more invasive due to the way the MMU needs to be managed, additionally multi-core support requires to add a thread scheduler in the runtime which is something we don't require for single-core ARM environments. So given the complexity and lack of interest on our side we used this prototype only to initially validate the concept and then move over to ARM. There are other efforts that implemented an amd64 port but they have been implemented in a way which I think it makes it too invasive for upstream adoption, see here and here. As our goal is to run embedded firmware without C we don't focus on hypervisor support (as the hypervisor is in C anyway) and running without one on amd64 is very challenging given the amount of drivers required imho. Not saying it's not doable ;), just not our focus. |
I checked docs. Ee would have no issues in providing a builder for this and maintain it. |
|
As mentioned in my initial issue message, I am quite sure that "tamago" needs to be changed for adoption as it's catchy ;) but too specific. Therefore in the spirit of #35956 I think we should settle on I can ensure that the proposed changes only kick in with GOOS=none, GOARCH=arm and GOARM=7 which is what this code currently targets. |
abarisani
changed the title
GOOS=tamago
|
I am not qualified to comment on the runtime and linker changes, but from the security point of view, this would enable fantastic applications for embedded deployments. Easy to write, fully memory-safe firmware on consumer hardware would not only replace a lot of HSMs, but make hardware security available to many more projects. For example, I'd make use of it immediately in age, and I'm sure a lot of people would have more creative uses for it. Secure elements and trusted hardware provide security properties that are simply impossible to replicate in software, like defending low-entropy passcodes from brute force. I am truly looking forward to this being a mature and robust target, which is why I'd love to see it supported in the main tree. |
|
This proposal seems like an exact duplicate of #35956. |
rsc
changed the title
|
I understand the Go team itself cannot support Go on bare metal. However, there are many people who do want to support this. This suggests to me that the GOOS and GOARCH should be made pluggable somehow. |
Yes it is, please see my earlier comment. We think that arm bare metal support is considerably less complicated, the principle is the same but the implementation is far easier. We also target embedded systems as we see much more value in running Go there than on amd64 based hypervisors, so also the vision is slightly different. |
We would be happy to maintain this just like other folks did for other ports (for example js/wasm). In fact the effort required to take this in is less, or comparable, to js/wasm in terms of LOC. |
|
Yes, exactly, this is an archtecture that has 3rd party support. It would be easier if third party architectures could be used with the main go compiler without having to be integrated in the main repository. |
|
Another difference from #35956 is that @abarisani indeed demoed an HTTPS server running on the bare platform, already crossing the threshold into being useful for applications. |
I honestly think this would add complications and open up to more "dirty hacks" if there is easy access in modifying the runtime. Given the simplicity of our implementation a GO3PARTY might open up more problems than what it needs to solve in my opinion. |
|
The main concerns I listed in #35956 (comment) are:
I read your linked reply and don't see any differences between ARM and x86 on these. If anything, there are almost certainly more low-cost oddly-behaved ARM devices than x86 devices, making the chip errata problem worse. (We spent so long debugging bad ARM chips just running builders for the linux/arm port.) This still seems like something that should be done out-of-tree, as a fork of the golang/go repo. Am I missing something? |
I'd encourage to please take a look at the diff and at my presentation and slides and the example application. The (small) complexity is separated from the changes required to upstream, as all drivers are in a separate package which is here, most drivers are very simple while some others have little more code, such as the USB one but still very compact and clean in my opinion. But yet again this code is not some we are requesting to be upstreamed. So in a nutshell chip erratas, if present, would be taken care in drivers that are outside the requested changes for the Go distribution. Useful implementations are possible (I am developing a pure Go smartcard right now with this). Drivers are already out-of-tree in our proposal and the changes to the Go distribution are minimal, isolated and clean to address all complexity/maintainability concerns within. |
|
To further add to my comments, this is a concrete example of what we accomplished using the proposed changes: Demo video: https://twitter.com/AndreaBarisani/status/1247177053482487809 |
|
This still seems beyond the scope of the core Go team. |
|
I respect your opinion, but to me it seems inconsistent with support for other architectures that are within go already. Then why not proposing the same approach for wasm ? What's the difference? Why is one within the scope of the core Go team and the other not ? What is the criteria? |
|
That's a fair question. I don't know that we have specific criteria. I looked at your patch. A lot of the file changes seem to be build tag additions. And a discouraging amount of the changes seem to be copying the NaCl code that we just got rid of. Here is the diffstat of the patch: Personally I think that we should start by trying to make this patch smaller. Let's try to think of some approach that reduces the number of build tag edits that are required. Perhaps we can introduce some sort of mapping by which build tags imply other build tags, as More generally, we want to minimize the maintenance burden on the Go maintainers. It's an essential step that you are volunteering to maintain the port. But the burden remains. We don't want to break ports as we go, which is why changes like https://golang.org/cl/171823 wind up changing 30 files. So lets start minimizing the new code required by any new port. I suggest that we take steps in that direction and see how far we can get. |
From https://github.com/golang/go/wiki/PortingPolicy:
So it's not just about the burden.
I have no opinion on this, just pointing out that the page already says that. |
|
Good point, I had missed that. Though I have to admit that's very subjective. If the "reaches enough users" judge is working on cloud computing, they're unlikely to care about embedded systems or unikernels, for example. Or if the judge is the open source community at large, it's unlikely they would care about platforms like mainframes. |
Indeed, but the reason cited for this being a likely decline is one of burden. Hence why @rasky, @mvdan and others are, I think quite understandably, asking for clarification, especially in the context of previous decisions on the js/wasm, riscv64 or plan9 ports. |
|
I understand why you want this in the distribution: because then it stays up-to-date. I agree that we should revise the porting policy doc to be clearer that it's no longer enough to just have one person willing to maintain the port. We've found that simply doesn't scale. Every new port imposes constraints on everyone checking code into the repo, not just the one maintainer. Today, we take on that maintenance burden for widely-used, broadly-applicable ports. Wasm can be used in any web browser. (There are many people with web browsers.) In contrast, tamago is only useful on arm systems that have chosen not to use Linux. There are comparatively many fewer of those. And again, we have had enough problems debugging Linux. We've also had problems debugging ARM chip errata in the user mode side on the linux/arm port. I really don't want to make us have to debug the kernel side too. But that's just one example. For what it's worth, if we were evaluating some of the existing ports today, I think some of them would not make the cut now either (most notably plan9 - if it got to the point where lack of some OS feature made Go on plan9 too hard to maintain, we might reasonably decide to drop it). |
|
I won't repeat most of the arguments I already made, but concerning erratas please see this comment, erratas are really not an issue in this port the way I see it. Concerning the number of maintainers which you mention, how many maintainers would it take? > 1 ? More? There are countless ARM systems out there (Raspberry PIs for instance, countless IoT devices), the reason why they are forced to chose to run Linux now is because something like this didn't exist before. This is why we think this is a strategic move for Go. |
|
@abarisani How hard would it be to maintain an out-of-tree patch if #30322 is solved? |
Not sure how that issue is related. |
|
This was marked as a likely decline two weeks ago (#37503 (comment)). As also noted last week, we should update the porting policy to better reflect the requirements for a new in-tree port. We should also work to make sure that out-of-tree ports are as easy as possible, whether that's #20322, #38364, or something else. It simply doesn't scale to put every port in the main tree. Out-of-tree ports need to be easy. Declining. |
|
@rsc we recently implemented changes that move away ARM exception handling and MMU initialization away from the Go distribution patches, delegating them to the application itself. This practically removes every hardware/architecture dependent modification to the Go distribution, making GOOS=tamago modifications to standard Go runtime essentially a basic template for bare metal execution. We are wondering if these changes can warrant reopening this discussion or (if that's preferred) creating a new proposal (please advise), thanks! You can see the current patch (against go1.16.3) here. A diff on removal of exception handling and MMU initialization can be found here. |
I don't think I got a response to why wrapping a static GOOS=linux Go program in a unikernel isn't good enough? We avoid adding another GOOS to the standard distribution, and Linux syscalls are a simple standard to follow. |
|
In my opinion it is an improper practice for more than one reason. The final glue code required to make that work is far higher to our approach, hijacking Finally doing so brings in several code that relies on Linux specific aspects/behavior (e.g. presence of certain files on the filesystem) and I think it's just wrong and impractical in the end. |
I have to agree that, from the code that I've seen and written in practice, a significant portion would break if GOOS=linux was replaced by just a set of syscalls. I can't decide if that's a bad thing or just inevitable. The reality is that, when testing and deploying, nearly everyone assumes that GOOS=linux is a proper Linux kernel. |
Can you elaborate? First, the syscall shim for Unik doesn't feel too bad (no networking support, however). Second, the syscalls have to be provided somehow, either in the Go distribution as GOOS=tamago or in the Linux shim. Note that any syscalls not used by Go itself can be implemented however you like. I assume that's how you implement exception handling and MMU setup in user code.
Sure, but the alternative is the maintenance burden of GOOS=tamago. You have to define some system API, and while the tamago interface may be simpler and/or faster, it is far from a standard. |
Programs that assume a proper Linux kernel presumably use the features from a proper Linux kernel, and thus wouldn't support GOOS=tamago either. And if you modify such a program to cope with GOOS=tamago, why not modify it to tolerate a shim Linux kernel? |
|
I don't share this logic, we are pulling in plenty of libraries that are portable, do not make any assumption of running under Linux and are the actual core benefit behind the tamago project. Why would we go the trouble to force use of a shim Linux kernel when using perfectly portable code? We have plenty of production firmware, based on TamaGo, using standard Go packages which are out there and without any need to modify then for supporting GOOS=tamago. |
That's a fair point, but then I argue that you do need a linux-related GOOS, similar to how we have GOOS=android. This doesn't mean that support for said GOOS needs to be part of Go, though. |
Why would code unaware of GOOS=tamago not work with a shim Linux kernel? |
How would you add another GOOS without support in the standard Go distribution, or in a fork of it? Neither of which is attractive if you can get away with a shim. |
|
Code unaware of GOOS=tamago of course works in a shim Linux kernel. What I am saying is that we don't want to run under GOOS=linux because that causes several issues (which I feel I clearly explained). My comment was a reply to your "wouldn't support GOOS=tamago either." to explain that this is an incorrect premise. There is plenty of code that works perfectly fine under GOOS=tamago without requiring modifications. We spent months to refine an implementation that doesn't simply hijack on GOOS=linux (like a few other bare metal frameworks did) because we experienced first hand how inappropriate that is. |
|
You might get more attention/input by discussing this in golang-dev. Also it might help to quantify the community of developers who'd benefit from a new GOOS type. And maybe get more of them to join the discussion. |
Hello,
I am unsure if to go directly with a CL (as this has been discussed already on golang-dev) or if to go through this issue first, apologies if I should have done otherwise.
On this golang-dev thread I described the new GOOS=tamago created to support bare metal ARMv7 architectures using the original Go compiler.
The use case for such support is to run native Go applications on Single Board Computers without and underlying Operating System. Currently we are testing this on our own USB armory but we are working on Raspberry PI Zero support as well.
We have working examples of SSH and HTTPS servers running on Ethernet over USB with all USB drivers and stack implemented in pure Go. We are also working on a pure Go OpenPGP card leveraging on this.
We think there is great value in having this kind of support in the Go language as it considerably improves the security and ease of development on embedded systems and allows to purge C, with ease, on otherwise complex firmware efforts.
Our work is split across the following 3 repositories:
For upstream consideration only the first repository matters, we intentionally kept the compiler changes separate from SoC specific drivers and ensure that Go applications can run mostly unencumbered and with a single import required for the board package.
You can find the following resources to understand more about our effort:
Our compiler modification approach tries to be as minimal and clean as possible, respecting or reusing existing Go runtime code. A new
GOOS=tamagois added only for GOARCH=arm, you can see the changes here.We are committed in maintaining the required patches long term, we also understand that any potential upstream inclusion would require renaming
tamagoto something more suited (others have been usingnoosin other proposals), which is totally fine.The patches should also be easy to revert if necessary.
To further prove the maintainability and isolation of this new GOOS from the existing code base, we began our work on go1.12, then rebased it to go1.13. Bumping it to 14.1 only took a small effort.
We built
GOOS=tamagospecifically to make it upstream friendly and have the minimum possible set of changes as our focus is trust for security applications and maintainability without reinventingor adding too much code to the existing runtime.
We would much like to start a conversation to have these changes accepted upstream and take responsibility to maintain them.
Thanks
The text was updated successfully, but these errors were encountered: