Editing the editor (tutorial platform)

[amlwwalker]

Hey,
This is super cool.
I have wanted for quite some time to build tutorials for the Go language using wasm (in browser) - you have probably jump started my work here by a mile.
The first step is to make a super interactive platform and a 'framework' for the tutorials so its fast and easy to add new ones.

Thoughts:

• left hand side panel where the instructions for this step/tutorial are
• pre written tests that can check the results are correct - I don't want to use regex parsing stdout or something, I want real Go tests to run against functions (students add their function call to the test harness so they can call functions whatever they want - part of learning is the tests.
• tutorials can use libraries that already exist (your platform offers this)
• I can use mockers (smocker) to mock out external services in tutorials
• I can run other Go apps/binaries that expose ports or something to interact with (e.g an API running locally)

Questions

left hand side panel where the instructions for this step/tutorial are

I looked through the code and see you are designing/rendering the UI in Go which was amazing/surprising. Two questions really - one is do you have a link to tutorial in how you are doing this, and secondly, could this be swapped out for HTML/CSS/JS or React or something easily? My guess is WASM means better to do all in Go, so question one is more the focus - i.e can you point me to anything to learn how to adjust the UI?

pre written tests that can check the results are correct - I don't want to use regex parsing stdout or something, I want real Go tests to run against functions (students add their function call to the test harness so they can call functions whatever they want - part of learning is the tests.

This sounds possible based on your "OS" - Would be cool if tests can run against code?

I can use mockers (smocker) to mock out external services in tutorials
I can run other Go apps/binaries that expose ports or something to interact with (e.g an API running locally)

I suspect these last two are closely related and its either "thats easy" or "thats hard" as the OS isn't really "real".....

Anyway would love just to hear your thoughts on this usecase for this platform.
Thanks

[JohnStarich]

Hey @amlwwalker, thanks for opening and the positive feedback!

I love those ideas. A learning platform written in Go for Go is a fantastic goal I can get behind 😄

Two questions really - one is do you have a link to tutorial in how you are doing this,

That's a great question. I actually do not have a tutorial for that one. Mostly, I pulled from my previous experience writing static sites to build up the look I wanted, and then later I used the APIs to build DOM elements for the entire UI instead of only part of it.

I typically insert some shared HTML and stylesheets using Go's embed package like this:

hackpad/cmd/editor/ide/settings.go

Lines 18 to 23 in be2b4e1

	var (
	//go:embed settings.html
	settingsHTML string
	//go:embed settings.css
	settingsCSS string
	)

Then connect some DOM elements with event listeners to parts of the application like the text input and button controls. You can find the whole editor is a Go program all stored in ./cmd/editor if you're curious.

and secondly, could this be swapped out for HTML/CSS/JS or React or something easily?

Yes, certainly! The only unique Hackpad piece is I start the editor program using a Node.js-like function call. That function can be called from JS or Go. Soon I'll call it from Go, but today I call it from JS here:

hackpad/server/src/App.js

Line 25 in be2b4e1

run('editor', '--editor=editor')

and here:

hackpad/server/src/Hackpad.js

Line 77 in be2b4e1

const subprocess = child_process.spawn(name, args, options)

The editor uses React under the hood today. In the future, I plan to remove as much of React as I can to simplify our dependencies a bit.

My guess is WASM means better to do all in Go, so question one is more the focus - i.e can you point me to anything to learn how to adjust the UI?

I think it may actually be easier to do in JS today, simply because really good development tools exist for those already. Doing it in Wasm is pretty fun though, so I wouldn't want to discourage you either way. 🙂

Turns out, you could write a UI using whichever tools you like! React certainly works (since we're using that now) and Go is an okay experience thus far. As mentioned above, it isn't quite as nice as a mature set of tools like React offers.

I have seen some interesting Go UI libraries out there though. You might want to give these a try:

• go-app – https://github.com/maxence-charriere/go-app
• GIO – https://github.com/gioui/gio
• Vugu – https://github.com/vugu/vugu

This sounds possible based on your "OS" - Would be cool if tests can run against code?

Yep, that works today! Give it a try and let me know if you hit any roadblocks.

I've just noticed new editor tabs aren't working at all like they should... oof. We should fix that. I worked around it by writing and running a test in the terminal though (yes, it looks very silly without the editor working!):

This is running the real Go CLI on a fake Node.js environment, so Go is totally convinced it's on a real system. There are some caveats, like binding to a port doesn't work, but generally most things work the way you'd expect.

I suspect these last two are closely related and its either "thats easy" or "thats hard" as the OS isn't really "real".....

Mocking out dependencies is 100% possible. Binding to a port for real isn't possible right now, but if you use a mock for that too, then you should be good to go.

You might be able to "hack" in your own net/http package with a Go Module replace line? I've not tried it before though. I'm actually curious to know if that really would work.

Anyway would love just to hear your thoughts on this usecase for this platform.
Thanks

Happy to provide! 😄

[amlwwalker]

Hey @JohnStarich
I have begun work on a webassembly editor to get myself going and to a place where i can use your code base

nothing too much yet but its working nicely using Go-App UI builder for web aseembly apps.

I'm now trying to understand what I need from your repository to run the code from my editor. Could you give me some support as to what I need to do to:

1. have a terminal where I can "read" the code in my editor (save to a file in the virtual OS) and compile that code to a binary?
2. Display the output of the terminal in my UI.

It may just be a case of you telling me which of your main.go's I should run or might even be able to turn into functions and call from within my application - or is it a case (for the terminal) of creating a new instance of terminalBuilder from here ?

if so I can probably work out how to get the output to display within my GUI if thats all I need.

Do I have to manually run the OS or does that happen as a result of running the terminal? I attempted to run the main.go at the root of the repo but I get build constraint errors so wondering if thats not supposed to be directly run perhaps.

Could you potentially give me a simple main.go that I can run from my terminal that builds a .wasm file that I can then fire up in my browser?

Apologies if this all seems a bit basic - trying to get my head around your setup is slightly slow on my part 😂 but once I understand how to save text to file and compile it from the terminal/run it I should be off - just the hackpad demo is a kitchen sink and hopefully can get a barebones equivalent runnning!

[amlwwalker]

hey @JohnStarich just pinging this because I actually see an awesome usecase for this in writing (and teaching to write) blockchain contracts (that can be written in Go). Would love to have a little help getting the terminal/compiler in a simple standalone format that i can then learn from. Thanks again!

[JohnStarich]

Hey, sorry for the delay!

I think all the necessary bits you need to run your own app are outlined in here:

hackpad/server/src/Hackpad.js

Lines 9 to 45 in 1f6b4af

	async function init() {
	const startTime = new Date().getTime()
	const go = new Go();
	const cmd = await WebAssembly.instantiateStreaming(fetch(`wasm/main.wasm`), go.importObject)
	go.env = {
	'GOMODCACHE': '/home/me/.cache/go-mod',
	'GOPROXY': 'https://proxy.golang.org/',
	'GOROOT': '/usr/local/go',
	'HOME': '/home/me',
	'PATH': '/bin:/home/me/go/bin:/usr/local/go/bin/js_wasm:/usr/local/go/pkg/tool/js_wasm',
	}
	go.run(cmd.instance)
	const { hackpad, fs } = window
	console.debug(`hackpad status: ${hackpad.ready ? 'ready' : 'not ready'}`)
	const mkdir = promisify(fs.mkdir)
	await mkdir("/bin", {mode: 0o700})
	await hackpad.overlayIndexedDB('/bin', {cache: true})
	await hackpad.overlayIndexedDB('/home/me')
	await mkdir("/home/me/.cache", {recursive: true, mode: 0o700})
	await hackpad.overlayIndexedDB('/home/me/.cache', {cache: true})
	await mkdir("/usr/local/go", {recursive: true, mode: 0o700})
	await hackpad.overlayTarGzip('/usr/local/go', 'wasm/go.tar.gz', {
	persist: true,
	skipCacheDirs: [
	'/usr/local/go/bin/js_wasm',
	'/usr/local/go/pkg/tool/js_wasm',
	],
	progress: percentage => {
	overlayProgress = percentage
	progressListeners.forEach(c => c(percentage))
	},
	})
	console.debug("Startup took", (new Date().getTime() - startTime) / 1000, "seconds")
	}

and here:

hackpad/server/src/App.js

Lines 19 to 27 in 1f6b4af

	window.editor = {
	newTerminal,
	newEditor,
	}
	Promise.all([ install('editor'), install('sh') ])
	.then(() => {
	run('editor', '--editor=editor')
	setLoading(false)
	})

where run is

hackpad/server/src/Hackpad.js

Lines 54 to 57 in 1f6b4af

	export async function run(name, ...args) {
	const process = await spawn({ name, args })
	return await wait(process.pid)
	}

The setup should be significantly easier after the big refactor lands, but that's taking quite a while so probably best not to wait for it.

If you run the JS I've linked above, you should be able to get everything up and running. Might be worth poking around in ./server/src to see how it's running now.

To get the wasm files, you can download prebuilt ones from the gh-pages branch. After the refactor, I think it may be reasonable to publish these as GitHub Releases too.

As a note: hackpad.org is using CodeMirror for the text editing portion and xterm.js for the terminals. Let me know if you need any pointers to hook those together.

[JohnStarich]

Do I have to manually run the OS or does that happen as a result of running the terminal? I attempted to run the main.go at the root of the repo but I get build constraint errors so wondering if thats not supposed to be directly run perhaps.

You can run this repo as a local copy using make watch. The first build will probably take a while to download and set up both Go and JS.

Could you potentially give me a simple main.go that I can run from my terminal that builds a .wasm file that I can then fire up in my browser?

This might be the best spot to start with Go and Wasm: https://github.com/golang/go/wiki/WebAssembly
Basically just set GOARCH and GOOS, then run go build as normal.

[amlwwalker]

Hey @JohnStarich - thanks for getting back to me. Since last speaking I've been doing a lot with Go-App and am itching to include your OS/terminal in there.
I had a read of your big-refactor. Looks fantastic what you're doing and I realise that you are going through the feeling of always being just around the corner but it sounds like it will take a proof of concept to a really practical solution.

I think I will wait and keep an eye on how the refactor goes because as you say I think it will be easier to follow the code through, but if you are working on it 😉 and during the refactor you are able to put simple examples of using the elements individually so we/I can build off of your components that would be epic.
For instance

• how to add a OS and terminal to an app (what do I need to call and what do I need to render) to have a simple terminal into the OS on a page
• how to send the compiler some code to compile/run

both from Go and JS/webasm would be interesting. Its a very cool set of components you have here.

I realise thats quite cheeky, as it is now its a lot to take in how its all put together and I think being able to build on top of your individual components will really make this an incredible bit of work

Going through your code snippets now