Idea: 'hands-free' build

[rhalkyard]

Just a little quality-of-life improvement that I've added to my personal build setup for durexforth:

The VICE -debugcart option makes VICE quit when a program writes to IO address $d7ff, with the return code being the byte that was written - the intent is to allow automated testing of C64 code. https://vice-emu.pokefinder.org/index.php/Debugcart

Invoking VICE with -debugcart and adding 0 $d7ff c! to the end of save-pack allows an unattended make deploy - on a successful build, VICE quits automatically and the build continues, but if something fails, it stays open so you can see the error message.

We probably don't want to make save-pack do this outside of the build environment in case it interferes with any unusual setups people have out there, but if we can somehow set things up so that it happens during the build but never on a built system, it would streamline things quite nicely. The C128 build means that another VICE instance has to run, and I'm forgetful enough that I often wonder why the build is stuck, until I find a VICE window I'd forgotten to close...

Probably a pretty low-priority thing, but I thought I'd share.

[rhalkyard]

Another thought: we could use the same mechanism to automatically run the test suite, to give some assurance that changes haven't broken compatibility or caused a performance hit.

[cminter]

@rhalkyard ,
Using your idea, I came up with a method to have the modified save-pack not be present in the final disk image. A working commit is in my f/debugcart_exit branch. If anyone has feedback (in advance of submitting a pull request) let me know.

1. The new default behavior is hands-free.
2. The original behavior can still be done with: make DEBUGCART=0 deploy
3. I also added a variable hook so we can generate d71 or d81 images when we want more free space available: make IMAGE_SUF=d71 deploy

For some reason the DEBUGCART mode fails for me when using a d81:

• make IMAGE_SUF=d81 deploy fails with an error it seems in the extracted durexforth bin according to the cartridge build step.
• make IMAGE_SUF=d81 DEBUGCART=0 deploy succeeds (of course need to manually close VICE)
• make IMAGE_SUF=d71 deploy succeeds

[pzembrod]

Hands-free build is something great; from my experience with cc64 and the CBM flavour of VolksForth I can highly recommend it.

Interesting stuff about the debug cartridge in VICE - I didn't know about that.

I chose a different path to implement stopping the emulator from within, basically using the file system as a signal path to the outside. The emulator is watched by a shell loop, and as soon as a certain file (called "notdone") disappears (because the code inside the emulator scratched it), the shell loop terminates VICE. Kinda nice: this approach also works with x16emu, the emulator for the Commander X16. Details are here, here, here and here if you want to look at some code.

chitselb in the 6502.org forum chose yet another pathto achieve the same goal: They use VICE's breakpoint and monitor feature to terminate VICE with a SYS command. See http://forum.6502.org/viewtopic.php?f=8&t=5628

[cminter]

@pzembrod, I happened across your presentation "Temporary headers in VolksForth" as I subscribe to the channel it was posted in. Nice work. The low level details go over my head. Very interesting. I'll have to look at it and cc64 in more detail.

But I've been wanting to build a scheme where I can swap in code from the C64 REU (i.e. overlays see Forth Dimensions v05 n1 p37). Maybe this is exactly solvable with your temporary headers.

[jkotlinski]

It would be nice to have CI setup through Github Actions. It seems like work to set it up, but in the ideal scenario, it could build & run test suite for every commit, and releases could be generated & published automatically.

[pzembrod]

@pzembrod, I happened across your presentation "Temporary headers in VolksForth" as I subscribe to the channel it was posted in. Nice work. The low level details go over my head. Very interesting. I'll have to look at it and cc64 in more detail.

@cminter thank you, and sorry, I completely forgot to reply earlier.

But I've been wanting to build a scheme where I can swap in code from the C64 REU (i.e. overlays see Forth Dimensions v05 n1 p37). Maybe this is exactly solvable with your temporary headers.

Thank you for pointing out this article to me. For cc64 I'll need to come up with a way to place Forth code into banked memory on the Commander X16 before I can continue any feature work as I am at the code size limit for the X16 version. For this, this FD article will surely be super-useful.

I could see temp headers being useful in this context, to save overlay or banked memory, but the core problem of banked or overlay code seems to be somewhat different from or even orthogonal to the problem that temp headers solve.

[pzembrod]

It would be nice to have CI setup through Github Actions. It seems like work to set it up, but in the ideal scenario, it could build & run test suite for every commit, and releases could be generated & published automatically.

@jkotlinski I've been thinking about this for cc64 and VolksForth, too. I think that I already have most of the scripting for CI actions. The one thing missing for proper batch processing would be a framebuffer X11 server to run the VICE emulator in a headless mode. But beyond that, a larger obstacle I would see is that you would have to check in C64 ROMs for VICE to run on github workers, and that might pose copyright challenges.

[jkotlinski]

@pzembrod Ooops, copyright challenges might be difficult! It was easier in the past when everyone was oblivious to such things!

[pzembrod]

@jkotlinski I've been thinking about this a bit more. Maybe https://github.com/MEGA65/open-roms could provide a viable option.

[jkotlinski]

I just upgraded Vice on my computer, to 3.6.0, and it seems ROMs came included. In that case, perhaps it is no longer an issue.

[pzembrod]

I just upgraded Vice on my computer, to 3.6.0, and it seems ROMs came included. In that case, perhaps it is no longer an issue.

That's great news!

[lonetech]

It is an issue for some people, but not all. I guess Vice has a license to distribute the unmodified ROMs somehow, but they are non-free binaries themselves, so won't ship in distributions like Debian or Guix. It is weird that Vice doesn't indicate under what conditions the ROMs are included, as they have been asked. Vice seems to have included full C64 ROMs since 0.14.2.26 was imported in 1998.
Improving open-roms until it runs durexforth is a potentially productive option.
Also, vice does have a headless mode with -console; they may have intended it for playing SID music, but nothing would stop us from using it to run a program whose effects show up in the file system.

[lonetech]

https://github.com/lonetech/durexforth/tree/faster-build demonstrates running the deploy step in an automated fashion using Vice. Vice itself checks the program finishes in a finite time, and saves a screenshot of the last stage.
On a real C64, the machine will simply hang after finishing the compilation and printing "ok!", at which point a reset is needed.

I should probably have read this thread properly to learn about the debugcart, but at least I get status output?

It's also interesting how the "ok!" didn't happen, but the interpreter was left awaiting input. It looks like turnkey interfered with the interpreting of the final words in base. I used a noname compiled section to work around that.

[jkotlinski]

@lonetech sorry for belated reply. What is "x64.exe" that you are using instead of "x64sc.exe"? It is not included in my Vice install.

[jkotlinski]

I now added so that "make deploy" exits Vice automatically. It also runs the tests automatically. Not totally hands-free though, as one test tests keyboard input.

The problem with "ok!" not happening, and the need to put the $d7ff write in save-pack, was because of a SAVEB bug which is now fixed.

Thank you for all the great advice given in this issue!

Anything else I should think of before I close the issue?