Skip to content
This repository has been archived by the owner on Mar 28, 2023. It is now read-only.

Build process/patch testing #39

Open
woodruffw opened this issue Sep 9, 2020 · 1 comment
Open

Build process/patch testing #39

woodruffw opened this issue Sep 9, 2020 · 1 comment
Assignees

Comments

@woodruffw
Copy link
Member

We need to talk, generally, about tourniquet's approach to building source files and testing patches.

As of #37, that approach looks like the following:

  • Patching: Patches are applied to the original source file (not a copy) within a context manager, and then rolled back when the context manager ends its scope.
  • Testing: Programs are assumed to be single-file, standalone executables with no special flags. They're passed directly into clang; builds that error are skipped. The auto-patcher assumes that the target program can be tested using inputs on stdin and exits with with an exit code on various different error conditions.

By and large, I think this approach to patching is fine for now: we'll probably want to not patch the original source file further down the line, but hiding it inside a context manager keeps things clean for the time being.

On the other hand, the approach to testing needs to be rethought and redesigned. In particular, we should think about the following:

  • How are we going to compile source files that belong to a larger (multi-file) program? Are we going to recompile the entire program each time, or attempt to recompile only the particular translation unit with the help of compilation records from blight? How are we going to ensure that we faithfully re-link the modified translation unit?

  • How are we going to test patched programs? We should probably implement a set of methodologies, with "throw stuff at stdin and see how it exits" being one of them. We should probably come up with an abstract "test" interface that each methodology refines; we can then make the process of selecting the appropriate methodology for a program something that requires human feedback.

@woodruffw
Copy link
Member Author

woodruffw commented Sep 9, 2020

On the testing front: a more aggressive idea I had is a translation one: given a pure function f, we can extract f's ABI and turn it into a standalone program, which we can then use the "stdin and exit code" strategy with.

For example:

int foo(int bar, int baz) {
  // Undefined for baz == 0
  return bar / baz;
}

Could be transformed into foo --bar 123 --baz 456 or something like that.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

2 participants