Proposal: send debug output to STDERR or a named file

[snim2]

Pydgin simulators can produce a trace of their state changes using the --debug switch. This is extremely useful, and it means that continuous integration and other automated testing tools can use the traces to detect errors. For example, the script diff_trace.py diffs a Pydgin trace against a trace from a simulator provided by a manufacturer and reports on any instructions which were executed incorrectly.

Unfortunately, this isn't as simple as it could be, because Pydgin prints all its debug output to STDOUT. This means that output produced by the program being simulated is mixed in with the trace, leading to output like this:

8e002a62 38e2800b movtimm32    1247     :: RD.RF[12] = 00002ac4 :: WR.RF[12] = 8e002ac4
8e002a66 0402114f jr32         1248     :: RD.RHello, world!
F[12] = 8e002ac4
8e002ac4 6dda1fe2 trap16       1249     :: RD.RF[3 ] = 00000005 :: RD.RF[0 ] = 00000001 :: RD.RF[1 ] = 8f000008 :: RD.RF[2 ] = 0000000e syscall_write( fd=1, buf=8f000008, count=e ) :: RD.MEM[8f000008] =  00000048 :: RD.MEM[8f000009] =  00000065 :: RD.MEM[8f00000a] =  0000006c :: RD.MEM[8f00000b] =  0000006c :: RD.MEM[8f00000c] =  0000006f :: RD.MEM[8f00000d] =  0000002c :: RD.MEM[8f00000e] =  00000020 :: RD.MEM[8f00000f] =  00000077 :: RD.MEM[8f000010] =  0000006f :: RD.MEM[8f000011] =  00000072 :: RD.MEM[8f000012] =  0000006c :: RD.MEM[8f000013] =  00000064 :: RD.MEM[8f000014] =  00000021 :: RD.MEM[8f000015] =  0000000a :: WR.RF[0 ] = 0000000e :: WR.RF[3 ] = 00000000

which is difficult to parse automatically.

It would be more helpful to send the trace output to a different stream (STDERR) or a named file.

Below is a proposal for resolving this issue. I have resisted sending this in as a pull request as there is a note about print statements in pydgin.debug which made me think I might have missed some subtleties here. However, if people think this, or something similar, is sensible, then I'll be very happy to code up this proposal and send the PR.

1. Adding to command line options

It would be nice to give the user the option of sending the trace to STDERR or a named file. STDERR is tricky to redirect to a file (compared to STDOUT), and saving output to a file makes it easier to automate testing. I would change the Pydgin command line switches to this:

help_message = """
Pydgin %s Instruction Set Simulator
usage: %s <args> <sim_exe> <sim_args>
<sim_exe>  the executable to be simulated
<sim_args> arguments to be passed to the simulated executable
<args>     the following optional arguments are supported:
  --help,-h       Show this message and exit
  --test          Run in testing mode (for running asm tests)
  --env,-e <NAME>=<VALUE>
                  Set an environment variable to be passed to the
                  simulated program. Can use multiple --env flags to set
                  multiple environment variables.
  --debug,-d <flags>[:<start_after>]
                  Enable debug flags in a comma-separated form (e.g.
                  "--debug syscalls,insts"). If provided, debugs starts
                  after <start_after> cycles. The following flags are
                  supported:
       insts              cycle-by-cycle instructions
       rf                 register file accesses
       mem                memory accesses
       regdump            register dump
       syscalls           syscall information
       bootstrap          initial stack and register state
  --trace, -t <filename> Save the output generated by --debug to 
                  a named file (default is STDERR).
  --max-insts <i> Run until the maximum number of instructions
  --jit <flags>   Set flags to tune the JIT (see
                  rpython.rlib.jit.PARAMETER_DOCS)
"""

2. Keeping track of the output file

Add an instance variable (or class variable?) called trace_fd to pydgin.debug.Debug which holds the file descriptor of the open output file (or stream). The file descriptor would need to be created either around line 289 of pydgin.sim.Sim (where the Debug) object is created or in the pydgin.debug.Debug.__init__().

3. Replacing all calls to print

I asked on the PyPy IRC channel, and Ronan said that os.write is the standard way to write to a named stream in RPython. All 81 calls to print would be replaced by calls to os.write. Newlines need to be added explicitly.

4. Closing pydgin.debug.Debug.trace_fd

A call to os.close(pydgin.debug.Debug.trace_fd) would need to be added around line 192 of pydgin.sim.Sim.