Reuse Hy compiler instances

[brandonwillard]

Every call to hy_eval and hy_compile will create a new HyASTCompiler instance, which incurs Hy standard library setup costs each time. When calls to these functions occur more than once (e.g. by use of HyREPL and eval-and-compile), the aforementioned work is performed unnecessarily for the same module. These changes avoid that unneeded and error-prone repetition.

• HyREPL calls hy_compile on every processed command #1698

[Kodiologist]

hy_eval is user-visible (as eval), so update its documentation in core.rst and NEWS.

[brandonwillard]

In order for a compiler instance to simply [re]use itself when calling hy_eval (e.g. during eval-and-compile), we need to make the compiler "stateless". Members like imports and temp_if attach state to a compiler instance that don't allow its immediate reuse.

To replace imports, my plan of attack involves more consistent tracking/use of Result objects and the imports in those. Basically, we can—and should—keep all "state" within Result objects. This looks like it will involve an interface-like change to the compiler, though; namely, that all HyASTCompiler.compile_* methods must return and take Result objects. While that's often the case (well, the former at least), it's not always true, and this inconsistency isn't good for the compiler's design.

For instance, HyASTCompiler.compile_symbol doesn't return a Result object, but it can be made to do so, and—with that change—it can assign stdlib auto-imports to the current state of compilation (via Result.imports) instead of the actual compiler instance. Likewise, with that in place, we can make HyASTCompiler.imports_as_stmts produce AST for a Results's imports and return those in another Result object—bringing it more in-line with its adjacent compile_* methods. Additionally, the logic currently in hy_compile could then be moved to HyASTCompiler.imports_as_stmts and disentangle essential compilation logic from hy_compile.

At the very least, these changes would get us closer to a compiler that better isolates its functionality and is significantly easier to extend and debug.

Regarding HyASTCompiler.temp_if, I'm having a hard time understanding exactly what its functional "intentions" are and the exact scope/constraints of its role. I can see that it's attempting to avoid the unnecessary use/creation of temporary variables in an if*, but how it's using the compiler state to do this isn't very clear or well contained. I can't tell whether or not the addition of a temp_if field to Result would work, or if there's something much simpler that would avoid such an undesirable addition.

Somewhat related issues: #1542, #1482. Also, this comment in #1697.

[brandonwillard]

@Kodiologist, good call; will do.

[Kodiologist]

I'm afraid this isn't rebasing cleanly after merging #1699. Could you take a look?

[brandonwillard]

Yeah, one minute.

[brandonwillard]

OK, that should do it.

[Kodiologist]

Nice. One more thing: the documentation of eval in core.rst. I know, it's a problem that we have both docstrings and the redundant documentation in those files.

[brandonwillard]

Didn't notice that; I'll update it.

Otherwise, it seems like we should really get started on #1044.

[Kodiologist]

Replace "fifth" with "fourth" and I'll merge.

Yep.