Optimize as-moses

[ngeiswei]

Overview

Currently asmoses is about 1.35x slower when atomspace port is enabled
without subprogram memoization, and 1.15x slower with subprogram
memoization, based on the following runs

[nilg@laptop as-moses]$ time asmoses -c1 -Hpa -k5 -m10000 --atomspace-port=0 --atomspace-store=1
-10 or(and(or(and($2 !$3) $5) or(and($2 !$3 $5) !$1 !$4) or($1 $4)) and(or(!$1 !$4 $5) !$2 $3)) 

real	0m7.215s
user	0m7.178s
sys	0m0.015s
[nilg@laptop as-moses]$ time asmoses -c1 -Hpa -k5 -m10000 --atomspace-port=1 --atomspace-store=1
-10 or(and(or(and($2 !$3) $5) or(and($2 !$3 $5) !$1 !$4) or($1 $4)) and(or(!$1 !$4 $5) !$2 $3)) 

real	0m8.328s
user	0m8.267s
sys	0m0.050s
[nilg@laptop as-moses]$ time asmoses -c1 -Hpa -k5 -m10000 --atomspace-port=1 --atomspace-store=0
-10 or(and(or(and($2 !$3) $5) or(and($2 !$3 $5) !$1 !$4) or($1 $4)) and(or(!$1 !$4 $5) !$2 $3)) 

real	0m9.708s
user	0m9.660s
sys	0m0.024s

What we would want is to have moses run at least as fast when
atomspace-port is enabled, and faster when subprogram memoization is
enabled.

Benchmark

First we need to define some benchmarks to measure asmoses speed. I have added the following script

https://github.com/singnet/as-moses/blob/master/scripts/benchmark/asmoses-bm.sh

as well as some usage example

https://github.com/singnet/as-moses/blob/master/scripts/benchmark/mb-example.sh

asmoses-bm.sh merely runs asmoses N times and return the mean
execution time. This can be useful when tweaking asmoses to avoid
confounding speed-up and system jitters.

I'm not sure what good benchmarks would be, maybe

1. a set of demo problems (as in mb-example.sh)
2. a set of data fitting problems, I think we need bigger data sets
   than what is available in the utests. I'll try to dig a few
   interesting ones.

It would be good to record the results in a diary file (maybe in the
same benchmark folder) to keep track of the impact of
optimizations.

Profiling

To my knowledge the best FOSS profiling tool is valgrind combined with
kcachegrind. It's very useful to find where the CPU is spending its
time.

Static Analysis

cppcheck can be used to discover inefficiencies in the code. I think
to analyze the whole code, the following command is enough (from the
asmoses root folder)

cppcheck --language=c++ --enable=all .

However it wasn't able to detect simple optimizations like passing
value by reference instead of copy in

https://github.com/singnet/as-moses/blob/master/opencog/atomese/interpreter/logical_interpreter.h#L89-L90

so beware, nothing that is available is gonna beat human ingenuity.

Initial Suggestions

I've already identify 2 optimizations that I believe can have an
impact on performance

1. Passing Value by const reference like in logical_interpreter
2. Don't repopulate the features if already there in
   complete_truth_table::populate

there are many more places for sure, but after profiling you'll get a
better idea of where to focus on.

[ngeiswei]

Here's some more specific guidance to use valgrind.

1. Call callgrind on asmoses with some test problem

valgrind --tool=callgrind asmoses -Hpa -k4 -m10000 --atomspace-port=1

This will produce a file looking like callgrind.out.3594.
2. Call kcachegrind on that file

kcachegrind callgrind.out.3594

3. On the left-hand side of the kcachegrind GUI look for the function local_moses. Then you'll the call tree with most functions involved in running moses.

Retry with different parameters, such k=3 to k=10, and m=100 to m=100000. (well valgrind might take too long when the values of k and m are too high, abort is that the case). Also compare with and without --atomspace-port=1, see if you find any significant differences.

After exploring that, then we can look into profiling with more real world problems.