Use source code in hash where possible (fix #2089)

[AdamGleave]

What do these changes do?

Rather than attempting to infer when source code for a function is available, we always try and omit it only if we find an inspection.

Related issue number

Fixes #2089 and maybe #1446 (do not have the setup to confirm)

[AmplabJenkins]

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/5466/
Test FAILed.

[AdamGleave]

@AmplabJenkins build seems to have failed due to lack of resources on the cluster, is there a way to poke it to retry?

[AdamGleave]

retest this please

[robertnishihara]

This doesn't seem to work for me in a Python 2 interpreter. It raises an IOError, which is related, but slightly different.

22:05 $ python2
Python 2.7.13 |Anaconda 4.3.1 (x86_64)| (default, Dec 20 2016, 23:05:08) 
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org
>>> import inspect
>>> def f():
...     pass
... 
>>> try:
...     inspect.getsource(f)
... except OSError as e:
...     pass
... 
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  File "/Users/rkn/anaconda2/lib/python2.7/inspect.py", line 700, in getsource
    lines, lnum = getsourcelines(object)
  File "/Users/rkn/anaconda2/lib/python2.7/inspect.py", line 689, in getsourcelines
    lines, lnum = findsource(object)
  File "/Users/rkn/anaconda2/lib/python2.7/inspect.py", line 537, in findsource
    raise IOError('could not get source code')
IOError: could not get source code

[AdamGleave]

Ah yes, this changed in Python 3.3 it seems: https://docs.python.org/3/library/inspect.html

I'll change it to catch both for compatibility.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/5468/
Test PASSed.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/5471/
Test PASSed.

[robertnishihara]

Looks like inspect.getsource actually raises a TypeError when run on a Cython function (see error below). I pushed a fix I think.

======================================================================
ERROR: test_numpy (__main__.CythonTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/cython_test.py", line 51, in test_numpy
    result = get_ray_result(cyth.masked_log, array)
  File "test/cython_test.py", line 13, in get_ray_result
    func = ray.remote(cython_func)
  File "/home/travis/.local/lib/python3.6/site-packages/ray-0.4.0-py3.6-linux-x86_64.egg/ray/worker.py", line 2647, in remote
    return make_decorator(worker=worker)(args[0])
  File "/home/travis/.local/lib/python3.6/site-packages/ray-0.4.0-py3.6-linux-x86_64.egg/ray/worker.py", line 2608, in decorator
    num_return_vals, max_calls)
  File "/home/travis/.local/lib/python3.6/site-packages/ray-0.4.0-py3.6-linux-x86_64.egg/ray/remote_function.py", line 74, in __init__
    self._function_id = compute_function_id(self._function).id()
  File "/home/travis/.local/lib/python3.6/site-packages/ray-0.4.0-py3.6-linux-x86_64.egg/ray/remote_function.py", line 33, in compute_function_id
    source = inspect.getsource(function).encode("ascii")
  File "/home/travis/miniconda/lib/python3.6/inspect.py", line 968, in getsource
    lines, lnum = getsourcelines(object)
  File "/home/travis/miniconda/lib/python3.6/inspect.py", line 955, in getsourcelines
    lines, lnum = findsource(object)
  File "/home/travis/miniconda/lib/python3.6/inspect.py", line 768, in findsource
    file = getsourcefile(object)
  File "/home/travis/miniconda/lib/python3.6/inspect.py", line 684, in getsourcefile
    filename = getfile(object)
  File "/home/travis/miniconda/lib/python3.6/inspect.py", line 666, in getfile
    'function, traceback, frame, or code object'.format(object))
TypeError: <cyfunction masked_log at 0x7fb078fb5778> is not a module, class, method, function, traceback, frame, or code object

======================================================================
ERROR: test_simple_func (__main__.CythonTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/cython_test.py", line 28, in test_simple_func
    self.assertEqualHelper(cyth.simple_func, 6, 1, 2, 3)
  File "test/cython_test.py", line 25, in assertEqualHelper
    self.assertEqual(get_ray_result(cython_func, *args), expected)
  File "test/cython_test.py", line 13, in get_ray_result
    func = ray.remote(cython_func)
  File "/home/travis/.local/lib/python3.6/site-packages/ray-0.4.0-py3.6-linux-x86_64.egg/ray/worker.py", line 2647, in remote
    return make_decorator(worker=worker)(args[0])
  File "/home/travis/.local/lib/python3.6/site-packages/ray-0.4.0-py3.6-linux-x86_64.egg/ray/worker.py", line 2608, in decorator
    num_return_vals, max_calls)
  File "/home/travis/.local/lib/python3.6/site-packages/ray-0.4.0-py3.6-linux-x86_64.egg/ray/remote_function.py", line 74, in __init__
    self._function_id = compute_function_id(self._function).id()
  File "/home/travis/.local/lib/python3.6/site-packages/ray-0.4.0-py3.6-linux-x86_64.egg/ray/remote_function.py", line 33, in compute_function_id
    source = inspect.getsource(function).encode("ascii")
  File "/home/travis/miniconda/lib/python3.6/inspect.py", line 968, in getsource
    lines, lnum = getsourcelines(object)
  File "/home/travis/miniconda/lib/python3.6/inspect.py", line 955, in getsourcelines
    lines, lnum = findsource(object)
  File "/home/travis/miniconda/lib/python3.6/inspect.py", line 768, in findsource
    file = getsourcefile(object)
  File "/home/travis/miniconda/lib/python3.6/inspect.py", line 684, in getsourcefile
    filename = getfile(object)
  File "/home/travis/miniconda/lib/python3.6/inspect.py", line 666, in getfile
    'function, traceback, frame, or code object'.format(object))
TypeError: <cyfunction simple_func at 0x7fb078fb5270> is not a module, class, method, function, traceback, frame, or code object

----------------------------------------------------------------------
Ran 3 tests in 5.992s

FAILED (errors=2)

travis_time:end:038101ed:start=1526624985818047460,finish=1526624992832056081,duration=7014008621
�[0K
�[31;1mThe command "python test/cython_test.py" exited with 1

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/5476/
Test PASSed.