TypeError: bytes or integer address expected instead of str instance

[ppwwyyxx]

Describe the bug
bps.push_pull on a naive tensor fails.

To Reproduce

In [1]: import tensorflow as tf

In [2]: import byteps.tensorflow as bps
WARNING: Logging before flag parsing goes to stderr.
W0627 11:36:47.010180 139917697820480 deprecation_wrapper.py:119] From /private/home/yuxinwu/.local/lib/python3.6/site-packages/byteps/tensorflow/__init__.py:79: The name tf.train.SessionRunHook is deprecated. Please use tf.estimator.SessionRunHook instead.

W0627 11:36:47.010504 139917697820480 deprecation_wrapper.py:119] From /private/home/yuxinwu/.local/lib/python3.6/site-packages/byteps/tensorflow/__init__.py:111: The name tf.train.Optimizer is deprecated. Please use tf.compat.v1.train.Optimizer instead.


In [3]: bps.push_pull(tf.constant([0.0]))
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-3-fc0e9eb4f2bb> in <module>()
----> 1 bps.push_pull(tf.constant([0.0]))

~/.local/lib/python3.6/site-packages/byteps/tensorflow/__init__.py in push_pull(tensor, scope, average, device_dense, device_sparse, compression)
     50         byteps_size = tf.cast(size(), dtype=tensor.dtype)
     51         tensor_compressed, ctx = compression.compress(tensor)
---> 52         summed_tensor_compressed = _push_pull(tensor_compressed, scope)
     53         summed_tensor = compression.decompress(summed_tensor_compressed, ctx)
     54         new_tensor = (tf.div(summed_tensor, byteps_size)

~/.local/lib/python3.6/site-packages/byteps/tensorflow/ops.py in _push_pull(tensor, scope, name)
     80     if name is None and not _executing_eagerly():
     81         name = 'BytePSPushPull_%s' % _normalize_name(tensor.name)
---> 82     TF_LIB_CTYPES.byteps_tensorflow_declare_tensor(ctypes.c_char_p(scope+name))
     83     return C_LIB.byteps_push_pull(tensor, name=name)
     84 

TypeError: bytes or integer address expected instead of str instance

Environment (please complete the following information):

• OS: ubuntu 18.04
• GCC version:
• CUDA and NCCL version:
• Framework (TF, PyTorch, MXNet): TF 1.14.0-rc1

[ppwwyyxx]

Looks like this issue is specific to Python 3:

In [2]: import ctypes

In [3]: ctypes.c_char_p("a")
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-3-741a469a5ca6> in <module>()
----> 1 ctypes.c_char_p("a")

TypeError: bytes or integer address expected instead of str instance

In [4]: ctypes.c_char_p(b"a")
Out[4]: c_char_p(140626871246832)

[bobzhuyb]

Thanks for localizing it. This should be fixed after the below is merged.
#12

[changlan]

#12 has been merged. @ppwwyyxx would you mind checking if this issue is fixed?

[ppwwyyxx]

Thanks! It works well

[jovepy]

I am having the same error