Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] User defined graph analysis algorithm error #2610

Open
ilovecomet opened this issue Apr 17, 2023 · 3 comments
Open

[BUG] User defined graph analysis algorithm error #2610

ilovecomet opened this issue Apr 17, 2023 · 3 comments
Assignees
@lidongze0629
Labels
bug Something isn't working component:gae component:python

Comments

@ilovecomet
Copy link

ilovecomet commented Apr 17, 2023 •
edited by sighingnow
Loading

Describe the bug
Using the official custom algorithm, an error occurred during operation

To Reproduce
Steps to reproduce the behavior:
code:

from graphscope.analytical.udf.decorators import pregel
from graphscope.framework.app import AppAssets

# 装饰器, 定义顶点数据和消息数据的类型
@pregel(vd_type="double", md_type="double")
class SSSP_Pregel(AppAssets):
    @staticmethod
    def Init(v, context):
        v.set_value(1000000000.0)

    @staticmethod
    def Compute(messages, v, context):
        src_id = context.get_config(b"src")
        cur_dist = v.value()
        new_dist = 1000000000.0
        if v.id() == src_id:
            new_dist = 0
        for message in messages:
            new_dist = min(message, new_dist)
        if new_dist < cur_dist:
            v.set_value(new_dist)
            for e_label_id in range(context.edge_label_num()):
                edges = v.outgoing_edges(e_label_id)
                for e in edges:
                    v.send(e.vertex(), new_dist + e.get_int(2))
        v.vote_to_halt()

    @staticmethod
    def Combine(messages):
        ret = 1000000000.0
        for m in messages:
            ret = min(ret, m)
        return ret

error:

2023-04-17 17:36:24,719 [ERROR][utils:276]: Failed to run command: cmake . -DNETWORKX=ON -DCMAKE_PREFIX_PATH='/usr/local/lib/python3.8/dist-packages/graphscope.runtime;/usr/local/lib/python3.8/dist-packages/graphscope.runtime/openmpi' -DCYTHON_PREGEL_APP=ON -DENABLE_PREGEL_COMBINE=ON, error message is: CMake Error at /usr/local/share/cmake-3.23/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
  Could NOT find Python (missing: Python_INCLUDE_DIRS Python_LIBRARIES
  Development Development.Module Development.Embed) (found version "2.7.17")
Call Stack (most recent call first):
  /usr/local/share/cmake-3.23/Modules/FindPackageHandleStandardArgs.cmake:594 (_FPHSA_FAILURE_MESSAGE)
  /usr/local/share/cmake-3.23/Modules/FindPython.cmake:561 (find_package_handle_standard_args)
  CMakeLists.txt:156 (find_package)



2023-04-17 17:36:24,751 [ERROR][rpc:188]: Runstep failed with code: ANALYTICAL_ENGINE_INTERNAL_ERROR, message: Error occurred during RunStep, The traceback is: Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/gscoordinator/utils.py", line 361, in _compile_on_local
    logger.debug(run_command(command, cwd=workdir))
  File "/usr/local/lib/python3.8/dist-packages/gscoordinator/utils.py", line 281, in run_command
    raise RuntimeError(f"Failed to run command: {args}, err: {err}")
RuntimeError: Failed to run command: cmake . -DNETWORKX=ON -DCMAKE_PREFIX_PATH='/usr/local/lib/python3.8/dist-packages/graphscope.runtime;/usr/local/lib/python3.8/dist-packages/graphscope.runtime/openmpi' -DCYTHON_PREGEL_APP=ON -DENABLE_PREGEL_COMBINE=ON, err: CMake Error at /usr/local/share/cmake-3.23/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
  Could NOT find Python (missing: Python_INCLUDE_DIRS Python_LIBRARIES
  Development Development.Module Development.Embed) (found version "2.7.17")
Call Stack (most recent call first):
  /usr/local/share/cmake-3.23/Modules/FindPackageHandleStandardArgs.cmake:594 (_FPHSA_FAILURE_MESSAGE)
  /usr/local/share/cmake-3.23/Modules/FindPython.cmake:561 (find_package_handle_standard_args)
  CMakeLists.txt:156 (find_package)




The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/gscoordinator/coordinator.py", line 309, in _RunStep
    head, bodies = self._operation_executor.run_on_analytical_engine(
  File "/usr/local/lib/python3.8/dist-packages/gscoordinator/monitor.py", line 193, in runOnAnalyticalEngineWarp
    res = func(instance, dag_def, dag_bodies, loader_op_bodies)
  File "/usr/local/lib/python3.8/dist-packages/gscoordinator/op_executor.py", line 167, in run_on_analytical_engine
    dag_def, dag_bodies = self.pre_process(dag_def, dag_bodies, loader_op_bodies)
  File "/usr/local/lib/python3.8/dist-packages/gscoordinator/op_executor.py", line 144, in pre_process
    op, _, _ = self._maybe_compile_app(op)
  File "/usr/local/lib/python3.8/dist-packages/gscoordinator/op_executor.py", line 257, in _maybe_compile_app
    compiled_path = self._compile_lib_and_distribute(
  File "/usr/local/lib/python3.8/dist-packages/gscoordinator/op_executor.py", line 395, in _compile_lib_and_distribute
    lib_path, java_jar_path, java_ffi_path, app_type = compile_func(
  File "/usr/local/lib/python3.8/dist-packages/gscoordinator/utils.py", line 540, in compile_app
    lib_path = compile_library(commands, library_dir, library_name, launcher)
  File "/usr/local/lib/python3.8/dist-packages/gscoordinator/utils.py", line 311, in compile_library
    return _compile_on_local(commands, workdir, output_name)
  File "/usr/local/lib/python3.8/dist-packages/gscoordinator/utils.py", line 368, in _compile_on_local
    raise CompilationError(
graphscope.framework.errors.CompilationError: Failed to compile f50bd1a6e93c0147cc21f356cd7595dbe1530149785c69a5b2b72efc7146b635 on platform system: Linux
machine: x86_64
platform: Linux-4.15.0-200-generic-x86_64-with-glibc2.27
uname: uname_result(system='Linux', node='c9c8c977cf55', release='4.15.0-200-generic', version='#211-Ubuntu SMP Thu Nov 24 18:16:04 UTC 2022', machine='x86_64', processor='x86_64')
kernel_ver: #211-Ubuntu SMP Thu Nov 24 18:16:04 UTC 2022
mac_ver: ('', ('', '', ''), '')
gcc_ver: b'gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0\nCopyright (C) 2017 Free Software Foundation, Inc.\nThis is free software; see the source for copying conditions.  There is NO\nwarranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.\n\n'
python_ver: 3.8.14


Traceback (most recent call last):
  File "/root/demo/code/graph_analysis.py", line 29, in <module>
    sssp_context = my_alg_pie(simple_graph, src=6)
  File "/usr/local/lib/python3.8/dist-packages/graphscope/analytical/udf/wrapper.py", line 63, in call
    return app_assets(graph, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/graphscope/framework/app.py", line 321, in __call__
    app_ = graph.session._wrapper(AppDAGNode(graph, self))
  File "/usr/local/lib/python3.8/dist-packages/graphscope/client/session.py", line 941, in _wrapper
    return self.run(dag_node)
  File "/usr/local/lib/python3.8/dist-packages/graphscope/client/session.py", line 977, in run
    return self.run_fetches(fetches)
  File "/usr/local/lib/python3.8/dist-packages/graphscope/client/session.py", line 987, in run_fetches
    response = self._grpc_client.run(fetch_handler.targets)
  File "/usr/local/lib/python3.8/dist-packages/graphscope/client/rpc.py", line 98, in run
    return self._run_step_impl(dag_def)
  File "/usr/local/lib/python3.8/dist-packages/graphscope/client/utils.py", line 156, in with_grpc_catch
    return fn(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/graphscope/client/rpc.py", line 198, in _run_step_impl
    raise exc
graphscope.framework.errors.CompilationError: Failed to compile f50bd1a6e93c0147cc21f356cd7595dbe1530149785c69a5b2b72efc7146b635 on platform system: Linux
machine: x86_64
platform: Linux-4.15.0-200-generic-x86_64-with-glibc2.27
uname: uname_result(system='Linux', node='c9c8c977cf55', release='4.15.0-200-generic', version='#211-Ubuntu SMP Thu Nov 24 18:16:04 UTC 2022', machine='x86_64', processor='x86_64')
kernel_ver: #211-Ubuntu SMP Thu Nov 24 18:16:04 UTC 2022
mac_ver: ('', ('', '', ''), '')
gcc_ver: b'gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0\nCopyright (C) 2017 Free Software Foundation, Inc.\nThis is free software; see the source for copying conditions.  There is NO\nwarranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.\n\n'
python_ver: 3.8.14

Environment (please complete the following information):

  • GraphScope version: [0.19.0]
  • OS: [ Linux]
@wenyuanyu
Copy link
Collaborator

@lidongze0629 Do you have time to look at this?

@lidongze0629
Copy link
Collaborator

I will take a look at together with this discussion.

@lidongze0629
Copy link
Collaborator

@ilovecomet Hi, from your error message, I think the wrong Python version(2.7.17) was found during the compilation. Could you please try the latest Graphscope, and copy the running code if this problem still exists, thanks very much.

pip3 install graphscope -U

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@lidongze0629 lidongze0629

Labels
bug Something isn't working component:gae component:python
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@sighingnow @wenyuanyu @lidongze0629 @ilovecomet