Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[<Ray component: Core|RLlib|etc...>] Support for Gradio version 4 on Ray Serve #49245

Open
SuryaaSeran opened this issue Dec 13, 2024 · 0 comments
Labels
enhancement Request for new feature and/or capability serve Ray Serve Related Issue triage Needs triage (eg: priority, bug/not-bug, and owning component)

Comments

@SuryaaSeran
Copy link

Description

Currently, we are facing an issue with using Gradio v4 on Ray 2.40.0 (latest version). Could you please help us with a fix to support the latest Gradio version, as many of our LLM demos require features from Gradio v4? If there are any workarounds, please do let us know. It would be great if you could update the documentation as well (https://docs.ray.io/en/latest/serve/tutorials/gradio-integration.html). I have pasted the error logs below.

The deployment failed to start 8 times in a row. This may be due to a problem with its constructor or initial health check failing. See controller logs for details. Retrying after 32.0 seconds. Error:
[36mray::ServeReplica:xyz_app:GradioUI.initialize_and_get_metadata()[39m (pid=771, ip=100.64.50.78, actor_id=1feda3fd4a666b60ece7612a01000000, repr=<ray.serve._private.replica.ServeReplica:conversational_search_app:GradioUI object at 0x7f23bdfed370>)
  File "/home/ray/anaconda3/lib/python3.8/concurrent/futures/_base.py", line 437, in result
    return self.__get_result()
  File "/home/ray/anaconda3/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result
    raise self._exception
  File "/home/ray/anaconda3/lib/python3.8/site-packages/ray/serve/_private/replica.py", line 455, in initialize_and_get_metadata
    raise RuntimeError(traceback.format_exc()) from None
RuntimeError: Traceback (most recent call last):
  File "/home/ray/anaconda3/lib/python3.8/site-packages/ray/serve/_private/replica.py", line 443, in initialize_and_get_metadata
    await self._initialize_replica()
  File "/home/ray/anaconda3/lib/python3.8/site-packages/ray/serve/_private/replica.py", line 182, in initialize_replica
    await sync_to_async(_callable.__init__)(*init_args, **init_kwargs)
  File "/tmp/ray/session_2024-12-09_15-16-36_544792_8/runtime_resources/working_dir_files/xyz/gradio_ui.py", line 57, in __init__
    submit_button.click(
  File "/tmp/ray/session_2024-12-09_15-16-36_544792_8/runtime_resources/pip/b6b53b3c285b9689fa850df780274a829a8f55a8/virtualenv/lib/python3.8/site-packages/gradio/events.py", line 550, in event_trigger
    return Dependency(block, dep.get_config(), dep_index, fn, timer)
  File "/tmp/ray/session_2024-12-09_15-16-36_544792_8/runtime_resources/pip/b6b53b3c285b9689fa850df780274a829a8f55a8/virtualenv/lib/python3.8/site-packages/gradio/blocks.py", line 581, in get_config
    "inputs": [block._id for block in self.inputs],
  File "/tmp/ray/session_2024-12-09_15-16-36_544792_8/runtime_resources/pip/b6b53b3c285b9689fa850df780274a829a8f55a8/virtualenv/lib/python3.8/site-packages/gradio/blocks.py", line 581, in <listcomp>
    "inputs": [block._id for block in self.inputs],
AttributeError: 'RunnableWithMessageHistory' object has no attribute '_id'

Use case

Please support Gradio v4 on latest Ray Serve (for LLM demos)

@SuryaaSeran SuryaaSeran added enhancement Request for new feature and/or capability triage Needs triage (eg: priority, bug/not-bug, and owning component) labels Dec 13, 2024
@jcotant1 jcotant1 added the serve Ray Serve Related Issue label Dec 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Request for new feature and/or capability serve Ray Serve Related Issue triage Needs triage (eg: priority, bug/not-bug, and owning component)
Projects
None yet
Development

No branches or pull requests

2 participants