-
Notifications
You must be signed in to change notification settings - Fork 7k
Closed
Labels
P1Issue that should be fixed within a few weeksIssue that should be fixed within a few weeksbugSomething that is supposed to be working; but isn'tSomething that is supposed to be working; but isn'tcoreIssues that should be addressed in Ray CoreIssues that should be addressed in Ray Core
Milestone
Description
What happened + What you expected to happen
Traceback (most recent call last):
File "/usr/local/bin/ray", line 8, in <module>
sys.exit(main())
File "/usr/local/lib/python3.7/site-packages/ray/scripts/scripts.py", line 2339, in main
return cli()
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1128, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1053, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1659, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1395, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 754, in invoke
return __callback(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/ray/autoscaler/_private/cli_logger.py", line 852, in wrapper
return f(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/ray/scripts/scripts.py", line 873, in start
ray_params, head=False, shutdown_at_exit=block, spawn_reaper=block
File "/usr/local/lib/python3.7/site-packages/ray/node.py", line 266, in __init__
self._ray_params.update_pre_selected_port()
File "/usr/local/lib/python3.7/site-packages/ray/_private/parameter.py", line 323, in update_pre_selected_port
f"Ray component {comp} is trying to use "
ValueError: Ray component worker_ports is trying to use a port number 17507 that is used by other components.
Port information: {'gcs': 'random', 'object_manager': 'random', 'node_manager': 'random', 'gcs_server': 'random', 'client_server': 10001, 'dashboard': 8265, 'dashboard_agent_grpc': 57208, 'dashboard_agent_http': 'random', 'metrics_export': 17507, 'redis_shards': 'random', 'worker_ports': '9998 ports from 10002 to 19999'}
If you allocate ports, please make sure the same port is not used by multiple components.
From error information,'metrics_export' uses port 17507, this is because if user does not set a port for metrics_export,it will be selected by: new_port = random.randint(port, 65535),L1, L2。If this random number happens to be in range: worker_ports[10002, 19999], in later code L3, L4, where it checks if there is a port being used repeatedly, an error will be thrown up。
Versions / Dependencies
nightly version
Reproduction script
No.
Issue Severity
Medium: It is a significant difficulty but I can work around it.
Metadata
Metadata
Assignees
Labels
P1Issue that should be fixed within a few weeksIssue that should be fixed within a few weeksbugSomething that is supposed to be working; but isn'tSomething that is supposed to be working; but isn'tcoreIssues that should be addressed in Ray CoreIssues that should be addressed in Ray Core