Command substitution: special characters inside XML comments causes launch to fail #136

jacobperron · 2020-02-05T20:18:58Z

Bug report

Required Info:

Operating System:
- Ubuntu 18.04
Installation type:
- from source
Version or commit hash:
- 4333910ba41977f3ab9867082c86a8698ada3a16

Specifically, I've noticed that colons (:) and tab characters are not working.

Steps to reproduce issue

I ran into this issue trying to pass a URDF from xacro to robot_state_publisher, like the example here. For some reason, if the URDF/xacro file contains a colon (:) or a tab character inside of an XML comment, then launch fails.

Here's my example URDF file foo.urdf:

<?xml version="1.0" ?>
<robot>
  <!--: -->
</robot>

And my launch file:

<launch>
  <node pkg="demo_nodes_cpp" exec="talker" output="screen">
    <param name="foo" value="$(command 'cat foo.urdf')" />
  </node>
</launch>

Expected behavior

The launch file succeeds and we see the talker node chatting.

Actual behavior

Error:

[ERROR] [launch]: Caught exception in launch (see debug for traceback): mapping values are not allowed here
  in "<unicode string>", line 3, column 7:
      <!--: -->
          ^

Additional information

The error seems to specifically occur if there is a colon followed by a space. E.g. making the following change works fine:

-      <!--: -->
+      <!--:-->

Replacing the colcon with a tab character also triggers the same error:

[ERROR] [launch]: Caught exception in launch (see debug for traceback): while scanning for the next token
found character '\t' that cannot start any token
  in "<unicode string>", line 3, column 7:
      <!--	-->
          ^

The text was updated successfully, but these errors were encountered:

ivanpauno · 2020-02-05T21:00:13Z

That seems extremely weird.
Can you run launch in debug mode (-d) and share the traceback?

hidmic · 2020-02-05T21:04:19Z

Wild guess: we're YAML parsing command output. Try adding quotes before and after the command substitution.

ivanpauno · 2020-02-05T21:11:31Z

Wild guess: we're YAML parsing command output. Try adding quotes before and after the command substitution.

Ouch! Yes, that's true, parameters values that are strings are parsed like yaml.
There's an issue about that #74 (I think I opened an issue proposing typed substitutions, but I haven't found it).
At the moment, I agree with Mich that the best solution is adding extra quotes.

jacobperron · 2020-02-05T21:55:50Z

Do you mean like this?

<launch>
  <node pkg="demo_nodes_cpp" exec="talker" output="screen">
    <param name="foo" value="'$(command 'cat foo.urdf')'" />
  </node>
</launch>

I get the same error.

jacobperron · 2020-02-05T22:07:16Z

Here's an example output with --debug:

[DEBUG] [launch.launch_context]: emitting event synchronously: 'launch.events.IncludeLaunchDescription'
[DEBUG] [launch.launch_context]: emitting event synchronously: 'launch.events.IncludeLaunchDescription'
[INFO] [launch]: All log files can be found below /home/jacob/.ros/log/2020-02-05-14-06-28-231429-warner-24708
[INFO] [launch]: Default logging verbosity is set to DEBUG
[DEBUG] [launch]: processing event: '<launch.events.include_launch_description.IncludeLaunchDescription object at 0x7f6fac362630>'
[DEBUG] [launch]: processing event: '<launch.events.include_launch_description.IncludeLaunchDescription object at 0x7f6fac362630>' ✓ '<launch.event_handlers.on_include_launch_description.OnIncludeLaunchDescription object at 0x7f6fac380fd0>'
[DEBUG] [launch]: processing event: '<launch.events.include_launch_description.IncludeLaunchDescription object at 0x7f6fac362748>'
[DEBUG] [launch]: processing event: '<launch.events.include_launch_description.IncludeLaunchDescription object at 0x7f6fac362748>' ✓ '<launch.event_handlers.on_include_launch_description.OnIncludeLaunchDescription object at 0x7f6fac380fd0>'
[DEBUG] [launch]: Traceback (most recent call last):
  File "/home/jacob/ws/latest_ws/install/launch/lib/python3.6/site-packages/launch/launch_service.py", line 385, in run_async
    await process_one_event_task
  File "/usr/lib/python3.6/asyncio/coroutines.py", line 126, in send
    return self.gen.send(value)
  File "/home/jacob/ws/latest_ws/install/launch/lib/python3.6/site-packages/launch/launch_service.py", line 295, in _process_one_event
    await self.__process_event(next_event)
  File "/usr/lib/python3.6/asyncio/coroutines.py", line 110, in __next__
    return self.gen.send(None)
  File "/home/jacob/ws/latest_ws/install/launch/lib/python3.6/site-packages/launch/launch_service.py", line 315, in __process_event
    visit_all_entities_and_collect_futures(entity, self.__context))
  File "/home/jacob/ws/latest_ws/install/launch/lib/python3.6/site-packages/launch/utilities/visit_all_entities_and_collect_futures_impl.py", line 45, in visit_all_entities_and_collect_futures
    futures_to_return += visit_all_entities_and_collect_futures(sub_entity, context)
  File "/home/jacob/ws/latest_ws/install/launch/lib/python3.6/site-packages/launch/utilities/visit_all_entities_and_collect_futures_impl.py", line 45, in visit_all_entities_and_collect_futures
    futures_to_return += visit_all_entities_and_collect_futures(sub_entity, context)
  File "/home/jacob/ws/latest_ws/install/launch/lib/python3.6/site-packages/launch/utilities/visit_all_entities_and_collect_futures_impl.py", line 45, in visit_all_entities_and_collect_futures
    futures_to_return += visit_all_entities_and_collect_futures(sub_entity, context)
  [Previous line repeated 1 more time]
  File "/home/jacob/ws/latest_ws/install/launch/lib/python3.6/site-packages/launch/utilities/visit_all_entities_and_collect_futures_impl.py", line 38, in visit_all_entities_and_collect_futures
    sub_entities = entity.visit(context)
  File "/home/jacob/ws/latest_ws/install/launch/lib/python3.6/site-packages/launch/action.py", line 108, in visit
    return self.execute(context)
  File "/home/jacob/ws/latest_ws/install/launch_ros/lib/python3.6/site-packages/launch_ros/actions/node.py", line 350, in execute
    self._perform_substitutions(context)
  File "/home/jacob/ws/latest_ws/install/launch_ros/lib/python3.6/site-packages/launch_ros/actions/node.py", line 321, in _perform_substitutions
    evaluated_parameters = evaluate_parameters(context, self.__parameters)
  File "/home/jacob/ws/latest_ws/install/launch_ros/lib/python3.6/site-packages/launch_ros/utilities/evaluate_parameters.py", line 145, in evaluate_parameters
    output_params.append(evaluate_parameter_dict(context, param))
  File "/home/jacob/ws/latest_ws/install/launch_ros/lib/python3.6/site-packages/launch_ros/utilities/evaluate_parameters.py", line 70, in evaluate_parameter_dict
    yaml_evaluated_value = yaml.safe_load(evaluated_value)
  File "/home/jacob/.local/lib/python3.6/site-packages/yaml/__init__.py", line 162, in safe_load
    return load(stream, SafeLoader)
  File "/home/jacob/.local/lib/python3.6/site-packages/yaml/__init__.py", line 114, in load
    return loader.get_single_data()
  File "/home/jacob/.local/lib/python3.6/site-packages/yaml/constructor.py", line 41, in get_single_data
    node = self.get_single_node()
  File "/home/jacob/.local/lib/python3.6/site-packages/yaml/composer.py", line 36, in get_single_node
    document = self.compose_document()
  File "/home/jacob/.local/lib/python3.6/site-packages/yaml/composer.py", line 58, in compose_document
    self.get_event()
  File "/home/jacob/.local/lib/python3.6/site-packages/yaml/parser.py", line 118, in get_event
    self.current_event = self.state()
  File "/home/jacob/.local/lib/python3.6/site-packages/yaml/parser.py", line 193, in parse_document_end
    token = self.peek_token()
  File "/home/jacob/.local/lib/python3.6/site-packages/yaml/scanner.py", line 129, in peek_token
    self.fetch_more_tokens()
  File "/home/jacob/.local/lib/python3.6/site-packages/yaml/scanner.py", line 223, in fetch_more_tokens
    return self.fetch_value()
  File "/home/jacob/.local/lib/python3.6/site-packages/yaml/scanner.py", line 579, in fetch_value
    self.get_mark())
yaml.scanner.ScannerError: mapping values are not allowed here
  in "<unicode string>", line 3, column 7:
      <!--: -->
          ^

[ERROR] [launch]: Caught exception in launch (see debug for traceback): mapping values are not allowed here
  in "<unicode string>", line 3, column 7:
      <!--: -->
          ^
[DEBUG] [launch.launch_context]: emitting event: 'launch.events.Shutdown'
[DEBUG] [launch]: processing event: '<launch.events.shutdown.Shutdown object at 0x7f6fac38f198>'
[DEBUG] [launch]: processing event: '<launch.events.shutdown.Shutdown object at 0x7f6fac38f198>' ✓ '<launch.event_handlers.on_shutdown.OnShutdown object at 0x7f6fa8039b70>'
[DEBUG] [launch]: processing event: '<launch.events.shutdown.Shutdown object at 0x7f6fac38f198>' ✓ '<launch.event_handlers.on_shutdown.OnShutdown object at 0x7f6fac3629e8>'
Executing <Task finished coro=<LaunchService._process_one_event() done, defined at /home/jacob/ws/latest_ws/install/launch/lib/python3.6/site-packages/launch/launch_service.py:293> result=None created at /home/jacob/ws/latest_ws/install/launch/lib/python3.6/site-packages/launch/launch_service.py:363> took 0.994 seconds

ivanpauno · 2020-02-06T12:59:10Z

Do you mean like this?

TL;DR no, you need duplicated single quotes 🤦‍♂️

<launch>
  <node pkg="demo_nodes_cpp" exec="talker" output="screen">
    <param name="foo" value="''$(command 'cat foo.urdf')''" />
  </node>
</launch>

long version:
why? well, each value read from an xml file is coerced with some rules to float/int/bool/str.
One of the rules, it's that outer single quotes are implicit in an string. If you find explicit quotes, they are dropped (similar of what yaml does).
After that, the Node action itself is parsing the string like yaml ....

We should definitly fix this, before the problem gets proliferated in multiple launch files.
I think that the easier fix, it's to specify the type of the parameter (somehow), instead of using yaml rules in the implementation of Node action (see #74 (comment)).

ddengster · 2020-02-12T02:13:03Z

<launch>
  <node pkg="demo_nodes_cpp" exec="talker" output="screen">
      <param name="foo" value="$(command 'pwd')" />
  </node>
</launch>

So I'm facing something similar to this issue. Has $(command '..') been deployed for ros2/launch#375 ? It seems anything I'm trying with $(command '..') doesn't work. I've built ros2/launch within my colcon workspace and confirmed that ros2 launch commands runs with what I've built.

ivanpauno · 2020-02-13T17:46:29Z

Currently, the following should work:

<launch>
  <node pkg="demo_nodes_cpp" exec="talker" output="screen">
      <param name="foo" value="''$(command 'pwd')''" />
  </node>
</launch>

mogumbo · 2020-03-09T21:41:41Z

I just got back to looking at this. It almost works for me. I'm doing:

<node name="robot_state_publisher" pkg="robot_state_publisher" exec="robot_state_publisher">
    <param name="robot_description" value="''$(command '$(find-pkg-prefix xacro)/bin/xacro $(find-pkg-share krex2_description)/urdf/rp1a_robot.xacro')''"/>
</node>

Which gives me this output:

[ERROR] [launch]: Caught exception in launch (see debug for traceback): expected '<document start>', but found '<scalar>'
  in "<unicode string>", line 96, column 11:
            I'm not sure if just specifying tw ... 
              ^

If I remove all the single-quotes from the comments in my xacro file, I can launch my robot. Any thoughts on what can be done about this?

ivanpauno · 2020-03-10T20:40:33Z

Which gives me this output:

[ERROR] [launch]: Caught exception in launch (see debug for traceback): expected '<document start>', but found '<scalar>'
  in "<unicode string>", line 96, column 11:
            I'm not sure if just specifying tw ... 
              ^

🤦‍♂️ That's a fun one.

If I remove all the single-quotes from the comments in my xacro file, I can launch my robot. Any thoughts on what can be done about this?

I will describe first the problem. The value of Node action parameters argument are read like yaml. That was proposed originally in #31, because substitutions are only resolved to a str.
#33 limited in which cases a parameter value was read like yaml (only if the original provided value was a substitution, avoided when it was a explicitly passed string).
To solve the problem, my idea was to stop automatically doing the yaml load, and provide an explicit way of specifying the parameter type (one of the options could be to use yaml to infer the type).

ivanpauno · 2020-03-10T20:41:54Z

To solve the problem, my idea was to stop automatically doing the yaml load, and provide an explicit way of specifying the parameter type (one of the options could be to use yaml to infer the type).

I've actually started to work in a fix , though I haven't had time to finish it.

ivanpauno · 2020-03-24T20:55:31Z

Moving to launch_ros, as the origin of the problem is that parameters Node argument is implicitly loading passed values like yaml.

ivanpauno · 2020-08-05T16:45:04Z

This can be fixed now by avoiding using yaml conversions:

<launch>
  <node pkg="demo_nodes_cpp" exec="talker" output="screen">
    <param name="foo" value="$(command 'cat foo.urdf')" type="str"/>
  </node>
</launch>

The fix was introduced in #137.

ivanpauno self-assigned this Mar 10, 2020

ivanpauno transferred this issue from ros2/launch Mar 24, 2020

ivanpauno added the bug Something isn't working label Mar 24, 2020

ivanpauno mentioned this issue Mar 24, 2020

Fix problems when parsing a Command Substitution as a parameter value #137

Merged

ivanpauno closed this as completed Aug 5, 2020

v-lopez mentioned this issue Jan 20, 2021

Evaluate parameters exception loading URDF #214

Closed

oKermorgant mentioned this issue Apr 17, 2021

Remove apostrophes in comments to workaround ros2/launch_ros#214 Liquid-ai/Plankton#32

Merged

relffok mentioned this issue Nov 26, 2021

Add support for Gazebo and Ignition simulation UniversalRobots/Universal_Robots_ROS2_Driver#245

Closed

5 tasks

shonigmann mentioned this issue Jan 27, 2022

Parameter robot_description as file path, not URDF string ros/robot_state_publisher#155

Closed

gavanderhoorn mentioned this issue Oct 2, 2023

launch_xml param tag, value attribute parsed as yaml ros2/launch#729

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Command substitution: special characters inside XML comments causes launch to fail #136

Command substitution: special characters inside XML comments causes launch to fail #136

jacobperron commented Feb 5, 2020

ivanpauno commented Feb 5, 2020 •

edited

Loading

hidmic commented Feb 5, 2020

ivanpauno commented Feb 5, 2020

jacobperron commented Feb 5, 2020

jacobperron commented Feb 5, 2020

ivanpauno commented Feb 6, 2020 •

edited

Loading

ddengster commented Feb 12, 2020

ivanpauno commented Feb 13, 2020 •

edited

Loading

mogumbo commented Mar 9, 2020 •

edited by jacobperron

Loading

ivanpauno commented Mar 10, 2020

ivanpauno commented Mar 10, 2020

ivanpauno commented Mar 24, 2020

ivanpauno commented Aug 5, 2020

Command substitution: special characters inside XML comments causes launch to fail #136

Command substitution: special characters inside XML comments causes launch to fail #136

Comments

jacobperron commented Feb 5, 2020

Bug report

Steps to reproduce issue

Expected behavior

Actual behavior

Additional information

ivanpauno commented Feb 5, 2020 • edited Loading

hidmic commented Feb 5, 2020

ivanpauno commented Feb 5, 2020

jacobperron commented Feb 5, 2020

jacobperron commented Feb 5, 2020

ivanpauno commented Feb 6, 2020 • edited Loading

ddengster commented Feb 12, 2020

ivanpauno commented Feb 13, 2020 • edited Loading

mogumbo commented Mar 9, 2020 • edited by jacobperron Loading

ivanpauno commented Mar 10, 2020

ivanpauno commented Mar 10, 2020

ivanpauno commented Mar 24, 2020

ivanpauno commented Aug 5, 2020

ivanpauno commented Feb 5, 2020 •

edited

Loading

ivanpauno commented Feb 6, 2020 •

edited

Loading

ivanpauno commented Feb 13, 2020 •

edited

Loading

mogumbo commented Mar 9, 2020 •

edited by jacobperron

Loading