[REVIEW]: SLAM Toolbox: SLAM for the dynamic world

[whedon]

Submitting author: @SteveMacenski (Steven Macenski)
Repository: https://github.com/SteveMacenski/slam_toolbox
Version: 2.3.0
Editor: @arfon
Reviewer: @mosteo, @carlosjoserg
Archive: 10.5281/zenodo.4749721

⚠️ JOSS reduced service mode ⚠️

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/92f7096f8cdf263c22dbcff980b785b2"><img src="https://joss.theoj.org/papers/92f7096f8cdf263c22dbcff980b785b2/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/92f7096f8cdf263c22dbcff980b785b2/status.svg)](https://joss.theoj.org/papers/92f7096f8cdf263c22dbcff980b785b2)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@mosteo & @carlosjoserg, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

1. Make sure you're logged in to your GitHub account
2. Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @arfon know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Review checklist for @mosteo

Conflict of interest

• I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

• I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• Repository: Is the source code for this software available at the repository url?
• License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• Contribution and authorship: Has the submitting author (@SteveMacenski) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
• Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines

Functionality

• Installation: Does installation proceed as outlined in the documentation?
• Functionality: Have the functional claims of the software been confirmed?
• Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
• Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
• A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• State of the field: Do the authors describe how this software compares to other commonly-used packages?
• Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
• References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

Review checklist for @carlosjoserg

Conflict of interest

• I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

• I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• Repository: Is the source code for this software available at the repository url?
• License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• Contribution and authorship: Has the submitting author (@SteveMacenski) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
• Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines

Functionality

• Installation: Does installation proceed as outlined in the documentation?
• Functionality: Have the functional claims of the software been confirmed?
• Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
• Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
• A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• State of the field: Do the authors describe how this software compares to other commonly-used packages?
• Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
• References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

[whedon]

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @mosteo, @carlosjoserg it looks like you're currently assigned to review this paper 🎉.

⚠️ JOSS reduced service mode ⚠️

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

⭐ Important ⭐

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

1. Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

2. You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf

[arfon]

@whedon generate pdf from branch joss

[whedon]

Attempting PDF compilation from custom branch joss. Reticulating splines etc...

[arfon]

@mosteo, @carlosjoserg - This is the review thread for the paper. All of our communications will happen here from now on.

Please read the "Reviewer instructions & questions" in the first comment above.

Both reviewers have checklists at the top of this thread (in that first comment) with the JOSS requirements. As you go over the submission, please check any items that you feel have been satisfied. There are also links to the JOSS reviewer guidelines.

The JOSS review is different from most other journals. Our goal is to work with the authors to help them meet our criteria instead of merely passing judgment on the submission. As such, the reviewers are encouraged to submit issues and pull requests on the software repository. When doing so, please mention #2783 so that a link is created to this thread (and I can keep an eye on what is happening). Please also feel free to comment and ask questions on this thread. In my experience, it is better to post comments/questions/suggestions as you come across them instead of waiting until you've reviewed the entire package.

We aim for the review process to be completed within about 4-6 weeks but please make a start well ahead of this as JOSS reviews are by their nature iterative and any early feedback you may be able to provide to the author will be very helpful in meeting this schedule.

[whedon]

👉📄 Download article proof 📄 View article proof on GitHub 📄 👈

[whedon]

👋 @carlosjoserg, please update us on how your review is going.

[whedon]

👋 @mosteo, please update us on how your review is going.

[SteveMacenski]

Hi, just wanted to touch base on this - any progress?

[arfon]

👋 @mosteo, @carlosjoserg - just checking in here to see how you're both getting on with your reviews?

[carlosjoserg]

Hi all, appologies for the delay, also got other reviews from IEEE that took me some time too... I hope within a week to finish this one. Best,

[mosteo]

Hi @arfon, for some reason I was not getting notifications from here. I'm going to review my settings to fix this for the future.

[mosteo]

Opened a PR with proofreading fixes: SteveMacenski/slam_toolbox#317

[mosteo]

Installation verified by installing the ros-foxy-slam-toolbox package.

[arfon]

👋 @mosteo, @carlosjoserg - happy new year. Just checking in on your reviews here? It looks like @SteveMacenski responded to the questions you asked in SteveMacenski/slam_toolbox#318 and SteveMacenski/slam_toolbox#320 ?

[mosteo]

I'm back on track, sorry for the delay. @arfon, in regard to the verification of the functionality/performance, this is intended to be verified by the reviewers? Or would evidence of usage by third parties be enough?

[SteveMacenski]

A few examples from people that aren't me and I've had no contact with (googled and found for reference) if that helps

• https://www.youtube.com/watch?v=ftfMsA-UykQ / https://www.notion.so/Tutorial-SLAM-toolbox-aac021ec21d24f898ce230c19def3b7b
• https://www.youtube.com/watch?v=s16269kol5M
• https://www.youtube.com/watch?v=Cgcl3LcFnEs
• http://www.robotandchisel.com/2020/08/19/slam-in-ros2/ (though note I do know Mike but I didn't ask/inform him on this)
• https://msadowski.github.io/hands-on-with-slam_toolbox/
• https://blog.pal-robotics.com/aris-wiki-ros-tutorials-on-slam/

[arfon]

I'm back on track, sorry for the delay. @arfon, in regard to the verification of the functionality/performance, this is intended to be verified by the reviewers? Or would evidence of usage by third parties be enough?

We would definitely prefer both reviewers to verify the functionality claims (performance is sometimes more challenging for folks).

[mosteo]

Understood. Then, I'm going to throw a ball to @SteveMacenski : I don't currently have access to my labs robots due to covid. Also, I'm exclusively using ROS2 these days. This brings me back to the issue of beginner tutorials. If I go to https://navigation.ros.org/tutorials/docs/navigation2_with_slam.html, I read:

we will use the turtlebot3. If you have another robot, replace with suitable instructions

I would really like to see there, instead of "replace with suitable", something like "use this previous tutorial to set up a simulated robot" or similar. Related to my earlier comment that people not current with ROS2 may hit a bump there if interested in a "quick" test.

[SteveMacenski]

Ok, makes sense - do you have a ROS2 bag file you can run it over? I'd be happy to share with you a ROS(1) bag file I have for the purposes of review and a set of instructions to reproduce, but unfortunately I don't have any ROS2 bag files (as you're aware until exceptionally recently, ROS2 bag has been very flaky). ROS 1 would be easier to see everything since that's what this article was written on but lets see what we can work out in ROS2.

---

Another option is as you've found in the tutorial, if you're OK installing Nav2 to run our canonical getting started demo then one of the parameters I have conveniently provided is slam. So

ros2 launch nav2_bringup tb3_simulation_launch.py slam:=true

will get you what you need (calls on a slam toolbox launch file rather than a map server / AMCL one). It will launch a TB3 in a sandbox world that you can initialize the pose with the rviz "Pose2D" tool and then request navigation goals with "goal pose" tool. You'll see the map update as you move.The SLAM specific tutorial is meant to be more abstract and separate out the navigation from the simulation from the SLAM since that tutorial is written with "bring your own robot" in mind - in which case using our one-stop-shop launch file tb3_simulation_launch.py isn't appropriate. I agree it leaves some to be desired, I'll update it later today to mention the types of things I mean (robot state publisher, interfaces, controllers, etc).

we will use the turtlebot3. If you have another robot, replace with suitable instructions

Is to mean your own robot state publisher, hardware / simulation interface, and any other robot-specific needs. It's hard to fully articulate the broad range of things that a particular company / robot might require, so we keep it abstract. It could be as little as the robot state publisher with URDF and drivers, but frequently its alot more.

Edit: its been updated to be more specific https://github.com/ros-planning/navigation.ros.org/blob/master/tutorials/docs/navigation2_with_slam.rst#0--launch-robot-interfaces thanks for the note. Always looking to make our docs better.

Edit2: the SLAM tutorial also includes the link to this https://github.com/ros-planning/navigation.ros.org/blob/master/tutorials/docs/navigation2_with_slam.rst#4--getting-started-simplification - while I don't think most people really need that, it is valuable to have that documented somewhere that isn't just tribal knowledge in my head. Again, thanks!

[arfon]

@whedon accept from branch joss

[whedon]

Attempting dry run of processing paper acceptance...

[whedon]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1177/1729881420910530 is OK
- 10.1016/j.procs.2015.12.336 is OK
- 10.1109/IROS.2010.5649043 is OK
- 10.1177/0278364906065387 is OK
- 10.1109/ROBOT.2006.1641929 is OK

MISSING DOIs

- 10.1109/iros45743.2020.9341207 may be a valid DOI for title: The Marathon 2: A Navigation System
- 10.36288/roscon2019-900903 may be a valid DOI for title: On Use of SLAM Toolbox
- 10.1109/icra.2016.7487258 may be a valid DOI for title: Real-Time Loop Closure in 2D LIDAR SLAM
- 10.1109/tro.2006.889486 may be a valid DOI for title: Improved Techniques for Grid Mapping With Rao-Blackwellized Particle Filters
- 10.1109/ssrr.2011.6106777 may be a valid DOI for title: A flexible and scalable SLAM system with full 3D motion estimation

INVALID DOIs

- None

[whedon]

👋 @openjournals/joss-eics, this paper is ready to be accepted and published.

Check final proof 👉 openjournals/joss-papers#2300

If the paper PDF and Crossref deposit XML look good in openjournals/joss-papers#2300, then you can now move forward with accepting the submission by compiling again with the flag deposit=true e.g.

@whedon accept deposit=true from branch joss 

[arfon]

@SteveMacenski - a couple of final things:

• Did you check the DOI suggestions from Whedon above? If any of them are correct, please update your BibTeX to include them.
• I made a small change to your paper in Minor paper change for clarity SteveMacenski/slam_toolbox#392 to make it read a little more clearly (I think).

[SteveMacenski]

Hi,

• Googling the DOIs all gave me the papers that are also matching the titles. I'm not exactly sure what the issue is there, but they seem to all be valid from my checking
• Merged

[arfon]

@whedon check references from branch joss

[whedon]

Attempting to check references... from custom branch joss

[whedon]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1177/1729881420910530 is OK
- 10.1016/j.procs.2015.12.336 is OK
- 10.1109/IROS.2010.5649043 is OK
- 10.1177/0278364906065387 is OK
- 10.1109/ROBOT.2006.1641929 is OK

MISSING DOIs

- 10.1109/iros45743.2020.9341207 may be a valid DOI for title: The Marathon 2: A Navigation System
- 10.36288/roscon2019-900903 may be a valid DOI for title: On Use of SLAM Toolbox
- 10.1109/icra.2016.7487258 may be a valid DOI for title: Real-Time Loop Closure in 2D LIDAR SLAM
- 10.1109/tro.2006.889486 may be a valid DOI for title: Improved Techniques for Grid Mapping With Rao-Blackwellized Particle Filters
- 10.1109/ssrr.2011.6106777 may be a valid DOI for title: A flexible and scalable SLAM system with full 3D motion estimation

INVALID DOIs

- None

[arfon]

Googling the DOIs all gave me the papers that are also matching the titles. I'm not exactly sure what the issue is there, but they seem to all be valid from my checking

Great! Then please update your paper.bib to include them. For example, for The Marathon 2: A Navigation System, if the suggested DOI from Whedon is correct then you need to add doi=10.1109/iros45743.2020.9341207 to this part of your BibTeX file (we need you to do this for all of the potential missing DOIs please).

[SteveMacenski]

SteveMacenski/slam_toolbox@7a333e5 Done!

[arfon]

@whedon check references from branch joss

[whedon]

Attempting to check references... from custom branch joss

[whedon]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1177/1729881420910530 is OK
- 10.1016/j.procs.2015.12.336 is OK
- 10.1109/IROS.2010.5649043 is OK
- 10.1109/iros45743.2020.9341207 is OK
- 10.36288/roscon2019-900903 is OK
- 10.1109/icra.2016.7487258 is OK
- 10.1177/0278364906065387 is OK
- 10.1109/ROBOT.2006.1641929 is OK
- 10.1109/tro.2006.889486 is OK
- 10.1109/ssrr.2011.6106777 is OK

MISSING DOIs

- None

INVALID DOIs

- None

[arfon]

@whedon accept from branch joss

[whedon]

Attempting dry run of processing paper acceptance...

[whedon]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1177/1729881420910530 is OK
- 10.1016/j.procs.2015.12.336 is OK
- 10.1109/IROS.2010.5649043 is OK
- 10.1109/iros45743.2020.9341207 is OK
- 10.36288/roscon2019-900903 is OK
- 10.1109/icra.2016.7487258 is OK
- 10.1177/0278364906065387 is OK
- 10.1109/ROBOT.2006.1641929 is OK
- 10.1109/tro.2006.889486 is OK
- 10.1109/ssrr.2011.6106777 is OK

MISSING DOIs

- None

INVALID DOIs

- None

[whedon]

👋 @openjournals/joss-eics, this paper is ready to be accepted and published.

Check final proof 👉 openjournals/joss-papers#2306

If the paper PDF and Crossref deposit XML look good in openjournals/joss-papers#2306, then you can now move forward with accepting the submission by compiling again with the flag deposit=true e.g.

@whedon accept deposit=true from branch joss 

[arfon]

@whedon accept deposit=true from branch joss

[whedon]

Doing it live! Attempting automated processing of paper acceptance...

[whedon]

🐦🐦🐦 👉 Tweet for this paper 👈 🐦🐦🐦

[whedon]

🚨🚨🚨 THIS IS NOT A DRILL, YOU HAVE JUST ACCEPTED A PAPER INTO JOSS! 🚨🚨🚨

Here's what you must now do:

0. Check final PDF and Crossref metadata that was deposited 👉 Creating pull request for 10.21105.joss.02783 joss-papers#2307
1. Wait a couple of minutes, then verify that the paper DOI resolves https://doi.org/10.21105/joss.02783
2. If everything looks good, then close this review issue.
3. Party like you just published a paper! 🎉🌈🦄💃👻🤘

Any issues? Notify your editorial technical team...

[arfon]

@mosteo, @carlosjoserg - many thanks for your reviews here! JOSS relies upon volunteer effort from folks like you and we simply wouldn't be able to do this without you! ✨

@SteveMacenski - your paper is now accepted and published in JOSS ⚡:rocket::boom:

[whedon]

🎉🎉🎉 Congratulations on your paper acceptance! 🎉🎉🎉

If you would like to include a link to your paper from your README use the following code snippets:

Markdown:
[![DOI](https://joss.theoj.org/papers/10.21105/joss.02783/status.svg)](https://doi.org/10.21105/joss.02783)

HTML:
<a style="border-width:0" href="https://doi.org/10.21105/joss.02783">
  <img src="https://joss.theoj.org/papers/10.21105/joss.02783/status.svg" alt="DOI badge" >
</a>

reStructuredText:
.. image:: https://joss.theoj.org/papers/10.21105/joss.02783/status.svg
   :target: https://doi.org/10.21105/joss.02783

This is how it will look in your documentation:

We need your help!

Journal of Open Source Software is a community-run journal and relies upon volunteer effort. If you'd like to support us please consider doing either one (or both) of the the following:

• Volunteering to review for us sometime in the future. You can add your name to the reviewer list here: https://joss.theoj.org/reviewer-signup.html
• Making a small donation to support our running costs here: https://numfocus.org/donate-to-joss

[SteveMacenski]

Amazing, thank you! I've added myself a bit ago as a reviewer for ROS/robotics papers, feel free to call on me if you have something in robotics/ROS!