DebugVisualizer Update

[aclegg3]

Motivation and Context

This PR refactors the DebugVisualizer (dbv) to improve overall utility and allow direct use for visual scene debugging in python with pdb.

Improvements:

• dbv creates a new agent and rgb sensor to avoid corrupting the active simulator state and allow precise camera control
• new DebugObservation wrapper for observation arrays allow easily showing and saving images without pulling in external APIs
• fixed and improved object peeking
• allow debug agent to be moved around interactively with translate and rotate functions

Also fixed ao global bounding box function (mistreating base links)

How Has This Been Tested

Locally using in pdb.

Try it out yourself!

1. set a breakpoint() in your python code
2. run your code
3. When the breakpoint is triggered use the API:

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PDB set_trace >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> /path/to/your/code.py
-> # this is the line after my `breakpoint()`
(Pdb) from habitat.sims.habitat_simulator.debug_visualizer import DebugVisualizer
(Pdb) dbv = DebugVisualizer(sim)
(Pdb) dbv.get_observation().show()
(Pdb) dbv.translate(mn.Vector3(1,0,0), show=True)
(Pdb) dbv.peek_articulated_object(my_ao, show=True)
(Pdb) dbv.peek_rigid_object(my_ro, show_all_axis=True, show=True)

For the easiest test, copy and paste the below code. Note, this assumes an object called sim. Instead, pass your local habitat_sim.Simulator object (e.g. self._sim):

from habitat.sims.habitat_simulator.debug_visualizer import DebugVisualizer
dbv = DebugVisualizer(sim)
dbv.get_observation().show()

Peek the scene:

dbv.peek_scene(show_all_axis=True, show=True)

Types of changes

• [Bug Fix] (non-breaking change which fixes an issue)
• [Development] A pull request that add new features to the habitat-lab task and environment codebase. Development Pull Requests must be small (less that 500 lines of code change), have unit testing, very extensive documentation and examples. These are typically new tasks, environments, sensors, etc... The review process for these Pull Request is longer because these changes will be maintained by our core team of developers, so make sure your changes are easy to understand!

Checklist

• My code follows the code style of this project.
• I have updated the documentation if required.
• I have read the CONTRIBUTING document.
• I have completed my CLA (see CONTRIBUTING)
• I have added tests to cover my changes if required.

[xavierpuigf]

Looks good! It'd be great to add a projection utility. Would you say this is the best way to get a static camera visualization when we run an episode?

[aclegg3]

Looks good! It'd be great to add a projection utility.

Thanks, agreed. Let's tack on additional functionality in future PRs!

Would you say this is the best way to get a static camera visualization when we run an episode?

This is certainly one way to do so. Try out the peek_scene and look_at functions to control the camera.

[xavierpuigf]

Thanks for this!