cogrob_final-project

Final project for Cognitive Robotics 2020/2021 - Group 26

Assignment

assignment.pdf

Team Members

• Giovanni Ammendola
• Edoardo Maffucci
• Vincenzo Petrone
• Salvatore Scala

Architecture

The project is organized into 4 ROS nodes:

• camera_node
  • Makes Pepper take pictures
• detector_node
  • Performs object detections on pictures taken
• head_node
  • Makes Pepper move the head left and right
• speaker_node
  • Makes Pepper speak, saying what she sees around

Documentation

Code

For futher information about the implementation, please refer to the in-code documentation. Every node is fully documentated in the related file.

Messages

The following messages are defined in the msg folder.

DetectionWithScore

This message is used by only one node:

• detector_node:
  • For each object it detects, it creates a message with the following fields:
    • clabel: the class label of the object
    • score: the confidence score for the detection
  • This message is then added to the detections component of the SayDetections service request

Services

The following services are defined in the srv folder.

TakePicture

This service is served by the camera node and is requested by the head node in order to take a picture. Since the picture will be forwarded to the DetectImage service, the requests take as parameter the current position of the head.

• camera_node upon receiving a request, takes the picture, forwards it to the DetectImage service along with the direction and sends a True response if and only if the operation is successful.
• head_node is the only service client. Before taking a picture, the node must be sure that the head is in the right position.

DetectImage

This service is served by the detector node and is requested by the camera node in order to detect objects in a picture. Since the objects will be forwarded to the SayDetections service, the requests take as parameter the direction at which the given image has been taken.

• detector_node upon receiving a request, gives the picture as input to the detector model, creates a SayDetections service request containing the detected objects and sends a True response if and only if the operation is successful.
• camera_node is the only service client. Creates a request by passing as parameter the image taken and its proper direction

SayDetections

This service is served by the speaker_node and is requested by the detector_node in order to convert a list of detected objects into a string.

• speaker_node: upon receiving a request, creates a string depending on the detected objects and the direction at which the objects have been detected and sends a True response if and only if the direction is valid
  • For example:

    I can see a bottle on the right, a PC in front of me, 2 TVs on the left.

• detector_node is the only service client. Creates a request by passing as parameters:
  • detections: the classes of the objects seen in the image, along with the confidence scores
  • direction: the direction at which the image has been taken

Usage

To download and run the software, you need the tool p7zip. To install it, run the following commands:

sudo apt-get update
sudo apt-get install p7zip-full

Setup

To set up the workspace and build all the dependencies, run the following commands:

curl -o setup.bash https://raw.githubusercontent.com/CR2020-team/cogrob_final-project/master/setup.bash
chmod +x setup.bash
./setup.bash

Launch

To use the software, first source the workspace with

cd CR2020_GRUPPO26_WS
source devel/setup.bash

Then run

roslaunch pepper_pkg pepper.launch pepper_id:=id

Where id is either 1 or 2 depending on which Pepper robot the software will be executed. If not specified, pepper_id is 1.