# KRR Setup Instructions
Practicum 1.1: Simulation Installation & Exploration

<div align="center">

**Knowledge Representation & Symbolic Reasoning**  
**Practicum 1.1: Simulation Installation & Exploration**

</div>

<div align="center">

**Practicum date:**  

</div>

**Course:** RO47014 Knowledge Representation and Symbolic Reasoning, TU Delft, CoR  


This practicum will help you install the software that we use for the course, especially a ROS 2 simulation with a Mirte Master robot for the Course Project. The setup is based on a Singularity image similar to the one used for Robot Software Practicals.  
In addition to the Singularity image, extra code is downloaded from online git repositories. After the installation instructions, this document guides you through exploring the robot simulation that you have to use for your project.

The idea of the whole course is to help you develop an architecture and solution for a real-world application task, such as tidying up an apartment in the RoboCup@Home competition. A topview of the apartment for this competition is presented in the figure below. Week by week, we will provide you with tools and methods on how to make a plan for such a task and how to execute it.  
You can check the solutions of some of last year's groups, like [Group 12](https://www.youtube.com/watch?v=Zx2PlRPjz5s&list=PL22tbRMI6v9A9nlR0sbOi6wwW1YMQ2uIt&index=1) and [Group 13](https://youtu.be/pcoaSkmnzlQ?si=3zuNWmgrrF18OcE0).

![Simulation environment with Mirte_master robot.](images/KRR_house2.png)

The final task and project have three subtasks that will be discussed in more detail later. However, for now, let's become more familiar with the simulation environment and singularity.

**You do not need to deliver anything about this practicum.**

---

## Pre-requisites

This course makes use of a Singularity container environment to run all software. The Singularity image provided for RO47014 Knowledge Representation and Symbolic Reasoning (KRR in short) is similar to what you may have used in Robot Software Practicals (RSP). As in RSP, we strongly recommend using a native installation of Ubuntu (22.04 LTS or 24.04 LTS).
While a virtual machine can be used, it is not preferred due to potential limitations in performance and hardware access.

If you do _not_ have the RSP setup available, you should follow the installation instructions in the document [setup_manual_rsp_2025_2026.pdf](https://brightspace.tudelft.nl/d2l/le/content/775417/viewContent/4674953/View) on BrightSpace. If you want to use the VM, follow only chapter 2(`Virtual Machines`). If you have a native installation, only follow chapter 1(`If you already have Linux`). Additional installation guidance for Singularity can also be found in the official documentation at:  
https://docs.sylabs.io/guides/4.3/user-guide/quick_start.html

After installing Singularity, verify your installation by running the following command in a terminal:

```bash
singularity --version
```

A correct installation should produce an output similar to:

```text
singularity-ce version 4.3.1-XXX
```

**Important:** You must use Singularity version **4.3.1 or newer**, but strictly **below 5.x**, to ensure compatibility with the course environment.


## Setup the KRR environment on your computer

To start working with the KRR software stack, you first need to create a ROS workspace for KRR in your `$HOME` folder in your machine (either your native Linux, dual boot, or a virtual machine), download the course Singularity image, download all the required software, and build the workspace.

> **Attention 1**  
> Although ROS is mentioned throughout the assignments, note that the actual version used by KRR is ROS 2 Humble.

### Create a ROS workspace

First, create your own ROS 2 workspace where you will develop **your** code.
```bash
mkdir -p ~/krr_ws/src   # create a directory for the krr workspace
cd ~/krr_ws             # navigate to the krr workspace source directory
touch .colcon_root      # this is only needed for colcon-top-level-workspace
```

### Download course Singularity image

You can download the Singularity image for the KRR course using this command:

```bash
# Change the tag(v3) when instructed to download a newer image:
singularity pull -F ro47014_humble_v3.sif oras://registry.tudelft.nl/cor_ro47014_krr_image/singularity_image:v3
```

> **Attention 2**  
> It's best to download the singularity images to a standard known location, like `~/Downloads`.
> When cleaning your workspace, you won't accidentally delete the images.

You can download an update by changing the tag to a newer version (`vN`, when instructed to do so). To get the version of your image, run:
```bash
singularity exec ro47014_humble_v3.sif cat /version.txt`
```

### Building the KRR course packages

> **Attention 3**  
> The KRR packages are installed in */krr/krr_base_ws/* and not under the default ROS installation path.
Please do not source ```/opt/ros/humble/setup.bash```, as this may override the required environment.
The KRR base workspace is already sourced in ```.bashrc```. This note is provided to avoid confusion and ensure a consistent setup.

:::{important}
All commands from here are run in a Singularity shell session if not stated otherwise!
So, first open a Singularity shell session, for example, from the directory where you have downloaded the course image:
```bash
singularity shell -p ro47014_humble_v3.sif
```

:::



To build the KRR base workspace:
```bash
source /krr/krr_base_ws/install/setup.bash
cd ~/krr_ws/
colcon build --symlink-install
source install/setup.bash
```

You should now have a `log`, `build` and `install` directory besides an empty `src` directory. After this you can just source your own workspace: `source ~/krr_ws/install/setup.bash`.
You should also have a `.colcon_root` file for [rhaschke/colcon-top-level-workspace](https://github.com/rhaschke/colcon-top-level-workspace).




---

## Using the KRR setup with Singularity

To use the KRR base workspace with Singularity, you simply need to start the Singularity image as follows:

```bash
cd ~/krr_ws   # navigate to the krr workspace source directory
singularity shell -p ro47014_humble_v3.sif
```

:::{important}
If you are running Ubuntu (or another Linux distribution) natively (no VM) and have an NVIDIA GPU and have installed the **official NVIDIA drivers** (via *Additional Drivers* on Ubuntu for instance), you can enable GPU acceleration. See the {ref}`GPU_acceleration` section for more information.
:::


Then, every time you want to use the KRR setup, simply start the Singularity image and run what you need in the Singularity shell.

---

## Simulation exercises

The following exercises are designed to familiarize you with the simulation and how to work with it.

:::{important}
- The following instructions assume that you are working inside a Singularity shell session. Remember, the command to start it is:  
   `singularity shell -p ro47014_humble_v3.sif`  
- You need to source your own workspace `setup.bash` file of your ROS2 workspace every time you open a new terminal and run a new command!
:::

### Launch the Mirte simulation

First, start the singularity image as explained above.

Then, within the singularity shell, source the KRR workspace:

```bash
source /krr/krr_base_ws/install/setup.bash
# or source ~/krr_ws/install/setup.bash if you already made your own workspace.
```

Now, you can launch the simulation with the following command:

```bash
ros2 launch mirte_gazebo gazebo_mirte_master_navigation.launch.py
```

This should launch Gazebo and RViz, and you should see something similar to this:

![Mirte simulation in gazebo plus RViz](images/mirte_gazebo_rviz.png)

### Moving Mirte

There are multiple ways to make Mirte navigate through the map. The simplest option is to control Mirte yourself with your keyboard, a second option is to use RViz to send navigation goals to MIRTE, and a third option is to send navigation goals to Mirte from a ROS node.

#### Keyboard teleoperation

Drive Mirte around with your keyboard keys by running in another terminal:

```bash
ros2 run teleop_twist_keyboard teleop_twist_keyboard --ros-args --remap cmd_vel:=/mirte_base_controller/cmd_vel_unstamped
```

After running the above command you will see some instructions on how you may command the robot.

#### Autonomous navigation with RVIZ

You can autonomously navigate Mirte to a goal with RViz. First, go into the RVIZ screen and set the initial position of the robot with the button "2D Pose Estimate", a bunch of green dots and red dots should appear. Then, use the button "Nav2 Goal" to send a navigation goal to Mirte and it should start moving.

#### Autonomous navigation with ROS Nodes

To make Mirte navigate with a ROS node, you need to create a new ROS node that sends navigation goals to the proper ROS action server.

The first step is to create a new ROS package to host the new node:

```bash
cd ~/krr_ws/src
ros2 pkg create --build-type ament_python --node-name navigate_to_pose mirte_skills
```

Now, you should have a new ROS package called `mirte_skills` in the folder `~/krr_ws/src`. Let's then work on our new ROS node. Open the KRR workspace in your preferred code editor on your computer (**not in Singularity**). For example, if you have vscode installed:

```bash
cd ~/krr_ws
code .
```

##### Adjust the Python script of the ROS node
Then lets edit the `navigate_to_pose.py` file in the folder `~/krr_ws/src/mirte_skills/mirte_skills/`. Add the following code to the file:

**Note:** The example code provided is a simplified version of Nav2's `example_nav_to_pose.py` as it is available here: https://github.com/ros-navigation/navigation2/blob/main/nav2_simple_commander/nav2_simple_commander/example_nav_to_pose.py


1. Import required lib and classes:

```python
from geometry_msgs.msg import PoseStamped
from nav2_simple_commander.robot_navigator import BasicNavigator, TaskResult
import rclpy
from rclpy.duration import Duration
```

2. Define the main function and initialize rclpy:

```python
def main():
    rclpy.init()
```

3. Then, still in the main function, instantiate a BasicNavigator and define the initial position of the robot:

```python
    navigator = BasicNavigator()

    initial_pose = PoseStamped()
    initial_pose.header.frame_id = 'map'
    initial_pose.header.stamp = navigator.get_clock().now().to_msg()
    initial_pose.pose.position.x = 0.89
    initial_pose.pose.position.y = 0.34
    initial_pose.pose.orientation.z = 0.0
    initial_pose.pose.orientation.w = 1.0
    navigator.setInitialPose(initial_pose)

    navigator.waitUntilNav2Active()
```

4. Continue further in the main function and define a goal pose for the robot, and send it:

```python
    goal_pose = PoseStamped()
    goal_pose.header.frame_id = 'map'
    goal_pose.header.stamp = navigator.get_clock().now().to_msg()
    goal_pose.pose.position.x = -1.62
    goal_pose.pose.position.y = 3.63
    goal_pose.pose.orientation.w = 1.0
    goal_pose.pose.orientation.z = 0.0

    navigator.goToPose(goal_pose)
```

5. Complete the main function by specifying what the robot should do while it is moving to its goal. You can substitute "pass" for some other code, for example, a print:

```python
    while not navigator.isTaskComplete():
      pass

    result = navigator.getResult()
    if result == TaskResult.SUCCEEDED:
        print('succeeded!')
    elif result == TaskResult.CANCELED:
        print('canceled!')
    elif result == TaskResult.FAILED:
        print('failed!')

    navigator.lifecycleShutdown()

    exit(0)
```

6. Finally, outside the scope of the main function, define that the main function should be executed when the file is executed:

```python
if __name__ == '__main__':
    main()
```

##### Build and run the new ROS node

After you are done editing the `navigate_to_pose.py` file, build the new package, start the simulation from scratch, and run the node you just created:

1. Build the new package (in the singularity shell):

```bash
cd ~/krr_ws/
colcon build --symlink-install --packages-select mirte_skills
source install/setup.bash
```

2. Start the simulation again:

```bash
ros2 launch mirte_gazebo gazebo_mirte_master_navigation.launch.py
```

3. Run the new node (in another singularity shell):

```bash
ros2 run mirte_skills navigate_to_pose
```

4. Watch Mirte move and be amazed!

This is an example of a robot `skill`. Later on in the course, you will learn how to have the robot decide when and how to execute these skills on its own! For now, let's explore some more skills.

---

## Controlling the arm and gripper

Moving the robotic arm and gripper of the Mirte robot can be done in various different ways. For now, let us move the arm and gripper by publishing messages with the desired joint positions to the arm and gripper controllers. In the following weeks, you will learn to move/control the arm with action servers to enable a more high-level manipulation of the arm and gripper. For example, by performing a "close gripper" action instead of sending the exact joint positions. Stay tuned!

The following commands will publish the joints of the robot and gripper on specific topics that are avialable in the ROS2 system that you have launched. After publishing on those topics the the arm and gripper shall move accordingly.

**Moving arm up:**

```bash
ros2 topic pub --once /mirte_arm_controller/joint_trajectory trajectory_msgs/msg/JointTrajectory "{joint_names: ['arm_Rot_joint', 'arm_Shoulder_joint', 'arm_Elbow_joint', 'arm_Wrist_joint'], points: [{positions: [0.0, 0.0, -1.56, 1.56], time_from_start:{ sec: 3, nanosec: 0}}]}"
```

**Moving arm down:**

```bash
ros2 topic pub --once /mirte_arm_controller/joint_trajectory trajectory_msgs/msg/JointTrajectory "{joint_names: ['arm_Rot_joint', 'arm_Shoulder_joint', 'arm_Elbow_joint', 'arm_Wrist_joint'], points: [{positions: [0.0, -1.56, -1.56, 1.56], time_from_start:{ sec: 3, nanosec: 0}}]}"
```

**Close gripper:**

```bash
ros2 topic pub --once /mirte_gripper_controller/joint_trajectory trajectory_msgs/msg/JointTrajectory "{joint_names: ['Gripper_joint'], points: [{positions: [0.1], time_from_start:{ sec: 1, nanosec: 0}}]}"
```

**Open gripper:**

```bash
ros2 topic pub --once /mirte_gripper_controller/joint_trajectory trajectory_msgs/msg/JointTrajectory "{joint_names: ['Gripper_joint'], points: [{positions: [-0.1], time_from_start:{ sec: 1, nanosec: 0}}]}"
```

**Note:** If it doesn't work the first time you publish a message, try publishing again.

---

## Contributing

If you want to contribute to a more realistic and interesting simulation you are more than welcome! If you find bugs or want to suggest new features, please create an issue in the [krr_bringup GitHub repo](https://github.com/kas-lab/krr_bringup) or in the repo related to the issue you faced. If you fix bugs or add new features, open a Pull Request on GitHub with your changes. The packages maintained by us are:

- [krr_bringup](https://github.com/kas-lab/krr_bringup)  
- [mirte-gazebo](https://github.com/Rezenders/mirte-gazebo)  
- [mirte-ros-packages](https://github.com/Rezenders/mirte-ros-packages)  
- [mirte_navigation](https://github.com/kas-lab/mirte_navigation)  
- [robocup_home_simulation](https://github.com/kas-lab/robocup_home_simulation)  
- [navigation_graph_map](https://github.com/kas-lab/navigation_graph_map)  
- [ros_typedb](https://github.com/kas-lab/ros_typedb)

You can also use the [discussion tab](https://github.com/kas-lab/krr_bringup/discussions/1) in the krr_bringup repo to ask questions and start discussions.

## Appendix

(gpu_acceleration)=
### GPU acceleration

If you are running native Ubuntu (without a Virtual Machine) and you have an NVIDIA GPU and are using the official NVIDIA drivers with it, you can use GPU acceleration to have a smoother simulation. 

Your Ubuntu host needs to use X or have Xwayland installed, this is often the case with Ubuntu 22.04 and 24.04.
Other modern Linux distributions also tend to have Xwayland available.

You need to install the [Nvidia Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html).

Then you need to configure Singularity:
edit two entries in the `singularity.conf` file, which should be at `/etc/singularity/singularity.conf` (note: editing this file requires `sudo`):

- Enable NVIDIA Container CLI:
  - set `use nvidia-container-cli = yes`
- Set the path to the `nvidia-container-cli` binary (the default value is empty):
  - `nvidia-container-cli path = /usr/bin/nvidia-container-cli`

The following command will launch a fully GPU accelerated shell session:

```bash
NVIDIA_DRIVER_CAPABILITIES="graphics,utility,video,display" \
  singularity \
    shell \
      --nvccli \
      -B $XAUTHORITY \
      /path/to/your_image.sif
```

:::{note}
The additional `-B $XAUTHORITY` argument is needed to allow programs inside the Singularity shell session access to X or Xwayland.
It's not strictly needed to enable GPU acceleration, but many ROS programs with a UI will not work correctly without it.
:::

For laptops with a dual-GPU configuration (NVIDIA Optimus or similar, such as in the TU Delft recommended laptop *HP ZBook X G1i*) you have to force the use of the NVIDIA GPU:
```bash
__NV_PRIME_RENDER_OFFLOAD=1 __GLX_VENDOR_LIBRARY_NAME=nvidia \
  NVIDIA_DRIVER_CAPABILITIES="graphics,utility,video,display" \
  singularity \
    shell \
      --nvccli \
      -B $XAUTHORITY \
      /path/to/your_image.sif
```

Note the two additional environment variables (starting with `__NV` and `__GLX`).

Obviously, using a suitably named and constructed alias would make this much more convenient.
You could consider adding something like this to your `.bashrc`:
```bash
alias singularity-shell-on-nvidia='__NV_PRIME_RENDER_OFFLOAD=1 __GLX_VENDOR_LIBRARY_NAME=nvidia NVIDIA_DRIVER_CAPABILITIES="graphics,utility,video,display" singularity shell --nvccli'
```

the command for all systems (extra variables are ignored when using normal GPU setup) would then become:

```bash
singularity-shell-on-nvidia -B $XAUTHORITY /path/to/your_image.sif
```