# Tutorial 3 - Utilizing the CAS and Annotators

In this tutorial you will learn how to use the CAS that contains 
the annotations created by the various Annotators.
A central annotation type for RoboKudo is the `ObjectHypothesis`.
It marks an object in the scene and many other annotators rely on this
hypothesis to add their own annotations to it.
This basically allows the system to gather more and more information over time
about the detected objects.

In this tutorial, you will work on an Annotator which uses the CAS to 
add your own annotation to an ObjectHypothesis.
Specifically, you will create an annotation based on the **Region Of Interest (ROI)** of an object.
The ROI can be seen as the rectangle around the Objects in the image as seen in the previous tutorial.
Your goal is to calculate its area.
The ROI is available in the `roi` field of an `ObjectHypothesis`. Its value is
an `ImageROI` object with which also has a field `roi` leading to a `Rect` object
with four attributes `x`, `y`, `width` and `height`.

The CAS is accessible in each annotator through its function `self.get_cas()`.
The annotations stored in the CAS are available through its field `annotations`,
which is a normal python list of all annotations.
Example to access the annotations in the CAS:

```python
cas = self.get_cas()
annotations: List[Annotation] = cas.annotations
```

The CAS also provides a helper function to retrieve only annotations of a certain
type. This can be used for example to only get `ObjectHypothesis` annotations:
```python
annotations: List[ObjectHypothesis] = cas.filter_annotations_by_type(
    robokudo.types.scene.ObjectHypothesis
)
```

Each `ObjectHypothesis` itself also has such a list of annotations.
This was for example used in the `ClusterColorAnnotator`, where the `SemanticColor`
annotations were added to the list of annotations on an `ObjectHypothesis`.
Like Annotators, annotations are also implemented as simple classes.

- <input type="checkbox"> **Task 3-1:** Look for the `annotators` directory and implement the `update` function in
`object_size.py` with the following criteria:
    - For each `ObjectHypothesis` in the CAS, calculate the area of the ROI.
    - For each `ObjectHypothesis` add a new annotation `Size` containing the ROI
    area in its field `value`. You can use the already existing annotation class `robokudo.types.annotation.Size` for that. 
    - Depending on the total size, also add one size classification out of 
    `["small", "medium", "large"]` to the `Size` annotation in the `class` field,
    by using the following values
        - `"small"` objects with a size/area of less than `35000`
        - `"medium"` objects with a size/area of more than `35000` and less than `50000`
        - `"large"` objects with a size/area of more than `50000`
    - At the end of the function return `py_trees.Status.SUCCESS` to signal that
    the execution was successful.

:::{admonition} Hint for roi data access
:class: dropdown hint
Access to the roi data of an annotation like this:
```python
annotation.roi.roi.width # Check the text above for other possible values.
```
:::

- <input type="checkbox"> **Task 3-2:** Similar to the previous tutorial, add the ObjectSizeAnnotator to the Analysis Engine. 
- <input type="checkbox"> **Task 3-3:** Restart RoboKudo and send the query of
type **detect** again. Afterwards, scroll down to the details tree next to the output
image and check if your annotations show up for each `ObjectHypothesis`.

:::{important}
Add the `ObjectSizeAnnotator` **after** the `ClusterColorAnnotator` so that the
`ObjectHypothesis` annotations are available in the CAS.
```python
object_detection.add_children(
    [
        # ...
        ClusterColorAnnotator(),
        ObjectSizeAnnotator(),
        # ...
    ]
)
```
:::

As an example the output in the details tree could look like this:
![](../img/03-object-size-annotation.png)

:::{admonition} The final code of `object_size.py` could look something like this:
:class: dropdown hint
```python
class ObjectSizeAnnotator(robokudo.annotators.core.BaseAnnotator):

    def __init__(self, name="ObjectSizeAnnotator") -> None:
        super().__init__(name=name)
        self.rk_logger.debug("%s.__init__()" % self.__class__.__name__)

    def update(self) -> py_trees.common.Status:
        # Get all ObjectHypothesis annotations from the CAS
        annotations: List[ObjectHypothesis] = self.get_cas().filter_annotations_by_type(
            robokudo.types.scene.ObjectHypothesis
        )

        for annotation in annotations:
            # Create a new Size annotation
            size = robokudo.types.annotation.Size()

            # Set the size value to be the size of the roi
            size.value = annotation.roi.roi.width * annotation.roi.roi.height

            # Classify the size
            if size.value < 35000:
                size.cls = "small"
            elif 35000 < size.value < 50000:
                size.cls = "medium"
            else:
                size.cls = "large"

            # Add the source annotator information to the annotation (optional)
            size.source = self.name

            # Add the annotation to the list of annotations on the ObjectHypothesis
            annotation.annotations.append(size)

        return py_trees.Status.SUCCESS
```
:::