tracklab.wrappers.detect_multiple package

Submodules

tracklab.wrappers.detect_multiple.bottomup_mmpose_api module

tracklab.wrappers.detect_multiple.bottomup_mmpose_api.BottomUpMMPose(cfg, device, batch_size)

tracklab.wrappers.detect_multiple.bottomup_mmpose_api.mmpose_collate(batch)

tracklab.wrappers.detect_multiple.mmdetection_api module

tracklab.wrappers.detect_multiple.openpifpaf_api module

tracklab.wrappers.detect_multiple.yolov8_api module

class tracklab.wrappers.detect_multiple.yolov8_api.YOLOv8(cfg, device, batch_size, **kwargs)

Bases: ImageLevelModule

collate_fn()

Function that takes in a batch of data and puts the elements within the batch into a tensor with an additional outer dimension - batch size. The exact output type can be a torch.Tensor, a Sequence of torch.Tensor, a Collection of torch.Tensor, or left unchanged, depending on the input type. This is used as the default function for collation when batch_size or batch_sampler is defined in DataLoader.

Here is the general input type (based on the type of the element within the batch) to output type mapping:

• torch.Tensor -> torch.Tensor (with an added outer dimension batch size)

• NumPy Arrays -> torch.Tensor

• float -> torch.Tensor

• int -> torch.Tensor

• str -> str (unchanged)

• bytes -> bytes (unchanged)

• Mapping[K, V_i] -> Mapping[K, default_collate([V_1, V_2, …])]

• NamedTuple[V1_i, V2_i, …] -> NamedTuple[default_collate([V1_1, V1_2, …]), default_collate([V2_1, V2_2, …]), …]

• Sequence[V1_i, V2_i, …] -> Sequence[default_collate([V1_1, V1_2, …]), default_collate([V2_1, V2_2, …]), …]

Parameters:

batch – a single batch to be collated

Examples

>>> # Example with a batch of `int`s:
>>> default_collate([0, 1, 2, 3])
tensor([0, 1, 2, 3])
>>> # Example with a batch of `str`s:
>>> default_collate(['a', 'b', 'c'])
['a', 'b', 'c']
>>> # Example with `Map` inside the batch:
>>> default_collate([{'A': 0, 'B': 1}, {'A': 100, 'B': 100}])
{'A': tensor([  0, 100]), 'B': tensor([  1, 100])}
>>> # Example with `NamedTuple` inside the batch:
>>> # xdoctest: +SKIP
>>> Point = namedtuple('Point', ['x', 'y'])
>>> default_collate([Point(0, 0), Point(1, 1)])
Point(x=tensor([0, 1]), y=tensor([0, 1]))
>>> # Example with `Tuple` inside the batch:
>>> default_collate([(0, 1), (2, 3)])
[tensor([0, 2]), tensor([1, 3])]
>>> # Example with `List` inside the batch:
>>> default_collate([[0, 1], [2, 3]])
[tensor([0, 2]), tensor([1, 3])]
>>> # Two options to extend `default_collate` to handle specific type
>>> # Option 1: Write custom collate function and invoke `default_collate`
>>> def custom_collate(batch):
...     elem = batch[0]
...     if isinstance(elem, CustomType):  # Some custom condition
...         return ...
...     else:  # Fall back to `default_collate`
...         return default_collate(batch)
>>> # Option 2: In-place modify `default_collate_fn_map`
>>> def collate_customtype_fn(batch, *, collate_fn_map=None):
...     return ...
>>> default_collate_fn_map.update(CustoType, collate_customtype_fn)
>>> default_collate(batch)  # Handle `CustomType` automatically

input_columns = []

output_columns = ['image_id', 'video_id', 'category_id', 'bbox_ltwh', 'bbox_conf']

preprocess(image, detections, metadata: Series)

Adapts the default input to your specific case.

Parameters:

• image – a numpy array of the current image

• detections – a DataFrame containing all the detections pertaining to a single image

• metadata – additional information about the image

Returns:

input for the process function

Return type:

preprocessed_sample

process(batch: Any, detections: DataFrame, metadatas: DataFrame)

The main processing function. Runs on GPU.

Parameters:

• batch – The batched outputs of preprocess

• detections – The previous detections.

• metadatas – The previous image metadatas

Returns:

Either a DataFrame containing the new/updated detections

or a tuple containing detections and metadatas (in that order) The DataFrames can be either a list of Series, a list of DataFrames or a single DataFrame. The returned objects will be aggregated automatically according to the name of the Series/index of the DataFrame. It is thus mandatory here to name correctly your series or index your dataframes. The output will override the previous detections with the same name/index.

Return type:

output

tracklab.wrappers.detect_multiple.yolov8_api.collate_fn(batch)

tracklab.wrappers.detect_multiple.yolov8_pose_api module

class tracklab.wrappers.detect_multiple.yolov8_pose_api.YOLOv8Pose(cfg, device, batch_size, **kwargs)

Bases: ImageLevelModule

collate_fn()

Function that takes in a batch of data and puts the elements within the batch into a tensor with an additional outer dimension - batch size. The exact output type can be a torch.Tensor, a Sequence of torch.Tensor, a Collection of torch.Tensor, or left unchanged, depending on the input type. This is used as the default function for collation when batch_size or batch_sampler is defined in DataLoader.

Here is the general input type (based on the type of the element within the batch) to output type mapping:

• torch.Tensor -> torch.Tensor (with an added outer dimension batch size)

• NumPy Arrays -> torch.Tensor

• float -> torch.Tensor

• int -> torch.Tensor

• str -> str (unchanged)

• bytes -> bytes (unchanged)

• Mapping[K, V_i] -> Mapping[K, default_collate([V_1, V_2, …])]

• NamedTuple[V1_i, V2_i, …] -> NamedTuple[default_collate([V1_1, V1_2, …]), default_collate([V2_1, V2_2, …]), …]

• Sequence[V1_i, V2_i, …] -> Sequence[default_collate([V1_1, V1_2, …]), default_collate([V2_1, V2_2, …]), …]

Parameters:

batch – a single batch to be collated

Examples

>>> # Example with a batch of `int`s:
>>> default_collate([0, 1, 2, 3])
tensor([0, 1, 2, 3])
>>> # Example with a batch of `str`s:
>>> default_collate(['a', 'b', 'c'])
['a', 'b', 'c']
>>> # Example with `Map` inside the batch:
>>> default_collate([{'A': 0, 'B': 1}, {'A': 100, 'B': 100}])
{'A': tensor([  0, 100]), 'B': tensor([  1, 100])}
>>> # Example with `NamedTuple` inside the batch:
>>> # xdoctest: +SKIP
>>> Point = namedtuple('Point', ['x', 'y'])
>>> default_collate([Point(0, 0), Point(1, 1)])
Point(x=tensor([0, 1]), y=tensor([0, 1]))
>>> # Example with `Tuple` inside the batch:
>>> default_collate([(0, 1), (2, 3)])
[tensor([0, 2]), tensor([1, 3])]
>>> # Example with `List` inside the batch:
>>> default_collate([[0, 1], [2, 3]])
[tensor([0, 2]), tensor([1, 3])]
>>> # Two options to extend `default_collate` to handle specific type
>>> # Option 1: Write custom collate function and invoke `default_collate`
>>> def custom_collate(batch):
...     elem = batch[0]
...     if isinstance(elem, CustomType):  # Some custom condition
...         return ...
...     else:  # Fall back to `default_collate`
...         return default_collate(batch)
>>> # Option 2: In-place modify `default_collate_fn_map`
>>> def collate_customtype_fn(batch, *, collate_fn_map=None):
...     return ...
>>> default_collate_fn_map.update(CustoType, collate_customtype_fn)
>>> default_collate(batch)  # Handle `CustomType` automatically

input_columns = []

output_columns = ['image_id', 'video_id', 'category_id', 'bbox_ltwh', 'bbox_conf', 'keypoints_xyc', 'keypoints_conf']

preprocess(image, detections, metadata: Series)

Adapts the default input to your specific case.

Parameters:

• image – a numpy array of the current image

• detections – a DataFrame containing all the detections pertaining to a single image

• metadata – additional information about the image

Returns:

input for the process function

Return type:

preprocessed_sample

process(batch: Any, detections: DataFrame, metadatas: DataFrame)

The main processing function. Runs on GPU.

Parameters:

• batch – The batched outputs of preprocess

• detections – The previous detections.

• metadatas – The previous image metadatas

Returns:

Either a DataFrame containing the new/updated detections

or a tuple containing detections and metadatas (in that order) The DataFrames can be either a list of Series, a list of DataFrames or a single DataFrame. The returned objects will be aggregated automatically according to the name of the Series/index of the DataFrame. It is thus mandatory here to name correctly your series or index your dataframes. The output will override the previous detections with the same name/index.

Return type:

output

tracklab.wrappers.detect_multiple.yolov8_pose_api.collate_fn(batch)