l5kit.data package

class l5kit.data.ChunkedStateDataset(path: str, key: str = '')

Bases: object

ChunkedDataSet is a dataset that lives on disk in compressed chunks, it has easy to use data loading and writing interfaces that involves making numpy-like slices.

Currently only .zarr directory stores are supported (i.e. the data will live in a folder on your local filesystem called <something>.zarr).

initialize(mode: str = 'w') → None

Initializes a new zarr dataset, creating the underlying arrays.

Keyword Arguments

mode (str) – Mode to open dataset in, should be something that supports writing. (default: {“w”})

open(mode: str = 'r', cached: bool = True, cache_size_bytes: int = 1000000000) → None

Opens a zarr dataset from disk from the path supplied in the constructor.

Keyword Arguments:

mode (str): Mode to open dataset in, default to read-only (default: {“r”}) cached (bool): Whether to cache files read from disk using a LRU cache. (default: {True}) cache_size (int): Size of cache in bytes (default: {1e9} (1GB))

Raises:

Exception: When any of the expected arrays (frames, agents, scenes) is missing or the store couldn’t be

opened.

class l5kit.data.DataManager

Bases: abc.ABC

abstract require(key: str) → str
class l5kit.data.LocalDataManager(local_data_folder: Optional[Union[str, pathlib.Path]] = None)

Bases: l5kit.data.local_data_manager.DataManager

LocalDataManager allows you to require data to be present in the subpath of a specific folder.

Example: Your data folder is set to "/tmp/my-data-folder", and you call local_data_manager.require("path/to/image.jpg"), it would check if "/tmp/my-data-folder/path/to/image.jpg" exists, and if so return that complete path ("/tmp/my-data-folder/path/to/image.jpg"), otherwise it raises an error.

In order of precedence, the local data folder is set by
  1. Passing in the path to the constructor of LocalDataManager

  2. Setting the L5KIT_DATA_FOLDER environment variable.

require(key: str) → str

Require checks whether the file with the given key is present in the local data folder, if it is not it raises an error. Returns the path to the file otherwise.

Parameters

key (str) – Path from the data folder where the file or folder should be present.

Returns

str – Filepath including the data folder where required key is present.

l5kit.data.filter_agents_by_frame(all_agents: numpy.ndarray, frame: numpy.ndarray) → numpy.ndarray

Given all agents, returns those in the given frame.

Parameters
  • all_agents (np.ndarray) – agents array

  • frame (np.ndarray) – frame whose agent_index_interval will be used to slice all_agents.

Returns

np.ndarray – An array of agents that is a subset of all_agents

l5kit.data.filter_agents_by_labels(agents: numpy.ndarray, threshold: float = 0.5) → numpy.ndarray

Filters an agents array, keeping those agents that meet the threshold.

Parameters

agents (np.ndarray) – Agents array

Keyword Arguments

threshold (float) – probability threshold for filtering (default: {0.5})

Returns

np.ndarray – A subset of input agents array.

l5kit.data.get_agent_by_track_id(all_agents: numpy.ndarray, frame: numpy.ndarray, track_id: int) → Optional[numpy.ndarray]

Return the agent object (np.ndarray) of a given track_id in a frame. Return None if the agent is not among those in the frame.

Parameters
  • all_agents (np.ndarray) – agents array

  • frame (np.ndarray) – given frame

  • track_id (int) – agent track id to select

Returns

Optional[np.ndarray] – Selected agent, or None if this agent is not present in given frame.

l5kit.data.get_combined_scenes(scenes: numpy.ndarray) → numpy.ndarray

Takes as input an np.ndarray or zarr array with scenes, and combines scenes that follow up eachother perfectly (i.e. from a single recording by the same host). Returns an np.ndarray of combined scenes.

Arguments:

scenes (np.ndarray): scenes

Returns:

np.ndarray – combined scenes where followup scenes have been merged.

l5kit.data.load_pose_to_ecef() → numpy.ndarray

Loads the pose to ECEF transformation matrix.

Returns

4x4 transformation matrix of dtype np.float64.

Return type

np.ndarray

l5kit.data.load_semantic_map(semantic_map_path: str) → dict

Loads and does preprocessing of given semantic map in binary proto format.

Parameters

semantic_map_path (str) – The path of the semantic map file to load, a binary proto.

Returns

A dict containing the semantic map contents.

Return type

dict