CyC dojo

The design, implementation, training and maintenance of the filters within a DataBlock is enabled by the CyC dojo system. The dojo is mainly written in Python and executed on dedicated computers, such as powerful HPC clusters. It is composed of a development and testing DataBlock and a set of libraries which can be used for the rapid prototyping of the filters.

The low-level AI libraries of choice for designing and training DNNs are PyTorch and ONNX. PyTorch has been chosen due to its flexibility and support for vision based neural networks, while ONNX has been chosen for its open-source format, stability and portability.

A common base DNN structure has been implemented on top of PyTorch using, CyC dnn, our AI inference engine. Each DNN exported from the dojo is inherited from this base structure. A DNN architecture is given in a configuration file which describes its input/output branches and tensor shapes. The same configuration file is loaded by the DNN filters in the CyC inference.

The training pipeline works by reading stored datastreams acquired from various DataBlocks running on robotic systems. In our CyC comm example, the Dojo's DataBlock stores the camera, LiDAR and control signals from the legged robot DataBlock, as well as the drone's camera. These are later used for training the perception DNN.

A neural network is trained by evaluating pairs of data samples \(\mathbf{x}^{\langle t-\tau_i, t \rangle}\) and labels \(\mathbf{y}^{\langle t+1, t+\tau_o \rangle}\). CyberCortex.AI treats both the samples and the labels as different datastreams. \(\mathbf{x}^{\langle t-\tau_i, t \rangle}\) are input datastreams, while the labels \(\mathbf{y}^{\langle t+1, t+\tau_o \rangle}\) are output datastreams.

An Annotator is provided for visualizing the recorded datastreams. The figure below shows a snapshot of the Annotator during the analysis of a saved DataBlock. The recorded datastreams are shown on the left side of the tool, while the center area shows annotated regions of interest corresponding to the objects present in the scene.