CyC dnn
Real-Time Embodied IntelligenceCyC dnn: Real-Time Embodied Intelligence
One of the main components in the Robotics Toolkit is the AI Inference Engine illustrated in the overall diagram of CyberCortex.AI. It is used for passing input datastreams through the layers of a Deep Neural Network (DNN). The architecture of the inference engine has been designed in conjunction with the CyberCortex.AI.dojo system, which is used for designing and training DNNs. Namely, DNNs constructed and trained on our CyC dojo are automatically sent to the DNN filters in the registered DataBlocks.
Temporal sequences of multiple input datastreams \(\mathbf{x}^{\langle t-\tau_i, t \rangle}\) are used to produce temporal sequences of output datastreams \(\mathbf{y}^{\langle t+1, t+\tau_o \rangle}\):
\[ \mathbf{x}^{\langle t-\tau_i, t \rangle} = [\mathbf{x}^{\langle t-\tau_i \rangle}, \dots, \mathbf{x}^{\langle t \rangle}] \]
\[ \mathbf{y}^{\langle t+1, t+\tau_o \rangle} = [\mathbf{y}^{\langle t+1 \rangle}, \mathbf{y}^{\langle t+2 \rangle}, \dots, \mathbf{y}^{\langle t+\tau_o \rangle}] \]
Where \(t\) is the discrete timestamp, \(\tau_i\) is the temporal length of the input samples and \(\tau_o\) is the temporal prediction horizon. The elements of \(\mathbf{x}^{\langle t-\tau_i, t \rangle}\) represent cached data from the Temporal Addressable Memory (TAM).
Each datastream is passed to the input layer of the DNN as a separate input branch of variable batch size. Similarly, the output layer of the DNN provides datastreams through its output branches. The Open Neural Network eXchange (ONNX) format was chosen for storing the structure and weights of the developed neural networks.