Ahmed Khamis Abdullah Al Ghadani

Ahmed Khamis Abdullah Al Ghadani

National University of Science and Technology

Ahmed Khamis Abdullah's lectures

lecture cover

Ahmed Khamis Abdullah Al Ghadani · AIAI 2020

Tensor-based CUDA Optimization for ANN Inferencing using Parallel Acceleration on Embedded GPU

With image processing, robots acquired visual perception skills; enabling them to become autonomous. Since the emergence of Artificial Intelligence (AI), sophis-ticated tasks such as object identification have become possible through inferenc-ing Artificial Neural Networks (ANN). Be that as it may, Autonomous Mobile Robots (AMR) are Embedded Systems (ESs) with limited on-board resources. Thus, efficient techniques in ANN inferencing are required for real-time perfor-mance. This paper presents the process of optimizing ANNs inferencing using tensor-based optimization on embedded Graphical Processing Unit (GPU) with Computer Unified Device Architecture (CUDA) platform for parallel acceleration on ES. This research evaluates renowned network, namely, You-Only-Look-Once (YOLO), on NVIDIA Jetson TX2 System-On-Module (SOM). The find-ings of this paper display a significant improvement in inferencing speed in terms of Frames-Per-Second (FPS) up to 3.5 times the non-optimized inferencing speed. Furthermore, the current CUDA model and TensorRT optimization tech-niques are studied, comments are made on its implementation for inferencing, and improvements are proposed based on the results acquired. These findings will contribute to ES developers and industries will benefit from real-time perfor-mance inferencing for AMR automation solutions