Towards Light Weight Object Detection System

Transformers are a popular choice for classification tasks and as backbones for object detection tasks. However, their high latency brings challenges in their adaptation to lightweight classification and object detection systems. We present an approximation of the self-attention layers used in the transformer architecture. This approximation significantly reduces the latency of the classification system while incurring minimal loss in accuracy. We also present a method that uses a transformer encoder layer for multi-resolution feature fusion for object detection. This feature fusion improves the accuracy of the state-of-the-art lightweight object detection system without significantly increasing the number of parameters. These modules can be easily integrated into existing CNN and Transformer architecture to reduce latency and increase the accuracy of the system. Finally, we provide an abstraction for the transformer architecture called Generalized Transformer (gFormer) that can guide the design of novel transformer-like architectures.