ViT-Slim: A Genetic Algorithm-based NAS Framework for Efficient Vision Transformer Design

In industrial applications, pre-trained models fine-tuned on custom datasets often exceed the specific requirements of their target domains due to their oversized and resource-intensive nature. To address these inefficiencies, this study introduces ViT-Slim, a Genetic Algorithm-based Neural Architecture Search (GA-NAS) framework designed for optimizing Vision Transformer (ViT) architectures. ViT-Slim balances model size and performance by leveraging the strengths of genetic algorithms to explore and optimize design configurations. The framework integrates Dense Relative Localization (DrLoc) to inject inductive bias, enhancing ViT's ability to process custom datasets efficiently, even with limited data. Experimental validation on a custom depth dataset demon-strates that ViT-Slim achieves significant resource efficiency, reducing memory usage by up to 79.18% and parameters by 83.09%, with less than 1% accuracy loss compared to the baseline ViT-Small.