Towards Memory-Efficient and Sustainable Machine Unlearning on Edge using Zeroth-Order Optimizer

Under increasing regulatory demands for data privacy, machine unlearning has emerged as a critical and effective technique for removing the influence of specific data points from a trained model. Although retraining from scratch typically yields the most ideal unlearning effect, it incurs prohibitive computational cost and resource waste. As a result, gradient ascent (GA)-based MU methods have become popular due to their efficiency. However, the reliance of GA on backpropagation leads to substantial memory overhead, rendering such methods impractical on memory-constrained devices such as mobile or edge platforms. In this paper, we propose a zeroth-order (ZO) alternative to conventional first-order-based backpropagation for performing GA-based unlearning. By eliminating the need for gradient computation, our approach significantly reduces memory consumption while maintaining the effectiveness of unlearning. Experiments demonstrate that ZO-GA achieves competitive MU performance, and notably exhibits greater training stability compared to conventional GA methods after unlearning is complete. Additionally, several intriguing observations are discussed, which may provide valuable insights for future research.