Treffer: One-step model agnostic meta-learning using two-phase switching optimization strategy

Conventional training mechanisms often encounter limited classification performance due to the need of large training samples. To counter such an issue, the field of meta-learning has shown great potential in fine tuning and generalizing to new tasks using mini dataset. As a variant derived from the concept of Model Agnostic Meta-Learning (MAML), an one-step MAML incorporated with the two-phase switching optimization strategy is proposed in this paper to improve performance using less iterations. One-step MAML uses two loops to conduct the training, known as the inner and the outer loop. During the inner loop, gradient update is performed only once per task. At the outer loop, gradient is updated based on losses accumulated by the evaluation set during each inner loop. Several experiments using the BERT-Tiny model are conducted to analyze and compare the performance of the one-step MAML with five benchmark datasets. The performance of evaluation shows that the best loss and accuracy can be achieved using one-step MAML that is coupled with the two-phase switching optimizer. It is also observed that this combination reaches its peak accuracy with the fewest number of steps.