Treffer: 基于问题导向式提示调优小样本文本分类.

In low-resource scenarios, prompt tuning has better classification performance than fine-tuning with additional classifiers, but it takes a lot of effort to design a better prompt template for the task. Aiming at this problem, this paper proposed a few-shot text classification method for question-oriented prompt-tuning. Firstly, constructing template for question form using dataset labels and trainable continuous prompts, and the optimal prompt template learned from pre-trained models. Then, using the template to fill in each input text, converted the text classification task into a cloze-style task. Finally, using external knowledge and two refinement methods constructed verbalizer, and the final classification results obtained by the mapping relationship between the predicted label words and the classification labels. Experiments on public datasets AG s News and IMDB demonstrate that the method improves performance on 5-shot, 10-shot and 20-shot tasks, and the accuracy improves by 0.81 and 1.36 percentage points on the 5-shot task, which is not only easy to implement but also achieves the optimal performance compared with the baseline model. [ABSTRACT FROM AUTHOR]

低资源场景下提示调优比带有额外分类器的通用微调方法 (fine-tuning) 分类性能好, 但提示调优中设计一个较好的提示模板和标签词映射器需要花费大量的精力。针对该问题, 提出问题导向式提示调优的小样本分类方法 (question-oriented prompt-tuning, QPT) 。首先, 利用数据集标签和可训练的连续提示构建提问形式的模板, 通过预训练模型学习到最优提示模板; 然后, 每条样本用模板进行填充, 将文本分类任务转换成完形填空任务; 最后, 使用外部知识及两种细化方法构建标签词映射器, 通过预测的标签词与分类标签的映射关系得出分类结果。在公开数据集AG’s News和IMDB上进行实验, 结果表明, 该方法在5-shot、10-shot和20-shot任务上性能均有所提升, 在5-shot任务上准确率分别提高了0.81和1.36百分点, 与基线模型相比, 不仅易于实现且性能取得了最优。 [ABSTRACT FROM AUTHOR]