Finetuing BERT with masking and having multiple correct labels
I aim to fine-tune a BERT model for a specific task involving simple arithmetic operations like "5 + 3 = 8" or "7 plus 2 equals 9". My dataset comprises thousands of examples where one operand, operator, or result is masked. For instance:
Masked: "1 added to [MASK] equals 7", Label: "1 added to 6 is equal
to 7"
Masked: "6 plus 5 [MASK] 11", Label: "6 plus 5 gives 11"
The challenge lies in ensuring that multiple correct labels are accepted for a masked sample during training. For instance,...