Have you checked whether variant prompts improve quality?

[Gurkenglas]

Out of 20 question-answer pairs you could pick one at random, put it at the end, and check whether shuffling the rest impacts quality of the completion, or more objectively the probability of generating the label you specified. Beyond shuffling, you could delete one random example, or have GPT generate more examples, descending towards a prompt that performs well on the original 20. Though I'm not sure how one'd keep from having to manually combat overfitting!