Fine-Tune training for new and rare Chinese characters in Traditional Chinese

[6692a]

Hello
Can I ask a question?
How to fine-tune training for new and rare Chinese characters in Traditional Chinese?
For rare words that are not established in the vocabulary, the official teaching is currently used, and the parameters of config.yaml are
FT: True, new_prediction: True

A small number of training sets/validation sets are generated, first generated with only one font, but with different background colors and tilts. Finally, the pre-trained model is used for fine-tuning training, but the results are almost always worse than the original pre-trained model.
In other words, The original official Traditional Chinese model worked well, but after the training steps, the text content that could have been recognized normally had errors