Finetuning Cross-Encoder model results in constant prediction but when validated on test set it shows 100 F1 and accuracy by evaluator function

While using cross-encoder (CrossEncoder('distilroberta-base',num_labels=1)), I'm getting F1 score of 1 on the validation set with labels 0 and 1 after fine tuning. Although fine tuning is not happening properly. While inferencing it gives a constant score of 0.4118. Then also in logs, it shows 100 accuracy and F1 score. CEBinaryClassificationEvaluator: Evaluating the model on fk-dev dataset after epoch 1: Accuracy: 100.00 (Threshold: 0.4118) F1: 100.00 (Threshold: 0.4118) Precision: 100.00 Recall: 100.00 Average Precision: 30.00

I'm not able to understand why it is not learning anything where I have 11k rows in the data with the target label equally distributed.