ValueError: pos_label=1 is not a valid label: array(['neg', 'pos'], dtype='<U3')

recall_average = recall_score(Y_test, y_predict, average="binary", pos_label="neg")

Use "neg" or "pos" as pos_label and this error won't raise again.

When you face this error it means the values of your target variable are not the expected one for recall_score(), which by default are 1 for positive case and 0 for negative case [This also applies to precision_score()]

From the error you mentioned:

pos_label=1 is not a valid label: array(['neg', 'pos']

It is clear that values for your positive scenarios is pos instead of 1 and for the negative neg instead of 0.

Then you have to options to fix this mismatch:

• Changing the value default in the recall_score() to consider positive scenarios when pos appears with:

recall_average = recall_score(Y_test, y_predict, average="binary", pos_label='pos') 

• Changing the values of the target variable in your dataset to be 1 or 0

Y_test = Y_test.map({'pos': 1, 'neg': 0}).astype(int)