Python Image Processing on Captcha how to remove noise

This is as far as I can get:

You probably know about medianBlur function which finds the median value in every kernel and substitute that value to kernel's center. We can do something similar to that but instead of the median, use the max value then the min value. With a median bluring too, I got some results. I know they are not perfect but I hope it gives you some ideas ( you can play with the sizes of the input image and the kernels, it may make the results a little better).

I don't have python installed right now, so I share the exact C++ code that I have used:

Mat im1 = imread("E:/1/3.jpg", 0);
Mat im2, im3;

im2 = Mat::zeros(im1.size(), CV_8U);
for (size_t i = 1; i < im1.rows-1; i++)
{
    for (size_t j = 1; j < im1.cols-1; j++)
    {
        double minVal, maxVal = 0;
        minMaxIdx(im1(Rect(j - 1, i - 1, 3, 3)), &minVal, &maxVal);
        im2.at<uchar>(i, j) = maxVal;
    }
}

imshow("(1) max bluring", im2);

medianBlur(im2, im2, 3);

imshow("(2) median bluring", im2);

im2.copyTo(im1);

im2 = Mat::zeros(im1.size(), CV_8U);
for (size_t i = 1; i < im1.rows - 1; i++)
{
    for (size_t j = 1; j < im1.cols - 1; j++)
    {
        double minVal, maxVal = 0;
        minMaxIdx(im1(Rect(j - 1, i - 1, 3, 3)), &minVal, &maxVal);
        im2.at<uchar>(i, j) = minVal;
    }
}

imshow("(3) min bluring", im2);

Mat tmp;
double st = threshold(im2, tmp, 10, 255, THRESH_OTSU);
threshold(im2, im2, st + 14, 255, THRESH_BINARY_INV);
//dilate(im2, im2, Mat::ones(3, 3, CV_8U));

imshow("(4) final", im2);

waitKey(0);

By the way in such cases, deep Learning methods like YOLO and RCNN are the best methods. Try them too.

Here is my solution,

Firstly I got the background pattern(Edited on paint by hand). From:

After that, I created a blank image to fill it with differences between the pattern and image.

img = Image.open("x.png").convert("RGBA")
pattern = Image.open("y.png").convert("RGBA")

pixels = img.load()
pixelsPattern = pattern.load()

new = Image.new("RGBA", (150, 50))
pixelNew = new.load()

for i in range(img.size[0]):
    for j in range(img.size[1]):
         if(pixels[i,j] != pixelsPattern[i,j]):
             pixelNew[i,j] = pixels[i,j]

new.save("differences.png")

Here are the differences..

and finally, I added blur and cleared the bits that is not black.

Result :

With pytesseract result is 2041, it is wrong for this image but the general rate is around %60.