How to change the color of the background of an image?

There are built in functions for this. Remove background

RemoveBackground[img]

Replace the background with a different color:

RemoveAlphaChannel[RemoveBackground[img], Green]

That works because removal of background is done with help of setting the alpha channel with mask:

AlphaChannel[RemoveBackground[img]]

img = ExampleData[{"TestImage", "House"}];

edge = ImageMultiply[EdgeDetect[ImageSubtract[#3, #] & @@ ColorSeparate[img], 4, 0.04],
    Image[SparseArray[{{i_, j_} /; Min[i, j] <= 100 -> 1}, ImageDimensions[img]]]]

binary = Binarize[ImageMultiply[ImageSubtract[#3, #] & @@ ColorSeparate[img],
    Image[SparseArray[{{i_, j_} /; Min[i, j] <= 100 -> 1}, ImageDimensions[img]]]], 0.12]

sky = Nest[ImageAdd[ImageFilter[Min, ImageFilter[Max, #, 1], 1], edge] &,
           ImageFilter[Min, ImageAdd[binary, edge], 1], 2]

ImageAdd[
   ImageMultiply[ImageApply[{#[[2]], #[[3]], #[[1]]} &, img], #],
   ImageMultiply[img, ColorNegate[#]]] & /@ {sky, sky - edge}

$\hspace{1.5cm}$

Here is how I would do it:

Get the Image:

img = ExampleData[{"TestImage", "House"}];

Use RegionBinarize to get a mask, and then close up the holes:

backgroundMask = ColorNegate[FillingTransform[Closing[RegionBinarize[img, ColorNegate @ Binarize @ img, 0.25], 5]]];

Choose the color to be targetted based on what occurs in the mask:

backgroundColor = RGBColor[Median[DeleteCases[Flatten[ImageData[backgroundMask img], 1], {0., 0., 0.}]]];

Make the replacement through the whole image:

colorReplacedImg = ColorReplace[img, backgroundColor -> Red];

Arithmetic so the replacement is only kept inside the mask:

resultImg = colorReplacedImg backgroundMask + ColorNegate[backgroundMask] img