How to merge several partially overlapping screenshots into one image?
Very roughly (using image1 and image2):
i1 = ImageData[image1];
i2 = ImageData[image2];
css = LongestCommonSubsequence[i1, i2];
s1 = SequencePosition[i1, css];
s2 = SequencePosition[i2, css];
Join[Take[i1, s1[[1,2]] ], Take[i2, {s2[[1,2]] + 1, -1}]] // Image
That seems to work for me and gives the images concatenated together.
This solves your "entangled images" case.
ClearAll[getMinRowFromImage, getBestMatch, findAllMatches, joinImages,
findImagesSequence, getIntensity];
getIntensity[i_Image, pos_List] :=
(* Calculates the intensity of a pixel for any image color space.*)
First@ColorConvert[Flatten@{PixelValue[i, pos]}, ImageColorSpace[i] -> "Grayscale"]
getMinRowFromImage[i_Image] :=
(* returns the min intensity value for an image and the row where it occurs*)
{getIntensity[i, #[[1]]], #[[1, 2]]} &@PixelValuePositions[i, "Min"]
getBestMatch[i_List, {m_Integer, n_Integer}, lines_Integer] :=
(* Finds the best match between i[[n]] and the last lines of i[[m]].
Note that "Padding -> None" is what allows this to run within
reasonable time, calculating the correlation only by rows and
thus returning single column image*)
getMinRowFromImage@ImageCorrelate[i[[n]], ImageTake[i[[m]], -lines],
CorrelationDistance, Padding -> None]
findAllMatches[i_List, lines_Integer, sens_Real] :=
(*forms all permutations {m,n} from the list of images
calculates the best match between all pairs and
select those below the sensitivity parameter
Could be done more efficiently, since we expect only one
valid continuation foreach image, so we don't really need to
calculate them all*)
With[{permutations = Position[IdentityMatrix[Length@i], 0]},
Select[{#, getBestMatch[i, #, lines]} & /@ permutations, #[[2, 1]] < sens &]
]
.
findImagesSequence[g_Graph] :=
(* g is a directed PathGraph whose wheights are the best match lines
for each pair. So, the Max of the distance matrix is at the
{head, tail} pair. We find the (only) path that goes from head
to tail, hence finding the right image sequence. There is a function
in Mma help that does the same for undirected Graphs, but I think
this one is better for our purpose *)
FindShortestPath[g, Sequence @@ VertexList[g][[First@Position[#, Max@#] &
[GraphDistanceMatrix@g /. ∞ -> 0]]]]
joinImages[i_List, linesToMatch_Integer, sens_Real] :=
(* Ensambles a full image from parts consisting in same column width
images matching from some row onwards for a minumum of "linesToMatch".
The sensitivity parameter seeme not really needed but perhaps useful
for very similar images
*)
Module[{matches, g, seqimg, fromline},
(* First find the pairs matching *)
matches = findAllMatches[i, linesToMatch, sens];
(*Now we build up the whole sequence, from head to tail.
Graph features are great for that*)
g = Graph[Rule @@@ #[[All, 1]], EdgeWeight -> Flatten@#[[All, 2, 2]]] &@matches;
seqimg = findImagesSequence[g];
fromline = PropertyValue[{g, DirectedEdge @@ #}, EdgeWeight] & /@
Partition[seqimg, 2, 1];
(*Finally assemble the whole thing*)
ImageAssemble@Join[{{i[[First@seqimg]]}},
MapThread[{ImageTrim[#1, {{0, #2}, {ImageDimensions[#1][[2]], 0}}]} &,
{i[[Rest@seqimg]], fromline}]]
]
Usage:
l = {"http://i.stack.imgur.com/IXFEq.png",
"http://i.stack.imgur.com/FMtjm.png",
"http://i.stack.imgur.com/aj8a1.png"};
(*Resize and GrayScale for speed*)
i = ImageResize[#, 500] & /@ (ColorConvert[#, "Grayscale"] & /@ Import /@ l);
sensitivity = .1;
lnsTomatch = 150;
joinImages[i, lnsTomatch, sensitivity]
Coloring not included in this code, used here to show how the image was composed from the three snapshots. The last step (the joining of images, excluding the import part) is instantaneous in my machine.
This answer is late but stronger,I have packed it in a custom function and name after CombineImage.And since for merge some cell phone screenshots,the height should be same.Considering some APP has bottom menu bar,this answer actually can adapt that more complicated case than my original question
CombineImage[imgs_List] :=
Module[{bottom, top, crop, data, threshold, height, bins, sortBins,
seq, oderImgs, order}, {bottom, top} =
Min /@ Transpose[
BlockMap[Last[BorderDimensions[ImageSubtract @@ #]] &, imgs, 2]];
crop = ImageTake[#, {top + 1,
Last[ImageDimensions[First[imgs]]] - bottom}] & /@ imgs;
data = ImageData /@ crop;
threshold =
Min[ImageMeasurements[ColorConvert[#, "Grayscale"] & /@ crop,
"Mean"]];
height = Last[ImageDimensions[First[crop]]];
bins = Binarize[#, threshold] & /@ crop;
sortBins =
FindHamiltonianPath[
RelationGraph[
Sign[N[Mean[#]/height] & /@
LongestCommonSubsequencePositions @@
ImageData /@ {##} - .5] === {1, -1} &, bins]];
order = FindPermutation[bins, sortBins]; {oderImgs, data} =
Permute[#, order] & /@ {imgs, data};
seq = Developer`PartitionMap[
LongestCommonSubsequencePositions @@ # &, ImageData /@ sortBins,
2, 1];
Image[Join[
Take[ImageData[First[oderImgs]], seq[[1, 1, 1]] + top - 1],
Sequence @@
MapThread[
Take, {data[[2 ;; -2]],
Partition[First /@ Catenate[seq][[2 ;; -2]], 2]}],
Take[ImageData[Last[oderImgs]],
seq[[-1, -1, 1]] - bottom - height - 1]]]]
Usage
I have provided four images to test in following
imgs = Import /@ {"https://i.stack.imgur.com/gmfPU.png",
"https://i.stack.imgur.com/sHJat.png",
"https://i.stack.imgur.com/LSbUk.png",
"https://i.stack.imgur.com/sOoGi.png"};
CombineImage[RandomSample[imgs]]
Note I use a RandomSample to show I don't care the order of image list.
Code interpretation
Get the crop images which delete the margin on top and the bottom.Note the notification bar are not very accordance
{bottom, top} =
Min /@ Transpose[
BlockMap[Last[BorderDimensions[ImageSubtract @@ #]] &, imgs, 2]];
crop = ImageTake[#, {top + 1,
Last[ImageDimensions[First[imgs]]] - bottom}] & /@ imgs
Calculate a threshold for binarize the image for speed.And for showing all valid information as much as possible,I will use a mean value.We cannot use Grayscale here like this anser by Dr. belisarius,which will make the imgs have little difference even the part is same tatally.
data = ImageData /@ crop;
threshold =
Min[ImageMeasurements[ColorConvert[#, "Grayscale"] & /@ crop,
"Mean"]];
0.824459
If the following page will continue the above page,the follow centre must be larger than $50\%height$ of the image,this is a explanatory drawing when the next page cover $90%$ of the above.The center of the next is $5.5(>50\% height)$ still.
Since this,we can sort the imgs,I will use RelationGraph to do this
height = Last[ImageDimensions[First[crop]]];
bins = Binarize[#, threshold] & /@ crop;
toImgs = Thread[bins -> imgs];
toCrop = Thread[bins -> crop];
sortImgs =
FindHamiltonianPath[
RelationGraph[Sign[N[Mean[#]/height] & /@
LongestCommonSubsequencePositions @@
ImageData /@ {##} - .5] == {1, -1} &, bins]]
Use LongestCommonSubsequencePositions to find the overlapping part.Note we process all of imgs which have been croped.So when we recover it,we should use toImgs and toCrop
seq = Developer`PartitionMap[LongestCommonSubsequencePositions @@ # &,
ImageData /@ sortImgs, 2, 1];
Image[Join[
Take[ImageData[First[sortImgs] /. toImgs],
seq[[1, 1, 1]] + top - 1],
Sequence @@
MapThread[
Take, {Map[ImageData, sortImgs /. toCrop][[2 ;; -2]],
Partition[First /@ Catenate[seq][[2 ;; -2]], 2]}],
Take[ImageData[Last[sortImgs] /. toImgs],
seq[[-1, -1, 1]] - bottom - height - 1]]]
Then we get the long combine image.And I also give clor image for comparison
