Finding CheckerBoard Points in opencv for any random ChessBoard( pattern size not known)

Short answer: you cannot.

The OpenCV checkerboard detection code assumes that the pattern is uniform (all squares have the same size) and therefore, in order to uniquely locate its position in the image, the following two conditions must be true:

1. The pattern is entirely visible.
2. The pattern has a known numbers of rows and columns.

If either 1 or 2 is violated there is no way to know which corner is, say, the "top left" one.

For a more general case, and in particular if you anticipate that the pattern may be partially occluded, you must use a different algorithm and a non-uniform pattern, upon which corners can be uniquely identified.

There are various way to do that. My favorite pattern is Matsunaga and Kanatani's "2D barcode" one, which uses sequences of square lengths with unique crossratios. See the paper here. In order to match it, once you have sorted the corners into a grid, you can use a simple majority voting algorithm:

• Precompute the crossratios of all the pattern's consecutive 4-tuples of corners, in both the horizontal and vertical directions.
• Do the above for the detected corners in the grid.
• For every possible horizontal shift
  • Over every row
    • Accumulate the number of crossratios that agree within a threshold
• Select the horizontal shift with the highest number of agreements.
• Repeat the above for every possible vertical shift, counting crossratios along the columns.

Placing the detected corners in a grid can be achieved in various ways. There is an often-rediscovered algorithm that uses topological proximity. The idea is to first associate each corner to all the squares within a small window of it, thus building a corner->squares table, and then traverse it as a graph to build a global table of the offsets of each corner from one another.