Cell-adjacency Graph of a Square Mesh

We can delete the rows in our incidence matrix that correspond to these edges of length 0.

pts = Flatten[Table[{i, j}, {i, 7}, {j, 5}], 1];

mesh = VoronoiMesh[pts, ImageSize -> Medium];
conn = mesh["ConnectivityMatrix"[1, 2]];

lens = PropertyValue[{mesh, 1}, MeshCellMeasure];
$threshold = 0.;
keep = Pick[Range[MeshCellCount[mesh, 1]], UnitStep[Subtract[$threshold, lens]], 0];
conn = conn[[keep]];

adj = Transpose[conn].conn;
centers = PropertyValue[{mesh, 2}, MeshCellCentroid];
g = AdjacencyGraph[adj, PlotTheme -> "Scientific", VertexCoordinates -> centers];

Show[mesh, g]

L1 = 2; L2 = 2;
mesh = VoronoiMesh @ Tuples[Range /@ {L1, L2}];
centers = Rationalize @ PropertyValue[{mesh, 2}, MeshCellCentroid];

g1 = VertexReplace[GridGraph[{L2, L1}, PlotTheme -> "Scientific", 
      VertexCoordinates -> centers[[Ordering @ centers]], 
    {v_ :> Ordering[centers][[v]]}];

Show[mesh, g1]

For L1 = 7; L2 = 5; the same approach gives

You can delete the unwanted edges using EdgeDelete:

Show[mesh, EdgeDelete[g, UndirectedEdge[a_, b_] /; 
   (FreeQ[0][Chop[Differences[PropertyValue[{g, #}, VertexCoordinates] & /@ {a, b}]]])]]

For g generated using pts = Flatten[Table[{i, j}, {i, 7}, {j, 5}], 1]; we get