Avoid duplicates in HashSet<double> for slightly different values

Well, the current implementation of Equals

   return a - b < 1e-3;

is incorrect one. Equals must be

1. Equals(a, a) == true;
2. Symmetric: Equals(a, b) == Equals(b, a);
3. Transitive Equals(a, b) && Equals(b, c) leads to Equals(a, c);

Conditions 2 and 3 are not hold in the current implementation. It's easy to repair the second condition with a help of Math.Abs; the third one is real difficulty: for arbitrary positive tolerance (which is 1e-3 in your case) we have

   a == a + tolerance == a + 2 * tolerance == ... == a + n * tolerance  

which means

   a == a + n * tolerance

for abitrary large n; thus a == b for all a and b (all numbers are equal).

As a partial (but valid) solution you can try rounding the values:

   class HeightEqualityComparer : IEqualityComparer<double>
   {
       public bool Equals(double a, double b)
       {
           return Math.Round(a, 3) == Math.Round(b, 3);
       }

       public int GetHashCode(double value)
       {
           return Math.Round(value, 3).GetHashCode();
       }
   } 

Note, that we have to change GetHashCode

You'd need to round your values in GetHashCode to the same precision you are eliminating in the equality.