Is there a specific way to use tuples as set elements in Swift?

Rather than using tuples to represent points, use the built in type CGPoint. You can extend CGPoint to be hashable by extending it:

import UIKit

extension CGPoint: Hashable {
    public var hashValue: Int {
        return self.x.hashValue << sizeof(CGFloat) ^ self.y.hashValue
    }
}

// Hashable requires Equatable, so define the equality function for CGPoints.
public func ==(lhs: CGPoint, rhs: CGPoint) -> Bool {
    return CGPointEqualToPoint(lhs, rhs)
}

Now that CGPoint is Hashable, you can use it in sets. For example:

let point1 = CGPoint(x: 0, y: 1)
let point2 = CGPoint(x: 0, y: 2)
let point3 = CGPoint(x: 1, y: 1)
let point4 = CGPoint(x: 3, y: 3)
let point5 = CGPoint(x: 3, y: 3)  // Intentionally the same as point4 to see the effect in union and difference.

let set1 = Set([point1, point2 , point5])
let set2 = Set([point4, point3])

let union = set1.union(set2) // -> {{x 0 y 2}, {x 3 y 3}, {x 0 y 1}, {x 1 y 1}}
let difference = set1.intersect(set2) // -> {{x 3 y 3}}