How do I reduce to a basis?

As long as you are just working with vectors, you can use RowReduce to remove linearly dependent elements in set.

Taking your example,

set = {X, Y, X + Y, X + Z, 2 X + Z}

you can, assuming X, Y and Z are some vectors in a vector space, define associated vectors via

setVecs = set /. {X -> {1, 0, 0}, Y -> {0, 1, 0}, Z -> {0, 0, 1}}

Edit: As suggested in the comments, this can be automated, e.g. via

 setVecs = set /. With[{vars = Variables[set]}, 
      MapIndexed[Rule[#1, UnitVector[Length@vars, First@#2]] &, vars]]

Since

MatrixRank[setVecs]

 3

we need to remove two elements. Which ones to remove can be found by looking at the pivots of

rr = setVecs // Transpose // RowReduce

{{1, 0, 1, 0, 1}, 
 {0, 1, 1, 0, 0}, 
 {0, 0, 0, 1, 1}}

The third element is therefore a linear combination of the first two, and the fifth element is the sum off the first and fourth element of set. To get the basis element positions in set we can use

basisElements = Flatten[FirstPosition[#, 1, Nothing] & /@ rr]

{1, 2, 4}

The associated elements of set can then be extracted via

basis = set[[basisElements]]]

{X, Y, X + Z}

One of the things I remember from my linear algebra class in 1983:

set = {X, Y, X + Y, X + Z, 2 X + Z};

Extract[set, 
 FirstPosition[#, 1, Nothing] & /@ 
  RowReduce@Transpose@CoefficientArrays[set, Variables@set][[2]]
 ]

(*  {X, Y, X + Z}  *)