Setting up uniform distribution over non-rectangular support

You can use Triangle as a region. I use Region here purely for visualization. It can be omitted.

reg = Region@Triangle[{{0, 0}, {1, 0}, {0, 1}}]

As Roman said, you can apply RegionMoment to it:

RegionMoment[reg, {1, 0}]
(* 1/6 *)

RegionMoment[reg, {1, 0}]/RegionMoment[reg, {0, 0}]
(* 1/3 *)

For more complicated things, you can evaluate any integral over the region:

Integrate[Indexed[x, 1], x \[Element] reg]
(* 1/6 *)

For numerical verification, you can use RandomPoint.

Moment[RandomPoint[reg, 10000], 1]
(* {0.332256, 0.335306} *)

A Dirichlet distribution with parameters $(1,1,1)$ will represent a uniform distribution over exactly the triangle you seek:

ListPlot[
  RandomVariate[DirichletDistribution[{1, 1, 1}], 10000],
  AspectRatio -> Automatic
]

You can then obtain expectations, moments, etc using the usual statistical machinery in MMA. So for instance, to reproduce the results shown in Roman's answer,

$E(X)$:

Moment[DirichletDistribution[{1, 1, 1}], {1, 0}]     (* Out: 1/3  *)

$E(Y^2)$:

Moment[DirichletDistribution[{1, 1, 1}], {0, 2}]     (* Out: 1/6  *)

$E(XY)$:

Moment[DirichletDistribution[{1, 1, 1}], {1, 1}]     (* Out: 1/12 *)

After some further investigation, it appears I can do what I want by using

dist = ProbabilityDistribution[2 Boole[0 <= x <= 1 && 0 <= y <= 1 - x], {x, 0, 1}, {y, 0, 1}]

Then e.g. $E[X]$, $E[Y^2]$, and $E[XY]$ are given respectively by

Moment[dist, {1, 0}]
(* 1/3 *)

Moment[dist, {0, 2}]
(* 1/6 *)

Moment[dist, {1, 1}]
(* 1/12 *)