When should I use Lazy<T>?

You typically use it when you want to instantiate something the first time its actually used. This delays the cost of creating it till if/when it's needed instead of always incurring the cost.

Usually this is preferable when the object may or may not be used and the cost of constructing it is non-trivial.

You should try to avoid using Singletons, but if you ever do need to, Lazy<T> makes implementing lazy, thread-safe singletons easy:

public sealed class Singleton
{
    // Because Singleton's constructor is private, we must explicitly
    // give the Lazy<Singleton> a delegate for creating the Singleton.
    static readonly Lazy<Singleton> instanceHolder =
        new Lazy<Singleton>(() => new Singleton());

    Singleton()
    {
        // Explicit private constructor to prevent default public constructor.
        ...
    }

    public static Singleton Instance => instanceHolder.Value;
}

A great real-world example of where lazy loading comes in handy is with ORM's (Object Relation Mappers) such as Entity Framework and NHibernate.

Say you have an entity Customer which has properties for Name, PhoneNumber, and Orders. Name and PhoneNumber are regular strings but Orders is a navigation property that returns a list of every order the customer ever made.

You often might want to go through all your customer's and get their name and phone number to call them. This is a very quick and simple task, but imagine if each time you created a customer it automatically went and did a complex join to return thousands of orders. The worst part is that you aren't even going to use the orders so it is a complete waste of resources!

This is the perfect place for lazy loading because if the Order property is lazy it will not go fetch all the customer's order unless you actually need them. You can enumerate the Customer objects getting only their Name and Phone Number while the Order property is patiently sleeping, ready for when you need it.