What is the difference between self-types and trait subclasses?

It is predominately used for Dependency Injection, such as in the Cake Pattern. There exists a great article covering many different forms of dependency injection in Scala, including the Cake Pattern. If you Google "Cake Pattern and Scala", you'll get many links, including presentations and videos. For now, here is a link to another question.

Now, as to what is the difference between a self type and extending a trait, that is simple. If you say B extends A, then B is an A. When you use self-types, B requires an A. There are two specific requirements that are created with self-types:

1. If B is extended, then you're required to mix-in an A.
2. When a concrete class finally extends/mixes-in these traits, some class/trait must implement A.

Consider the following examples:

scala> trait User { def name: String }
defined trait User

scala> trait Tweeter {
     |   user: User =>
     |   def tweet(msg: String) = println(s"$name: $msg")
     | }
defined trait Tweeter

scala> trait Wrong extends Tweeter {
     |   def noCanDo = name
     | }
<console>:9: error: illegal inheritance;
 self-type Wrong does not conform to Tweeter's selftype Tweeter with User
       trait Wrong extends Tweeter {
                           ^
<console>:10: error: not found: value name
         def noCanDo = name
                       ^

If Tweeter was a subclass of User, there would be no error. In the code above, we required a User whenever Tweeter is used, however a User wasn't provided to Wrong, so we got an error. Now, with the code above still in scope, consider:

scala> trait DummyUser extends User {
     |   override def name: String = "foo"
     | }
defined trait DummyUser

scala> trait Right extends Tweeter with User {
     |   val canDo = name
     | }
defined trait Right 

scala> trait RightAgain extends Tweeter with DummyUser {
     |   val canDo = name
     | }
defined trait RightAgain

With Right, the requirement to mix-in a User is satisfied. However, the second requirement mentioned above is not satisfied: the burden of implementing User still remains for classes/traits which extend Right.

With RightAgain both requirements are satisfied. A User and an implementation of User are provided.

For more practical use cases, please see the links at the start of this answer! But, hopefully now you get it.

Self types allow you to define cyclical dependencies. For example, you can achieve this:

trait A { self: B => }
trait B { self: A => }

Inheritance using extends does not allow that. Try:

trait A extends B
trait B extends A
error:  illegal cyclic reference involving trait A

In the Odersky book, look at section 33.5 (Creating spreadsheet UI chapter) where it mentions:

In the spreadsheet example, class Model inherits from Evaluator and thus gains access to its evaluation method. To go the other way, class Evaluator defines its self type to be Model, like this:

package org.stairwaybook.scells
trait Evaluator { this: Model => ...

Hope this helps.

One additional difference is that self-types can specify non-class types. For instance

trait Foo{
   this: { def close:Unit} => 
   ...
}

The self type here is a structural type. The effect is to say that anything that mixes in Foo must implement a no-arg "close" method returning unit. This allows for safe mixins for duck-typing.