Immutability and reordering
UPDATE Feb10
I'm getting convinced that we should separate 2 phases: compilation and execution.
I think that the decision factor whether it is allowed to return null or not is what the bytecode is. I made 3 examples:
Example 1:
The original source code, literally translated to bytecode:
if (resource == null)
resource = new Resource(); // unsafe publication
return resource;
The bytecode:
public static Resource getInstance();
Code:
0: getstatic #20; //Field resource:LResource;
3: ifnonnull 16
6: new #22; //class Resource
9: dup
10: invokespecial #24; //Method Resource."<init>":()V
13: putstatic #20; //Field resource:LResource;
16: getstatic #20; //Field resource:LResource;
19: areturn
This is the most interesting case, because there are 2 reads (Line#0 and Line#16), and there is 1 write inbetween (Line#13). I claim that it is not possible to reorder, but let's examine it below.
Example 2:
The "complier optimized" code, which can be literally re-converted to java as follows:
Resource read = resource;
if (resource==null)
read = resource = new Resource();
return read;
The byte code for that (actually I produced this by compiling the above code snippet):
public static Resource getInstance();
Code:
0: getstatic #20; //Field resource:LResource;
3: astore_0
4: getstatic #20; //Field resource:LResource;
7: ifnonnull 22
10: new #22; //class Resource
13: dup
14: invokespecial #24; //Method Resource."<init>":()V
17: dup
18: putstatic #20; //Field resource:LResource;
21: astore_0
22: aload_0
23: areturn
It is obvious, that if the compiler "optimizes", and the byte code like above is produced, a null read can occur (for example, I refer to Jeremy Manson's blog)
It is also interesting to see that how a = b = c is working: the reference to new instance (Line#14) is duplicated (Line#17), and the same reference is stored then, first to b (resource, (Line#18)) then to a (read, (Line#21)).
Example 3:
Let's make an even slighter modification: read the resource only once! If the compiler starts to optimize (and using registers, as others mentioned), this is better optimization than above, because Line#4 here is a "register access" rather than a more expensive "static access" in Example 2.
Resource read = resource;
if (read == null) // reading the local variable, not the static field
read = resource = new Resource();
return read;
The bytecode for Example 3 (also created with literally compiling the above):
public static Resource getInstance();
Code:
0: getstatic #20; //Field resource:LResource;
3: astore_0
4: aload_0
5: ifnonnull 20
8: new #22; //class Resource
11: dup
12: invokespecial #24; //Method Resource."<init>":()V
15: dup
16: putstatic #20; //Field resource:LResource;
19: astore_0
20: aload_0
21: areturn
It is also easy to see, that it is not possible to get null from this bytecode since it is constructed the same way as String.hashcode(), having only 1 read of the static variable of resource.
Now let's examine Example 1:
0: getstatic #20; //Field resource:LResource;
3: ifnonnull 16
6: new #22; //class Resource
9: dup
10: invokespecial #24; //Method Resource."<init>":()V
13: putstatic #20; //Field resource:LResource;
16: getstatic #20; //Field resource:LResource;
19: areturn
You can see that Line#16 (the read of variable#20 for return) most observe the write from Line#13 (the assignation of variable#20 from the constructor), so it is illegal to place it ahead in any execution order where Line#13 is executed. So, no reordering is possible.
For a JVM it is possible to construct (and take advantage of) a branch that (using certain extra conditions) bypasses the Line#13 write: the condition is that the read from variable#20 must not be null.
So, in neither case for Example 1 is possible to return null.
Conclusion:
Seeing the examples above, a bytecode seen in Example 1 WILL NOT PRODUCE null. An optimized bytecode like in Example 2 WILL PROCUDE null, but there is an even better optimization Example 3, which WILL NOT PRODUCE null.
Because we cannot be prepared for all possible optimization of all the compilers, we can say that in some cases it is possible, some other cases not possible to return null, and it all depends on the byte code. Also, we have shown that there is at least one example for both cases.
Older reasoning: Referring for the example of Assylias: The main question is: is it valid (concerning all specs, JMM, JLS) that a VM would reorder the 11 and 14 reads so, that 14 will happen BEFORE 11?
If it could happen, then the independent Thread2could write the resource with 23, so 14 could read null. I state that it is not possible.
Actually, because there is a possible write of 13, it would not be a valid execution order. A VM may optimize the execution order so, that excludes the not-executed branches (remaining just 2 reads, no writes), but to make this decision, it must do the first read (11), and it must read not-null, so the 14 read cannot precede the 11 read. So, it is NOT possible to return null.
Immutability
Concerning immutability, I think that this statement is not true:
UnsafeLazyInitialization is actually safe if Resource is immutable.
However, if the constructor is unpredictable, interesting results may come out. Imagine a constructor like this:
public class Resource {
public final double foo;
public Resource() {
this.foo = Math.random();
}
}
If we have tho Threads, it may result, that the 2 threads will receive a differently-behaving Object. So, the full statement should sound like this:
UnsafeLazyInitialization is actually safe if Resource is immutable and its initialization is consistent.
By consistent I mean that calling the constructor of the Resource twice we will receive two objects that behave exactly the same way (calling the same methods in the same order on both will yield the same results).
The confusion I think you have here is what the author meant by safe publication. He was referring to the safe publication of a non-null Resource, but you seem to get that.
Your question is interesting - is it possible to return a null cached value of resource?
Yes.
The compiler is allowed to reorder the operation like such
public static Resource getInstance(){
Resource reordered = resource;
if(resource != null){
return reordered;
}
return (resource = new Resource());
}
This doesn't violate the rule of sequential consistency but can return a null value.
Whether or not this is the best implementation is up for debate but there is no rules to prevent this type of reordering.
After applying the JLS rules to this example, I have come to the conclusion that getInstance can definitely return null. In particular, JLS 17.4:
The memory model determines what values can be read at every point in the program. The actions of each thread in isolation must behave as governed by the semantics of that thread, with the exception that the values seen by each read are determined by the memory model.
It is then clear that in the absence of synchronization, null is a legal outcome of the method since each of the two reads can observe anything.
Proof
Decomposition of reads and writes
The program can be decomposed as follows (to clearly see the reads and writes):
Some Thread
---------------------------------------------------------------------
10: resource = null; //default value //write
=====================================================================
Thread 1 | Thread 2
----------------------------------+----------------------------------
11: a = resource; | 21: x = resource; //read
12: if (a == null) | 22: if (x == null)
13: resource = new Resource(); | 23: resource = new Resource(); //write
14: b = resource; | 24: y = resource; //read
15: return b; | 25: return y;
What the JLS says
JLS 17.4.5 gives the rules for a read to be allowed to observe a write:
We say that a read r of a variable v is allowed to observe a write w to v if, in the happens-before partial order of the execution trace:
- r is not ordered before w (i.e., it is not the case that hb(r, w)), and
- there is no intervening write w' to v (i.e. no write w' to v such that hb(w, w') and hb(w', r)).
Application of the rule
In our example, let's assume that thread 1 sees null and properly initialises resource. In thread 2, an invalid execution would be for 21 to observe 23 (due to program order) - but any of the other writes (10 and 13) can be observed by either read:
- 10 happens-before all actions so no read is ordered before 10
- 21 and 24 have no hb relationship with 13
- 13 does not happens-before 23 (no hb relationship between the two)
So both 21 and 24 (our 2 reads) are allowed to observe either 10 (null) or 13 (not null).
Execution path that returns null
In particular, assuming that Thread 1 sees a null on line 11 and initialises resource on line 13, Thread 2 could legally execute as follows:
24: y = null(reads write 10)21: x = non null(reads write 13)22: false25: return y
Note: to clarify, this does not mean that T2 sees non null and subsequently sees null (which would breach the causality requirements) - it means that from an execution perspective, the two reads have been reordered and the second one was committed before the first one - however it does look as if the later write had been seen before the earlier one based on the initial program order.
UPDATE 10 Feb
Back to the code, a valid reordering would be:
Resource tmp = resource; // null here
if (resource != null) { // resource not null here
resource = tmp = new Resource();
}
return tmp; // returns null
And because that code is sequentially consistent (if executed by a single thread, it will always have the same behaviour as the original code) it shows that the causality requirements are satisfied (there is a valid execution that produces the outcome).
After posting on the concurrency interest list, I got a few messages regarding the legality of that reordering, which confirm that null is a legal outcome:
- The transformation is definitely legal since a single-threaded execution won't tell the difference. [Note that] the transformation doesn't seem sensible - there's no good reason a compiler would do it. However, given a larger amount of surrounding code or perhaps a compiler optimization "bug", it could happen.
- The statement about intra-thread ordering and program order is what made me question the validity of things, but ultimately the JMM relates to the bytecode that gets executed. The transformation could be done by the javac compiler in which case null will be perfectly valid. And there are no rules for how javac has to convert from Java source to Java bytecode so...