Computed Column Index Not Used

Try with COALESCE instead of ISNULL. With ISNULL, SQL Server doesn't seem capable of pushing a predicate against the narrower index, and therefore has to scan the clustered to find the information.

CREATE TABLE dbo.Diffs
    (
    Id int NOT NULL IDENTITY (1, 1),
    DataA int NULL,
    DataB int NULL,
    DiffPersisted  AS COALESCE(convert(bit, case when [DataA] is null 
      and [DataB] is not null then 1 when [DataA] <> [DataB] 
      then 1 else 0 end), 0) PERSISTED ,
    DiffComp  AS COALESCE(convert(bit, case when [DataA] is null 
      and [DataB] is not null then 1 when [DataA] <> [DataB] 
      then 1 else 0 end), 0),
    DiffStatic bit not null,
    Primary Key (Id)
    );

That said, if you stick with a static column, a filtered index might make more sense, and will have lower I/O costs (all depending on how many rows typically match the filter predicate) e.g.:

CREATE INDEX ix_DiffStaticFiltered 
  ON dbo.Diffs(DiffStatic)
  WHERE DiffStatic = 1;

This is a specific limitation of the SQL Server computed column matching logic, when an outermost ISNULL is used, and the datatype of the column is bit.

Bug report

To avoid the issue, any of the following workarounds may be employed:

  1. Do not use an outermost ISNULL (the only way to make a computed column NOT NULL).
  2. Do not use the bit data type as the final type of the computed column.
  3. Make the computed column PERSISTED and enable trace flag 176.

Details

The heart of the issue is that without trace flag 176, all computed column references in a query (even persisted) are always expanded into the underlying definition very early in query compilation.

The idea of expansion is that it could enable simplifications and rewrites that can only work on the definition, not on the column name alone. For example, there may be predicates in the query referencing that computed column that could make part of the calculation redundant, or otherwise more constrained.

Once early simplifications and rewrites are considered, query compilation attempts to match expressions in the query to computed columns (all computed columns, not only those originally found in the query text).

Unchanged computed column expressions match back to the original computed column without issue in most cases. There appears to be a bug when specific to matching an expression of bit type, with an outermost ISNULL. Matching is unsuccessful in this specific case, even where a detailed examination of the internals shows that it should succeed.