Does ALTER INDEX ALL REBUILD use more transaction log space with simple recovery model than rebuilding each index individually?

I assumed that following each index operation performed by the "ALL" form of the command, the transaction log data would be flushed prior to the next index rebuild. Is that how it actually works, or are the index rebuilds logged as if they are part of a single transaction?

1) Log flushing: the SIMPLE recovery model does not clear the log after every transaction, but at checkpoints. (link for more info)

2a) REBUILD ALL: yes, REBUILD ALL works as a single transaction. The index rebuilds within have their own transactions, but the overall operation isn't fully committed until the end. So yes, you might limit log file growth by rebuilding individual indexes (and possibly issuing CHECKPOINT commands).

2b) Proof! Here, have a demo script. (Built in 2016 dev) First, set up a test db, with table and indexes:

USE master
GO

CREATE DATABASE Test_RebuildLog
GO

ALTER DATABASE Test_RebuildLog
SET RECOVERY SIMPLE
GO

USE Test_RebuildLog
GO

CREATE TABLE IndexTest
(ID int identity(1,1),
a char(1),
b char(1))

CREATE CLUSTERED INDEX CIX_IndexTest_ID ON IndexTest(ID)
CREATE INDEX IX_IndexTest_a ON IndexTest(a)
CREATE INDEX IX_IndexTest_b ON IndexTest(b)

INSERT IndexTest
(a,b)
VALUES ('a','b'),('z','y'),('s','r')

Now you can compare log activity between REBUILD ALL and rebuilding individually

CHECKPOINT
GO
ALTER INDEX ALL ON IndexTest REBUILD

SELECT *
FROM sys.fn_dblog(NULL,NULL)
WHERE Operation = 'LOP_COMMIT_XACT'
OR Operation = 'LOP_BEGIN_XACT'
GO

CHECKPOINT
GO
ALTER INDEX CIX_IndexTest_ID ON IndexTest REBUILD
ALTER INDEX IX_IndexTest_a ON IndexTest REBUILD
ALTER INDEX IX_IndexTest_b ON IndexTest REBUILD

SELECT *
FROM sys.fn_dblog(NULL,NULL)
WHERE Operation = 'LOP_COMMIT_XACT'
OR Operation = 'LOP_BEGIN_XACT'
GO

Note how the first open transaction (Transaction ID 0000:000002fa for me) isn't committed until the end of the REBUILD ALL, but for the index-by-index rebuilds, they are successively committed.


As it stands, this is single transaction.


The question is trivial for an offline rebuild. Of course is a single transaction. Imagine the havoc that would ensue if the operation split each index into its own transaction, as it would have to release the locks when committing and then re-acquire them. While the critical table SCH-M lock was released, indexes may be dropped and new indexes could be created, how would the statement handle such cases? Not to mention that table may be dropped, and even re-created between the two transactions! Including the case when the table gets dropped and a different table gets created with the same object id (yes, it can happen)...

What if you augment the question to say what happens if the index rebuild is an online rebuild? Is it a single transaction or many? The answer is complex, as there are actually several internal transactions involved. However, the key point is that there is an overall arching transaction that spans the entire operation (the ALTER statement) and this pins the log in place (cannot truncate), thus the operation needs to be accordingly planned to allow for ~1.6x data size for FULL recovery mode, or 0.2x data size for BULK_LOGGED/SIMPLE mode. See linked paper for more details.

You can argue that why doesn't the offline build employ the same internal transactions the online mode does, and split the operation? The issues I mentioned about table being altered/dropped between the individual index operations (ie. table 'schema stability') would still require that there is an encompassing transaction that holds a SCH-S on the table for the entire duration of the statement. As this transaction must hold the SCH-S also during recovery, it must be logged, and as such there will be a BEGIN XACT log record that will pin the log and prevent truncation for the whole duration of the statement. I know this particular issue was being addressed in SQL 2016-2017 time frame (because of SQL Azure log size issues), but I'm not sure what progress was made. Looks like is in preview now: Resumable Online Index Rebuild is in public preview for SQL Server 2017 CTP 2.0.