Writing Large Shapefiles using FME?

No, there's no functionality that detects this and splits the file up for you. It's already registered as an enhancement request with our developers, and the reference number is PR#34128. I added a note linked to this question.

NB: Technically I believe the limit specified by the format is 2.14gb, so even 4.29 is stretching the format more than it should really be.

Of course, the question is whether a Shape dataset of that size is a good idea in the first place! Maybe you could use a different format?

If you are stuck on Shape, then I'd do as others have suggested. The ModuloCounter transformer is a good way to divide data (on a non-geographic basis) or you could split the data with either the Tiler transformer or the Clipper.


A good guide on how to approximate when you are reaching a ShapeFile limit is mentioned in this documentation by Esri

http://help.arcgis.com/en/arcgisdesktop/10.0/help/index.html#//002t0000000m000000

"There is a 2 GB size limit for any shapefile component file, which translates to a maximum of roughly 70 million point features. The actual number of line or polygon features you can store in a shapefile depends on the number of vertices in each line or polygon (a vertex is equivalent to a point)."

Now in FME you could count the vertices but that would really slow down your process, and is not advisable.

Preferred way would be fanout by some attribute, or limit to a number of features per shapefile.