Scrapy : storing the data

You can view a list of available commands by typing scrapy crawl -h from within your project directory.

scrapy crawl spidername -o items.json -t json

• -o specifies the output filename for dumped items (items.json)
• -t specifies the format for dumping items (json)

scrapy crawl spidername --set FEED_URI=output.csv --set FEED_FORMAT=csv

• --set is used to set/override a setting
• FEED_URI is used to set the storage backend for the item dumping. In this instance it is set to "output.csv" which is using the local filesystem ie a simple output file.(for current example - output.csv)
• FEED_FORMAT is used to set the serialization format for the (output) feed ie (for current example csv)

References (Scrapy documentation):

1. Available tool commands (for the command line)
2. Feed exports