Data Loader Command Line Tool slow

In the folder that contains the process-conf file, do you also have a file called log-conf.xml? Because if you don't, I know DataLoader starts logging everything, and creates a huge file...

Do you also have a config file in the folder? If not specified in the bean, that will also set parameters...

I have the command line setup on my machine for various projects - happy to to do a comparison test to check whether it is tool or machine specific - which version are you using?

AMENDED: I added the log-conf.xml and set the priorities to 'OFF'. Not sure why the setting in the process-conf doesnt override this, I am not sure.


Your bean actually looks legit, but I know this is a pain to get it working alright. I can't tell why it's wrong, unfortunately. In fact, I felt so much pain I went through quite some hoops to create a tool around the data loader functionality to make it work from the command line each time. It highly simplified the scripting of the Data Loader.

I hope you don't mind if I show you how you'd do it with NoFrillsTransformation. As a prerequisite, you will need to "install" NoFrillsTransformation. If you're on Windows (which I assume as you are using the DataLoader in the first place), this is fairly simple and is described in the Installation Guide.

First, create a sfdc_config.xml file which contains the general information on end points, credentials and such. It has the following syntax:

<?xml version="1.0" encoding="utf-8"?>
<SfdcConfig>
  <DataLoaderDir>[path to Data Loader]</DataLoaderDir>
  <LogFileDir>[log file path]</LogFileDir>
  <SuccessFileName>[full path to success log]</SuccessFileName>
  <ErrorFileName>[full path to error log]</ErrorFileName>
  <SfdcUsername>[Salesforce user name]</SfdcUsername>
  <SfdcPassword>[Salesforce Password + Token]</SfdcPassword>
  <SfdcEncryptedPassword>[encrypted password</SfdcEncryptedPassword>
  <SfdcEndPoint>[Salesforce endpoint]</SfdcEndPoint>
  <LoadBatchSize>[batch size]</LoadBatchSize>
  <KeepTempFiles>[true|false]</KeepTempFiles>
  <FailOnErrors>[true|false]</FailOnErrors>
  <UseBulkApi>[true|false]</UseBulkApi>
  <BulkApiSerialMode>[true|false]</BulkApiSerialMode>
</SfdcConfig>

These are the ones you need to fill in; you should have all of this information already at hand, otherwise you wouldn't have been able to create your bean XML. In case you need additional help, you can find more information on that inside the Wiki (above link).

Then you continue to describe as another XML file what you are trying to do, in this case you're upserting OrderEntity records. So, this will be your template:

<?xml version="1.0" encoding="utf-8" ?>
<Transformation>
  <Source config="delim=','">file://[path to your CSV file]</Source>
  <Target config="sfdc_config.xml">sfdc://OrderItem.upsert:SlId__c</Target>

  <Fields appendSource="true" />
</Transformation> 

After doing this, you can call NoFrillsTransformation.exe like so:

NoFrillsTransformation.exe upsert_orderentity.xml

If you have specified that NFT should use the bulk API, it will do so. Otherwise you can switch it on and off using the command line switches usebulkapi (usebulkapi=true) and bulkapiserialmode (bulkapiserialmode=true).

Disclaimer: I wrote this tool, and I cannot guarantee it does not contain any bugs. I can actually guarantee it does. But I successfully used it to migrate 60k Accounts, 300k Contacts, 150k E-Mails, 60k Tasks, 500 Users, and a lot more from SAP/CRM to Salesforce.