Troubleshooting

If you are experiencing issues with DataGrab please read this entire guide before creating a support ticket.

File validation

Make sure your import files are valid. Use an online validation checker to ensure there are no errors with the file formatting. DataGrab can not read invalid XML, JSON, or CSV files.

Import Configuration

Most of the issues encountered with DataGrab occur when configuring an import. Usually it's because the import file is invalid or inaccessible.

If you see an error on this page, usually something to do with cURL, there is a 99.9% chance that this is not a DataGrab issue. For some reason, DataGrab can't read your import file. Check the following:

  • The path in the config is correct

  • Make sure your environment has cURL enabled, and make sure you're not running into a self-referential request failure (e.g. making requests to URLs on the same server)

  • Make sure $config['datagrab_verify_peer'] = 'y'; is not in your config.php file, or it is set to 'n'

  • Open the import feed in a separate browser window, if it does not load and you do not see the XML or JSON data, neither can DataGrab.

  • If you are importing from a local file or URL and using basic auth to block access to your site while in development, then be sure to include the credentials in your import URL as well. The import process does not use the same authenticated session as your browser. For example: http://user:pass@mystagingsite.com/import-file.json

  • If you are importing from a remote file or URL that is behind a basic auth, make sure it is correct, and make sure the username and password don't contain an @. If you open the feed with the username and password in a separate browser window and you receive a login prompt, then DataGrab sees the same thing and can not read the file. This likely indicates your username or password is wrong, or something else is misconfigured on the server.

  • Make sure your site and feed protocols match. Don't load a feed from https if your site is running from http.

Check the logs

If your import file is valid and everything is configured and you are imports checking the log file is the best place to go to monitor the import activity. By default DataGrab will log to the system/user/cache/DataGrab-import.log file. Each time an import starts, it will purge old logged items from the file to avoid compounding the file size. You can change the logging type by adding the following option to your config.php file:

$config['datagrab_log_type'] = 'file|developer|php|off';

'file' is the default value. You do not need to add this to your config file unless you want to use one of the other options.

If you set the log type to 'developer' it will log to ExpressionEngine's Developer Log that is accessible in the control panel. Go to Tools > Logs >Developer Log and review the log for any error messages. It may be helpful to fully clear the entire log, then run an import again to ensure you have up-to-date log messages.

If you set the log type to 'php' it will write to your server's PHP log file. Depending on your system and how you have PHP configured, this file path may not be in a standard location such as /var/log/. If you do not want to log to your database (e.g. ExpressionEngine's Developer Log) or the user/cache directory, then you should use the php log option. If you choose 'php' as your logging option you can also change the path if you use the following config option:

$config['datagrab_log_file'] = '/var/www/log/DataGrab-import.log';

Note that if you change the log file make sure the directory is writable. We do not provide support for system configuration settings if the log is not outputting to the value you've defined. This option will not set your entire php error logging to this file. It will only log DataGrab events and errors to this file. Other ExpressionEngine or similar PHP errors will continue to be logged to your system defined log file.

If you do not set this option in your config file DataGrab will continue logging to the DataGrab-import.log file in your user/cache directory as mentioned above. You can turn off all logging by setting the log type to 'off';

If you disable logging and create a support ticket, you will be asked to re-enable and provide a log output of the import issue you're experiencing. The php or default DataGrab-import.log file is preferred for support requests since I can not read your Developer log without control panel access.

Try the Import URL

If you attempt to run an import and the Status column does not render a progress bar, but instead renders an error message, or something that looks like the screenshot below, then use the Import URL option and attempt to run the import in a new browser tab. From there, the full error message will be revealed. It is also highly possible the error will be in your import log. Please include the full error message if you're creating a support ticket.

Last updated