Importing with cron (Deprecated)

Old method of importing with cron using the ACT url.

This method of importing with cron is no longer supported. It functionally still works, but we will not be offering add-on support for this method. Please use the new CLI commands.

What is cron?

cron is a service available on Unix-based servers that can execute commands at regular intervals.

How to set up ExpressionEngine

In order to activate a DataGrab import, you will need to simulate a visit to a URL. If you want to run a single import, or a small number of imports, the easiest method is to use DataGrab’s import action.

To run a single import, the url to access is can be found from the DataGrab module’s front page, and is in the form:

http://example.com/index.php?ACT=20&id=123

Your value for ACT= might be different from 20. The value for id= is the id of the saved import you wish to run.

Visiting this URL in a browser will run the specified import.

An alternative method, which is more useful if you have a number of imports to run, is to create an EE template and use a tag to select the import to run. Inside this template, add the tags as follows:

{exp:datagrab:run_saved_import id="2"}
{exp:datagrab:run_saved_import id="9"}

This would run the import with id=2, followed by the import with id=9. You could add as many imports as you wish.

To run these imports, you would need to visit this URL. So if your template group was ‘feeds’ and the template name ‘batch’, you would go to:

http://example.com/index.php/feeds/batch

Setting up cron

Depending on your web hosting, you may need to set up cron jobs through the command-line or through a web-based back-end like cPanel or Plesk.

I won’t go into detail here on each individual method. The key to setting it up correctly is to remember that to run an import you need get cron to visit a webpage on your site.

In order to do this, you’ll need to use one of the following commands (replacing the URL to your URL):

/usr/bin/wget -O - -q -t 1 "http://example.com/index.php?ACT=20&id=id"

curl --silent --compressed "http://example.com/index.php?ACT=20&id=id"

/usr/bin/lynx -source "http://example.com/index.php?ACT=20&id=id"

These are 3 alternative options (note: you only need to use one of them) to do the same thing. You might find that your webhost might not have, say, lynx, installed, in which case you can use wget or curl.

It is important to note that because the URL might contain an ampersand (&), you should put quotation marks around the URL.

DataGrab defaults to importing 1 entry at a time with the ?limit=1 parameter, which makes a new http request. Cron requests do not use http, so you may need to set your limit value to a value slightly more than the number of entries you expect to import.