Running Imports

Imports can be executed directly in the ExpressionEngine control panel, or by requesting the Import URL. Note that the Import URL does not require the user to be logged in to execute. Therefore, if a random user discovers your Import URL, they can run a import for you without you knowing! For this reason, it is recommended to use a passkey, which you can set when configuring the import.

If you encounter any issues running an import, the first step you should take is look at your import log, as described in the troubleshooting section. DataGrab has pretty extensive logging, so there is a good chance you will find what you are looking for in the logs.

Since DataGrab operates using a queue, there may be times where you view the DataGrab imports list and a status changes, such as displaying a loading bar, on one of the imports without you taking action. This may happen if an item from that import is in the queue and DataGrab takes action on it. When this page is open in the browser the it is always checking for status updates of imports.

Toolbar Actions

starts, or continues an import.

restarts an import from the beginning. This button is only visible if an import is WAITING.

lets you edit the import settings, such as it's name, description, and passkey.

lets you configure the import itself, e.g. mapping import file properties to fields in ExpressionEngine.

Import URL simply gives you the URL to the import so you can run it in a cron, or outside of the control panel.

If an import was halted for any reason, such as reloading the control panel page while it was in progress, it will switch to the WAITING status. You can resume the import at any time by clicking on the icon.

Memory Management and timeout issues

DataGrab uses import queues. When an import initiated it reads your import file and creates records in the queue database table for all the items it needs to import. This could be 5 or 5000 entries. It then uses consumer processes to import the entries in batches, defined by your batch import size setting (defaults to 50). Consumers will stop importing when it hits that limit, or if it reaches a PHP script max execution time. DataGrab will start new consumers when the import is executed through the control panel (see the screenshot above), or if using the ACT url to import. However, if you are importing with the CLI, you will need to manually start new consumers. Fortunately there are things like cron and supervisord that will automatically start new consumers on whatever interval you define. Read more about this in the Automatic Imports section of the docs.