Config File Options

If you're making a request to an external import file/url and it requires additional validation you can use the datagrab_custom_headers config option. In this example 6 is the import ID to assign these headers to.

$config['datagrab_custom_headers'] = [
    6 => [
        'Content-Type: application/json',
        'Authorization: Bearer YOUR_ACCESS_TOKEN'
    ]
];

If you want to disable verifying peers when making a cURL request you can set this to 'n'.

$config['datagrab_verify_peer'] = 'n';

By default DataGrab will write to the system/user/cache/DataGrab-import.log file each time an import is executed, and erase the log contents from the previous import and replace it with the currently executing import. If you want to keep a backup of the log files you will want to enable log rotate, which will append a timestamp to the end of of the file. If you do this you may want to consider deleting old backups, which will require a simple bash script.

$config['datagrab_rotate_log'] = 'y';

Starting with DataGrab 6.1, remote import files are fetched through a new URL guard that only allows http and https URLs and refuses to connect to hosts that resolve to private, loopback, or otherwise reserved IP addresses. This prevents a saved import from being pointed at internal infrastructure or cloud metadata endpoints. The following config options let you tune that behavior.

If you import from a local development domain (such as a .ddev.site or .test address) or from a trusted host on your internal network, those requests would normally be blocked because they resolve to private IPs. You can allow specific hosts through with the datagrab_allowed_hosts config option. It accepts an array of host names, and a leading *. will match any subdomain. Your site's own domain (from base_url) is always trusted automatically, so same-host imports keep working without any configuration.

$config['datagrab_allowed_hosts'] = [
    'localhost',
    '*.ddev.site',
    '*.test',
];

If you'd rather not maintain a host list and simply want to allow requests to private and reserved addresses everywhere, you can disable the private-address block entirely by setting datagrab_allow_private_urls to 'y'. The datagrab_allowed_hosts option above is the safer, more targeted choice and should be preferred on production sites.

$config['datagrab_allow_private_urls'] = 'y';

If you fetch import files from a URL that issues redirects, DataGrab now follows each redirect manually and re-checks the destination against the same security rules before continuing. The number of redirects it will follow is capped, and you can adjust that limit with the datagrab_max_redirects config option. This only applies when redirect following is already enabled via datagrab_follow_redirects; the default is 5.

$config['datagrab_max_redirects'] = 5;

If your import is run from outside the Control Panel using its ?ACT= URL, the passkey can now be sent as an HTTP request header instead of as a query-string parameter, which keeps the secret out of server logs, browser history, and Referer headers. Send the passkey in the X-Datagrab-Passkey header; the existing passkey= query-string parameter continues to work for backwards compatibility.