BoldMinded Docs
  • Overview
  • Advanced Categories
    • Docs
      • Installation
      • Requirements
      • Setup & Configuration
        • Hidden Config Options
        • Multi-Site Manager
      • Template Tags
      • FAQs
  • Ansel
    • Docs
      • Installation
      • Upgrading
      • Requirements
      • Settings
      • Usage
        • Channel:Form
      • Troubleshooting
      • Template Tags
    • FAQs
  • Bloqs
    • Docs
      • Why Bloqs?
      • Features
        • Add Bloq Menu
        • Nesting
        • Cloning
        • Drafts
        • Bloq Usage
        • Deprecated Bloqs
        • Caching
        • Profiling
        • Bloq Components
        • Bloq Groups
      • Installation
      • Requirements & Compatibility
      • Setup & Configuration
        • Hidden Config Options
      • Creating Bloqs
      • Template Tags
        • Twig
        • Examples
      • Context Variables
      • Add-on Developers
    • FAQs
  • Carson
    • Docs
      • Installation
      • Requirements
      • Fields
        • Omni
          • Hidden Config
        • Assistant
        • SEO
      • Troubleshooting
  • Custom System Messages
    • Docs
      • Installation
      • Requirements
      • Variables
      • Template Tags
  • DataGrab
    • Docs
      • Installation & Upgrading
      • Requirements
      • Troubleshooting
      • Configuring Imports
      • Running Imports
      • Automatic Imports
        • Importing with cron
        • Importing with cron (Deprecated)
      • Endpoints
      • Configuration Options
        • Config File Options
        • Increasing PHP memory limit
      • Import Types
        • CSV
          • Importing into Grid or Matrix field
        • JSON
          • Example file
        • WordPress
        • XML
          • Example file
        • Creating your own import type
      • Assigning Authors
      • Field Types
        • Assets
        • Ansel
        • Bloqs
        • Calendar
        • Channel Images
        • Date
        • File
        • File Grid
        • Fluid
        • Grid
        • Low Events
        • Relationships
        • Simple Grids & Tables
        • Tag & Tagger
        • Matrix (Deprecated)
        • Creating your own fieldtype
      • Publisher
      • Version 5.0
      • Version 6.0
    • FAQs
  • Feature Flags
    • Docs
      • Installation & Upgrading
      • Requirements
      • Configuration
      • Template Tags
      • A/B Testing
    • FAQs
  • Fluidity
    • Docs
      • Installation & Upgrading
      • Requirements
      • Configuration
      • Demos
    • FAQs
  • Logit
    • Docs
      • Installation
      • Requirements
      • Configuration
    • FAQs
  • Publisher
    • Docs
      • Installation
      • Requirements
      • Issues & Tips
      • Languages
      • Template Tags
        • Forms
        • Email Notification Templates
        • Channel:Form
        • Twig
      • URL Translations
      • Auto Translations
      • Diffs
      • Drafts
      • Categories
      • Phrases
      • Persistence
      • Performance
      • Add-ons
        • First Party
        • Third Party
      • Hidden Config
      • Extending Publisher
    • FAQs
  • Reading Time PRo
    • Docs
      • Installation
      • Requirements
      • Configuration
      • Template Tags
  • Reel
    • Docs
      • Installation
      • Requirements
      • Settings
      • Field Tags
    • FAQs
  • Simple Grids & Tables
    • Docs
      • Installation
      • Requirements
      • Template Tags
      • CSV File Imports
      • Field Settings
      • GraphQL
      • Advanced Configuration (deprecated)
    • FAQs
  • Sitemap
    • Docs
      • Installation
      • Requirements
      • Configuration & Usage
    • FAQs
  • Snaptcha
    • Docs
      • Installation
      • Requirements
      • Configuration
      • Template Tags
      • Developers
    • FAQs
  • Speedy
    • Docs
      • Installation & Updating
      • Requirements
      • Configuration
      • Template Tags
      • Static Caching
        • Real World Example
      • Frontedit Support
      • Control Panel
      • Migrating from CE Cache
      • Diagnostics
      • CLI Commands
      • Reverse Proxy Purging
    • FAQs
  • Trek (unreleased)
    • Docs
      • Configuration
    • FAQs
  • Queue
    • Docs
Powered by GitBook
On this page
  • To use endpoints:
  • Endpoints require:
  • Creating an Endpoint

Was this helpful?

  1. DataGrab
  2. Docs

Endpoints

Last updated 4 months ago

Was this helpful?

New in DataGrab 5.3 are Endpoints. DataGrab Imports are designed to seek out and pull data from an external file or remote URL. Endpoints are a way for DataGrab to receive incoming data from an external application. Usually this is referred to as a webhook. An event happens in an application and it triggers a webhook, which POSTs data to a URL you've configured. Now with DataGrab you can accept incoming data to create or update entries in ExpressionEngine.

Endpoints are always paired with a previously configured Import. Imports usually handle many entries at a time from a remote file. Endpoints can also handle many entries, but generally the payload will be much smaller, such as 1 entry at a time. Regardless of the number of entries, the format of the payload sent by the external application must match the format of an already configured import.

To use endpoints:

  1. Create an Import: Begin by creating an import using a static or remote JSON or XML file that matches the sender’s data format. Run the import through the DataGrab interface and verify it works as expected.

  2. Create the Endpoint: Assign the import to a new endpoint. The sender will POST data to the Endpoint URL, including proper authentication.

Endpoints require:

  1. A configured JSON or XML-based import.

  2. POST requests must include body content that matches the import’s format.

That last step is very important. You must configure a crontab to consume the data from an endpoint. Otherwise it will sit in the queue and never become an ExpressionEngine entry.

Creating an Endpoint

Creating an Endpoint is very easy. Give it a name and associate it to an existing Import. You must also provide some method of authentication. Endpoints can accept authentication via GET variables, or HTTP Headers. You can add more than 1 parameter, though 1 or 2 is all that is usually required. If the parameters do not match exactly what is sent the request will fail.

Add the POST body content. This should match the value of a full import, except it will contain data for 1 entry.

Most importantly, add your authentication values. You don't want to openly accept any data sent to your endpoint.

A with the --consumer flag to process data received by the endpoint. If you have multiple endpoints with different imports, each import requires its own dedicated consumer process.

To test an endpoint you can use an application such as , , or any other rest client. In these screenshots we're using Bruno. First start with defining the query parameters to the Endpoint URL, which is generated after you create an Endpoint.

After submiting the test request from your preferred rest client you should see a row in your exp_datagrab_jobs table (if you are using the database queue driver, which is the default). If you do not see anything in this table, and you have the DataGrab module page open in a browser tab it is possible that DataGrab already picked up the job from the queue. In this case you should see the new entry in ExpressionEngine. If it still does not appear to be working, with the log file.

crontab process running
Bruno
Insomnia
troublehoot it