cs-icon.svg

Get started with Contentstack DataSync

With this step-by-step guide, you will learn how to start Contentstack DataSync on your localhost.

Prerequisites

You need the following tools to run Contentstack DataSync:

Installation and Setup

  1. Clone the boilerplate

    Clone the boilerplate by using the following git clone command.
    git clone https://github.com/contentstack/datasync-boilerplate
    

    Note: By default, the boilerplate uses @contentstack/datasync-content-store-filesystem for storing JSON data, and @contentstack/datasync-asset-store-filesystem for storing media files.

  2. Add stack API key and delivery token

    Go to the config folder and open the all.js file. Add your stack's API key and the delivery token of the environment:
    const config = {
      contentStoreModule: '@contentstack/datasync-content-store-filesystem',
      assetStoreModule: '@contentstack/datasync-asset-store-filesystem',
      listenerModule: '@contentstack/webhook-listener',
      contentstack: {
        apiKey: '<<Your API Key>>',
        deliveryToken:'<<Delivery Token>>',
       },                
    }
    
    For Europe, Azure NA, or Azure EU Region:
    By default, the app runs in the NA region. If you are hosting your app in the Europe, Azure NA, or Azure EU region, you will have to specify the host in the all.js file as shown below:
    const config = {
      contentStoreModule: '@contentstack/datasync-content-store-filesystem',
      assetStoreModule: '@contentstack/datasync-asset-store-filesystem',
      listenerModule: '@contentstack/webhook-listener',
      contentstack: {
        apiKey: '<<your_api_key>>',
        deliveryToken:'<<your_delivery_token>>',
         host: '<<your_region_host_url>>' // For Europe region set host: 'eu-cdn.contentstack.com'. For Azure NA region set host: 'azure-na-cdn.contentstack.com'. For Azure EU region set host: 'azure-eu-cdn.contentstack.com'
    .
    },
    }
    
  3. Install dependencies

    Open the command prompt and navigate to the project directory. Now, install npm dependencies by using the following command:
    npm install
    
  4. Run Contentstack DataSync

    Now start Contentstack DataSync by executing the following command.

    On Linux:

    NODE_ENV=development SYNC_ENV=development npm start
    

    On Windows:

    set NODE_ENV=development 
    set SYNC_ENV=development
    npm start
    

    Note: Set NODE_ENV to the name of the environment for which the delivery token was added in Step 2.

    After running the above command, the server starts on port 5000 and all the published items of your stack syncs on your infrastructure.

    If your environment has any published content, the sync utility creates a _contents folder in your project, and stores assets inside the _content/en-us/assets folder and entries inside the _content/en-us/data folder.

  5. Install and run ngrok (for localhost only)

    Note: If you use a deployment environment, such as production, you don't need ngrok. Skip this step and move to creating a webhook. Else, follow this step.

    Contentstack DataSync is a webhook-based utility, and webhooks get triggered only on public domains. So, if you are setting up Contentstack DataSync on your local machine, you will require a public domain to receive webhook notifications.

    ngrok creates a public URL for your localhost and creates a secure tunnel between the two. So, you need to:

    Warning: Make a note of the "Forwarding URL," you will need this URL while creating webhooks. Also, DO NOT close the ngrok console during the process, else Contentstack DataSync will not receive webhook notifications.

    ngrok.gif
  6. Create a webhook

    1. Log in to your Contentstack account, go to your stack, and click on the "Settings" icon on the left navigation panel.
    2. Select Webhooks and click on + New Webhook.
    3. On the Create Webhook page, enter a specific name for your webhook and inside the URL to notify field, provide the ngrok URL (generated in the above step) and append "/notify" to the URL (for example, "https://ca1a27e9.ngrok.io/notify").
    4. Tip: If you using a deployment environment, such as "production," you will have to provide the public URL of your environment inside the URL to notify field (for example, "https://(your_environment_name).com/notify")

    5. In the When section, click on the Code View option and paste the following code snippet in it:
      content_types.entries.environments.development.publish.success,
      assets.environments.development.publish.success,
      assets.environments.development.unpublish.success,
      content_types.entries.environments.development.unpublish.success,
      content_types.entries.delete,
      content_types.delete,
      assets.delete
      

      Note: We have used the "development" environment in the above code snippet. Please replace it with your environment name in your code snippet.

    6. Click on Save.
    By adding this code, we have added webhook events that get triggered when data is published, updated, or deleted, and when any content type is deleted.
  7. Test the execution

    Now try publishing new entries or assets and test if it syncs as expected. Check whether the _contents folder inside your project has all the synced data that you just published.

Was this article helpful?
^