Adding Diffbot Source

Curator connects with Diffbot to extract structured content from web pages. This allows you to pull articles, products, discussions, and other public web content into your feed alongside your social media sources.

Diffbot converts web pages into structured data, making it easier to display clean, consistent posts inside Curator.

Connecting a Diffbot Source

1. On the Organize page, click Add Source

2. Select Diffbot

3. Choose Crawlbot as your source type

4. Enter the URL for the content you want to extract

5. Click next to complete the adding source process

Diffbot source pulls posts when you add it, then refreshes based on your plan's update frequency (Free: 24 hours, Pro: 60 minutes, Business: 15 minutes).

Common Issues

No content appearing — Ensure the URL or query you entered returns valid content. Test the URL directly in Diffbot's API or dashboard to confirm it extracts data.

Connection failed — Check that the URL is publicly accessible. Diffbot cannot extract content from pages behind authentication or paywalls without additional configuration.

Missing posts — Confirm you haven't hit your plan's post limit (Free: 1,000 posts). Also verify there are no recent outages or rate limit issues in your Diffbot account.

Limitations

• Content extraction quality depends on page structure and Diffbot's recognition algorithms

• Pages requiring authentication or behind paywalls need special Diffbot configuration

• Each Diffbot source counts as 1 source toward your Curator plan limit (Free: 3, Pro: 5, Business: 15)

• Available on all Curator plans (not a premium source)

• Update frequency limited by your Curator plan, not Diffbot