< Go back to blog
Tutorials

February 3, 2025

How to Use Dumpling AI’s Scrape URL Endpoint in Make.com

The Scrape URL module in Dumpling AI enables you to extract data from any webpage in a clean, structured, and customizable format. Whether you need plain text, HTML, or screenshots, this module simplifies web data collection and ensures efficiency. In this guide, we’ll explore its configuration, inputs, outputs, and potential use cases, and provide a detailed setup process in Make.com using the HTTP Request module.

Understanding the API

Endpoint
  • URL: POST /api/v1/scrape
Headers
  • Content-Type: application/json
  • Authorization: Bearer <API_KEY>

Request Body Parameters

  1. url (Required): The URL of the webpage to scrape.
  2. format (Optional): Specifies the output format (markdown, html, screenshot).
  3. cleaned (Optional): Boolean value to clean unnecessary HTML or ads.
  4. renderJs (Optional): Enables or disables JavaScript rendering (default: true).

Step-by-Step Guide to Set Up in Make.com

Step 1: Add the HTTP Request Module
  1. Open your Make.com scenario.
  2. Add the HTTP Request module to your workflow.
Step 2: Configure the HTTP Request Module
  1. Method: POST.
  2. URL: Enter the API endpoint:
    https://app.dumplingai.com/api/v1/scrape.
  3. Headers:
    • Content-Type: application/json
    • Authorization: Bearer <Your_API_Key>
  4. Request Body:
    Use JSON to specify the scraping options. 
Step 3: Test the Module
  1. Click Run Once to test the configuration.
  2. If successful, the response will include:
    • title: Title of the webpage.
    • metadata: Metadata of the page (e.g., description, keywords).
    • content: Scraped data in the specified format.

Outputs and Applications

Outputs
  1. title: The webpage’s title.
  2. metadata: Metadata information, such as meta tags and descriptions.
  3. content: Scraped data in the chosen format (markdown, html, or raw content).
Applications
  1. Content Extraction
    • Extract blog posts, articles, or product descriptions for analysis.
  2. SEO Analysis
    • Gather metadata from competitor websites to improve your SEO strategy.
  3. Web Data Research
    • Scrape structured content for reports or data-driven decisions.

Conclusion

Dumpling AI’s Scrape URL module is a versatile tool for extracting clean and structured data from web pages. By integrating this feature with Make.com’s HTTP Request module, you can automate complex web scraping tasks with ease. Use it for content analysis, SEO monitoring, or research, and take advantage of its flexibility in handling different formats and rendering options. Start scraping smarter today!

Related Posts

How to Use Dumpling AI’s Get YouTube Transcript Endpoint in Make.com
Tutorials

How to Use Dumpling AI’s Get YouTube Transcript Endpoint in Make.com

January 31, 2025

How to Add Content to a Knowledge Base Using Dumpling AI’s API
Tutorials

How to Add Content to a Knowledge Base Using Dumpling AI’s API

January 28, 2025

Understanding the Dumpling AI Generate AI Image with FLUX.1 Dev in Make.com
Tutorials

Understanding the Dumpling AI Generate AI Image with FLUX.1 Dev in Make.com

January 25, 2025

Understanding the Dumpling AI Generate AI Images with FLUX.1 Schnell
Tutorials

Understanding the Dumpling AI Generate AI Images with FLUX.1 Schnell

January 22, 2025

Understanding Dumpling AI Search Knowledge Base Module in Make.com
Tutorials

Understanding Dumpling AI Search Knowledge Base Module in Make.com

January 19, 2025

How to Find All URLs on a Domain for Scraping
Tutorials

How to Find All URLs on a Domain for Scraping

January 16, 2025