November 18, 2024
How To Automate Website Monitoring Using Dumpling AI and Make.com
Staying updated with competitors’ website changes can be time-consuming and challenging, especially when done manually. Dumpling AI’s Screenshot URL feature offers a solution by automating the process of capturing periodic screenshots of any webpage. This is incredibly useful for businesses looking to track competitors’ updates, such as changes in product offerings, pricing, or marketing strategies.
In this automation, we’ll show you how to set up a workflow in Make.com to capture screenshots of competitors’ webpages and store them in Google Drive for easy access. This process helps you stay ahead of the competition with minimal effort.
Step 1: Setting Up Google Sheets to Monitor URLs
Google Sheets will act as the central source for URLs you want to monitor. Whenever a new URL is added, it will trigger this automation to capture a screenshot.
- Create a New Scenario in Make.com
Log in to Make.com and start a New Scenario. - Add Google Sheets Module
- Search for Google Sheets: Watch Rows and add it to your scenario.
- Connection: Choose your connected Google account or authenticate if this is your first time.
- Spreadsheet ID: Select the spreadsheet where URLs are listed.
- Sheet Name: Enter “Sheet1” (or the specific sheet name).
Limit: Set to 1 to process one row per run.
Step 2: Adding the Dumpling AI Screenshot URL Module
Dumpling AI’s Screenshot URL feature is designed to capture high-quality, full-page images of web content. This is a critical step in setting up your automation. This module allows you to capture clean and comprehensive webpage screenshots, avoiding elements like cookie banners that may obscure content.
- Add Dumpling AI Screenshot URL Module
In Make.com, search for Dumpling AI: Screenshot URL and add it. - Configure Screenshot Settings
- Connection: Choose your Dumpling AI connection (add it if you haven’t already by entering your API key).
- URL: Map the URL from Google Sheets ({{1.URL}}).
- Screenshot Full Page: Set to true (optional)if you want the entire webpage captured.
- Auto Scroll: Enable to capture content as it scrolls, ensuring you capture content that loads while scrolling.
- Block Cookie Banners: Enable if you want to block cookie pop-ups from appearing in screenshots.
- Viewport Settings: Set width and height to 1024×1024 for standard-sized images, or adjust based on your needs.
Step 3: Retrieving the Screenshot with HTTP
- This step fetches the screenshot file, preparing it to be saved in your storage solution.
- Add HTTP Module
After the screenshot is generated, use the HTTP: Get a File module to retrieve it.- URL: Map the screenshot URL from the Dumpling AI module ({{2.screenshotUrl}}).
- Method: Choose GET to download the file.
- Evaluate All States as Errors: Set to false to avoid errors unless there’s a real issue.
Step 4: Storing the Screenshot in Google Drive
Storing screenshots in Google Drive provides a centralized, accessible archive for all web page captures.
- Add Google Drive Module
Add Google Drive: Upload a File to save the screenshot in a designated folder.- Connection: Use your Google Drive connection.
- Folder ID: Enter the ID of the folder where screenshots will be stored.
- File Name: Map to “Website Screenshot – {{1.URL}} – {{now}}” for organized storage.
- Data: Use the output file data from the HTTP module ({{3.data}}).
Step 5: Testing and Activating Your Automation
- Run a Test Scenario
- Add a URL to Google Sheets and manually run the scenario.
- Ensure that each module executes as expected:
- Google Sheets triggers with the URL.
- Dumpling AI captures the screenshot.
- HTTP retrieves the screenshot file.
- Google Drive saves the screenshot correctly.
- Activate the Scenario
Once everything is confirmed working, activate the scenario. This will automate the process, running each time a new URL is added to Google Sheets.
This workflow saves time by automating the capture and storage of web content. You’ll have a regularly updated archive for tracking webpage changes, useful for analysis, documentation, or monitoring competitor activity.
By following these steps, you’ll have a fully automated system for monitoring and archiving webpage screenshots with Dumpling AI and Make.com.
Get the Blueprint Featured in This Guide
Access the full blueprint here to get started on setting up this automation effortlessly!