Capturing competitor sites that block page monitors
Why does my page monitor fail on some competitor sites?
Many websites block simple automated fetchers, so a basic page monitor gets an error, a challenge page or nothing at all instead of the content. PMM Tools records when a capture was blocked and keeps serving your agent the latest good capture of each page. On Growth and above, a blocked capture is retried automatically with heavier capture methods, one step at a time. A page that still can't be captured can be captured from your own browser with the PMM Tools Chrome extension, into the same competitor collection your agent reads.
How it works
1. Every capture is checked
PMM Tools records whether each capture was blocked. Your agent is served each page's latest good capture, and the app shows which pages were blocked.
2. Blocked captures are retried
On Growth and above, a blocked capture is retried automatically with a heavier capture method, one step at a time, up to what your plan includes. Plans include 1 to 10 bot-protected competitors, and a competitor takes one of those slots the first time it needs the heavier methods.
3. Capture the rest from your browser
If a page still can't be captured automatically, open it in Chrome and capture it with the PMM Tools extension. The capture is saved to the same competitor, and your agent reads it like any other.
Questions
Why do page monitors fail on some competitor sites?
Many sites check whether a visitor is an automated tool before they serve the page. A simple fetcher then receives an error, a challenge page or an empty response, and the monitor either reports a failure or saves that response as if it were the page.
Which plans retry blocked captures?
Growth and above. Plans include 1 to 10 bot-protected competitors. When every slot is taken, blocked pages from other competitors are not retried with the heavier methods; you can change which competitors are protected in the app or with pmm_competitor_protect. Starter does not include bot-protected competitors.
Can my agent tell whether pages were blocked?
Yes. pmm_run_status reports a run's progress, including its blocked and escalated captures, and pmm_competitor_protect shows which competitors hold a protected slot.
Will every page be captured automatically?
No. Some pages still can't be captured after the retries. The app shows those pages as blocked, and you can capture them from your browser with the Chrome extension.