6 guides
scraping basics
How to scrape
Learn the core scraper workflow: entering domains, focusing subfolders, and starting from sitemaps.
Read guidecrawler controls
Spider settings
Understand the core crawler settings, including user agent, crawl politeness, and scope controls.
Read guidecrawl planning
Spider vs. preview
Use preview mode to parse sitemaps and choose specific folders before running a focused spider.
Read guideai workflows
Agents
See common agent actions and learn how to use Pagebrain agents to analyze, organize, and ship SEO work.
Read guideintegrations
Connection tools
Connect Google Search Console and Google Drive so crawl evidence can become richer inputs and cleaner deliverables.
Read guideconnected ai
MCP
Connect Pagebrain MCP to tools like Claude and ChatGPT so agents can query crawl workspaces directly.
Read guide