scraping basics
How to scrape
Learn the core scraper workflow: entering domains, focusing subfolders, and starting from sitemaps.
Start with the right entry point
Use a domain when you want Pagebrain to discover the site broadly. Use a subfolder when you already know the section you want to inspect. Use a sitemap when the site publishes a clean URL list and you want tighter control over what gets crawled.
Keep the crawl focused
Focused scrapes are easier to review and easier for agents to reason over. Start with the part of the site that matches your question, then expand the crawl if you need more context.
Review before you analyze
Once the scrape finishes, use the crawl table, page context, and filters to understand what was collected before turning the data into findings.