crawl planning
Spider vs. preview
Use preview mode to parse sitemaps and choose specific folders before running a focused spider.
Preview first when scope is unclear
Preview helps you inspect discovered URLs before committing to a full scrape. It is useful when a sitemap contains many sections or when you want to focus on only a few folders.
Spider when you are ready to collect
Spider mode performs the full crawl and collects the page evidence Pagebrain needs for analysis, agents, exports, and comparison workflows.
Use both together
Preview lets you choose a cleaner target. Spider turns that target into crawl evidence your team and agents can use.