Powerful Features for Complete Website Documentation

Everything you need to understand, document, and automate any website.

Start Free Trial

AI-Powered Navigation

Claude-powered intelligent crawling that explores your site like a human would. Handles JavaScript, clicks buttons, fills forms, and navigates complex UI patterns.

  • Intelligent link discovery and following
  • Form detection and auto-filling
  • Modal and popup handling
  • Dynamic content waiting

Comprehensive Screenshots

Capture full-page screenshots of every unique page and state. Multiple formats supported with automatic thumbnails.

  • Full-page and viewport captures
  • PNG, JPEG, and WebP formats
  • Before/after state captures
  • Automatic thumbnail generation

Auto Documentation

Generate human-readable descriptions and technical documentation for every page automatically.

  • Page purpose descriptions
  • Technical element inventory
  • Accessibility information
  • Form field documentation

Playwright Scripts

Generate ready-to-use Playwright automation scripts for every navigation path discovered.

  • Complete page navigation scripts
  • Form interaction templates
  • Master crawl replay script
  • Customizable selectors

Sitemap Generation

Create complete XML sitemaps and interactive visualizations of your entire site structure.

  • Standard XML sitemap format
  • Interactive tree visualization
  • Page hierarchy mapping
  • Link relationship tracking

Authentication Support

Crawl protected pages with full authentication support. Multiple auth methods available.

  • Form-based login
  • Basic authentication
  • Cookie-based sessions
  • Custom header injection

REST API

Full programmatic access to manage sites, start crawls, and retrieve results. Perfect for CI/CD integration.

  • Complete CRUD operations for sites
  • Start and monitor crawls programmatically
  • Retrieve results, pages, and artifacts
  • Secure API key authentication

Webhooks

Real-time notifications when crawls complete, fail, or reach milestones. Stay informed without polling.

  • Crawl started/completed/failed events
  • HMAC-SHA256 signature verification
  • Retry logic for failed deliveries
  • Test webhooks before going live

And So Much More

Additional features to supercharge your workflow

Fast Execution

Crawl hundreds of pages in minutes with our optimized engine.

Real-time Progress

Watch your crawl progress with live updates and activity logs.

Easy Downloads

Download all artifacts as a ZIP or access individual files.

Secure Storage

Your credentials are encrypted with AES-256-GCM.

Console Logs

Capture browser console output to identify errors and warnings.

Visual Review

Browse screenshots in a beautiful gallery interface.

Ready to Document Your Entire Site?

Start your free trial today. No credit card required.

Features | SiteGulp | SiteGulp