Does Orsa render JavaScript?

Yes. Scrape calls run through a real headless browser path so SPAs and client-rendered content are included by default.

How does Orsa handle Cloudflare and bot detection?

We use patched browser fingerprints and tiered proxy escalation so most targets succeed without you operating infrastructure.

Can I tune aggressiveness per call?

Yes. You can opt into stricter behavior when you need it; robots.txt is respected by default.

What are the default size limits?

Large pages are supported with sensible defaults; higher limits are available for long-form content when your workload needs them.

★ WEB EXTRACTION / 04 OF 15

Every URL on any domain.

Discover the full shape of a website in one call. Recursive index handling, gzip, and malformed XML are handled silently.

api.orsa.dev

Discover all URLs from a website's sitemap.

Live API response — no signup required.

0ms

p50 latency

1.9s

p99 latency

High recall on standard and nested sitemap indexes

quality

credits per call

One call in. The full URL graph out.

Integration

Three lines to structured data.

Drop this into your codebase. TypeScript, Python, cURL — pick your language.

example

Response

{
  "domain": "nytimes.com",
  "urls": [
    "https://www.nytimes.com/",
    "https://www.nytimes.com/section/world"
  ],
  "sources": ["https://www.nytimes.com/sitemap.xml"]
}

Who builds with this

Real jobs this endpoint solves.

Crawl planning

Seed crawls with the real surface area of a site, not just the homepage.

SEO ops

Diff sitemaps over time to catch indexing regressions early.

Docs mirrors

Pull every docs path before you snapshot content to Markdown.

Combine with

Crawl Website Scrape Markdown Scrape HTML

FAQ

Common questions.

Can't find your answer? Check the docs or reach out.

Try this endpoint. Free.

No credit card. Most teams integrate in under 10 minutes.