How do I read results?

Poll the crawl status endpoint with the returned job id until pages are ready.

Yes — set max pages and depth to cap scope before you start a job.

Does Orsa render JavaScript?

Yes. Scrape calls run through a real headless browser path so SPAs and client-rendered content are included by default.

How does Orsa handle Cloudflare and bot detection?

We use patched browser fingerprints and tiered proxy escalation so most targets succeed without you operating infrastructure.

Do you respect robots.txt?

By default, yes — you can request different behavior when your use case allows.

★ WEB EXTRACTION / 05 OF 15

The whole site. Markdown, end to end.

Point Orsa at a domain and walk away. We crawl pages, normalize HTML, and hand back a structured corpus you can feed straight into RAG.

api.orsa.dev

Discover all URLs from a website's sitemap.

Live API response — no signup required.

Job start < 400ms

p50 latency

Varies by depth

p99 latency

Per-page Markdown via `web/crawl` pipeline

quality

credits per call

One job start. A crawl ID out.

Integration

Three lines to structured data.

Drop this into your codebase. TypeScript, Python, cURL — pick your language.

example

Response

{
  "job_id": "crawl_01jqz7y8k3v9s2r4t5u6v7w8x",
  "status": "running",
  "url": "https://docs.stripe.com/",
  "max_pages": 25
}

Who builds with this

Real jobs this endpoint solves.

Docs ingestion

Turn a public docs tree into embeddings without maintaining your own crawler fleet.

Competitive research

Snapshot how a product surface changes week over week.

Support deflection

Keep an always-fresh Markdown mirror of help centers for agents.

Combine with

Scrape Markdown Scrape Sitemap Scrape HTML

FAQ

Common questions.

Can't find your answer? Check the docs or reach out.

Try this endpoint. Free.

No credit card. Most teams integrate in under 10 minutes.