Crawl overview
POST /v1/crawl/web/crawl expands from one seed URL into a bounded list of scraped pages.
Use it when:
- you already know the seed URL
- you want actual page content, not just discovered URLs
- you want a hard cap with
limit - you need regex or de-duplication controls before downstream processing
Bounded crawl
```bash
curl https://api.faircompany.ai/v1/crawl/web/crawl \
-X POST \
-H "Authorization: Bearer fc_live_xxx" \
-H "Content-Type: application/json" \
-d '{
"url": "https://docs.stripe.com",
"limit": 20,
"render": false
}'
```
```ts
const crawl = await fc.scrape.crawl({
url: "https://docs.stripe.com",
limit: 20,
render: false,
});
console.log(crawl.pagesScraped);
console.log(crawl.pages[0]?.url);
```
```python
crawl = fc.crawl(
"https://docs.stripe.com",
limit=20,
render=False,
)
print(crawl.pages_scraped)
print(crawl.pages[0].url if crawl.pages else None)
```
```json
{
"tool": "faircrawl_crawl",
"arguments": {
"url": "https://docs.stripe.com",
"limit": 20,
"render": false
}
}
```
Result model
The crawl route returns the seed, scraped pages, discovered links, and a normalized _meta billing block. FairCrawl does not require a poll loop for this route in the current public contract.