Data freshness
Freshness is a companion system for source-backed datasets. It does not scrape by itself. It tracks whether the source set tied to an entity is fresh, aging, stale, expired, or errored.
Register sources
Register a source
```bash
curl https://api.faircompany.ai/v1/freshness/sources \
-X POST \
-H "Authorization: Bearer fc_live_xxx" \
-H "Content-Type: application/json" \
-d '{
"entity_id": "company_stripe",
"entity_type": "company",
"source_key": "pricing_page",
"source_url": "https://stripe.com/pricing",
"refresh_cadence": "7 days",
"priority": 90
}'
```
```ts
const source = await fc.freshness.registerSource({
entity_id: "company_stripe",
entity_type: "company",
source_key: "pricing_page",
source_url: "https://stripe.com/pricing",
refresh_cadence: "7 days",
priority: 90,
});
console.log(source.source.id);
```
```python
import httpx
import os
api_key = os.environ.get("FAIRCRAWL_API_KEY", "fc_live_xxx")
response = httpx.post(
"https://api.faircompany.ai/v1/freshness/sources",
headers={"Authorization": f"Bearer {api_key}"},
json={
"entity_id": "company_stripe",
"entity_type": "company",
"source_key": "pricing_page",
"source_url": "https://stripe.com/pricing",
"refresh_cadence": "7 days",
"priority": 90,
},
timeout=30,
)
response.raise_for_status()
print(response.json()["source"]["id"])
```
```json
{
"tool": "faircrawl_freshness_check",
"arguments": {
"entity_id": "company_stripe"
}
}
```
Inspect one entity
Check entity freshness
```bash
curl https://api.faircompany.ai/v1/freshness/entities/company_stripe \
-H "Authorization: Bearer fc_live_xxx"
```
```ts
const dashboard = await fc.freshness.entitySources("company_stripe");
console.log(dashboard.summary);
console.log(dashboard.sources[0]?.freshnessState);
```
```python
status = fc.freshness_check("company_stripe")
print(status.summary)
print(status.sources[0].freshness_state if status.sources else None)
```
```json
{
"tool": "faircrawl_freshness_check",
"arguments": {
"entity_id": "company_stripe"
}
}
```
Read the aggregate dashboard
Read dashboard state
```bash
curl https://api.faircompany.ai/v1/freshness/dashboard \
-H "Authorization: Bearer fc_live_xxx"
```
```ts
const dashboard = await fc.freshness.dashboard();
console.log(dashboard.total_sources);
console.log(dashboard.by_state);
```
```python
import httpx
import os
api_key = os.environ.get("FAIRCRAWL_API_KEY", "fc_live_xxx")
response = httpx.get(
"https://api.faircompany.ai/v1/freshness/dashboard",
headers={"Authorization": f"Bearer {api_key}"},
timeout=30,
)
response.raise_for_status()
print(response.json()["by_state"])
```
```json
{
"tool": "faircrawl_freshness_check",
"arguments": {
"entity_id": "company_stripe"
}
}
```
Recommended pattern
- Register stable, canonical sources like pricing pages, docs roots, or product changelogs.
- Use short cadences only for sources that materially affect downstream decisions.
- Let freshness tell you what is due before you schedule expensive scrape or research jobs.