Search, then full details

This recipe is the workhorse pipeline: run a search for summaries, collect the job_key of every hit, batch-fetch the full role objects, and export everything as CSV. One runnable script, using httpx (pip install httpx).

What is the flow?

1. POST /v1/search with max_pages: 2 — synchronous, since 2 ≤ 3 pages. Costs 2 credits.
2. Collect job_key from each summary.
3. POST /v1/roles/batch with those keys — returns 202 with a job. Costs 1 credit per role.
4. Poll GET /v1/jobs/{id} until completed.
5. Download GET /v1/jobs/{id}/results?format=csv.

If you would rather have the server do steps 1–3 in one call, use POST /v1/search/with-details; this recipe keeps the steps separate so you can filter between search and detail (skipping roles you already have is the main credit saver).

The script

"""Search Indeed postings, then batch-fetch full details and save CSV."""

import sys
import time

import httpx

BASE = "https://api.rolesapi.com"
API_KEY = "rk_live_your_key"  # read from an env var in real code
HEADERS = {"Authorization": f"Bearer {API_KEY}"}

KEYWORD = "backend engineer"
LOCATION = "Austin, TX"


def search_two_pages(client: httpx.Client) -> list[dict]:
    """POST /v1/search with max_pages=2 -- sync at <=3 pages."""
    r = client.post(
        f"{BASE}/v1/search",
        json={
            "keyword": KEYWORD,
            "location": LOCATION,
            "country": "us",
            "sort": "date",
            "max_pages": 2,
        },
    )
    r.raise_for_status()
    return r.json()["data"]


def submit_batch(client: httpx.Client, job_keys: list[str]) -> str:
    """POST /v1/roles/batch -- always async, 202 with a job id."""
    r = client.post(
        f"{BASE}/v1/roles/batch",
        json={"job_keys": job_keys, "country": "us"},
    )
    r.raise_for_status()
    job = r.json()["data"]
    print(f"Job {job['job_id']} queued")
    return job["job_id"]


def wait_for_job(client: httpx.Client, job_id: str) -> None:
    """Poll GET /v1/jobs/{id} until it reaches a terminal state."""
    while True:
        r = client.get(f"{BASE}/v1/jobs/{job_id}")
        r.raise_for_status()
        job = r.json()["data"]
        p = job["progress"]
        print(f"  {job['status']}: {p['done']}/{p['total']}")
        if job["status"] == "succeeded":
            return
        if job["status"] in ("failed", "timed_out", "aborted"):
            sys.exit(f"Job {job['status']}: {job['error']}")
        time.sleep(3)


def download_csv(client: httpx.Client, job_id: str, path: str) -> None:
    """GET /v1/jobs/{id}/results?format=csv -- whole result set, no envelope."""
    r = client.get(f"{BASE}/v1/jobs/{job_id}/results", params={"format": "csv"})
    r.raise_for_status()
    with open(path, "w", encoding="utf-8") as f:
        f.write(r.text)
    print(f"Wrote {path}")


def main() -> None:
    with httpx.Client(headers=HEADERS, timeout=30) as client:
        summaries = search_two_pages(client)
        print(f"Search returned {len(summaries)} roles")

        job_keys = [s["job_key"] for s in summaries]
        if not job_keys:
            sys.exit("No results for this query.")

        job_id = submit_batch(client, job_keys)
        wait_for_job(client, job_id)
        download_csv(client, job_id, "roles.csv")


if __name__ == "__main__":
    main()

What does the output look like?

roles.csv has one row per role, with nested fields flattened to dot-path columns and arrays joined with | — the rules in Output formats:

job_key,title,company.name,company.rating,location,remote,employment_type,salary.min,salary.max,salary.currency,salary.period,benefits,posted_at,country
a1b2c3d4e5f60718,Senior Backend Engineer,Acme Logistics,4.1,"Austin, TX",false,Full-time,140000,175000,USD,year,Health insurance|401(k) matching|Paid time off,2026-06-28,us

How many credits did this use?

2 (search pages) + 1 per role fetched in the batch. If the search returned 30 roles: 2 + 30 = 32 credits.

How can I adapt it?

• Dedupe before the batch. Filter job_keys against roles already in your database — that line is where this pipeline beats with-details on cost.
• Skip the poll loop. Register a webhook with POST /v1/webhooks and let RolesAPI call you — see the Webhooks guide.
• Trim the payload. Add params={"format": "csv", "fields": "job_key,title,company.name,salary.min"} to the results request to control the columns — see Field projection.