Document workloads run vision-capable models over scans and PDFs in large batches. Nemo Router keeps every request on a vision model, paces the batch against provider quotas, and tracks cost per job.
A document batch in flight
Requests stay on vision-capable models
Per-key RPM/TPM limits throttle the loop
Spend attributed by job-id metadata
models on Google Vertex AI
Document processing has three constants: it needs vision, it runs in bulk, and it has a budget. Nemo Router handles all three behind one key.
Extracting fields from a scanned invoice or classifying a PDF needs a vision model. The catalog tags vision capability, and tag-filtered routing guarantees a document request lands on a vision-capable model.
Document processing runs in bulk — thousands of pages per job. Each page is a standard request, so a batch is a loop. Per-key rate limits pace it, and failover absorbs a provider blip mid-run.
A batch job has a budget. LiteLLM reports the real cost of every call; tag the batch with a job id and the dashboard attributes total spend to that job — cost-per-document becomes a measured number.
A batch can run for hours. A provider degradation halfway through shouldn’t mean restarting from page one. The fallback chain retries the next link transparently and the run continues.
A batch is a loop of standard requests. Each document routes to a vision-capable model, the loop is rate-paced and failover-protected, and total spend rolls up by job id.
Document batch flow
Batch job
job-id in metadata
Thousands of documents — scans, PDFs, images.
Per document
one /chat/completions
Each page is a standard vision request.
Vision tag route
catalog · vision filter
Tag filter pins the request to a vision-capable model.
Rate-paced + failover
RPM/TPM · fallback
Limits pace the loop; failover survives a provider blip.
Cost per job
spend analytics
Total spend rolled up by the job-id tag.
Tag-filtered routing is the safety rail — a request tagged vision can never silently fall back to a text-only model.
Capability tags
Document work breaks if a request meant for a vision model lands on a text-only one. Tag a request with the capability it needs and the candidate pool is filtered to models that have it. Combine vision with long-context for multi-page PDFs; a mismatched tag set returns a 400 instead of a wrong answer.
Tag filter in action
A document batch is a loop of standard chat.completions calls — image content in, structured fields out. These snippets are generated from the SDK examples the playground uses; add a job-id to metadata to attribute the batch.
pip install openai| 1 | # Cache: enabled (org default). Pass nemo_cache: false to skip. |
| 2 | from openai import OpenAI |
| 3 | import os |
| 4 | |
| 5 | client = OpenAI( |
| 6 | api_key=os.environ["NEMOROUTER_API_KEY"], |
| 7 | base_url="https://api.nemorouter.ai/v1", |
| 8 | ) |
| 9 | |
| 10 | response = client.chat.completions.create( |
| 11 | model="gemini-2.5-flash", |
| 12 | temperature=1, |
| 13 | max_tokens=1024, |
| 14 | top_p=1, |
| 15 | messages=[ |
| 16 | {"role": "user", "content": "Hello! What models do you support?"}, |
| 17 | ], |
| 18 | extra_body={ |
| 19 | # "nemo_cache": False, # Uncomment to skip cache |
| 20 | }, |
| 21 | ) |
| 22 | |
| 23 | print(response.choices[0].message.content) |
Tag each call with a job id in request metadata — spend analytics rolls the whole batch up to that job.
Vision, at volume, with a cost number
Tag-filtered vision routing, batch-friendly rate limits, failover, and per-job cost tracking — all unlocked on every plan.