Run Functions at scale

Execute published Functions over production datasets with Sutro Batch.

Sutro Batch is the production execution path for published Functions. Use it to apply the same optimized prompt, model, schema, and runtime settings to one row or millions of rows asynchronously.

Batch jobs cannot search the web or call tools. Inputs must contain all context the Function needs.

Batch uses machine-time pricing, measured in GPU-hours of inference.

Install and authenticate

pip install sutro
export SUTRO_API_URL="https://your-sutro-deployment.example.com"
export SUTRO_API_KEY="sk_..."

Use the URL where you published the Function and a deployment API key. See Authentication to obtain a key and verify that Batch is available before submitting work.

Submit a Function

import polars as pl
import sutro as so

reviews = pl.DataFrame({
    "review": [
        "The battery life is terrible.",
        "Great camera and build quality.",
    ]
})

job_id = so.batch_run_function(
    name="review-sentiment",
    data=reviews,
    job_priority=1,
    stay_attached=False,
)

results = so.await_job_completion(job_id)
print(results)

name is the published Function name. Input fields must match its declared inputs; the published Function supplies its prompt, model, output schema, and sampling defaults. Submission returns a job ID, and successful results return as a DataFrame.

Job priority and SLA

Job priority selects the workload class and service level for each submission. It is not a request to move one job ahead of another.

Priority	Use	Completion target
0	Prototyping on smaller samples	Within 10 minutes
1	Production execution at scale	Within one hour

Priority 1 is the production SLA for full-dataset Function runs and has higher workload quotas. Completion time still depends on input size and model. Check the quotas configured for your deployment with sutro quotas or so.get_quotas() before submitting a large job.

Sutro can handle workloads with millions of rows. Reach out to [e-mail] to request higher quotas for your deployment.

Supported inputs

• Lists of dictionaries
• Pandas and Polars DataFrames
• Local CSV and Parquet files
• CSV and Parquet datasets in Amazon S3, Cloudflare R2, or other S3-compatible storage, submitted with a presigned HTTPS GET URL

Batch does not accept s3:// URIs directly. For large production datasets, generate a read-only presigned HTTPS GET URL, preserve a stable ID column, and follow Production Batch for estimation, monitoring, and resumable downloads.

Custom models and embeddings

Batch can also execute custom open-source generation and embedding models for specialized workloads. Contact [e-mail] if you need this capability enabled for your deployment.