Production Batch

Estimate, submit, monitor, and download large Batch jobs.

Production Batch

Prototype with priority 0. Use priority 1 for the one-hour production SLA and higher quotas. Production jobs normally execute a published Function so the behavior you evaluated is the behavior applied to the full dataset.

Machine-time pricing

Successful Batch jobs are billed for recorded GPU inference time. The default rate is $3.00 per GPU-hour, saved when each job is created.

GPU hours = sum(worker inference seconds × worker GPU count) / 3600
Charge in USD = GPU hours × the job's saved hourly rate

For example, two GPUs running inference for 30 minutes use one GPU-hour and cost $3.00 at the default rate. Startup, queueing, and failed retries are excluded. Charges are rounded to four decimal places.

Estimates, samples, and restarts retain the job’s saved rate. Later rate changes apply to new jobs. See the job’s machine_time_pricing fields for its measured GPU hours, hourly rate, and charge in the job response.

Estimate before submitting

estimate_job_id = so.batch_run_function(
    name="document-summary",
    data="https://storage.example.com/input.parquet?signature=...",
    job_priority=1,
    dry_run=True,
    id_column="record_id",
)

A dry run creates an estimate job and returns its job ID. It does not launch the full workload.

Submit without loading the file locally

For large inputs, pass a short-lived HTTPS GET URL for a CSV or Parquet object. We recommend using a presigned URL with a 1-hour expiration, as Sutro needs to be able to download the file within that time frame.

job_id = so.batch_run_function(
    name="document-summary",
    data="https://storage.example.com/input.parquet?signature=...",
    job_priority=1,
    stay_attached=False,
    id_column="record_id",
)

Use an immutable object key and a read-only signing identity. Treat the full presigned URL as a temporary credential.

CSV or Parquet columns must match the published Function’s inputs. See your Function’s configuration for the required columns.

Wait and download

Avoid materializing a large result as JSON:

so.await_job_completion(job_id, obtain_results=False)

if so.get_job_status(job_id) != "SUCCEEDED":
    raise RuntimeError("Job did not succeed")

so.download_job_results(
    job_id,
    output_path="results.parquet",
    include_inputs=True,
)

Parquet downloads support resuming after interruption. Keep your source object until you have reconciled the result rows.