Pipeline Runs

Monitor pipeline execution progress, download validation errors, and retry failed runs with checkpoint support.

GET /api/v1/pipelines//runs//progress

Returns detailed progress information for a pipeline run including row counts, partition details, ETA, validation errors (first 20), and log entries.

Path Parameters

Parameter	Type	Description
id	string	Pipeline ID
runId	string	Pipeline run ID

Response

{
  "status": "running",
  "totalRows": 50000,
  "processedRows": 32000,
  "rowsIn": 32000,
  "rowsOut": 31988,
  "failedRows": 12,
  "skippedRows": 3,
  "partitions": 4,
  "completedPartitions": 2,
  "partitionDetails": [
    { "index": 0, "status": "completed", "rows": 12500 },
    { "index": 1, "status": "completed", "rows": 12500 },
    { "index": 2, "status": "running", "rows": 7000, "progress": 56 },
    { "index": 3, "status": "pending", "rows": 0 }
  ],
  "validationErrors": [
    { "row": 142, "column": "email", "value": "not-an-email", "error": "Invalid email format" }
  ],
  "eta": "3m",
  "durationMs": 45200,
  "loadStrategy": "upsert",
  "log": [
    { "stage": "extract", "message": "Read 50000 rows from source", "timestamp": "2026-03-30T10:00:05Z" }
  ]
}

Response Fields

Field	Type	Description
status	string	pending, running, completed, failed, completed_with_errors
totalRows	integer	Total rows to process
processedRows	integer	Rows processed so far
rowsIn	integer or null	Rows read from the source (populated when the run reaches completed / completed_with_errors)
rowsOut	integer or null	Rows landed in the target after transforms, validation, and load-mode rules
failedRows	integer	Rows that failed validation
skippedRows	integer	Rows skipped (e.g., duplicates)
partitions	integer	Total partition count
completedPartitions	integer	Partitions finished
partitionDetails	array	Per-partition status breakdown
validationErrors	array	First 20 validation errors (use the errors endpoint for full list)
eta	string or null	Estimated time remaining (e.g., "3m", "1.2h")
durationMs	integer	Elapsed time in milliseconds
loadStrategy	string	Load strategy used (append, upsert, replace)
log	array	Execution log entries

Roles

admin, editor, viewer

---

GET /api/v1/pipelines//runs//errors

Returns all validation errors for a pipeline run. Supports both JSON and CSV formats via the Accept header.

Path Parameters

Parameter	Type	Description
id	string	Pipeline ID
runId	string	Pipeline run ID

Headers

Header	Value	Description
Accept	application/json	Returns JSON (default)
Accept	text/csv	Returns downloadable CSV file

Response (JSON)

{
  "runId": "run_001",
  "total": 12,
  "errors": [
    { "row": 142, "column": "email", "value": "not-an-email", "error": "Invalid email format" },
    { "row": 305, "column": "age", "value": "abc", "error": "Expected integer" }
  ]
}

Response (CSV)

When Accept: text/csv is set, returns a downloadable CSV file with headers: Row, Column, Value, Error.

Content-Type: text/csv; charset=utf-8
Content-Disposition: attachment; filename="pipeline-run-{runId}-errors.csv"

Roles

admin, editor, viewer

---

POST /api/v1/pipelines//runs//retry

Retry a failed or partially-failed pipeline run. Creates a new run with checkpoint support for resumable load strategies.

Path Parameters

Parameter	Type	Description
id	string	Pipeline ID
runId	string	Original run ID to retry

Retry Behavior

Load Strategy	Resume Support	Behavior
append	Yes	Resumes from the last processed row
upsert	Yes	Resumes from the last processed row
replace	No	Full re-run from the beginning

Only runs with status failed or completed_with_errors can be retried. Attempting to retry a running or completed run returns 422.

Response (201)

{
  "id": "run_002",
  "pipelineId": "pipe_001",
  "status": "pending",
  "loadStrategy": "upsert",
  "metadata": {
    "retryOf": "run_001",
    "resumeFromRow": 32000,
    "resumable": true
  },
  "log": [
    {
      "stage": "retry",
      "message": "Resuming from row 32000 (original run: run_001)",
      "timestamp": "2026-03-30T12:00:00Z"
    }
  ]
}

Error Codes

Code	Reason
404	Pipeline or run not found
422	Run status does not allow retry (must be failed or completed_with_errors)

Roles

admin, editor

---

Role Summary

Endpoint	Method	Allowed Roles
/pipelines/{id}/runs/{runId}/progress	GET	admin, editor, viewer
/pipelines/{id}/runs/{runId}/errors	GET	admin, editor, viewer
/pipelines/{id}/runs/{runId}/retry	POST	admin, editor

See also: Pipelines | Data Platform