One engine, one HTTPS API, one audit trail. Two endpoints do the work: POST /api/v1/validate (one record) and POST /api/v1/validate/batch (up to 10,000 records per call). Everything below is the honest pattern for each platform — including what each one can and cannot gate.
Export from anything that can make a spreadsheet — ERP, estates system, supplier portal. Drop the CSV in the dashboard, pick a contract, read the gap report record by record. A weekly run lands the same sealed evidence as a pipeline. Day one needs nothing else.
The reference recipe: a kept-alive session, explicit timeouts, retries on transport errors only.
import requests
s = requests.Session() # keep-alive: one TLS handshake, many calls
r = s.post(f"{CELL}/api/v1/validate/batch",
headers={"Authorization": f"Bearer {TOKEN}"},
json={"contract": "customer", "records": chunk}, # 100–1,000 per call
timeout=(2, 10))
r.raise_for_status() # fail-closed: an outage stops the write
out = r.json()
good = [x for x in out["results"] if x["valid"]]
quarantine = [x for x in out["results"] if not x["valid"]]
1. Guided entry, synchronous, no code: register OpenDQV Cloud as an External Service (import this OpenAPI file), build a Screen Flow — Action → Decision → Create Records — and override the standard New button per object. GA platform features since 2017. 2. Integration & bulk volume: gate in your middleware or API layer before the upsert — one HTTP step, true fail-closed. 3. The coverage net: writes born inside the platform (inline edits, third-party packages) are validated seconds after commit via Change Data Capture and quarantined. No external service can synchronously intercept a SaaS platform's own save path — we won't pretend otherwise.
Validate before you land: chunk each partition through the batch endpoint, split pass and quarantine, then write. In Structured Streaming do the same inside foreachBatch. Keep your DLT expectations — they inspect in-pipeline; this gates before landing.
def gate(rows):
# one session per partition — never per row
for chunk in chunks(rows, 500):
out = post_batch(chunk) # /validate/batch
yield from split(out) # pass → write · block → quarantine table
df.rdd.mapPartitions(gate)
Gate in the loader before COPY INTO (cleanest — the loader is the door), or validate in-warehouse from Snowpark via External Network Access with a chunked stored procedure. Streams + Tasks gate promotion raw→silver rather than first landing — we say that plainly.
Best gate: a consumer between topic and sink — one batch call per poll, pass records to the validated topic, blocks to a quarantine topic, commit offsets after both. Producer-side validation is the true door where you control every producer. Connect SMTs with per-record HTTP calls collapse throughput — put the gate upstream of the sink connector instead.
Keep your dbt tests — they inspect what landed. Add the gate where rows are born: the snippet in the dashboard shows a dbt-adjacent batch call for pre-load validation.
Paste https://mcp.opendqv.com into any MCP-compatible assistant and approve — then ask "what failed validation this week?" in plain English. Agent calls default to dry-run so an exploring assistant never pollutes your audit trail.
Copy-paste snippets for curl, Python, Node, Apex, Salesforce Flow, and dbt — with your tenant URL and token filled in — are in the dashboard from the moment you sign up. Start free trial →