Scheduling & drift
Schedule syncs and detect when source schemas drift.
By the end you’ll be able to
- Schedule an incremental sync respecting the adapter's `rateLimitHintRps`.
- Capture a schema snapshot and read a drift report.
- Know when drift means 'safe to ignore' vs 'rules need review'.
Once a source is connected and mapped, you put it on a schedule. Incremental-capable adapters (`supportsIncremental: true` — most of the estate, per `docs/CONNECTOR_MATRIX.md`) read only what changed since the last watermark, which is what makes nightly cycles tractable on a multi-million-row student dataset. Each adapter also publishes a `rateLimitHintRps` so the scheduler can pace itself without tripping vendor throttles.
The companion concern is schema drift. A source might rename a column, add an enum value, change a type, or quietly retire a resource. If you find out only when an audit fails, you've already shipped a bad return. DataBridge captures a schema snapshot on a schedule (dictionary-capable adapters do this for free; others compare against the last observed shape) and diffs every new capture against the baseline.
Drift reports live under `/drift`, with history at `/drift/history` and the cadence configured at `/drift/scheduler`. Each report classifies findings: additive changes are usually safe, type changes and removals are not. Removals on a field your mapper consumes are a hard signal — review the resource map and any rules that touch the field before the next ingest.
The recommended cadence: incremental sync nightly, full re-sync weekly (where the volume allows), drift capture on every full sync. That gives you fresh data and an alarm when the contract changes — the two things integration engineers actually own.
Walkthrough
- Open Drift
1.Open the drift dashboard
The Drift screen lists the most recent drift reports across connected systems. Severity tells you where to look first.
- Open Drift history
2.Browse drift history
Open the history view to see drift captures over time. The first capture is the baseline; subsequent captures diff against it.
- Open Drift scheduler
3.Configure the scheduler
Open the drift scheduler to see how the capture cadence is configured. Pair this with your ingest schedule so drift is detected before the next ingest.
Your turn
Open the Drift screen and identify the most recent drift report in the demo. Note whether it's additive (safe) or destructive (needs review), and which connected system it belongs to.
Hint: The 'Open Drift' step lands you on the dashboard; the history view shows the diff per capture.