Schema Drift Detection

Kadoa tracks the structure of each web source it scrapes and flags a job when that structure changes enough to break the extraction. A Beam agent checks job status on each run and, for a known and previously seen change, applies the account's approved fix, such as remapping a renamed field, then resumes the job. For a source that fails validation repeatedly or looks fully redesigned, the agent stops the job and routes it to a person instead of guessing at the layout. It never redefines a schema on its own, it only reapplies fixes the team has already approved for that kind of change.

Field-Level Validation Rules

Kadoa lets an account set validation rules for each field it pulls, like a price that must be numeric or a date in a set format. A Beam agent reads each newly scraped record against those rules and, for a record that passes, loads it into the connected warehouse or spreadsheet. Records that fail a rule, such as a missing price or a malformed date, get set aside with the failure noted, and a person reviews the batch before anything questionable reaches downstream systems. The agent applies only the validation rules already configured, it does not loosen or add rules to push more records through.

Scheduled Job Monitoring

Kadoa runs extraction jobs on a schedule the account sets, and each run reports whether it finished, failed, or returned fewer rows than expected. A Beam agent checks that run status after each schedule and, when a job finishes cleanly, notifies the team that new data is ready in the connected destination. When a run fails outright or returns a row count far below the recent average, the agent holds the notification and routes the run to a person to check before anyone treats the output as complete. The agent does not change job schedules or sources, it only reports on runs already configured.