Reference design for an enterprise HR data platform that integrates with Greenhouse (recruiting) and Workday (HCM) to support People Systems and employee lifecycle analytics. The integration layer uses OAuth authentication, retry logic, rate limiting, and robust error handling to reliably ingest candidate, employee, and recruitment records into a Medallion-architected lakehouse with centralized governance.
Recruiting and HR platforms hold two halves of the employee lifecycle. Greenhouse covers the path from job requisition to accepted offer: candidates, applications, interviews, scorecards and offers. Workday covers everything after hire: worker records, positions, organisations, compensation and terminations. Joining the two lets People Analytics answer questions neither system answers alone, such as time from requisition to productive employee, offer-acceptance rates by source, or retention of hires by interview panel.
The design separates concerns. A dedicated ingestion layer handles each vendor's authentication (API keys or OAuth 2.0 client credentials), pagination, incremental extraction by updated timestamp, rate limiting with exponential backoff, and dead-letter handling for records that fail validation. Raw responses land unchanged in a Bronze layer for replay. Silver tables normalise and deduplicate records and keep history with slowly changing dimension Type 2, so a worker's department on any past date can be reconstructed. Gold tables publish metrics with access controls appropriate to sensitive personnel data.
| Concern | Greenhouse | Workday |
|---|---|---|
| Primary data | candidates, applications, offers | workers, positions, compensation |
| Access style | REST (Harvest API), paginated JSON | REST and SOAP web services, reports-as-a-service |
| Incremental loads | updated_after filters | effective-dated changes, transaction logs |
| Join key | candidate or employee id mapping | worker id; map from Greenhouse hire records |
| Sensitivity | candidate PII, demographics | PII, compensation, performance |
Treat identity resolution as a first-class step: the link between a Greenhouse candidate and a Workday worker is rarely a clean shared key and usually relies on hire events, email addresses or an integration-managed mapping table. Apply column-level masking and least-privilege roles from the Bronze layer onward, because raw HR payloads contain the most sensitive fields. The detailed page below covers endpoints, authentication flows, PySpark ingestion code, SCD Type 2 history and anomaly detection. Related material: medallion architecture and API engineering.