The Collective Agreement Compliance Validator is built on a Supervisor-Worker multi-agent architecture using LangGraph. Seven specialized agents process each validation request in sequence, with a typed state schema ensuring deterministic data flow between agents. The system is designed to be reusable across any of the Government of Canada's 150 collective agreements.
RAG query to Pinecone. Fetches the relevant collective agreement clause chunks using metadata-filtered semantic search.
Extracts structured legal requirements from the raw clause text. Outputs a typed JSON schema of requirements.
Analyzes the Dayforce pay rule configuration parameters against the parsed requirements schema.
Line-by-line comparison of the configuration against each legal requirement. Identifies discrepancies.
Assigns severity levels (Critical/High/Medium/Low) to each discrepancy based on financial and legal impact.
Compiles all findings into a structured compliance report with remediation guidance.
LLM-as-a-Judge: scores the report against the original clause for hallucinations. Below 0.9 triggers human review.
LangGraph (Python)
Manages the 7-agent state machine. Enforces a typed TypedDict state schema — every agent reads from and writes to a single shared state object, ensuring deterministic data flow between agents.
Why this tool
Chosen over n8n's visual canvas because LangGraph provides strict type contracts between agents and native support for conditional routing and human-in-the-loop checkpoints — non-negotiable at government enterprise scale.
GPT-4o (OpenAI)
Powers the Rule Parser, Compliance Checker, Risk Assessor, Report Writer, and Evaluator agents. Used for complex legal reasoning, structured JSON output, and LLM-as-a-Judge evaluation.
Why this tool
GPT-4o's structured output mode (JSON schema enforcement) is critical for ensuring agents return parseable, typed data rather than free-form text.
Claude 3.5 Sonnet (Anthropic)
Powers the Config Analyzer agent. Claude excels at analyzing long structured documents and configuration parameters with high precision.
Why this tool
Dual-LLM strategy: different models have different strengths. Using the best model for each task rather than a single model for everything is a key enterprise design principle.
Pinecone
Stores collective agreement clause embeddings. The Context Retriever agent queries Pinecone with metadata filtering on agreement_id and clause_id, ensuring agents only retrieve context for the specific clause being validated.
Why this tool
Scoped retrieval is a fundamental guardrail — without metadata filtering, agents could hallucinate context from a different collective agreement's clauses.
OpenAI text-embedding-3-small
Converts collective agreement clause text into vector embeddings for semantic search. Enables the Context Retriever to find relevant clauses even when the query doesn't use exact legal terminology.
Why this tool
Cost-effective and integrates natively with LangChain's vector store interface. At government scale (150 collective agreements), embedding cost matters.
FastAPI (Python)
Wraps the LangGraph workflow in an HTTP endpoint (/run-validation). Enables GitHub Actions and other CI/CD systems to trigger validation runs programmatically via webhook.
Why this tool
FastAPI's automatic OpenAPI documentation generation means every team consuming this service gets interactive API docs for free — important for adoption across 300 scrum teams.
GitHub Actions
Triggers the validation workflow automatically via webhook when a developer opens a pull request to the qa branch. Integrates the agent directly into the SDLC — developers don't need to remember to run it manually.
Why this tool
The workflow is environment-agnostic by design. The GitHub Actions trigger can be replaced with GitLab CI, Jenkins, or any webhook-capable CI system without changing the agent architecture.
LangSmith
Provides full tracing and spans for every agent call. Every validation run generates a complete trace showing inputs, outputs, latency, and token usage for each of the 7 agents.
Why this tool
In a government program, auditability is non-negotiable. LangSmith's traces provide the equivalent of distributed tracing in a microservices architecture — essential for debugging and compliance audits.
The validator is intentionally designed as a reusable scaffold. To validate a different collective agreement or clause, you simply ingest the new agreement text into Pinecone with the appropriate metadata tags. No agent code changes are required. The same 7-agent pipeline handles any of the Government of Canada's 150 collective agreements — making it deployable across all 100+ departments and agencies without modification.