HR Automation — Design Concept
Mapped out a design for automating HR record-keeping for an organization with 400+ staff spread across multiple sites — currently split across scattered spreadsheets and email attachments, with no reliable way to track visa, labor card, or contract expiries. This is a design only; nothing below has been implemented yet.
The core decision is to not build custom. Frappe HR, an open-source HRIS on the Frappe Framework (Python, MariaDB, Redis for cache/jobs), already covers employee records, custom fields, notifications, and role-based field permissions — weeks of work to replicate from scratch. Deployment would use the official frappe_docker Compose stack (site + MariaDB + Redis + internal Nginx, managed via bench). Visa expiry, labor card expiry, and Emirates ID expiry become custom fields on the Employee doctype through the Customize Form UI — no code changes. Reminders ride on Frappe's built-in Notification doctype, triggered off those date fields N days out, able to call a webhook instead of just Frappe's own email/in-app alerts.
Hosting sits on a dedicated VPS, isolated from other infrastructure, with no public 80/443 exposed at the firewall level at all — reachable only over a WireGuard-based private overlay (the same protocol behind Twingate). That's network-layer isolation, not just an app login page. Data-at-rest encryption on the volume holding MariaDB's data directory sits on top of that as a second, independent layer.
Getting scattered data in has two paths. The base directory (name, branch, joining date) is a one-time bulk import through Frappe's own Data Import tool — CSV/Excel columns mapped to fields, no script needed. Expiry dates are the harder case, since they live on scanned documents, not structured text: the existing accounts-report pipeline already authenticates against the Microsoft Graph API and pulls email attachments for OCR, so the same auth flow and attachment handling can point at ID/visa scans instead, writing parsed dates to Frappe over its REST API (PUT /api/resource/Employee/<name>) rather than building a second ingestion path.
The part of the design that actually matters is the interface, because a web portal is only as good as people's willingness to log into it. Alerts and updates route through a WhatsApp group instead. Outbound: a scheduled job queries Frappe for records with an expiry inside the alert window and posts a message. Inbound: a plain-language reply (a visa renewal, a new joiner) gets parsed into a structured change, matched against the Employee list, and echoed back as a confirmation prompt — only an explicit reply triggers the actual write. Salary fields specifically sit behind Frappe's Role Permission Manager, scoped per role rather than per user, so the confirm-before-write step and the access model are two separate safeguards on the same sensitive data.
Threads worth pulling on further, none of it decided yet:
Trust, not just access. Confirm-before-write stops accidental bad parses, but it doesn't stop a bad actor with WhatsApp access. Salary changes specifically might warrant a second approver rather than a single self-confirm — and every change, regardless of size, should probably land in an immutable audit log (old value, new value, who confirmed, timestamp) rather than just overwriting the field.
Escalation, not just alerting. A single reminder N days out assumes someone reads it and acts. A tiered version — 30/14/7/1 days, escalating to a different person (branch admin → ultimately the owner) if nothing changes in the record after the first alert — turns "we sent a message" into "someone is actually accountable."
Keep the source document, not just the extracted date. Frappe supports file attachments per record. Attaching the actual scanned visa/labor card/Emirates ID to the Employee record — not just the OCR'd expiry date — means the source document is one click away when it's time to actually renew something, instead of a date with no paper trail behind it.
Cost forecasting. Every visa/labor card renewal has a government fee attached. A cost-per-renewal-type field, multiplied against the year's known expiries, turns this from a pure compliance tracker into a rough annual HR-compliance budget line — and it's a natural bridge to the existing beancount ledger rather than a fully separate system.
Onboarding and offboarding as workflows, not just directory edits. Frappe HR already ships Onboarding and Separation doctypes — checklists for what a new joiner needs (visa sponsorship steps, documents) and what a leaver triggers (clearance, final settlement). Worth using those directly instead of treating every hire/exit as a manual directory edit.
Retention policy. Once someone leaves, their salary and ID data doesn't just get deleted — labor law typically requires records be kept for a minimum period. Worth deciding upfront what "departed staff" looks like in the schema (archived vs. deleted, and for how long) rather than retrofitting it later.
The system itself needs a backup story. This is now infrastructure holding sensitive data on hardware separate from everything else — it needs its own place in the existing nightly backup routine, encrypted at the same standard as the live data, not an afterthought once it's running.
Updated Sep 29, 2026 18:48 UTC — a few more threads worth adding to the list above.
A dedicated number for the bot, not a personal-linked one. The WhatsApp interface should run on its own number, isolated from any personally-linked account, the same way a Telegram bot has its own separate identity. That keeps the compliance-sensitive group boundaries clean and avoids a scenario where confirmations for salary changes appear to come from a personal account rather than the system.
Don't trust every OCR read equally. The Graph-API-plus-OCR pipeline pulling expiry dates off scanned documents will occasionally misread a digit. Low-confidence extractions should route to manual review instead of flowing straight into a confirm-before-write prompt — a bad OCR read paired with a rubber-stamped confirmation is exactly how a wrong expiry date ends up as the system of record.
Push the audit log somewhere, don't just store it. An immutable change log is only useful if someone looks at it. A periodic digest — who's expiring soon, what changed since last time — timed to line up with the existing compliance committee cadence turns the audit trail into something that actually feeds that process, instead of a table nobody queries until something goes wrong.
Updated Sep 30, 2026 10:30 UTC — VPS sizing and the agent-hosting question, now resolved.
Sizing the VPS. Landed on a 4 vCPU / 8GB RAM NVMe-backed instance running Debian. At this scale the constraint isn't data volume — a few hundred employee records plus scanned documents is trivial — it's the number of always-on services a typical Frappe Docker Compose stack runs simultaneously: web server, several worker processes, a scheduler, cache, queue, and the database itself, each wanting its own slice of memory whether or not anything is actively happening.
A second encryption layer without a boot-unlock headache. Rather than full-disk LUKS encryption on the boot volume — which typically means entering a passphrase on every reboot, awkward for an unattended headless box — MariaDB's data directory and the attached scanned documents live on a separate network-attached block storage volume that the provider encrypts at rest by default (AES-256), no customer-managed key or boot-time unlock step required. That satisfies the "second independent layer" from the original design without adding an operational failure mode every time the box restarts.
Where the WhatsApp interface runs matters as much as where the database sits. The obvious shortcut was to add the WhatsApp bot as one more account on the automation gateway already running elsewhere, rather than standing up a second instance just for this. Wrong call: the isolation goal covers the whole conversational path, not only the database. Employee names, expiry dates, confirmation prompts, and the bot's own session credentials would otherwise flow through that other gateway's logs and its existing nightly backup routine — alongside unrelated personal data — which is exactly the mixing the dedicated VPS exists to prevent. Corrected design: the WhatsApp interface runs as its own agent instance, directly on the same isolated VPS as the database it talks to, with its own logs and its own backup path to the private backup target, independent of anything else.
Extending an existing agent, not inventing a third one. There's already a sandboxed, read-only business-automation agent running elsewhere with its own WhatsApp binding. Rather than designing a new agent identity from scratch for this, that existing one gets extended into the new environment: a fresh instance on the isolated VPS, a newly assigned WhatsApp number (not yet provisioned), and write access to Frappe's API layered in behind the confirm-before-write step described above — a capability upgrade on a known, already-sandboxed agent rather than a fourth thing to reason about.
Updated Oct 08, 2026 UTC — the design moved from concept to a running system over the following days.
From design to a running system. Deployed on the isolated VPS described above, running in Docker. The employee directory came in from several spreadsheets, one per site, each with its own column layout — so the import needed per-sheet header detection rather than one fixed column mapping. Recovering expiry dates from the same source required reading formula-evaluated values rather than raw cell contents, since several sheets computed dates via formulas instead of storing a literal date.
Salary reconciliation, done in passes. Matching a separate payroll export against the directory went exact-match first, then fuzzy name-matching with manual confirmation on anything ambiguous, then a one-by-one pass on what was left — new joiners not yet in the directory, or departed staff being incorrectly reintroduced. A batch-creation bug during the original import re-scanned entire sheets instead of only the unmatched rows, creating a wave of duplicate records mixed in with genuinely new ones. Cleanup ran in two passes: a creation-timestamp cutoff plus loose token-matching for the bulk of it, then a string-similarity pass to catch the near-duplicates — typos, spacing differences, OCR-style variants — the first pass missed. Where a duplicate carried data the original lacked, that data got merged in before the duplicate was deleted, never the reverse.
Matching gets harder than exact-or-fuzzy. A later reconciliation round needed two refinements: de-duplicating repeated name tokens before scoring similarity, and falling back to first-and-last-name matching when a record has no middle name rather than failing the comparison outright. A dedicated check — independent of the overall similarity score — caught a same-surname, different-person collision that would otherwise have applied one person's pay figure to someone else's record; the overall score alone scored it high enough to pass. A handful of genuine gaps, people absent from payroll data across two consecutive months, went to manual follow-up rather than being force-matched at a lower confidence threshold — in keeping with the original design principle of flagging low-confidence results instead of auto-applying them.
Two payroll snapshots, not one replacing the other. A later month's payroll file used a different column layout than the first — no net-pay column; gross had to be computed from basic pay plus allowances instead — so rather than overwrite the first month's figures, both now sit side by side as separate snapshots, useful for cross-checking one month against another rather than trusting either in isolation.
Housekeeping on the record layout. Reorganized the custom fields so salary and employment-status data live under their logical sections instead of one catch-all block, and hid — not deleted — the fields that weren't part of daily use. The reordering needed the platform's own form-customization mechanism rather than a direct field edit; a direct edit silently misplaced one block while appearing to work on another.
The interface needed its own isolated agent, not just its own database. Resolved the open question from the design above: rather than adding one more account to the automation agent already running elsewhere, the messaging interface now runs as a fully separate agent instance, directly on the isolated VPS, through its own subscription login with a different LLM provider than the main instance — different enough to count as a genuinely separate environment, not a second account on a shared one. It has its own persona, deliberately exacting and procedural in tone given what it handles, and inherited the standing caution rules (read-only on sensitive config, prefer-safe-delete, ask-first on infrastructure changes) from the main assistant's own rulebook, for consistency.
Messaging landed on a different platform than planned first. WhatsApp pairing got as far as a scannable QR code before being deferred. In the meantime, a Telegram bot connection went live instead — far simpler to wire up without an interactive pairing flow — so the alerts below are flowing there today, with WhatsApp still the eventual target.
The confirm-before-write mechanism, for real. A small queue now sits between the agent and the database: a proposed change gets recorded but not written, requires one confirmation for most fields and two independent confirmations for salary changes specifically, and only writes through the API once that threshold is met. Tested end-to-end against real records, including the two-confirmation salary path, before going live — a genuine gate the agent can't bypass unilaterally, not just an instruction it's been told to follow.
Alerts and a weekly digest, both live. Tiered reminders (30/14/7 days out, plus already-expired) run daily across every tracked expiry type, de-duplicated so the same alert doesn't repeat but does escalate if it crosses into a more urgent tier. The first real run surfaced a meaningful backlog of already-expired records, some stale for years — reported plainly rather than softened, which is the tone the whole system is built around. A weekly digest adds compliance counts, pending-queue activity, and a genuine audit trail pulled from the platform's own built-in version history rather than a separately bolted-on log.
Still open: an escalation contact beyond the primary reviewer, a private network path to reach the new instance's dashboard remotely, a backup routine for this instance's own data, and the WhatsApp migration itself.