Inherited Platform Audit and Remediation

The scheduling and dispatch platform was three years old, entirely functional, and quietly frozen. The contractor who wrote it had delivered on time, been paid, and moved on. Everything he knew about the system went with him, and what remained ran every crew, every job, and every invoice the business issued.

The engagement did not begin because the system was broken. It began because a change was required and no one was willing to make it.

What Was Inherited

Read access to a repository, a set of AWS credentials, and a login. That is a normal starting position and it is more than some.

The codebase was not poor work. That distinction matters, because the reflex in this situation is to declare an inherited system a disaster and quote for a rewrite, which is usually both incorrect and extremely profitable. What was present was code written quickly and competently by one person who held the entire design in his head and was never asked to record any of it, which is a different failure and one the buyer participates in.

The resulting condition is consistent enough to predict before reading. No test suite, so no change can be made with confidence. No architecture notes, so the reasoning behind every non-obvious decision is unavailable. Infrastructure constructed by hand in a console across three years and reproducible nowhere. A deployment consisting of a shell script on one machine. Credentials in a shared spreadsheet, including a database password that had never been rotated. And a set of business rules existing only as behavior: the invoicing logic in particular handled part-day callouts in a specific and undocumented way that proved to be correct, load-bearing, and understood by exactly one person in the accounts function.

The Audit

Two weeks, fixed fee, with a document rather than any code as the deliverable.

The repository was read in full, the infrastructure was walked, real users were observed at work, and the system was run under load against a copy of production. Findings were then recorded and ranked, not by engineering severity but by what it would cost the business to be wrong about each item. An unindexed query that makes a page slow is an irritation. A backup that has never been restored is the company.

That ranking is the portion clients find most useful and the portion that requires understanding the operation rather than the code. The database password in a spreadsheet and the absent rate limit on the public booking endpoint moved to the top of the list, and neither was the item anyone had been concerned about.

The document was delivered as the client’s property, with an estimate against each item, to be acted on with this team or with any other. Some clients take the document and hire elsewhere. That is a legitimate outcome, and pricing the audit as a standalone piece of work is what keeps it honest.

Ranking by the Cost of Being Wrong

Four items were selected for remediation. They were the same four the audit had prioritized.

A means of holding the behavior still. Not comprehensive coverage, but characterization tests: tests asserting what the system currently does, including the parts suspected of being wrong, because the objective at this stage is not correctness but the ability to detect change. The undocumented callout invoicing rule was pinned by a test before any work went near it, and the test comment states plainly that the behavior is unverified and deliberately preserved. Once that net exists, everything else becomes possible. Until it exists, every change is a gamble.

A deployment that does not depend on a person. The shell script became a GitHub Actions pipeline that runs the tests and refuses to ship when they fail. The hand-built infrastructure was imported into Terraform in place, resource by resource, without recreating anything. That route is slower and considerably less satisfying than starting from a clean definition, and it is the only one that does not involve an unauthorized outage.

Credential control. Everything in the spreadsheet was rotated into a secrets manager, the previous keys were revoked rather than merely left unused, and access was reduced to what each role actually required. This was the least expensive item on the list by a wide margin and the one that had been sitting at the highest exposure for the longest.

Somewhere to look during an incident. Structured logging and tracing, so that a report of slow dispatch becomes a query rather than a conversation. The first week of traces identified an N+1 on the job list that had been degrading steadily as the company grew, which had never been reported because it had never been fast enough for the degradation to be noticed.

Work Deliberately Excluded

A considerable amount. The front end is not how this team would have built it and it works, so it remained. The queue is a database table under polling, which is unfashionable and entirely adequate at the current volume; replacing it would have cost real money to purchase capacity for a scale with no forecast date. The invoicing rule remains undocumented in origin and is now documented in behavior, with a note naming the person in accounts who understands the reasoning.

Restraint is most of the skill in this class of work. Every hour spent modernizing something absent from the risk list is an hour not spent on something on it, and the client pays either way.

Handover

A week of sessions, followed by the test: the client’s developer deployed a real change to production while the engagement team sat in the room and answered questions. Their hands, their release. Where that cannot be completed, the handover has not occurred, whatever the invoice states.

They retained runbooks for deployment, rollback, and the three failures assessed as most likely; architecture notes explaining reasoning rather than structure; and a restored backup, performed in front of them, because an unrestored backup is a hypothesis rather than a recovery plan.

Assessment Before Commitment

An audit is the appropriate first purchase when a system is inherited. Two weeks of a competent engineer reading the system and reporting where the risk sits is the least expensive decision available, it retains its value regardless of who is hired next, and it is the only route to establishing whether the answer is a rewrite before a rewrite has already been paid for.

A recommendation to start again in the first conversation warrants scrutiny. It is the easiest position in this industry to assert and the hardest to justify.