How to add AI to a legacy system
The instinct is to assume the old system has to change first. It almost never does. What matters is not how modern the system is but whether anything can be read out of it and written back, and there are usually more ways in than the team believes.
Does a legacy system need an API for this to work?
No. An API is the most convenient surface, not the required one. Anything that lets you observe state and record a result is enough: a database you can read, a scheduled export, a file drop, a message queue, even a report that already lands somewhere on a schedule.
That matters because the systems most in need of help are usually the ones least able to expose an endpoint. A twenty-year-old line-of-business application with no integration layer still writes to a database, and that database is a perfectly good surface to work against.
The age of the system is rarely the constraint. The constraint is whether anyone still understands its data model well enough to say which table means what, and that is a people problem rather than a technical one.
| Surface | How it works | Best when |
|---|---|---|
| API or webhook | React to events, write results back directly | One exists and is documented |
| Database read | Poll or use change tracking, write to a side table | No API, but schema access is available |
| Scheduled export | Process the file, return results by import or report | Nightly batch is fast enough |
| Message queue | Consume events already being published | The system already emits them |
| Screen or RPA layer | Drive the interface as a user would | Genuine last resort, brittle and slow |
Why not just modify the legacy system?
Because the risk is disproportionate to the benefit. Legacy systems are load-bearing, often poorly tested, and frequently maintained by nobody in particular. Changing one to add a feature means accepting the chance of breaking something the business depends on, in exchange for a capability that could have sat alongside it.
The pattern that works is a service beside the legacy system rather than inside it. It reads what it needs, does the model work, and writes results to a new table, a new field, or a queue the legacy system already consumes. The old code is untouched, which means the rollback is turning the new service off.
There is a second reason, and it is commercial. A change inside a legacy system needs whoever owns that system to approve, schedule and regression-test it. A service alongside it needs read access. Those are very different conversations, and the second one happens in weeks rather than quarters.
What about systems where the data cannot leave?
Common with legacy estates, because they tend to sit in regulated environments or on networks that were never designed to reach the internet. It is a solvable problem but it has to be settled first, since it rules out hosted model APIs entirely.
The options are running open-weight models on hardware inside the network, or redacting and tokenising the sensitive fields before anything leaves. Which one applies depends on whether the restriction is about the data itself or about the network path, and those are often confused in the initial conversation.
Ask the question precisely: is the rule that this data cannot be processed by a third party, or that this network cannot make outbound connections? The first constrains the model. The second constrains the architecture. They have different answers.
What does a realistic first project look like?
Read-only, batch, and beside the system. Something like classifying a nightly export of records and writing the classifications to a new table, where nothing the legacy system does changes and nothing it relies on is touched.
That is deliberately unambitious, and it is the point. It proves the data is reachable and usable, it produces something measurable, and it establishes the integration surface that every later project reuses. Most of the difficulty in legacy work is the first extraction; once that path exists, the second project is straightforward.
Resist starting with a write-back into a field the business acts on. Reads are reversible and invisible when they go wrong. Writes into a system nobody fully understands are neither.
What usually goes wrong?
Nobody can say what the data means. Field names abbreviated twenty years ago, status codes whose meaning lives in one person's head, records that are technically duplicates for reasons that made sense once. This is the real cost of legacy AI work and it is almost never in the estimate.
The second is assuming data quality that is not there. Models are sensitive to inconsistency in a way that reports are not, because a report is read by a person who silently corrects for it. Expect a cleaning pass, and budget it separately rather than discovering it mid-build.
The third is access taking longer than the build. Getting read credentials to a production database in a regulated environment can take longer than everything else combined. Start that request on day one, before any technical work, because it is the dependency with the longest lead time and the least ability to be accelerated.
Common questions
- Can AI be integrated with a system that has no API?
- Yes. An API is convenient but not required. A readable database, a scheduled export, a file drop or an existing message queue all work as integration surfaces. The practical requirement is that the system's state can be observed and a result can be recorded somewhere, not that it exposes an endpoint.
- Do we have to modify our legacy system?
- Usually not, and usually you should not. The pattern that works is a service alongside the legacy system: it reads what it needs, does the model work, and writes results to a new table or queue. The old code stays untouched, so rolling back means switching the new service off rather than reverting a change to load-bearing software.
- What if our data cannot leave our network?
- Then open-weight models run on hardware inside the network, rather than a hosted API. Worth establishing precisely what the restriction is first: whether the data cannot be processed by a third party, or the network cannot make outbound connections. Those sound alike and have different architectural answers.
- What is the biggest hidden cost in legacy AI work?
- Understanding the data. Abbreviated field names, status codes whose meaning is undocumented, and duplicate records that exist for historical reasons all have to be decoded before a model can use them, and that work is rarely in the original estimate. Access approvals are a close second, since production database credentials in a regulated environment can take longer to obtain than the build takes to write.
