Containers or serverless: which fits your workload?

What actually decides it?

Four traits, in this order: how variable the traffic is, how long one unit of work takes, whether the workload holds connections open, and how tolerant the caller is of an occasional slow first response. Everything else, including language, team preference and vendor relationship, is secondary to those.

The reason traffic variability leads is that it determines which pricing model is even in your favour. Serverless charges for execution and nothing for idle. A container platform charges for capacity whether or not it is doing anything. A workload that is busy for two hours a day is paying twenty-two hours of nothing on a container, and a workload busy all day is paying a premium per request on a function.

What is the cold start problem really?

The delay between a request arriving and your code being able to answer it, when no warm instance exists. The platform has to allocate a sandbox, load your package, start the runtime and run whatever your code does before the handler, which typically includes reading configuration, opening a database connection and constructing dependency graphs.

The size of the penalty depends more on your code than on the platform. A small handler in a runtime that starts quickly is often unnoticeable. A large application on a runtime with a heavy startup path, loading a framework and a full object relational mapper before serving anything, is a different experience entirely, and the fix is usually to move work out of the initialisation path rather than to change platform.

The workaround worth knowing about is provisioned or pre-warmed concurrency, where the provider keeps instances hot. It removes the latency and removes the cost argument at the same time, because you are now paying for idle capacity, which is what a container was. If your design requires it permanently, that is a signal to reconsider the choice rather than to buy your way out.

Where does serverless start costing more than a server?

At the point where sustained request volume multiplied by per-request cost exceeds the price of capacity you would otherwise keep running. That crossover is computable from your own numbers and worth computing before the architecture is fixed rather than after the first surprising bill.

The calculation people get wrong is using average traffic. What matters is the ratio between peak and mean. A service whose peak is many times its average has long idle stretches that a container pays for and a function does not, so it can be well past the raw request crossover and still be cheaper on functions. A service with a flat profile has no such advantage and crosses early.

Two costs are routinely left out of the model. Per-invocation charges apply to every call including health checks, retries and the fan-out from one user action into several internal calls, so a chatty design multiplies the bill in a way a monolithic handler does not. And functions that sit idle waiting on a slow downstream call are billed for the waiting, which makes an unoptimised third-party integration expensive in a way it never was on a server.

Why do database connections cause so much trouble with functions?

Because a connection pool assumes a long-lived process and a function is not one. Each concurrent invocation is effectively its own process, so a workload that scales to hundreds of concurrent executions tries to open hundreds of database connections, and most managed databases have a connection ceiling far lower than the concurrency ceiling of the function platform.

The result is a failure mode that only appears under load: the application works perfectly in testing and then, during the exact traffic spike serverless was chosen to handle, every new invocation fails to connect while the database sits at low CPU. Autoscaling makes it worse rather than better, which is what makes it confusing the first time.

The standard fix is a connection proxy or pooler between the functions and the database, which multiplexes many short-lived clients onto a small number of real connections. It works, and it is another component to run and monitor. Alternatively, use a data service designed for per-request connections. Either way the decision belongs at design time, because retrofitting it is a change to every data access path.

Can you sensibly run both?

Yes, and most mature systems do. The split that holds up is to run the request path and anything holding connections on containers, and put genuinely event-shaped work on functions: reacting to a file upload, processing a webhook, running a scheduled report, handling an occasional integration that fires a few hundred times a day.

The reason this works is that those workloads have the profile serverless is priced for, and they are also the workloads that are annoying to host on a container platform because they require capacity to sit waiting for something that may not happen for hours.

The cost of mixing is two deployment models, two sets of local development ergonomics and two places to look during an incident. That is a real overhead and it is worth accepting for a clear split along workload lines. It is not worth accepting when the split runs through the middle of one request path, because then every debugging session crosses the boundary.

What should you check before committing?

Pull a week of request logs and plot requests per minute. If the line is flat, functions have no idle advantage to offer and you are choosing them for operational reasons rather than cost ones, which is a legitimate but different argument. If the line is mostly floor with occasional peaks, functions are probably cheaper and the peaks are the thing to size for.

Then measure the longest unit of work you have, including the slowest third-party call in the path. If anything approaches the platform execution limit, that path needs restructuring into steps before it can go on functions at all, and the restructuring is the actual project.

Finally, count the maximum concurrent executions you would expect at peak and compare that to the connection limit of your database. If the first number is larger than the second, you have a pooling problem to solve before the first deploy rather than during the first incident.