Running It in Production · Field Guide
Shipping isn't the finish line.
It's the day you start operating.
The discipline the delivery guide hands off to: keeping shipped software alive, reliable, and affordable in production. Hosting and releases, monitoring and updates, backups and incidents — the unglamorous operational work that decides whether customers trust your software enough to keep paying for it.
Introduction & how to use this guide
Purpose & audience
The moment software goes live, its nature changes. It stops being a thing you're building and becomes a thing you're responsible for — every hour, whether you're watching or not.
The Delivery Guide takes software up to release. This guide takes it from there: the ongoing work of operating software in production — keeping it running, reliable, current, and affordable. It covers where and how to host, how to release changes without breaking things, how to know your system is healthy before your customers tell you it isn't, how to update live systems safely, how to protect and manage data, how to handle incidents calmly, and how to keep the operational bill from quietly eating your margin. It is the operational half of running a software business, distinct from the security posture of Security & Compliance and the commercial support of From Project to Product.
It is written for the developer or small firm running software that other people depend on — whether a product you sell or a system you built and maintain for a client. Everything here is generic and reusable: the operational principles apply across platforms, from a single server to a cloud deployment, whatever the specific tools.
The core principle
The delivery guide argues that quality is built in, not tested in. Operations has a harder cousin: reliability is not a state you reach but a practice you keep — and in production, the boring, disciplined, invisible work is the work that matters most.
Good operations is almost entirely invisible. When it works, nothing happens — the software just runs, quietly, and no one thinks about it. This is precisely why it's neglected: there's no applause for the outage that didn't occur, the data loss that was prevented, the incident that resolved in minutes because you were ready. But that invisible reliability is exactly what customers are paying for and trusting you with, and its absence is catastrophic in a way its presence is silent. Operating well means valuing the unglamorous, preventive, disciplined work whose whole reward is that nothing goes wrong — and treating that as an achievement, not an afterthought.
Two failure modes recur. The first is neglect — shipping and then ignoring the running system until it breaks, at which point you're firefighting under pressure with no preparation. The second is over-engineering — a solo developer building the elaborate operational apparatus of a huge company for a system with ten users, drowning in complexity no one needed. The right path, as everywhere in this library, is proportion: solid, boring fundamentals sized to what you actually run.
Where this sits in the library
This guide picks up where the Custom Software Delivery Guide's deployment section ends. It's the operational partner to two others: Security & Compliance handles the security side of running systems (this handles the reliability side), and From Project to Product handles the commercial side of ongoing support (this handles the technical side that support rests on). Together they cover the full reality of software that's live and being paid for.
You do not need a large operations team or an elaborate toolchain. You need a small set of disciplined practices — reliable releases, real monitoring, tested backups, a calm incident habit — that keep the system trustworthy without consuming your life. This guide is about the proportionate version of each.
Hosting & infrastructure
Where your software runs shapes how much of your life it consumes. The central trade-off — how much you hand off versus how much you control — is worth deciding deliberately.
Managed vs. self-run
The most consequential infrastructure decision is how much of the running you delegate. Managed services hand the operational burden — keeping servers patched, databases backed up, infrastructure healthy — to a provider, in exchange for a higher bill and less control. Self-run infrastructure gives you full control and often lower direct cost, at the price of doing all that operational work yourself. For a small business, the trade is usually not really about money — it's about your time and your risk. Managed services cost more in dollars but far less in the hours and expertise required to run infrastructure well, and they shift a large class of operational risk onto someone whose whole job is handling it. For most small vendors, paying a provider to handle the undifferentiated heavy lifting is money well spent; your scarce time is better on your product than on server administration.
Right-sizing
The other common trap is running infrastructure sized for a scale you don't have. It is easy, and expensive, to provision for imagined future load — paying every month for capacity that sits idle against traffic that hasn't arrived. Right-sizing means matching your infrastructure to your actual current needs, with enough headroom to be safe but not so much that you're funding a hypothetical. Start modest, watch the actual load, and scale up when reality demands it — which, with modern infrastructure, is usually quick to do. Paying for the scale you have, not the scale you dream of, keeps the operational bill honest and is the infrastructure version of the lean discipline that runs through the whole library.
Deployment & release management
How you get changes into production determines how often you break things. A repeatable, low-drama release process is one of the highest-return investments in operational sanity you can make.
Repeatable releases
The way changes reach production should be repeatable and boring — the same reliable steps every time, ideally automated, so that releasing is a routine non-event rather than a nerve-wracking manual ritual. When deploying is ad-hoc and manual, every release is a chance to forget a step, and releases become rare, large, and dangerous precisely because they're scary. When deploying is automated and consistent, releases become small, frequent, and safe — and small frequent changes are far easier to get right and to fix than large infrequent ones. You don't need elaborate tooling to start; you need a defined, consistent process that removes improvisation from the moment your changes touch live systems. The goal is that shipping a change is dull, and dull is exactly what you want it to be.
Environments & rollback
Two practices make releases dramatically safer. The first is separate environments — somewhere that is not production where you can test changes against realistic conditions before they reach real users. Testing in production, on your live customers, is a recipe for public failure; a place to catch problems first is basic operational hygiene. The second is a reliable rollback — a fast, known way to undo a release that's gone wrong and return to the last good state. Things will occasionally break despite your best testing; what separates a minor blip from a prolonged outage is whether you can quickly revert. Knowing you can undo a bad release — and having practised it — turns deployment from a high-stakes gamble into a reversible, low-fear routine. One caution keeps this honest: not every change is cleanly reversible. Once a database migration has altered or dropped data, or a message has already gone out to customers, “return to the last good state” may no longer be available. Where that is a risk, plan the reverse path deliberately — keep migrations backward-compatible (expand first, contract later) so old and new code both run, rely on point-in-time database backups, and put risky changes behind feature flags you can switch off without a full redeploy. Sometimes the fastest safe recovery is not rolling back at all, but rolling forward with a small, targeted fix.
Reliability & monitoring
Reliability is the product feature customers never mention until it's missing. You cannot deliver it without knowing, at any moment, whether your system is actually healthy.
What uptime really costs
Uptime — the software being available and working when customers need it — is quietly one of the most important things you sell, because unreliable software erodes trust faster than almost any other failure. But perfect uptime is expensive and, past a point, subject to diminishing returns; the effort to go from very reliable to essentially never-down is large and rarely worth it for a small vendor. The practical question is not “how do I achieve perfection?” but “what level of reliability do my customers actually need, and what does the gap between that and where I am cost me?” Match your reliability effort to what your customers genuinely require and what an outage would actually cost them — the same proportionate, risk-sized thinking the library applies everywhere. Aim for reliable enough that trust holds, not for a perfection that bankrupts you.
Knowing before customers do
The single most important operational capability is to know your system has a problem before your customers tell you. Learning about an outage from an angry customer is both a failure of operations and a blow to trust; learning about it from your own monitoring, and often fixing it before anyone notices, is the difference between a professional operation and a hopeful one.
Monitoring — automated watching of whether your system is up and working, with alerts when it isn't — is not optional infrastructure but the foundation of reliable operations, because you cannot respond to what you can't see. For a small vendor this need not be elaborate: even basic checks that tell you promptly when the system is down, or behaving abnormally, transform your position from blind to informed. The goal is simple and worth a great deal: never to be the last to know that your software is broken. A modest monitoring setup that reliably alerts you is one of the highest-value operational investments you can make.
Updates in production
Live software is never finished. The world beneath it — dependencies, platforms, security threats — keeps moving, and keeping up with it safely is an ongoing operational discipline.
A patching cadence
Software you've shipped sits on a foundation of other software — libraries, frameworks, operating systems, platforms — all of which keep changing, and some of whose changes fix security holes you're exposed to until you apply them. Keeping current is not a one-off but a cadence: a regular, scheduled habit of applying updates, especially security ones, before the gap between a fix existing and you applying it becomes the window an attacker uses. Falling behind is easy and comfortable — nothing breaks immediately — right up until an old, unpatched flaw is exploited or an update deferred so long it becomes a massive, risky leap. A steady, unglamorous patching rhythm keeps the system current at low cost and low risk; neglect converts that into rare, dangerous, expensive catch-ups. This is the technical spine of the maintenance that From Project to Product teaches you to sell.
Changing live systems safely
Every change to a running system that real people depend on carries the risk of breaking something they rely on, so changes to production deserve more care than changes to something no one is using yet. The practices from earlier sections converge here: test changes somewhere safe first, release them through a repeatable process, watch your monitoring closely after each change, and keep the ability to roll back if something goes wrong. The mindset that matters is respect for the live system — treating a change to production not as a casual edit but as an operation with real stakes for real people, warranting a moment's caution and a plan for if it misbehaves. Most production disasters are not exotic; they're ordinary changes made carelessly to systems that people were depending on.
Data operations
Of everything you operate, the data is the part you cannot afford to lose. Systems can be rebuilt; lost customer data often cannot, and its loss can end a business.
Backups you've tested
The most important data practice is also the most neglected: reliable, tested backups. Keeping regular backups of your data, stored safely and separately from the live system, is the fundamental protection against the many ways data can be lost — hardware failure, human error, a bad deployment, an attack. But a backup you have never restored from is not a safety net; it is a hope, and hopes have a way of failing exactly when relied upon.
The single most dangerous assumption in operations is that your backups work when you've never checked. Backups fail silently — misconfigured, incomplete, corrupted — and the moment you discover this is almost always the moment you desperately need them, after data is already lost. The discipline that separates safe operations from a disaster waiting to happen is periodically restoring from your backups to confirm they actually work. A restore you've practised is genuine protection; a backup you've merely configured and never tested is a comforting illusion. Test the restore, on a schedule, before you ever need it for real.
Migrations & integrity
As software evolves, its data usually must change shape too — a migration, altering the structure or content of live data to match new code. Migrations are among the most dangerous operations there are, because they touch the irreplaceable data directly and a mistake can corrupt or destroy it. They deserve the greatest care: a tested backup taken immediately before, a rehearsal on non-production data first, and a plan for what to do if the migration goes wrong. Beyond migrations, ongoing data integrity — the data staying accurate and consistent as the system runs — is worth watching for, since silent corruption can accumulate unnoticed until it causes visible harm. The data is the crown jewels of most software businesses; operate on it with corresponding respect.
Incident operations
Things will go wrong — that's not a failure of operations, it's the reason operations exists. What separates a professional operation from a panicked one is not that incidents never happen, but how calmly they're handled when they do.
On-call for small teams
Software that people depend on may break at any hour, which raises an uncomfortable question for a small vendor: who responds, and when? Large companies run rotations of on-call staff; a solo developer or tiny team cannot, and pretending otherwise leads to burnout or broken promises. The honest approach is to be clear and realistic about what response you actually offer — the hours you cover, how fast you'll respond, what counts as an emergency — and to make sure that's what your contracts and SLAs promise (the From Project to Product guide covers structuring those commitments). Over-promising round-the-clock response you can't sustainably provide sets up both failure and exhaustion. Set expectations you can actually meet, build in real rest, and price higher availability as the premium it genuinely is rather than quietly absorbing it.
The calm response
When something does break, the quality of your response depends less on brilliance in the moment than on preparation before it. A prepared operator handles an incident calmly because they've thought it through in advance.
- Stabilize first. Get the system working again — even a temporary fix or rollback — before hunting for the root cause. Stop the bleeding, then diagnose.
- Communicate. Tell affected customers what's happening. A calm, honest update during an outage preserves trust; silence destroys it.
- Diagnose once stable. With service restored and the pressure off, find what actually went wrong.
- Learn. Afterwards, a short, blameless look at what happened and the one or two changes that stop it recurring turns an incident into an improvement.
The goal is not to never have incidents — that's impossible — but to handle them so competently that they strengthen customer trust rather than eroding it. A well-handled outage can leave a customer more confident in you than before.
Operational cost
Running software costs money every month, forever, whether or not anyone is looking at the bill. Left unwatched, operational cost quietly erodes the margin the rest of the business works to build.
Watching the bill
Operational costs — hosting, services, tools, the whole apparatus of running your software — are recurring, easy to accumulate, and easy to stop noticing. A service added for a good reason and forgotten, a resource over-provisioned and never trimmed, a tool no longer used but still billing: individually small, collectively a steady leak from your margin. The habit worth building is to periodically review what you're actually paying to run the business and ask whether each cost still earns its place. This is the operational sibling of the financial discipline in The Money Behind the Business and the tool discipline in Setting Up Shop: costs left unwatched drift upward, and a light regular review keeps them honest. What you don't look at, you overpay for.
Cost that scales with revenue
The healthiest operational cost is one that grows in proportion to what you're earning, not ahead of it. A cost structure where serving more customers costs more only as those customers pay you more keeps the business's unit economics sound as it grows; a structure where costs balloon regardless of revenue — heavy fixed infrastructure sized for scale you don't have — can turn growth into loss. The aim is for your operational spending to track your actual usage and revenue, so that the business remains profitable at every size rather than gambling on future scale to justify present cost. This connects directly to the unit economics that The Money Behind the Business examines and the profitability the whole library works to protect: operations that cost more only as they earn more are operations that keep the business healthy while it grows.
Templates & checklists
The guide compressed into working instruments: what to have in place before you're truly live, a release you can run without fear, a calm incident response, and the vocabulary of operations.
The ops-readiness checklist
- Hosting decided — managed vs. self-run chosen deliberately, and right-sized to real needs.
- Repeatable deploys — a consistent, ideally automated release process; no improvised manual steps.
- A safe environment — somewhere that isn't production to test changes first.
- Rollback ready — a fast, tested way to undo a bad release.
- Monitoring live — automated checks that alert you before customers notice a problem.
- Tested backups — regular, separate, and actually restored at least once.
- A patching cadence — a regular habit of applying updates, especially security ones.
- Realistic support terms — response commitments you can actually sustain.
The release checklist
- Tested first — the change verified somewhere that isn't your live customers.
- Backup current — especially if the change touches data, a fresh backup exists.
- Released cleanly — through the repeatable process, not by hand.
- Watched after — monitoring checked closely once the change is live.
- Rollback at hand — you know exactly how to undo it if it misbehaves.
The incident runbook
- Stabilize — restore service first, even temporarily; stop the bleeding before diagnosing.
- Communicate — tell affected customers honestly what's happening.
- Diagnose — once stable and the pressure is off, find the real cause.
- Recover fully — restore from tested backups if data was affected; verify before resuming.
- Learn — a short, blameless review and the change that prevents a repeat.
Glossary
| Term | Meaning |
|---|---|
| Production | The live environment where real customers use the software. |
| Managed services | Infrastructure where a provider handles the operational burden, for a higher fee. |
| Right-sizing | Matching infrastructure to actual needs, not imagined future scale. |
| Deployment / release | The process of getting a change from development into production. |
| Environment | A place the software runs; keeping non-production ones lets you test safely. |
| Rollback | Undoing a release to return to the last known-good state. |
| Uptime | The software being available and working when customers need it. |
| Monitoring | Automated watching of system health, with alerts when something is wrong. |
| Patching cadence | A regular habit of applying updates, especially security fixes. |
| Migration | Changing the structure or content of live data to match new code. |
| Incident | An unplanned disruption to the service that needs a response. |
| Unit economics | Whether serving each additional customer makes or loses money. |
This is the operational engine room beneath a live software business. It picks up where the Custom Software Delivery Guide ships, runs alongside the security posture of Security & Compliance, provides the technical backbone for the support and maintenance that From Project to Product turns into revenue, and feeds its costs into The Money Behind the Business. Operate well — reliably, safely, affordably — and the software keeps the trust that everything else is built on.