Running MongoDB in production

A single mongod running on your laptop is wonderful for learning, and exactly the wrong thing to put in front of real users. The moment a database holds data people depend on, four questions arrive that a lone server cannot answer well. What happens when that one machine dies? What happens when one machine can no longer hold all the data or all the traffic? Why did a query that was instant last week now take two seconds? And who, exactly, is allowed to read this data, and over what kind of connection?

This module is about those four questions, and MongoDB has a distinct answer for each. High availability comes from a replica set — several copies of your data that elect a new leader automatically when one fails. Horizontal scale comes from sharding — splitting one logical collection across many machines so the dataset and the load divide. Performance comes from understanding where time goes: profiling slow queries, keeping the working set in memory, and reaching for the right index. Security comes from turning on the protections that are off by default: authentication, encryption in transit, and a network that does not expose the database to the open internet.

The shape of a production deployment

The two structural ideas — replication and sharding — stack on top of each other. A replica set gives one copy of the data resilience. A sharded cluster is several replica sets, each holding a slice of the data, with a router in front that knows which slice is where:

flowchart TB
  App["Application"] --> Router["mongos router"]
  Router --> CSRS["Config servers (replica set)"]
  Router --> ShardA
  Router --> ShardB
  subgraph ShardA["Shard A — replica set"]
    PA["Primary"] --> S1A["Secondary"]
    PA --> S2A["Secondary"]
  end
  subgraph ShardB["Shard B — replica set"]
    PB["Primary"] --> S1B["Secondary"]
    PB --> S2B["Secondary"]
  end

A sharded cluster is many replica sets behind a router; each shard is itself highly available

Read that diagram from the bottom up. Each shaded box is a replica set: one primary plus secondaries holding the same data, so a single failed node does not lose anything. Stack two of those side by side, give each a different slice of the collection, and put a mongos router in front to direct every request to the right slice — that is a sharded cluster. You do not have to start sharded; most deployments live happily as a single replica set for a long time and only shard once one machine genuinely cannot keep up.

What this module covers

The lessons follow the same order as the four questions:

Running MongoDB in production — this overview of availability, scale, performance, and security.
Replica sets — primaries, secondaries, the oplog that keeps them in sync, and the automatic election that picks a new primary when one fails.
Sharding — partitioning a collection by a shard key across shards, the role of the balancer and the config servers, and how to choose a shard key you will not regret.
Performance and profiling — finding slow queries with the database profiler, re-reading explain, and the common fixes that make queries fast again.
Security and Atlas — authentication and roles, TLS, locking down the network, encryption at rest, and how MongoDB Atlas hands you all of it as a managed service.

Tips and gotchas

These features compose rather than compete. You almost always run a replica set first for availability, and only add sharding later when scale forces it. Security applies to both from day one.
Sharding is not a performance band-aid for a slow query. A missing index hurts just as much across ten shards as on one. Fix performance first, scale second.
The single most common production mistake is the easiest to avoid: leaving a database reachable on the internet with authentication switched off. The security lesson exists so you never do that.

Trade-offs

Choice	Benefit	Cost
A single `mongod`	Simplest to set up, lowest cost	No high availability — one machine is a single point of failure
Replica set	Automatic failover, can spread reads across secondaries	Writes always still go to one primary; more infrastructure to maintain
Sharded cluster	Scales both writes and storage horizontally	Much higher operational complexity; a badly chosen shard key is hard to fix later
Self-managed	Full control over infrastructure	You own every patch, backup, and security task yourself
MongoDB Atlas	Less operational burden, secure by default	Less low-level control, usage-based cost

What the lessons measured

Every number here was captured on this Mac with mongo 8.3.11, mongosh 2.11.1 and Node driver 7.7.0, on throwaway Docker containers (each capped at 512 MB) that were removed afterwards.

Replica sets — a crashed primary stalled one write for about 10.8 s (the docs say the median election “should not typically exceed 12 seconds”); a cleanly stopped one for 0.3 s; with two of three members gone the survivor refused writes.
Sharding — a timestamp shard key put 100 000 of 100 000 documents on one shard, a hashed key split them 48 900 / 51 100; enableSharding was not needed.
Performance — a profile entry with COLLSCAN and 49 999 documents examined for 5 000 returned; a client per request multiplied new connections.
Security — the default container accepted an insert with no credentials; Command find requires authentication; a missing authSource failed with code 18.

Mental Model

Four independent questions, four independent mechanisms: copies and votes (availability), key ranges (scale), counters (performance), doors (security). None substitutes for another — a replica set does not scale writes, sharding does not fix a missing index, TLS does not replace authentication.

Common Misconceptions

“Shard from day one because the data will grow.” — a single replica set handles a lot; sharding adds four processes even in the smallest lab and a permanent key decision (changeable, but at least five minutes per resharding).
“Security is a one-time task.” — certificates expire (the lab CA was valid for 2 days on purpose), users and roles drift.
“HA that has never failed over is HA.” — rehearse it: a crash and a clean stop paused writes for 10.8 s versus 0.3 s in the lab.