Handling Duplicate Key Errors and Unique Constraints in MongoDB
Every MongoDB developer eventually meets this line in their logs:
MongoServerError: E11000 duplicate key error collection: shop.users index: email_1 dup key: { email: "alice@example.com" }
The first reaction is often to treat it as a crash to be prevented: check whether the email exists before inserting, and hope the error goes away. That instinct is backwards. A duplicate key error is MongoDB doing its job, enforcing a unique index that guarantees no two documents share a value. The right move is to let the database enforce the rule and teach your application to handle the error gracefully.
This guide covers how unique indexes work (including the surprising behavior around missing fields and arrays), how to create one on a collection that already has duplicates, how to catch and interpret duplicate key errors in Node.js, Mongoose, and Python, why "check then insert" is a race condition, how duplicates interact with upserts and bulk inserts, and the extra rules for sharded clusters.
Unique Indexes: The Only Real Guarantee
MongoDB has no UNIQUE column constraint in a schema. Uniqueness is a property of an index:
db.users.createIndex({ email: 1 }, { unique: true });
Once this index exists, any insert or update that would produce a second document with the same email fails with error code 11000. The check happens atomically inside the storage engine, so it holds under any amount of concurrency. No application-level check can make that promise.
Every collection already has one unique index: _id. That's why inserting a document with an existing _id produces the same error.
Compound Unique Indexes
A unique index on multiple fields enforces uniqueness of the combination:
db.memberships.createIndex({ orgId: 1, userId: 1 }, { unique: true });
A user can belong to many organizations, and an organization can have many users, but the same user can't be added to the same organization twice. This pattern is everywhere: one vote per user per poll, one review per customer per product, one slug per tenant.
The field order doesn't affect what's unique, but it does affect which queries the index can serve, so put the field you filter on most often first.
Surprising Behavior You Need to Know
Missing Fields Count as null
A unique index treats a missing field as null. That means only one document in the collection can omit the field:
db.users.createIndex({ phone: 1 }, { unique: true });
db.users.insertOne({ email: "alice@example.com" }); // phone missing: indexed as null
db.users.insertOne({ email: "bob@example.com" }); // phone missing again
MongoServerError: E11000 duplicate key error collection: app.users index: phone_1 dup key: { phone: null }
This bites hard for optional fields. The fix is a partial unique index that only applies to documents where the field actually has a value:
db.users.createIndex(
{ phone: 1 },
{ unique: true, partialFilterExpression: { phone: { $type: "string" } } },
);
Now any number of users can skip the phone number, but two users can't share the same one. You'll see older advice to use sparse: true instead. Sparse indexes skip documents where the field is missing but still index explicit null values, so a document with phone: null can still collide. Partial indexes give you precise control and are the better choice for new indexes.
The same technique powers unique constraints that ignore soft-deleted documents, as covered in implementing soft deletes.
Arrays Enforce Uniqueness Across Documents
A unique index on an array field (a multikey index) prevents the same element from appearing in two different documents:
db.accounts.createIndex({ aliases: 1 }, { unique: true });
db.accounts.insertOne({ name: "Acme", aliases: ["acme", "acme-inc"] });
db.accounts.insertOne({ name: "Other", aliases: ["acme"] }); // E11000: "acme" already used
It does not prevent duplicates within a single document's array. aliases: ["acme", "acme"] inserts fine. To keep an array's elements unique within a document, use $addToSet instead of $push when updating it.
Case Sensitivity
By default, Alice@Example.com and alice@example.com are different keys. If uniqueness should ignore case, create the index with a collation:
db.users.createIndex(
{ email: 1 },
{ unique: true, collation: { locale: "en", strength: 2 } },
);
The post on case-insensitive queries and collation explains strengths and how to make sure queries use the collated index.
Adding a Unique Index to Existing Data
If the collection already contains duplicates, createIndex fails with a duplicate key error and the index isn't created. Find the offenders first:
db.users.aggregate(
[
{
$group: {
_id: { $toLower: "$email" },
count: { $sum: 1 },
ids: { $push: "$_id" },
},
},
{ $match: { count: { $gt: 1 } } },
{ $sort: { count: -1 } },
],
{ allowDiskUse: true },
);
[
{
_id: "alice@example.com",
count: 2,
ids: [ObjectId("66e0a1..."), ObjectId("66e3f7...")],
},
];
Group by the same normalization your new index will use: $toLower if the index will be case-insensitive, the raw field otherwise. Then decide how to resolve each group. For users, that usually means merging accounts or contacting the owners, not blindly deleting the newer one. Once the collection is clean, create the index.
On a busy production collection, there's a window between cleaning up and the index build finishing when new duplicates can sneak in. Deploy application code that prevents new duplicates (or handles them) first, clean up, then build the index, and re-run the duplicate check if the build fails.
Handling Duplicate Key Errors in Code
The goal is to recognize error 11000, figure out which constraint was violated, and turn it into a meaningful response, usually HTTP 409 Conflict with a helpful message.
Node.js Driver
Duplicate key errors are thrown as MongoServerError with code === 11000. The error also carries keyPattern and keyValue, which tell you which index was violated:
import { MongoServerError } from "mongodb";
export async function createUser(db, { email, username }) {
try {
const { insertedId } = await db.collection("users").insertOne({
email,
username,
createdAt: new Date(),
});
return { ok: true, id: insertedId };
} catch (err) {
if (err instanceof MongoServerError && err.code === 11000) {
const field = Object.keys(err.keyPattern ?? {})[0] ?? "value";
return { ok: false, conflict: field };
}
throw err;
}
}
In an Express route:
app.post("/users", async (req, res, next) => {
try {
const result = await createUser(req.app.locals.db, {
email: String(req.body.email),
username: String(req.body.username),
});
if (!result.ok) {
return res
.status(409)
.json({ error: `That ${result.conflict} is already taken.` });
}
res.status(201).json({ id: result.id });
} catch (err) {
next(err);
}
});
Re-throw every error that isn't a duplicate key. Swallowing all errors as "already exists" hides real outages.
Mongoose
A common misconception is that unique: true in a Mongoose schema is a validator. It isn't. It's an index definition, and violations surface as the same MongoServerError with code 11000, not as a ValidationError:
const userSchema = new mongoose.Schema({
email: { type: String, required: true, unique: true },
});
try {
await User.create({ email: "alice@example.com" });
} catch (err) {
if (err.code === 11000) {
// err.keyValue => { email: "alice@example.com" }
throw new ConflictError("Email already registered");
}
throw err;
}
Two practical notes. Mongoose's autoIndex builds indexes when the model is first used, which is convenient in development but can be slow or fail silently on large production collections; many teams disable it in production and manage indexes explicitly. And plugins that "validate" uniqueness by querying first reintroduce the race condition described below, so treat them as a friendly early warning at best.
PyMongo
PyMongo raises a dedicated exception class, DuplicateKeyError, a subclass of WriteError:
from datetime import datetime, timezone
from pymongo.errors import DuplicateKeyError
def create_user(db, email, username):
try:
result = db.users.insert_one({
"email": email,
"username": username,
"createdAt": datetime.now(timezone.utc),
})
return {"ok": True, "id": result.inserted_id}
except DuplicateKeyError as e:
field = next(iter(e.details.get("keyPattern", {})), "value")
return {"ok": False, "conflict": field}
e.details contains the server's error document, including keyPattern, keyValue, and errmsg.
Why "Check Then Insert" Is a Race Condition
This pattern appears in countless codebases:
// Don't do this as your only protection
const existing = await users.findOne({ email });
if (existing) throw new ConflictError("Email taken");
await users.insertOne({ email });
Two requests for the same email arrive a few milliseconds apart. Both run findOne, both see nothing, and both insert. Without a unique index, you now have duplicates. With a unique index, one of the inserts fails with E11000, and if your code doesn't handle it, the user gets a 500 error.
The correct approach is insert and handle the error. The database resolves the race atomically, and your error handler produces the same friendly message the pre-check would have. A pre-check is still fine for UX, like showing "username taken" as someone types, but it must never be the thing that actually enforces uniqueness.
Duplicates and Upserts
Upserts (updateOne with upsert: true) seem like they should be immune to duplicates, since they either update the existing document or insert a new one. Under concurrency, they aren't quite:
await db
.collection("counters")
.updateOne(
{ name: "page-views", day: "2026-09-18" },
{ $inc: { count: 1 } },
{ upsert: true },
);
If two of these run at the same moment and no document exists yet, both can decide to insert. With a unique index on { name: 1, day: 1 }, one of them fails with E11000. Without the unique index, you get two counter documents, which is worse.
In recent versions, the server automatically retries an upsert that hits a duplicate key error in the common case where the filter is an equality match on the unique index's fields, so you'll rarely see this in practice. It's still worth having a simple retry in application code for upserts on hot keys:
async function upsertWithRetry(collection, filter, update, attempts = 2) {
for (let i = 0; i < attempts; i++) {
try {
return await collection.updateOne(filter, update, { upsert: true });
} catch (err) {
if (err.code !== 11000 || i === attempts - 1) throw err;
}
}
}
On retry, the document exists, so the upsert takes the update path and succeeds. More on this in upserts in MongoDB.
Duplicates in Bulk Inserts
When importing data, you often want to insert everything new and skip what already exists. An unordered insertMany does exactly that: it attempts every document, and duplicates fail individually without stopping the batch.
import { MongoBulkWriteError } from "mongodb";
try {
await db.collection("products").insertMany(docs, { ordered: false });
} catch (err) {
if (!(err instanceof MongoBulkWriteError)) throw err;
const writeErrors = Array.isArray(err.writeErrors)
? err.writeErrors
: [err.writeErrors];
const nonDuplicate = writeErrors.filter((e) => e.code !== 11000);
if (nonDuplicate.length > 0) throw err;
console.log(
`Inserted ${err.result.insertedCount}, skipped ${writeErrors.length} duplicates`,
);
}
With the default ordered: true, the batch stops at the first duplicate, and everything after it is never attempted. That's almost never what you want for imports.
In PyMongo, the equivalent is insert_many(docs, ordered=False) and catching BulkWriteError, whose details["writeErrors"] list contains each failure with its code and index.
If you want duplicates to update the existing documents rather than be skipped, use bulkWrite with updateOne and upsert: true for each record instead of inserts. The post on bulk write operations covers both patterns in depth.
Using Uniqueness for Idempotency
Unique indexes are also a clean way to make operations idempotent, which is essential for payments, webhooks, and anything a client might retry. Have the client send an idempotency key, and store it under a unique index:
await db
.collection("payments")
.createIndex({ idempotencyKey: 1 }, { unique: true });
export async function recordPayment(
db,
{ idempotencyKey, amount, customerId },
) {
try {
const doc = {
idempotencyKey,
amount,
customerId,
status: "pending",
createdAt: new Date(),
};
await db.collection("payments").insertOne(doc);
return { created: true, payment: doc };
} catch (err) {
if (err.code !== 11000) throw err;
const existing = await db
.collection("payments")
.findOne({ idempotencyKey });
return { created: false, payment: existing };
}
}
A retried request with the same key returns the original payment instead of charging twice. The same trick works for webhook deduplication: store each provider's event ID under a unique index and ignore repeats.
Unique Indexes in Sharded Clusters
Sharding adds one hard rule: a unique index on a sharded collection must include the shard key as a prefix. Each shard enforces uniqueness only among its own documents, so MongoDB can only guarantee global uniqueness when the unique fields determine which shard a document lives on.
If you shard users on { tenantId: 1 }, you can have a unique index on { tenantId: 1, email: 1 }, meaning "unique email per tenant," but not a globally unique index on { email: 1 }. For a globally unique value on a collection sharded by something else, a common pattern is a small, separate collection keyed by that value (for example, emails with _id set to the email) that you insert into first, often within a transaction.
The _id field is a special case: its uniqueness is only guaranteed across the whole cluster if _id is the shard key, or if your application generates values that can't collide (which default ObjectIds effectively do).
Common Pitfalls
Relying on application checks. Only a unique index guarantees uniqueness. Pre-checks are for UX; the index is for correctness.
Unique indexes on optional fields without a partial filter. Every document missing the field is indexed as null, and the second one fails. Use partialFilterExpression with $type or $exists: true.
Catching every error as a duplicate. Check code === 11000 (or DuplicateKeyError in Python) and re-throw everything else.
Ordered bulk imports. One duplicate stops the whole batch. Use ordered: false and inspect the individual write errors.
Ignoring which index failed. A users collection might have unique indexes on email, username, and phone. Use keyPattern to tell the user which one is the problem.
Forgetting case. Without a collation, uniqueness is case-sensitive, and "Alice" and "alice" coexist.
Conclusion
Duplicate key errors are a feature. A unique index is the only way to guarantee uniqueness under concurrency, and error code 11000 is the signal that it worked. Design your indexes carefully (partial filters for optional fields, collations for case, compound keys for scoped uniqueness), then write code that inserts first and translates E11000 into a clear 409 response, a skipped import row, or an idempotent replay.
Search your codebase for findOne calls that are immediately followed by an insert on the same field. Each one is a race condition. Make sure a unique index backs it, then add a code === 11000 handler to the insert so the pre-check becomes optional.
