Setting the file. One moment.
Chapter 06 · MongoDB Schema Design
Subchapter 6.5
references/fundamental-document-size.mdMarkdown10 KBView on GitHub
MongoDB documents cannot exceed 16 megabytes. This is a hard BSON limit, not a guideline — writes fail once a document reaches it.
However, practical documents should be much smaller than 16MB. As a rule of thumb, aim for documents . Smaller documents mean:
The 16MB ceiling is a safety net, not a design target.
activityLog array receiving entries on every user action: 100,000 events × ~150 bytes ≈ 15MB, growing until writes are rejected.$push rewrites the growing document, and a multikey index fans out to one entry per element.db.products.find({}, {name: 1, price: 1})) still reads the full document from storage.BinData PDF attachment of 10MB+; additional attachments push the document past the limit.Keep the parent document small. Store children in their own collection with a reference field and a compound index for efficient queries.
// Parent stays lean
{ _id: "user123", name: "Alice", activityCount: 48210, lastActivity: ISODate("...") }
// Children in separate collection with efficient index
// Index: { userId: 1, ts: -1 }
{ userId: "user123", action: "login", ts: ISODate("...") }For large binary blobs, use GridFS for in-database storage, or — often more efficient — store them in external object storage and keep only a reference in MongoDB.
Keep frequently-accessed (hot) data in the main document; store rarely-accessed (cold) data in a separate collection. This dramatically improves cache density for hot-path queries.
Incorrect (all data in one document): A movie document with all 10,000 reviews embedded (~1MB of cold data alongside ~1KB of hot data like title, rating, plot) means every page load pulls ~1MB into RAM. Most page views only need title + rating + plot, so this reduces how many movies fit in cache (e.g. 1GB RAM ≈ 1,000 movies instead of ~1,000,000 if only hot data were loaded).
Correct (subset pattern): The movie document (~2KB) contains only hot fields: title, year, rating, plot, reviewStats (count, avgRating, distribution), and a bounded featuredReviews array (top 5 only, ~500 bytes). Full reviews live in a separate reviews collection with movieId reference, loaded only when the user clicks “Show all reviews.”
Similarly, a product document should keep only hot fields in the main document (~500 bytes): name, price, thumbnail URL, avgRating, reviewCount, inStock. Move cold data to separate collections — products_details (description, fullSpecs), products_images (images array), products_reviews (paginated reviews).
How to identify hot vs cold data:
| Hot Data (embed) | Cold Data (separate) |
|---|---|
| Displayed on every page load | Only on user action (click, scroll) |
| Used for filtering/sorting | Historical/archival |
| Small relative size | Large relative size |
| Bounded small subsets | Large or unbounded sets |
| Changes rarely | Changes frequently |
Maintaining an embedded subset:
// When a new review is added:
// 1. Insert full review into reviews collection
db.reviews.insertOne({ movieId: "movie123", user: "newUser", rating: 5, text: "Amazing!", date: new Date(), helpful: 0 })
// 2. Update movie stats
db.movies.updateOne(
{ _id: "movie123" },
{ $inc: { "reviewStats.count": 1, "reviewStats.distribution.5": 1 } }
)
// 3. Periodically refresh featured reviews (background job)
const topReviews = db.reviews.find({ movieId: "movie123" }).sort({ helpful: -1 }).limit(5).toArray()
db.movies.updateOne({ _id: "movie123" }, { $set: { featuredReviews: topReviews } })For arrays, atomic $slice keeps the embedded subset bounded without a background job:
db.posts.updateOne(
{ _id: "post123" },
{
$push: {
recentComments: {
$each: [newComment],
$slice: -20,
$sort: { ts: -1 }
}
},
$inc: { commentCount: 1 }
}
)
// Also insert into overflow comments collection
db.comments.insertOne({ postId: "post123", ...newComment })// Only transfers ~500 bytes instead of 665KB over the network
db.products.find(
{ category: "electronics" },
{ name: 1, price: 1, thumbnail: 1 }
)Projection reduces network transfer but still loads full documents into memory unless the query is fully covered by an index. For real working-set reduction, split hot and cold data into separate collections.
// 1. Schema validation with array limits
db.createCollection("users", {
validator: {
$jsonSchema: {
properties: {
addresses: { maxItems: 10 },
tags: { maxItems: 100 }
}
}
}
})
// (See fundamental-schema-validation.md for full validation guidance).
// 2. Application-level checks before write
const doc = await db.users.findOne({ _id: userId })
const currentSize = BSON.calculateObjectSize(doc)
if (currentSize > 200 * 1024) { // 200KB warning — well before trouble
logger.warn("Document size exceeding recommended threshold")
}
// 3. Use $slice to cap arrays
db.users.updateOne(
{ _id: userId },
{
$push: {
activityLog: {
$each: [newActivity],
$slice: -1000 // Keep only last 1000
}
}
}
)| Signal | Action |
|---|---|
| Array cardinality keeps growing | Cap with $slice or move to separate collection |
| Array field is heavily indexed | Review multikey fan-out; move cold data out |
| Reads only need recent subset | Embed recent N, reference full history |
| Updates slow as array grows | Switch to referenced write path |
| Documents routinely exceed ~200KB | Reassess schema — consider splitting hot/cold |
| WiredTiger cache pressure is high | Check for bloated documents; split candidates |
tags: ["a", "b", "c"] is much cheaper than arrays of objects.// Find largest documents in collection
db.collection.aggregate([
{ $project: { size: { $bsonSize: "$$ROOT" } } },
{ $sort: { size: -1 } },
{ $limit: 10 }
])
// Check specific field sizes to find bloat
db.collection.aggregate([
{ $project: {
total: { $bsonSize: "$$ROOT" },
activitySize: { $bsonSize: { $ifNull: ["$activityLog", []] } },
profileSize: { $bsonSize: { $ifNull: ["$profile", {}] } }
}}
])
// Find documents with large arrays
db.collection.aggregate([
{ $project: {
size: { $bsonSize: "$$ROOT" },
arrayLen: { $size: { $ifNull: ["$myArray", []] } }
}},
{ $match: { arrayLen: { $gt: 100 } } },
{ $sort: { arrayLen: -1 } },
{ $limit: 10 }
])
// Find documents with hot/cold imbalance
db.collection.aggregate([
{ $project: {
totalSize: { $bsonSize: "$$ROOT" },
coldSize: { $bsonSize: { $ifNull: ["$reviews", []] } },
hotSize: { $subtract: [
{ $bsonSize: "$$ROOT" },
{ $bsonSize: { $ifNull: ["$reviews", []] } }
]}
}},
{ $match: {
$expr: { $gt: ["$coldSize", { $multiply: ["$hotSize", 10] }] }
}},
{ $limit: 10 }
])
// Check working set vs RAM
db.serverStatus().wiredTiger.cache
// "bytes currently in the cache" vs "maximum bytes configured"Atlas Schema Suggestions flags: “Array field may grow without bound”, “Document size exceeds recommended limit”
References: