Skip to content

Commit eaaae09

Browse files
DOCSP-38000 Merge Capped Collections IA feature branch into master (#7033)
* DOCS-15096 Capped collections landing page updates (#6079) * DOCS-15096 Capped collections landing page updates * move capped collections page and update snooty.toml * add facet * formatting * reorg * fix ref * update use cases * review edits * shuffle * wording * address review comments * paragraph split * typo * undo toc update * following > these * review edits * typo * wording * clarify update note * edit * (DOCSP-37995): Change max docs for capped collection (#6948) * (DOCSP-37995): Change max docs for capped collection * edit include * add ref link * add comma * (DOCSP-38002): Change byte size of a capped collection (#6953) * (DOCSP-38002): Change byte size of a capped collection * add unit * fix command * fix command * (DOCSP-38010): convert collection to capped (#6955) * (DOCSP-38010): convert collection to capped * typo * add copyable option * change learn more link * build error * add detail for db lock * edits * remove quotes around field name * (DOCSP-38019): Check if collection is capped (#6957) * (DOCSP-38019): Check if collection is capped * fix formatting for collection names * formatting * formatting * add page meta info * fix indentation * (DOCSP-38041): Create a capped collection (#6965) * (DOCSP-38041): Create a capped collection * edits * formatting * review edits * (DOCSP-38027): Query a capped collection (#6962) * (DOCSP-38027): Query a capped collection * add code block language * edits * add page metadata * typo * add natural order and use include * edits * formatting * add learn more * add step lead-in * review edits * add detail for multiple concurrent writes * add period * nit
1 parent a030acc commit eaaae09

11 files changed

+650
-216
lines changed

snooty.toml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -46,6 +46,7 @@ toc_landing_pages = [
4646
"/core/authentication",
4747
"/core/authorization",
4848
"/core/backups",
49+
"/core/capped-collections",
4950
"/core/crud",
5051
"/core/csfle",
5152
"/core/csfle/fundamentals/",

source/core/capped-collections.txt

Lines changed: 95 additions & 216 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
.. _manual-capped-collection:
2+
.. _capped_collections_remove_documents:
23

34
==================
45
Capped Collections
@@ -12,271 +13,149 @@ Capped Collections
1213
:depth: 2
1314
:class: singlecol
1415

15-
:term:`Capped collections <capped collection>` are fixed-size
16-
collections that support high-throughput operations that insert
17-
and retrieve documents based on insertion order. Capped
18-
collections work in a way similar to circular buffers: once a
19-
collection fills its allocated space, it makes room for new documents
20-
by overwriting the oldest documents in the collection.
21-
22-
See :method:`~db.createCollection()` or :dbcommand:`create`
23-
for more information on creating capped collections.
24-
25-
As an alternative to capped collections, consider :ref:`TTL (Time To
26-
Live) indexes <index-feature-ttl>`. TTL indexes allow you to expire and
27-
remove data from normal collections based on the value of a date-typed
28-
field and a TTL value for the index. You can also use a TTL index on a
29-
capped collection to remove expired documents even if the capped
30-
collection hasn't exceeded its size limit. For details, :ref:`ttl-collections`.
31-
32-
33-
Behavior
34-
--------
35-
36-
Insertion Order
37-
~~~~~~~~~~~~~~~
38-
39-
Capped collections guarantee preservation of the insertion order. As a
40-
result, queries do not need an index to return documents in insertion
41-
order. Without this indexing overhead, capped collections can support
42-
higher insertion throughput.
43-
44-
.. _capped_collections_remove_documents:
45-
46-
Automatic Removal of Oldest Documents
47-
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
48-
49-
To make room for new documents, capped collections automatically remove
50-
the oldest documents in the collection without requiring scripts or
51-
explicit remove operations.
52-
53-
Consider the following potential use cases for capped
54-
collections:
55-
56-
- Store log information generated by high-volume systems. Inserting
57-
documents in a capped collection without an index is close to the
58-
speed of writing log information directly to a file
59-
system. Furthermore, the built-in *first-in-first-out* property
60-
maintains the order of events, while managing storage use.
61-
For example, the :ref:`oplog <capped-collections-oplog>`
62-
uses a capped collection.
63-
64-
- Cache small amounts of data in a capped collections. Since caches
65-
are read rather than write heavy, you would either need to ensure
66-
that this collection *always* remains in the working set (i.e. in
67-
RAM) *or* accept some write penalty for the required index or
68-
indexes.
69-
70-
.. _capped-collections-oplog:
71-
72-
Oplog Collection
73-
~~~~~~~~~~~~~~~~
74-
75-
The :term:`oplog.rs <oplog>` collection that stores a log
76-
of the operations in a :term:`replica set` uses a capped collection.
77-
78-
Starting in MongoDB 4.0, unlike other capped collections, the oplog can
79-
grow past its configured size limit to avoid deleting the :data:`majority
80-
commit point <replSetGetStatus.optimes.lastCommittedOpTime>`.
81-
82-
.. note::
83-
84-
MongoDB rounds the capped size of the oplog up to the nearest
85-
integer multiple of 256, in bytes.
86-
87-
.. note::
88-
89-
MongoDB rounds the capped size of the oplog
90-
up to the nearest integer multiple of 256, in bytes.
91-
92-
``_id`` Index
93-
~~~~~~~~~~~~~
94-
95-
Capped collections have an ``_id`` field and an index on the ``_id``
96-
field by default.
16+
.. facet::
17+
:name: genre
18+
:values: reference
9719

98-
.. _capped-collections-recommendations-and-restrictions:
20+
Capped collections are fixed-size collections that insert and retrieve
21+
documents based on insertion order. Capped collections work similarly to
22+
circular buffers: once a collection fills its allocated space, it makes
23+
room for new documents by overwriting the oldest documents in the
24+
collection.
9925

100-
Restrictions and Recommendations
101-
--------------------------------
102-
103-
Reads
104-
~~~~~
105-
106-
.. include:: /includes/extracts/transactions-capped-collection-read-change.rst
107-
108-
Updates
109-
~~~~~~~
110-
111-
If you plan to update documents in a capped collection, create an index
112-
so that these update operations do not require a collection scan.
113-
114-
Sharding
115-
~~~~~~~~
116-
117-
You cannot shard a capped collection.
118-
119-
Query Efficiency
120-
~~~~~~~~~~~~~~~~
26+
Restrictions
27+
------------
12128

122-
Use natural ordering to retrieve the most recently inserted elements
123-
from the collection efficiently. This is similar to using the ``tail``
124-
command on a log file.
29+
- Capped collections cannot be sharded.
12530

126-
Aggregation ``$out``
127-
~~~~~~~~~~~~~~~~~~~~
31+
- You cannot create capped collections on :atlas:`serverless instances
32+
</tutorial/create-serverless-instance/>`.
12833

129-
The aggregation pipeline stage :pipeline:`$out`
130-
cannot write results to a capped collection.
34+
- Capped collections are not supported in :ref:`Stable API <stable-api>`
35+
V1.
13136

132-
.. include:: /includes/replacement-mms.rst
37+
- You cannot write to capped collections in :ref:`transactions
38+
<transactions>`.
13339

134-
Transactions
135-
~~~~~~~~~~~~
40+
- The :pipeline:`$out` aggregation pipeline stage cannot write results
41+
to a capped collection.
13642

137-
.. include:: /includes/extracts/transactions-capped-collection-change.rst
43+
- You cannot use read concern :readconcern:`"snapshot"` when reading
44+
from a capped collection.
13845

139-
Stable API
140-
~~~~~~~~~~
46+
Command Syntax
47+
--------------
14148

142-
Capped collections are not supported in :ref:`Stable API
143-
<stable-api>` V1.
144-
145-
Procedures
146-
----------
147-
148-
Create a Capped Collection
149-
~~~~~~~~~~~~~~~~~~~~~~~~~~
150-
151-
You must create capped collections explicitly using the
152-
:method:`db.createCollection()` method, which is a
153-
:binary:`~bin.mongosh` helper for the :dbcommand:`create` command.
154-
When creating a capped collection you must specify the maximum size of
155-
the collection in bytes, which MongoDB will pre-allocate for the
156-
collection. The size of the capped collection includes a small amount of
157-
space for internal overhead.
49+
The following example creates a capped collection called ``log`` with a
50+
maximum size of 100,000 bytes.
15851

15952
.. code-block:: javascript
16053

16154
db.createCollection( "log", { capped: true, size: 100000 } )
16255

163-
.. note::
164-
165-
The value that you provide for the ``size`` field
166-
must be greater than ``0`` and less than or equal to
167-
``1024^5`` (1 {+pb+}). MongoDB rounds the ``size`` of all capped
168-
collections up to the nearest integer multiple of 256, in bytes.
169-
170-
Additionally, you may also specify a maximum number of documents for the
171-
collection using the ``max`` field as in the following document:
172-
173-
.. code-block:: javascript
56+
For more information on creating capped collections, see
57+
:method:`~db.createCollection()` or :dbcommand:`create`.
17458

175-
db.createCollection("log", { capped : true, size : 5242880, max :
176-
5000 } )
59+
Use Cases
60+
---------
17761

178-
.. important::
179-
180-
The ``size`` field is *always* required, even when
181-
you specify the ``max`` number of documents. MongoDB removes older
182-
documents if a collection reaches the maximum size limit before it
183-
reaches the maximum document count.
62+
.. include:: /includes/capped-collections/use-ttl-index.rst
18463

185-
.. see::
64+
The most common use case for a capped collection is to store log
65+
information. When the capped collection reaches its maximum size, old
66+
log entries are automatically overwritten with new entries.
18667

187-
:method:`db.createCollection()` and :dbcommand:`create`.
68+
Get Started
69+
-----------
18870

189-
.. _capped-collections-options:
71+
To create and query capped collections, see these pages:
19072

191-
Query a Capped Collection
192-
~~~~~~~~~~~~~~~~~~~~~~~~~
73+
- :ref:`capped-collections-create`
19374

194-
If you perform a :method:`~db.collection.find()` on a capped collection
195-
with no ordering specified, MongoDB guarantees that the ordering of
196-
results is the same as the insertion order.
75+
- :ref:`capped-collections-query`
19776

198-
To retrieve documents in reverse insertion order, issue
199-
:method:`~db.collection.find()` along with the :method:`~cursor.sort()`
200-
method with the :operator:`$natural` parameter set to ``-1``, as shown
201-
in the following example:
77+
- :ref:`capped-collections-check`
20278

203-
.. code-block:: javascript
79+
- :ref:`capped-collections-convert`
20480

205-
db.cappedCollection.find().sort( { $natural: -1 } )
81+
- :ref:`capped-collections-change-size`
20682

207-
Check if a Collection is Capped
208-
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
83+
- :ref:`capped-collections-change-max-docs`
20984

210-
Use the :method:`~db.collection.isCapped()` method to determine if a
211-
collection is capped, as follows:
85+
.. _capped-collections-recommendations-and-restrictions:
21286

213-
.. code-block:: javascript
87+
Details
88+
-------
21489

215-
db.collection.isCapped()
90+
Consider these behavioral details for capped collections.
21691

217-
Convert a Collection to Capped
218-
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
92+
.. _capped-collections-oplog:
21993

220-
You can convert a non-capped collection to a capped collection with
221-
the :dbcommand:`convertToCapped` command:
94+
Oplog Collection
95+
~~~~~~~~~~~~~~~~
22296

223-
.. code-block:: javascript
97+
The :term:`oplog.rs <oplog>` collection that stores a log
98+
of the operations in a :term:`replica set` uses a capped collection.
22499

225-
db.runCommand({"convertToCapped": "mycoll", size: 100000});
100+
Unlike other capped collections, the oplog can grow past its configured
101+
size limit to avoid deleting the :data:`majority commit point
102+
<replSetGetStatus.optimes.lastCommittedOpTime>`.
226103

227-
The ``size`` parameter specifies the size of the capped collection in
228-
bytes.
104+
.. note::
229105

230-
.. include:: /includes/fact-database-lock.rst
106+
MongoDB rounds the capped size of the oplog up to the nearest
107+
integer multiple of 256, in bytes.
231108

232-
Change a Capped Collection's Size
233-
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
109+
_id Index
110+
~~~~~~~~~
234111

235-
.. versionadded:: 6.0
112+
Capped collections have an ``_id`` field and an index on the ``_id``
113+
field by default.
236114

237-
You can resize a capped collection using the :dbcommand:`collMod` command's
238-
``cappedSize`` option to set the ``cappedSize`` in bytes. ``cappedSize`` must be
239-
greater than ``0`` and less than or equal to ``1024^5`` (1 {+pb+}).
115+
Updates
116+
~~~~~~~
240117

241-
.. note::
118+
Avoid updating data in a capped collection. Because capped collections
119+
are fixed-size, updates can cause your data to expand beyond the
120+
collection's allocated space, which can cause unexpected behavior.
242121

243-
Before you can resize a capped collection, you must have already set
244-
the :ref:`featureCompatibilityVersion <set-fcv>` to at least version
245-
``"6.0"``.
122+
Query Efficiency
123+
~~~~~~~~~~~~~~~~
246124

247-
For example, the following command sets the maximum size of the ``"log"`` capped
248-
collection to 100000 bytes:
125+
.. include:: /includes/capped-collections/query-natural-order.rst
249126

250-
.. code-block:: javascript
127+
Tailable Cursor
128+
~~~~~~~~~~~~~~~
251129

252-
db.runCommand( { collMod: "log", cappedSize: 100000 } )
130+
You can use a :term:`tailable cursor` with capped collections. Similar to the
131+
Unix ``tail -f`` command, the tailable cursor "tails" the end of a
132+
capped collection. As new documents are inserted into the capped
133+
collection, you can use the tailable cursor to continue retrieving
134+
documents.
253135

254-
Change the Maximum Number of Documents in a Capped Collection
255-
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
136+
For information on creating a tailable cursor, see
137+
:ref:`tailable-cursors-landing-page`.
256138

257-
.. versionadded:: 6.0
139+
Multiple Concurrent Writes
140+
~~~~~~~~~~~~~~~~~~~~~~~~~~
258141

259-
To change the maximum number of documents in a capped collection, use the
260-
:dbcommand:`collMod` command's ``cappedMax`` option. If ``cappedMax`` is less
261-
than or equal to ``0``, there is no maximum document limit. If
262-
``cappedMax`` is less than the current number of documents in the
263-
collection, MongoDB removes the excess documents on the next insert operation.
142+
.. include:: /includes/capped-collections/concurrent-writes.rst
264143

265-
For example, the following command sets the maximum number of documents in the
266-
``"log"`` capped collection to 500:
144+
Learn More
145+
----------
267146

268-
.. code-block:: javascript
147+
- :ref:`index-feature-ttl`
269148

270-
db.runCommand( { collMod: "log", cappedMax: 500 } )
149+
- :ref:`index-properties`
271150

272-
Tailable Cursor
273-
~~~~~~~~~~~~~~~
151+
- :ref:`indexing-strategies`
274152

275-
You can use a :term:`tailable cursor` with capped collections. Similar to the
276-
Unix ``tail -f`` command, the tailable cursor "tails" the end of a
277-
capped collection. As new documents are inserted into the capped
278-
collection, you can use the tailable cursor to continue retrieving
279-
documents.
153+
.. toctree::
154+
:titlesonly:
280155

281-
See :doc:`/core/tailable-cursors` for information on creating
282-
a tailable cursor.
156+
/core/capped-collections/create-capped-collection
157+
/core/capped-collections/query-capped-collection
158+
/core/capped-collections/check-if-collection-is-capped
159+
/core/capped-collections/convert-collection-to-capped
160+
/core/capped-collections/change-size-capped-collection
161+
/core/capped-collections/change-max-docs-capped-collection

0 commit comments

Comments
 (0)