[mongodb-user] WiredTiger cache usage

[Yos Tj]

Hello,

With mongostat, %used (of WiredTiger cache) never goes over 80% or so. Why?

Though it is not said clearly in MongoDB manual, there exist three
variables
related to eviction mechanism internally in my understanding.

eviction_dirty_target (default 80)
eviction_target (default 80)
eviction_trigger (default 95)

As I use read -only workload this time, I can put aside
eviction_dirty_target.
When I read large collections for example, I expect that %used fluctuates
between 80 and 95, but I have never seen such activity.

Regards,
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+***@googlegroups.com.
To post to this group, send email to mongodb-***@googlegroups.com.
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/f8a110e4-4411-45c0-a9f4-67961c72c5e2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Yos Tj]

Does anyone know about this?

Post by Yos Tj
Hello,
With mongostat, %used (of WiredTiger cache) never goes over 80% or so. Why?
Though it is not said clearly in MongoDB manual, there exist three
variables
related to eviction mechanism internally in my understanding.
eviction_dirty_target (default 80)
eviction_target (default 80)
eviction_trigger (default 95)
As I use read -only workload this time, I can put aside
eviction_dirty_target.
When I read large collections for example, I expect that %used fluctuates
between 80 and 95, but I have never seen such activity.
Regards,

--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+***@googlegroups.com.
To post to this group, send email to mongodb-***@googlegroups.com.
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/fe7ddf84-34a1-499f-80c6-3923ac2aed93%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Ankur Raina]

Hi Yos,

This looks like the expected behavior. There are two possibilities:

1) Your working set only takes up 80% of cache. In which case that is all
the memory WiredTiger will use.

2) WiredTiger is keeping up with eviction easily, and as such is keeping
the cache at the configured *eviction_target*.

You can figure out which of the two is the case by inspecting the output of
*db.serverStatus* over time - if statistics about the number of pages
evicted changes over time it is the second case. We don’t recommend users
configuring targets a lot higher than 80 - since if the cache does become
entirely full it can cause operations to slow down in less predictable
ways. Generally we would suggest leaving the eviction parameters at the
default settings unless you’ve done extensive testing with your production
workload.

Regards

Ankur
​
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: https://docs.mongodb.org/manual/support/
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+***@googlegroups.com.
To post to this group, send email to mongodb-***@googlegroups.com.
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/aa48ba03-f926-4bf9-a1cc-63be2b883c9e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Yos Tj]

Hi,

Post by Ankur Raina
2) WiredTiger is keeping up with eviction easily, and as such is keeping

the cache at the configured *eviction_target*.

Yes, I'm faced with the second case. To make sure, I
checked db.serverStatus().wiredTiger.cache.

Just after starting mongod,
"bytes currently in the cache" : 145253,
"maximum bytes configured" : 7516192768, /* half of physical memory */
"unmodified pages evicted" : 0,
"modified pages evicted" : 0,
"internal pages evicted" : 0,

After starting col-scan of a very big collection, bytes-in-the-cache starts
growing.
"bytes currently in the cache" : 3875183925, /* growing */
"maximum bytes configured" : 7516192768,
"unmodified pages evicted" : 0,
"modified pages evicted" : 0,
"internal pages evicted" : 0,

At last %used reached 80%.
"bytes currently in the cache" : 6008865084, /* reached 80% of maximum
bytes*/
"maximum bytes configured" : 7516192768,
"unmodified pages evicted" : 5545,
"modified pages evicted" : 3,
"internal pages evicted" : 0,

While col-scan is still running, number of eviction pages increases.
"bytes currently in the cache" : 6015283698, /* keeping 80% */
"maximum bytes configured" : 7516192768,
"unmodified pages evicted" : 112087, /* increasing */
"modified pages evicted" : 3,
"internal pages evicted" : 823,

It continues until con-scan finishes.
"bytes currently in the cache" : 6024869133, /* keeping 80% */
"maximum bytes configured" : 7516192768,
"unmodified pages evicted" : 219019, /* increasing */
"modified pages evicted" : 3,
"internal pages evicted" : 1712,

I hope the ducumentation becomes clearer about the eviction mechanism.

Regards,
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: https://docs.mongodb.org/manual/support/
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+***@googlegroups.com.
To post to this group, send email to mongodb-***@googlegroups.com.
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/801306be-2a74-4d67-9091-01f9cb55648a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Ankur Raina]

Hi Yos,

Thank you for your explanatory example with db.serverStatus() output for
WiredTiger cache, which confirms that this is expected behavior. I’ll make
a suggestion to include more detail in the documentation.

Regards
Ankur
​
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: https://docs.mongodb.org/manual/support/
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+***@googlegroups.com.
To post to this group, send email to mongodb-***@googlegroups.com.
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/628ffd8e-0fa7-4603-a7f2-997161a68584%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.