CPU overload and instability when running 3.1.0 with .compact files from earlier versions.

[kathy1121]

Couchdb consumes 95% cpus event though we extend our disk space.

Description

We upgrade couchdb to the latest version but it consumes thousands of cpus every moment.
The highest process is beam.smp.
We have many indexers in active tasks.

Steps to Reproduce

Expected Behaviour

Suggest how to optimize the performance and reduce the cpus.

Your Environment

Additional Context

It consumes lots cpus in our prod environment. Please help solve this issue. Thanks.

[janl]

To diagnose this issue, we need to know a lot more details. Can you explain:

• your setup
• how CouchDB is used by your application(s)
• what your traffic looks like?
• share any error messages from your logs

[kathy1121]

you can not set rereduce to anything, CouchDB uses that to tell you that it is in a rereduce phase. Can you share your reduce function here? The issue is still that your reduce function is not written correctly.

We just upgarded the couchdb from old version to laterst version. But before it worked well without issue.
Why the new version consumes lots of cpus?And the couchdb eats lots of disks every moment.

We also have this kind of errors:
mfa: fabric_rpc:map_view/5 throw:shutdown
emulator -------- Error in process
{{badmatch,{error,{noproc,{gen_server,call

Could you please help on this?

[kathy1121]

my first guess is that the emsort errors are just follow-up errors from this, so let’s get that resolved first and then see if any other errors remain

Hi Jan,

checked with team, we have quite a few design docs which have custom reduce function.
But in the earlier version we didn't met this issue. But now the new couchdb eats lots of disk space every moment.
Could you please give some suggestion?

[wohali]

If you check the output of ps or Process Explorer (Windows), do you see lots of couchjs processes? Are they eating all the CPU?

CouchDB 3.1 will be building every view on every database automatically for you now, even ones you never accessed before. This is new functionality in CouchDB 3.x. Perhaps it is building views that were never used before, including views with errors.

You can configure how this auto-building process works, it's fully documented. You may want to disable some views from auto-building (by adding autoupdate: false to their ddocs), or disable ken entirely by setting [ken] batch_channels = 0 and [ken] incremental_channels = 0 in your etc/local.ini file.

[kathy1121]

If you check the output of ps or Process Explorer (Windows), do you see lots of couchjs processes? Are they eating all the CPU?

CouchDB 3.1 will be building every view on every database automatically for you now, even ones you never accessed before. This is new functionality in CouchDB 3.x. Perhaps it is building views that were never used before, including views with errors.

You can configure how this auto-building process works, it's fully documented. You may want to disable some views from auto-building (by adding autoupdate: false to their ddocs), or disable ken entirely by setting [ken] batch_channels = 0 and [ken] incremental_channels = 0 in your etc/local.ini file.

Thank you for your reply.

Yes I checked the couch.js process and it doesn't eat all CPU.
Regarding the ken configuration, I checked but the result is {} Does it mean we don't have the related configure?

Besides, we found that another core issue is the new version consumes a mess of disks from 40G to 220G in several days. But the old version is stable. Why now eats lots of disks?

Could you please give some advice?

[kathy1121]

Hi Team,

regard this case, #2941
I found that maybe my errors are similar to him. Also our db is very large and now compact failed.

How to solve this issue?

[kathy1121]

Refering to #2941, we also delete old .compact files. The disk usage decrease.
But now we still face issue in a pod:
[error] CRASH REPORT Process
[error]rexi_server: mfa: fabric_rpc:map_view/5 throw:shutdown
[error]gen_server <0.17218.33> terminated with reason: bad return value {read_beyond_eof,"./data/shards

[janl]

Perfect!

[kathy1121]

but we still have error in a pod… one pod still consumes a lot