[mongodb-user] Cannot Start MongoDB - WT_ERROR: non-specific WiredTiger error

[Victor Inocentes]

Hello everyone,

Hope there is someone who can help me in this situation.

I am running a MongoDB EC2 Instance and ran out of space so the service has
shutdown. I have taken it up with the Bitnami Community but we've failed to
reach a resolution to this problem. We tried restoring the permissions of
the files affected to the original state of the instance prior to shutdown.

The log below is the last message so I believe that we have narrowed down
the error to its true source but I still cannot restart the service

***@ip-10-10-222-193:/home/bitnami$ /opt/bitnami/mongodb/bin/mongod --repair
--dbpath /opt/bitnami/mongodb/data/db
2018-11-21T13:10:00.866+0000 I CONTROL [initandlisten] MongoDB starting :
pid=24158 port=27017 dbpath=/opt/bitnami/mongodb/data/db 64-bit host=ip-10-
10-222-193
2018-11-21T13:10:00.866+0000 I CONTROL [initandlisten] db version v3.4.10
2018-11-21T13:10:00.866+0000 I CONTROL [initandlisten] git version:
078f28920cb24de0dd479b5ea6c66c644f6326e9
2018-11-21T13:10:00.866+0000 I CONTROL [initandlisten] allocator: tcmalloc
2018-11-21T13:10:00.866+0000 I CONTROL [initandlisten] modules: none
2018-11-21T13:10:00.866+0000 I CONTROL [initandlisten] build environment:
2018-11-21T13:10:00.866+0000 I CONTROL [initandlisten] distarch: x86_64
2018-11-21T13:10:00.866+0000 I CONTROL [initandlisten] target_arch:
x86_64
2018-11-21T13:10:00.866+0000 I CONTROL [initandlisten] options: { repair:
true, storage: { dbPath: "/opt/bitnami/mongodb/data/db" } }
2018-11-21T13:10:00.888+0000 I - [initandlisten] Detected data files
in /opt/bitnami/mongodb/data/db created by the 'wiredTiger' storage engine,
so setting the active storage engine to 'wiredTiger'.
2018-11-21T13:10:00.888+0000 I STORAGE [initandlisten]
2018-11-21T13:10:00.888+0000 I STORAGE [initandlisten] ** WARNING: Using
the XFS filesystem is strongly recommended with the WiredTiger storage
engine
2018-11-21T13:10:00.888+0000 I STORAGE [initandlisten] ** See http
://dochub.mongodb.org/core/prodnotes-filesystem
2018-11-21T13:10:00.888+0000 I STORAGE [initandlisten] Detected WT journal
files. Running recovery from last checkpoint.
2018-11-21T13:10:00.888+0000 I STORAGE [initandlisten] journal to
nojournal transition config: create,cache_size=3479M,session_max=20000,
eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),
log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager
=(close_idle_time=100000),checkpoint=(wait=60,log_size=2GB),statistics_log=(
wait=0),
2018-11-21T13:10:00.896+0000 E STORAGE [initandlisten] WiredTiger error (-
31802) [1542805800:896663][24158:0x7f3fc3466d40], file:WiredTiger.wt,
connection: /opt/bitnami/mongodb/data/db/WiredTiger.wt: handle-read: pread:
failed to read 4096 bytes at offset 53248: WT_ERROR: non-specific WiredTiger
error
2018-11-21T13:10:00.897+0000 I - [initandlisten] Assertion: 28718:-
31802: WT_ERROR: non-specific WiredTiger error src/mongo/db/storage/
wiredtiger/wiredtiger_kv_engine.cpp 253
2018-11-21T13:10:00.897+0000 I STORAGE [initandlisten] exception in
initAndListen: 28718 -31802: WT_ERROR: non-specific WiredTiger error,
terminating
2018-11-21T13:10:00.897+0000 I NETWORK [initandlisten] shutdown: going to
close listening sockets...
2018-11-21T13:10:00.897+0000 I NETWORK [initandlisten] removing socket file
: /tmp/mongodb-27017.sock
2018-11-21T13:10:00.897+0000 I NETWORK [initandlisten] shutdown: going to
flush diaglog...
2018-11-21T13:10:00.897+0000 I CONTROL [initandlisten] now exiting
2018-11-21T13:10:00.897+0000 I CONTROL [initandlisten] shutting down with
code:100


Sincerely appreciate the assistance and thank you in advance for the help!!
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: https://docs.mongodb.com/manual/support/
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+***@googlegroups.com.
To post to this group, send email to mongodb-***@googlegroups.com.
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/93b04c27-993e-44a2-b37a-6f622b6180e7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

['Kevin Adistambha' via mongodb-user]

Hi Victor,

The error message you posted seems to indicate that the WiredTiger.wt file
is corrupted.

Having said that, MongoDB 4.0 has improved --repair behaviour. If you are
able to, could you run --repair on a copy of the data files using the 4.0
binary and see if it allows MongoDB to start:

1. Download the MongoDB 4.0 binary
2. Take a backup of the dbpath
3. Run mongod --repair on the dbpath using the MongoDB 4.0 binary
4. Try starting with MongoDB 3.4.10

Note that this is not guaranteed to be successful, and should be done as a
last resort if you don’t have a recent backup.

Best regards,
Kevin
​
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: https://docs.mongodb.com/manual/support/
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+***@googlegroups.com.
To post to this group, send email to mongodb-***@googlegroups.com.
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/ba19cbd8-64e9-442c-9e7c-8033d1f27751%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.