Discussion:
call mongocxx functions when process quit
(too old to reply)
li ning
2018-04-01 09:32:38 UTC
Permalink
There is a segfault when I kill -2 my process:

(gdb) bt
#0 0x0000000000801866 in _mongoc_queue_pop_head ()
#1 0x00000000007ec949 in mongoc_client_pool_pop ()
#2 0x00000000007e320c in mongocxx::v_noabi::pool::acquire() ()

My code is like:

int main() {
mongocxx::instance mongo_driver;
MySingleton obj;
return 0;
}

When the process quit, the MySingleton desctor will call "auto conn =
_pool.acquire();", _pool is data member of MySingleton class.
During the destruction of MySingleton obj, the mongo_driver still there, so
it should not cause a segfault in mongocxx driver.

In this coredump file, there is a thread still running:
#0 pthread_cond_timedwait@@GLIBC_2.3.2 () at
../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
#1 0x0000000000809065 in _mongoc_topology_run_background ()
#2 0x00007f34e3faf6ba in start_thread (arg=0x7f34e1d50700) at
pthread_create.c:333
#3 0x00007f34e323c3dd in clone () at
../sysdeps/unix/sysv/linux/x86_64/clone.S:109

So what's the problem? Can I use mongocxx like this?

mongocxx version 3.1.1
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: https://docs.mongodb.com/manual/support/
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+***@googlegroups.com.
To post to this group, send email to mongodb-***@googlegroups.com.
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/d664b0ca-2423-4b15-90a0-5d13122e15f7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
'Andrew Morrow' via mongodb-user
2018-04-02 20:20:48 UTC
Permalink
Hi -

Could you please post a complete and compilable example that reproduces the
behavior you are seeing? It will help greatly with diagnosis if I can run
the same code you are running.

Thanks,
Andrew
Post by li ning
(gdb) bt
#0 0x0000000000801866 in _mongoc_queue_pop_head ()
#1 0x00000000007ec949 in mongoc_client_pool_pop ()
#2 0x00000000007e320c in mongocxx::v_noabi::pool::acquire() ()
int main() {
mongocxx::instance mongo_driver;
MySingleton obj;
return 0;
}
When the process quit, the MySingleton desctor will call "auto conn =
_pool.acquire();", _pool is data member of MySingleton class.
During the destruction of MySingleton obj, the mongo_driver still there,
so it should not cause a segfault in mongocxx driver.
x86_64/pthread_cond_timedwait.S:225
#1 0x0000000000809065 in _mongoc_topology_run_background ()
#2 0x00007f34e3faf6ba in start_thread (arg=0x7f34e1d50700) at
pthread_create.c:333
#3 0x00007f34e323c3dd in clone () at ../sysdeps/unix/sysv/linux/
x86_64/clone.S:109
So what's the problem? Can I use mongocxx like this?
mongocxx version 3.1.1
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
https://docs.mongodb.com/manual/support/
---
You received this message because you are subscribed to the Google Groups
"mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/
msgid/mongodb-user/d664b0ca-2423-4b15-90a0-5d13122e15f7%40googlegroups.com
<https://groups.google.com/d/msgid/mongodb-user/d664b0ca-2423-4b15-90a0-5d13122e15f7%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: https://docs.mongodb.com/manual/support/
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+***@googlegroups.com.
To post to this group, send email to mongodb-***@googlegroups.com.
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/CAHX05qFF92iXaHbHbM5NPuJmskTQmucObx8vrRAnRFPv2xtPfA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
li ning
2018-04-03 09:22:37 UTC
Permalink
I try to reproduce this problem with a simplified code. I am working on
this now...

The code have a Database singleton class MyDb which is like:
class MyDb {
public:
static MyDb& instance() {
static MyDb instance;
return instance;
}
void update_x(...) {
auto conn = _pool.acquire();
....
}
private:
MyDb(host_str) : _pool(url(host_str)...{}
mongocxx::uri _uri;
mongocxx::pool _pool;
mongocxx::database _db;
}

and an singleton manager class 'IoServiceMgr' to manage boost io_service
class IoServiceMgr {
public:
static IoServiceMgr & instance() {
static IoServiceMgr instance;
return instance;
}
boost::io_service& io() const {
random return one io
}
private:
std::vector<shared_ptr<io_serivce>> _ios;
};

in main func:
int main() {
mongocxx::instance mongo_driver;
MyDb::instance();
boost::asio::signal_set signals(IoServiceMgr::instance().io());
signals.add(SIGINT);
signals.add(SIGTERM);
signals.add(SIGABRT);
signals.async_wait(shutdown_server);
IoServiceMgr::instance().run();
}

void shutdown_server() {
do clean up work
}

It has 50% chance when I kill -2, the process crash in libmongoc code. In
the crash time, the process only have 2 thread, one is mongo thread, other
is the main thread
(main function already exit). When boost io service destruction, that will
cause my business logic object shared_ptr go destory
then in my business logic object destructor, will call
MyDb::instance().update_x(...). In this update_x _pool.acquire(), the
libmongoc code will cause an segv.

First, I thought the reason is mongo_driver got destroy before
'MyDb::instance().update_x(...)' call in destructor chain.
However I add static to mongo_driver, it is same. I guess this is related
to multi thread. I can not reproduce this in single thread code.



圚 2018幎4月3日星期二 UTC+8䞊午4:21:17acm写道
Post by 'Andrew Morrow' via mongodb-user
Hi -
Could you please post a complete and compilable example that reproduces
the behavior you are seeing? It will help greatly with diagnosis if I can
run the same code you are running.
Thanks,
Andrew
Post by li ning
(gdb) bt
#0 0x0000000000801866 in _mongoc_queue_pop_head ()
#1 0x00000000007ec949 in mongoc_client_pool_pop ()
#2 0x00000000007e320c in mongocxx::v_noabi::pool::acquire() ()
int main() {
mongocxx::instance mongo_driver;
MySingleton obj;
return 0;
}
When the process quit, the MySingleton desctor will call "auto conn =
_pool.acquire();", _pool is data member of MySingleton class.
During the destruction of MySingleton obj, the mongo_driver still there,
so it should not cause a segfault in mongocxx driver.
../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
#1 0x0000000000809065 in _mongoc_topology_run_background ()
#2 0x00007f34e3faf6ba in start_thread (arg=0x7f34e1d50700) at
pthread_create.c:333
#3 0x00007f34e323c3dd in clone () at
../sysdeps/unix/sysv/linux/x86_64/clone.S:109
So what's the problem? Can I use mongocxx like this?
mongocxx version 3.1.1
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
https://docs.mongodb.com/manual/support/
---
You received this message because you are subscribed to the Google Groups
"mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an
<javascript:>.
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit
https://groups.google.com/d/msgid/mongodb-user/d664b0ca-2423-4b15-90a0-5d13122e15f7%40googlegroups.com
<https://groups.google.com/d/msgid/mongodb-user/d664b0ca-2423-4b15-90a0-5d13122e15f7%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: https://docs.mongodb.com/manual/support/
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+***@googlegroups.com.
To post to this group, send email to mongodb-***@googlegroups.com.
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/1ed58594-1a35-41dd-ae4a-70cc83bc8cb2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
'Andrew Morrow' via mongodb-user
2018-04-03 14:31:35 UTC
Permalink
Hi

I'd strongly advise you to run your program under valgrind, or build it
with the various sanitizers (AddressSanitizer, UndefinedSanitizer,
ThreadSanitizer, etc.). My suspicion is that you have an issue with the
order of static destructors, which are not sequenced across translation
units.

Thanks,
Andrew
Post by li ning
I try to reproduce this problem with a simplified code. I am working on
this now...
class MyDb {
static MyDb& instance() {
static MyDb instance;
return instance;
}
void update_x(...) {
auto conn = _pool.acquire();
....
}
MyDb(host_str) : _pool(url(host_str)...{}
mongocxx::uri _uri;
mongocxx::pool _pool;
mongocxx::database _db;
}
and an singleton manager class 'IoServiceMgr' to manage boost io_service
class IoServiceMgr {
static IoServiceMgr & instance() {
static IoServiceMgr instance;
return instance;
}
boost::io_service& io() const {
random return one io
}
std::vector<shared_ptr<io_serivce>> _ios;
};
int main() {
mongocxx::instance mongo_driver;
MyDb::instance();
boost::asio::signal_set signals(IoServiceMgr::instance().io());
signals.add(SIGINT);
signals.add(SIGTERM);
signals.add(SIGABRT);
signals.async_wait(shutdown_server);
IoServiceMgr::instance().run();
}
void shutdown_server() {
do clean up work
}
It has 50% chance when I kill -2, the process crash in libmongoc code. In
the crash time, the process only have 2 thread, one is mongo thread, other
is the main thread
(main function already exit). When boost io service destruction, that will
cause my business logic object shared_ptr go destory
then in my business logic object destructor, will call
MyDb::instance().update_x(...). In this update_x _pool.acquire(), the
libmongoc code will cause an segv.
First, I thought the reason is mongo_driver got destroy before
'MyDb::instance().update_x(...)' call in destructor chain.
However I add static to mongo_driver, it is same. I guess this is related
to multi thread. I can not reproduce this in single thread code.
圚 2018幎4月3日星期二 UTC+8䞊午4:21:17acm写道
Post by 'Andrew Morrow' via mongodb-user
Hi -
Could you please post a complete and compilable example that reproduces
the behavior you are seeing? It will help greatly with diagnosis if I can
run the same code you are running.
Thanks,
Andrew
Post by li ning
(gdb) bt
#0 0x0000000000801866 in _mongoc_queue_pop_head ()
#1 0x00000000007ec949 in mongoc_client_pool_pop ()
#2 0x00000000007e320c in mongocxx::v_noabi::pool::acquire() ()
int main() {
mongocxx::instance mongo_driver;
MySingleton obj;
return 0;
}
When the process quit, the MySingleton desctor will call "auto conn =
_pool.acquire();", _pool is data member of MySingleton class.
During the destruction of MySingleton obj, the mongo_driver still there,
so it should not cause a segfault in mongocxx driver.
../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
#1 0x0000000000809065 in _mongoc_topology_run_background ()
#2 0x00007f34e3faf6ba in start_thread (arg=0x7f34e1d50700) at
pthread_create.c:333
#3 0x00007f34e323c3dd in clone () at ../sysdeps/unix/sysv/linux/x86
_64/clone.S:109
So what's the problem? Can I use mongocxx like this?
mongocxx version 3.1.1
--
You received this message because you are subscribed to the Google
Groups "mongodb-user"
group.
https://docs.mongodb.com/manual/support/
---
You received this message because you are subscribed to the Google
Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/ms
gid/mongodb-user/d664b0ca-2423-4b15-90a0-5d13122e15f7%40googlegroups.com
<https://groups.google.com/d/msgid/mongodb-user/d664b0ca-2423-4b15-90a0-5d13122e15f7%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
https://docs.mongodb.com/manual/support/
---
You received this message because you are subscribed to the Google Groups
"mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/
msgid/mongodb-user/1ed58594-1a35-41dd-ae4a-70cc83bc8cb2%40googlegroups.com
<https://groups.google.com/d/msgid/mongodb-user/1ed58594-1a35-41dd-ae4a-70cc83bc8cb2%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: https://docs.mongodb.com/manual/support/
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+***@googlegroups.com.
To post to this group, send email to mongodb-***@googlegroups.com.
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/CAHX05qErQhv3T_cV0NGy71O00R2ezB0i9YyFnJ7rx1Y%3DTiEf-Q%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
li ning
2018-04-04 08:35:37 UTC
Permalink
You are right. It is the order problem. Even I arrange them in order in
main, but I forget some other singleton ctor already called
IoServiceMgr::instance so that this cause incorrect destruction order

圚 2018幎4月3日星期二 UTC+8䞋午10:32:06acm写道
Post by 'Andrew Morrow' via mongodb-user
Hi
I'd strongly advise you to run your program under valgrind, or build it
with the various sanitizers (AddressSanitizer, UndefinedSanitizer,
ThreadSanitizer, etc.). My suspicion is that you have an issue with the
order of static destructors, which are not sequenced across translation
units.
Thanks,
Andrew
Post by li ning
I try to reproduce this problem with a simplified code. I am working on
this now...
class MyDb {
static MyDb& instance() {
static MyDb instance;
return instance;
}
void update_x(...) {
auto conn = _pool.acquire();
....
}
MyDb(host_str) : _pool(url(host_str)...{}
mongocxx::uri _uri;
mongocxx::pool _pool;
mongocxx::database _db;
}
and an singleton manager class 'IoServiceMgr' to manage boost io_service
class IoServiceMgr {
static IoServiceMgr & instance() {
static IoServiceMgr instance;
return instance;
}
boost::io_service& io() const {
random return one io
}
std::vector<shared_ptr<io_serivce>> _ios;
};
int main() {
mongocxx::instance mongo_driver;
MyDb::instance();
boost::asio::signal_set signals(IoServiceMgr::instance().io());
signals.add(SIGINT);
signals.add(SIGTERM);
signals.add(SIGABRT);
signals.async_wait(shutdown_server);
IoServiceMgr::instance().run();
}
void shutdown_server() {
do clean up work
}
It has 50% chance when I kill -2, the process crash in libmongoc code. In
the crash time, the process only have 2 thread, one is mongo thread, other
is the main thread
(main function already exit). When boost io service destruction, that
will cause my business logic object shared_ptr go destory
then in my business logic object destructor, will call
MyDb::instance().update_x(...). In this update_x _pool.acquire(), the
libmongoc code will cause an segv.
First, I thought the reason is mongo_driver got destroy before
'MyDb::instance().update_x(...)' call in destructor chain.
However I add static to mongo_driver, it is same. I guess this is related
to multi thread. I can not reproduce this in single thread code.
圚 2018幎4月3日星期二 UTC+8䞊午4:21:17acm写道
Post by 'Andrew Morrow' via mongodb-user
Hi -
Could you please post a complete and compilable example that reproduces
the behavior you are seeing? It will help greatly with diagnosis if I can
run the same code you are running.
Thanks,
Andrew
Post by li ning
(gdb) bt
#0 0x0000000000801866 in _mongoc_queue_pop_head ()
#1 0x00000000007ec949 in mongoc_client_pool_pop ()
#2 0x00000000007e320c in mongocxx::v_noabi::pool::acquire() ()
int main() {
mongocxx::instance mongo_driver;
MySingleton obj;
return 0;
}
When the process quit, the MySingleton desctor will call "auto conn =
_pool.acquire();", _pool is data member of MySingleton class.
During the destruction of MySingleton obj, the mongo_driver still
there, so it should not cause a segfault in mongocxx driver.
../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
#1 0x0000000000809065 in _mongoc_topology_run_background ()
#2 0x00007f34e3faf6ba in start_thread (arg=0x7f34e1d50700) at
pthread_create.c:333
#3 0x00007f34e323c3dd in clone () at
../sysdeps/unix/sysv/linux/x86_64/clone.S:109
So what's the problem? Can I use mongocxx like this?
mongocxx version 3.1.1
--
You received this message because you are subscribed to the Google
Groups "mongodb-user"
group.
https://docs.mongodb.com/manual/support/
---
You received this message because you are subscribed to the Google
Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit
https://groups.google.com/d/msgid/mongodb-user/d664b0ca-2423-4b15-90a0-5d13122e15f7%40googlegroups.com
<https://groups.google.com/d/msgid/mongodb-user/d664b0ca-2423-4b15-90a0-5d13122e15f7%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
https://docs.mongodb.com/manual/support/
---
You received this message because you are subscribed to the Google Groups
"mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an
<javascript:>.
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit
https://groups.google.com/d/msgid/mongodb-user/1ed58594-1a35-41dd-ae4a-70cc83bc8cb2%40googlegroups.com
<https://groups.google.com/d/msgid/mongodb-user/1ed58594-1a35-41dd-ae4a-70cc83bc8cb2%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: https://docs.mongodb.com/manual/support/
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+***@googlegroups.com.
To post to this group, send email to mongodb-***@googlegroups.com.
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/262d54c8-38e2-4b44-9ae6-189d0def699f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
li ning
2018-04-04 09:13:11 UTC
Permalink
After I fix the construct order, the process will not crash, but hang
there. gdb shows 2 threads:

thread 1: (the main thread)
#0 __pthread_mutex_lock_full (mutex=0x4012150) at
../nptl/pthread_mutex_lock.c:343
#1 0x00000000007f3b33 in mongoc_client_pool_pop ()
#2 0x00000000007ea41c in mongocxx::v_noabi::pool::acquire() ()

thread 2:
#0 pthread_cond_timedwait@@GLIBC_2.3.2 () at
../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
#1 0x0000000000810275 in _mongoc_topology_run_background ()
#2 0x00007f6352c226ba in start_thread (arg=0x7f63509c3700) at
pthread_create.c:333
#3 0x00007f6351eaf3dd in clone () at
../sysdeps/unix/sysv/linux/x86_64/clone.S:109

It is not deadlock but seems thread 1 can not lock mutex=0x4012150. Why?
Any suggestion?

圚 2018幎4月3日星期二 UTC+8䞋午10:32:06acm写道
Post by 'Andrew Morrow' via mongodb-user
Hi
I'd strongly advise you to run your program under valgrind, or build it
with the various sanitizers (AddressSanitizer, UndefinedSanitizer,
ThreadSanitizer, etc.). My suspicion is that you have an issue with the
order of static destructors, which are not sequenced across translation
units.
Thanks,
Andrew
Post by li ning
I try to reproduce this problem with a simplified code. I am working on
this now...
class MyDb {
static MyDb& instance() {
static MyDb instance;
return instance;
}
void update_x(...) {
auto conn = _pool.acquire();
....
}
MyDb(host_str) : _pool(url(host_str)...{}
mongocxx::uri _uri;
mongocxx::pool _pool;
mongocxx::database _db;
}
and an singleton manager class 'IoServiceMgr' to manage boost io_service
class IoServiceMgr {
static IoServiceMgr & instance() {
static IoServiceMgr instance;
return instance;
}
boost::io_service& io() const {
random return one io
}
std::vector<shared_ptr<io_serivce>> _ios;
};
int main() {
mongocxx::instance mongo_driver;
MyDb::instance();
boost::asio::signal_set signals(IoServiceMgr::instance().io());
signals.add(SIGINT);
signals.add(SIGTERM);
signals.add(SIGABRT);
signals.async_wait(shutdown_server);
IoServiceMgr::instance().run();
}
void shutdown_server() {
do clean up work
}
It has 50% chance when I kill -2, the process crash in libmongoc code. In
the crash time, the process only have 2 thread, one is mongo thread, other
is the main thread
(main function already exit). When boost io service destruction, that
will cause my business logic object shared_ptr go destory
then in my business logic object destructor, will call
MyDb::instance().update_x(...). In this update_x _pool.acquire(), the
libmongoc code will cause an segv.
First, I thought the reason is mongo_driver got destroy before
'MyDb::instance().update_x(...)' call in destructor chain.
However I add static to mongo_driver, it is same. I guess this is related
to multi thread. I can not reproduce this in single thread code.
圚 2018幎4月3日星期二 UTC+8䞊午4:21:17acm写道
Post by 'Andrew Morrow' via mongodb-user
Hi -
Could you please post a complete and compilable example that reproduces
the behavior you are seeing? It will help greatly with diagnosis if I can
run the same code you are running.
Thanks,
Andrew
Post by li ning
(gdb) bt
#0 0x0000000000801866 in _mongoc_queue_pop_head ()
#1 0x00000000007ec949 in mongoc_client_pool_pop ()
#2 0x00000000007e320c in mongocxx::v_noabi::pool::acquire() ()
int main() {
mongocxx::instance mongo_driver;
MySingleton obj;
return 0;
}
When the process quit, the MySingleton desctor will call "auto conn =
_pool.acquire();", _pool is data member of MySingleton class.
During the destruction of MySingleton obj, the mongo_driver still
there, so it should not cause a segfault in mongocxx driver.
../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
#1 0x0000000000809065 in _mongoc_topology_run_background ()
#2 0x00007f34e3faf6ba in start_thread (arg=0x7f34e1d50700) at
pthread_create.c:333
#3 0x00007f34e323c3dd in clone () at
../sysdeps/unix/sysv/linux/x86_64/clone.S:109
So what's the problem? Can I use mongocxx like this?
mongocxx version 3.1.1
--
You received this message because you are subscribed to the Google
Groups "mongodb-user"
group.
https://docs.mongodb.com/manual/support/
---
You received this message because you are subscribed to the Google
Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit
https://groups.google.com/d/msgid/mongodb-user/d664b0ca-2423-4b15-90a0-5d13122e15f7%40googlegroups.com
<https://groups.google.com/d/msgid/mongodb-user/d664b0ca-2423-4b15-90a0-5d13122e15f7%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
https://docs.mongodb.com/manual/support/
---
You received this message because you are subscribed to the Google Groups
"mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an
<javascript:>.
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit
https://groups.google.com/d/msgid/mongodb-user/1ed58594-1a35-41dd-ae4a-70cc83bc8cb2%40googlegroups.com
<https://groups.google.com/d/msgid/mongodb-user/1ed58594-1a35-41dd-ae4a-70cc83bc8cb2%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: https://docs.mongodb.com/manual/support/
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+***@googlegroups.com.
To post to this group, send email to mongodb-***@googlegroups.com.
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/62a9eb4a-253a-48c8-8f61-ba4ae2d17574%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
'Andrew Morrow' via mongodb-user
2018-04-04 12:26:03 UTC
Permalink
Without working code to compile and run that reproduces the problem, it is
going to be very hard for me to debug this. I suggest that you carefully
trace the execution of your program with respect to things like
mongocxx::instance::[~]instance, mongoc_init and mongoc_cleanup, etc. These
sorts of startup/shutdown bugs are subtle, and there isn't really a better
way forward than to use the tools available (as suggested before) or work
in your debugger.

Also, I wanted to point out that your original problem description included
the fact that you attempted to obtain an element from the pool during
static destruction. I'd suggest that you reconsider this design. Presumably
you are obtaining a pool member because you intend to access the database,
but doing so at that point in program execution is strange. What if the
operation takes a long time? What if it fails? What is your error handling
strategy?

Thanks,
Andrew
Post by li ning
After I fix the construct order, the process will not crash, but hang
thread 1: (the main thread)
#0 __pthread_mutex_lock_full (mutex=0x4012150) at
../nptl/pthread_mutex_lock.c:343
#1 0x00000000007f3b33 in mongoc_client_pool_pop ()
#2 0x00000000007ea41c in mongocxx::v_noabi::pool::acquire() ()
x86_64/pthread_cond_timedwait.S:225
#1 0x0000000000810275 in _mongoc_topology_run_background ()
#2 0x00007f6352c226ba in start_thread (arg=0x7f63509c3700) at
pthread_create.c:333
#3 0x00007f6351eaf3dd in clone () at ../sysdeps/unix/sysv/linux/
x86_64/clone.S:109
It is not deadlock but seems thread 1 can not lock mutex=0x4012150. Why?
Any suggestion?
圚 2018幎4月3日星期二 UTC+8䞋午10:32:06acm写道
Post by 'Andrew Morrow' via mongodb-user
Hi
I'd strongly advise you to run your program under valgrind, or build it
with the various sanitizers (AddressSanitizer, UndefinedSanitizer,
ThreadSanitizer, etc.). My suspicion is that you have an issue with the
order of static destructors, which are not sequenced across translation
units.
Thanks,
Andrew
Post by li ning
I try to reproduce this problem with a simplified code. I am working on
this now...
class MyDb {
static MyDb& instance() {
static MyDb instance;
return instance;
}
void update_x(...) {
auto conn = _pool.acquire();
....
}
MyDb(host_str) : _pool(url(host_str)...{}
mongocxx::uri _uri;
mongocxx::pool _pool;
mongocxx::database _db;
}
and an singleton manager class 'IoServiceMgr' to manage boost io_service
class IoServiceMgr {
static IoServiceMgr & instance() {
static IoServiceMgr instance;
return instance;
}
boost::io_service& io() const {
random return one io
}
std::vector<shared_ptr<io_serivce>> _ios;
};
int main() {
mongocxx::instance mongo_driver;
MyDb::instance();
boost::asio::signal_set signals(IoServiceMgr::instance().io());
signals.add(SIGINT);
signals.add(SIGTERM);
signals.add(SIGABRT);
signals.async_wait(shutdown_server);
IoServiceMgr::instance().run();
}
void shutdown_server() {
do clean up work
}
It has 50% chance when I kill -2, the process crash in libmongoc code.
In the crash time, the process only have 2 thread, one is mongo thread,
other is the main thread
(main function already exit). When boost io service destruction, that
will cause my business logic object shared_ptr go destory
then in my business logic object destructor, will call
MyDb::instance().update_x(...). In this update_x _pool.acquire(), the
libmongoc code will cause an segv.
First, I thought the reason is mongo_driver got destroy before
'MyDb::instance().update_x(...)' call in destructor chain.
However I add static to mongo_driver, it is same. I guess this is
related to multi thread. I can not reproduce this in single thread code.
圚 2018幎4月3日星期二 UTC+8䞊午4:21:17acm写道
Post by 'Andrew Morrow' via mongodb-user
Hi -
Could you please post a complete and compilable example that reproduces
the behavior you are seeing? It will help greatly with diagnosis if I can
run the same code you are running.
Thanks,
Andrew
Post by li ning
(gdb) bt
#0 0x0000000000801866 in _mongoc_queue_pop_head ()
#1 0x00000000007ec949 in mongoc_client_pool_pop ()
#2 0x00000000007e320c in mongocxx::v_noabi::pool::acquire() ()
int main() {
mongocxx::instance mongo_driver;
MySingleton obj;
return 0;
}
When the process quit, the MySingleton desctor will call "auto conn =
_pool.acquire();", _pool is data member of MySingleton class.
During the destruction of MySingleton obj, the mongo_driver still
there, so it should not cause a segfault in mongocxx driver.
../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
#1 0x0000000000809065 in _mongoc_topology_run_background ()
#2 0x00007f34e3faf6ba in start_thread (arg=0x7f34e1d50700) at
pthread_create.c:333
#3 0x00007f34e323c3dd in clone () at ../sysdeps/unix/sysv/linux/x86
_64/clone.S:109
So what's the problem? Can I use mongocxx like this?
mongocxx version 3.1.1
--
You received this message because you are subscribed to the Google
Groups "mongodb-user"
group.
https://docs.mongodb.com/manual/support/
---
You received this message because you are subscribed to the Google
Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit
https://groups.google.com/d/msgid/mongodb-user/d664b0ca-2423
-4b15-90a0-5d13122e15f7%40googlegroups.com
<https://groups.google.com/d/msgid/mongodb-user/d664b0ca-2423-4b15-90a0-5d13122e15f7%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google
Groups "mongodb-user"
group.
https://docs.mongodb.com/manual/support/
---
You received this message because you are subscribed to the Google
Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/ms
gid/mongodb-user/1ed58594-1a35-41dd-ae4a-70cc83bc8cb2%40googlegroups.com
<https://groups.google.com/d/msgid/mongodb-user/1ed58594-1a35-41dd-ae4a-70cc83bc8cb2%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
https://docs.mongodb.com/manual/support/
---
You received this message because you are subscribed to the Google Groups
"mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/
msgid/mongodb-user/62a9eb4a-253a-48c8-8f61-ba4ae2d17574%40googlegroups.com
<https://groups.google.com/d/msgid/mongodb-user/62a9eb4a-253a-48c8-8f61-ba4ae2d17574%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: https://docs.mongodb.com/manual/support/
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+***@googlegroups.com.
To post to this group, send email to mongodb-***@googlegroups.com.
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/CAHX05qGKrnSaTu9_pi8VxZv0LqPbWNCmZKNkmw0hq3igGgHD4w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
li ning
2018-04-13 03:05:44 UTC
Permalink
They are all destruction order problem. I already fix it.
My program is online game server. When the server need to shutdown, I need
to disconnect all client connections and save player's data to db.
I do this by register the quit signal. When the signal receive, the clean
up function are called. However, some object persist action is done in its
desctor.

For your questions, the answer is:
1. all data saving operations are all simple, just given a _id, replace
whole data part to new one. The update frequency is about 30 second one
time.
2. due to 1, so it is very fast. and impossible to fail
So there is no error handling strategy, even all update action, we use non
ack mode (make blocking time small)
We doing this only require 1. mongodb is keep running. 2. the connection to
db is stable, if an reconnect happen, as long as quick, it is acceptable.

圚 2018幎4月4日星期䞉 UTC+8䞋午8:26:33acm写道
Post by 'Andrew Morrow' via mongodb-user
Without working code to compile and run that reproduces the problem, it is
going to be very hard for me to debug this. I suggest that you carefully
trace the execution of your program with respect to things like
mongocxx::instance::[~]instance, mongoc_init and mongoc_cleanup, etc. These
sorts of startup/shutdown bugs are subtle, and there isn't really a better
way forward than to use the tools available (as suggested before) or work
in your debugger.
Also, I wanted to point out that your original problem description
included the fact that you attempted to obtain an element from the pool
during static destruction. I'd suggest that you reconsider this design.
Presumably you are obtaining a pool member because you intend to access the
database, but doing so at that point in program execution is strange. What
if the operation takes a long time? What if it fails? What is your error
handling strategy?
Thanks,
Andrew
Post by li ning
After I fix the construct order, the process will not crash, but hang
thread 1: (the main thread)
#0 __pthread_mutex_lock_full (mutex=0x4012150) at
../nptl/pthread_mutex_lock.c:343
#1 0x00000000007f3b33 in mongoc_client_pool_pop ()
#2 0x00000000007ea41c in mongocxx::v_noabi::pool::acquire() ()
../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
#1 0x0000000000810275 in _mongoc_topology_run_background ()
#2 0x00007f6352c226ba in start_thread (arg=0x7f63509c3700) at
pthread_create.c:333
#3 0x00007f6351eaf3dd in clone () at
../sysdeps/unix/sysv/linux/x86_64/clone.S:109
It is not deadlock but seems thread 1 can not lock mutex=0x4012150. Why?
Any suggestion?
圚 2018幎4月3日星期二 UTC+8䞋午10:32:06acm写道
Post by 'Andrew Morrow' via mongodb-user
Hi
I'd strongly advise you to run your program under valgrind, or build it
with the various sanitizers (AddressSanitizer, UndefinedSanitizer,
ThreadSanitizer, etc.). My suspicion is that you have an issue with the
order of static destructors, which are not sequenced across translation
units.
Thanks,
Andrew
Post by li ning
I try to reproduce this problem with a simplified code. I am working on
this now...
class MyDb {
static MyDb& instance() {
static MyDb instance;
return instance;
}
void update_x(...) {
auto conn = _pool.acquire();
....
}
MyDb(host_str) : _pool(url(host_str)...{}
mongocxx::uri _uri;
mongocxx::pool _pool;
mongocxx::database _db;
}
and an singleton manager class 'IoServiceMgr' to manage boost io_service
class IoServiceMgr {
static IoServiceMgr & instance() {
static IoServiceMgr instance;
return instance;
}
boost::io_service& io() const {
random return one io
}
std::vector<shared_ptr<io_serivce>> _ios;
};
int main() {
mongocxx::instance mongo_driver;
MyDb::instance();
boost::asio::signal_set signals(IoServiceMgr::instance().io());
signals.add(SIGINT);
signals.add(SIGTERM);
signals.add(SIGABRT);
signals.async_wait(shutdown_server);
IoServiceMgr::instance().run();
}
void shutdown_server() {
do clean up work
}
It has 50% chance when I kill -2, the process crash in libmongoc code.
In the crash time, the process only have 2 thread, one is mongo thread,
other is the main thread
(main function already exit). When boost io service destruction, that
will cause my business logic object shared_ptr go destory
then in my business logic object destructor, will call
MyDb::instance().update_x(...). In this update_x _pool.acquire(), the
libmongoc code will cause an segv.
First, I thought the reason is mongo_driver got destroy before
'MyDb::instance().update_x(...)' call in destructor chain.
However I add static to mongo_driver, it is same. I guess this is
related to multi thread. I can not reproduce this in single thread code.
圚 2018幎4月3日星期二 UTC+8䞊午4:21:17acm写道
Post by 'Andrew Morrow' via mongodb-user
Hi -
Could you please post a complete and compilable example that
reproduces the behavior you are seeing? It will help greatly with diagnosis
if I can run the same code you are running.
Thanks,
Andrew
Post by li ning
(gdb) bt
#0 0x0000000000801866 in _mongoc_queue_pop_head ()
#1 0x00000000007ec949 in mongoc_client_pool_pop ()
#2 0x00000000007e320c in mongocxx::v_noabi::pool::acquire() ()
int main() {
mongocxx::instance mongo_driver;
MySingleton obj;
return 0;
}
When the process quit, the MySingleton desctor will call "auto conn
= _pool.acquire();", _pool is data member of MySingleton class.
During the destruction of MySingleton obj, the mongo_driver still
there, so it should not cause a segfault in mongocxx driver.
../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
#1 0x0000000000809065 in _mongoc_topology_run_background ()
#2 0x00007f34e3faf6ba in start_thread (arg=0x7f34e1d50700) at
pthread_create.c:333
#3 0x00007f34e323c3dd in clone () at
../sysdeps/unix/sysv/linux/x86_64/clone.S:109
So what's the problem? Can I use mongocxx like this?
mongocxx version 3.1.1
--
You received this message because you are subscribed to the Google
Groups "mongodb-user"
group.
https://docs.mongodb.com/manual/support/
---
You received this message because you are subscribed to the Google
Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it,
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit
https://groups.google.com/d/msgid/mongodb-user/d664b0ca-2423-4b15-90a0-5d13122e15f7%40googlegroups.com
<https://groups.google.com/d/msgid/mongodb-user/d664b0ca-2423-4b15-90a0-5d13122e15f7%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google
Groups "mongodb-user"
group.
https://docs.mongodb.com/manual/support/
---
You received this message because you are subscribed to the Google
Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit
https://groups.google.com/d/msgid/mongodb-user/1ed58594-1a35-41dd-ae4a-70cc83bc8cb2%40googlegroups.com
<https://groups.google.com/d/msgid/mongodb-user/1ed58594-1a35-41dd-ae4a-70cc83bc8cb2%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
https://docs.mongodb.com/manual/support/
---
You received this message because you are subscribed to the Google Groups
"mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an
<javascript:>.
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit
https://groups.google.com/d/msgid/mongodb-user/62a9eb4a-253a-48c8-8f61-ba4ae2d17574%40googlegroups.com
<https://groups.google.com/d/msgid/mongodb-user/62a9eb4a-253a-48c8-8f61-ba4ae2d17574%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: https://docs.mongodb.com/manual/support/
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+***@googlegroups.com.
To post to this group, send email to mongodb-***@googlegroups.com.
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/519e2b7f-5c35-4178-8db8-cbb6ac975cf9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Loading...