Discussion:
Mongodb bulk insert operation is crashing in incrementalSize during execute
(too old to reply)
Sumanta Dutta
2016-01-07 22:38:02 UTC
Permalink
Hi, I am trying to bulk insert large amount of documents (~222K). The
execute() crashes with the following. If size is the issue, what's the best
way of managing it?

#0 mongo::InsertWriteOperation::incrementalSize (this=0x14626a0) at
src/mongo/client/insert_write_operation.cpp:45
#1 0x000000000064da85 in mongo::CommandWriter::_fits (this=0xc3adc0,
builder=0x1960320, operation=<optimized out>) at
src/mongo/client/command_writer.cpp:116
#2 0x000000000064e5c5 in mongo::CommandWriter::write (this=0xc3adc0,
ns=..., write_operations=..., ordered=false, writeConcern=0xc13d60
<mongo::WriteConcern::acknowledged>, writeResult=0x7fffffffbe40)
at src/mongo/client/command_writer.cpp:53
#3 0x00000000005a8c7c in mongo::DBClientBase::_write (this=0xc47a00,
ns=..., writes=..., ordered=<optimized out>,
writeConcern=***@entry=0xc13d60
<mongo::WriteConcern::acknowledged>,
writeResult=***@entry=0xc13d60
<mongo::WriteConcern::acknowledged>) at src/mongo/client/dbclient.cpp:2060
#4 0x00000000005a5cf5 in mongo::BulkOperationBuilder::execute
(this=***@entry=0x7fffffffbdc0, writeConcern=0xc13d60
<mongo::WriteConcern::acknowledged>,
writeResult=0xc13d60 <mongo::WriteConcern::acknowledged>,
***@entry=0x7fffffffbe40) at
src/mongo/client/bulk_operation_builder.cpp:73
#5 0x0000000000506521 in copyData (mgr=..., fromTICollection=...,
fromTRCollection=..., toTICollection=..., toTRCollection=...)
at
/model/home/sdutta/build/oas/CLEAN_HEAD/cmo_oas_meth/dev/trancheinfo/tiutils.h:48
#6 0x0000000000507cfb in main (argc=<optimized out>, argv=0x7fffffffc788)
at ./ticopymongo.cpp:108


Thanks!
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+***@googlegroups.com.
To post to this group, send email to mongodb-***@googlegroups.com.
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/61ea8a44-d319-4534-8574-e169b064bb54%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Andrew Morrow
2016-01-07 22:45:40 UTC
Permalink
Hi -

I'm guessing you are using the C++ driver. However, there is a lot more
information we will need to help. Could you please provide some of the
information requested here:
https://github.com/mongodb/mongo-cxx-driver/wiki/Download-and-Compile-the-Legacy-Driver#how-to-ask-for-help

Thanks,
Andrew
Post by Sumanta Dutta
Hi, I am trying to bulk insert large amount of documents (~222K). The
execute() crashes with the following. If size is the issue, what's the best
way of managing it?
#0 mongo::InsertWriteOperation::incrementalSize (this=0x14626a0) at
src/mongo/client/insert_write_operation.cpp:45
#1 0x000000000064da85 in mongo::CommandWriter::_fits (this=0xc3adc0,
builder=0x1960320, operation=<optimized out>) at
src/mongo/client/command_writer.cpp:116
#2 0x000000000064e5c5 in mongo::CommandWriter::write (this=0xc3adc0,
ns=..., write_operations=..., ordered=false, writeConcern=0xc13d60
<mongo::WriteConcern::acknowledged>, writeResult=0x7fffffffbe40)
at src/mongo/client/command_writer.cpp:53
#3 0x00000000005a8c7c in mongo::DBClientBase::_write (this=0xc47a00,
<mongo::WriteConcern::acknowledged>,
<mongo::WriteConcern::acknowledged>) at src/mongo/client/dbclient.cpp:2060
#4 0x00000000005a5cf5 in mongo::BulkOperationBuilder::execute
<mongo::WriteConcern::acknowledged>,
writeResult=0xc13d60 <mongo::WriteConcern::acknowledged>,
src/mongo/client/bulk_operation_builder.cpp:73
#5 0x0000000000506521 in copyData (mgr=..., fromTICollection=...,
fromTRCollection=..., toTICollection=..., toTRCollection=...)
at
/model/home/sdutta/build/oas/CLEAN_HEAD/cmo_oas_meth/dev/trancheinfo/tiutils.h:48
#6 0x0000000000507cfb in main (argc=<optimized out>, argv=0x7fffffffc788)
at ./ticopymongo.cpp:108
Thanks!
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups
"mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit
https://groups.google.com/d/msgid/mongodb-user/61ea8a44-d319-4534-8574-e169b064bb54%40googlegroups.com
<https://groups.google.com/d/msgid/mongodb-user/61ea8a44-d319-4534-8574-e169b064bb54%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+***@googlegroups.com.
To post to this group, send email to mongodb-***@googlegroups.com.
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/CAHX05qGMAQAT8o8ph0NMxYx%2B7gzBJngx9GY0retZcUZzV6cjew%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Sumanta Dutta
2016-01-07 23:16:59 UTC
Permalink
- The version of the driver you are trying to build (branch or tag).
- C++, *legacy-1.x*
- Host OS, version, and architecture.
- *Redhat Linux 5.2 64-bit x86*
- C++ Compiler and version.
- *GCC 4.9.1*
- Boost version.
- *boost 1.56*
- How boost was built or installed.
- *built from source*
- Most likely used standard instructions (./bootstrap.sh & ./b2
install). I don't have full detail at this time.

Thanks!
Post by Sumanta Dutta
Hi, I am trying to bulk insert large amount of documents (~222K). The
execute() crashes with the following. If size is the issue, what's the best
way of managing it?
#0 mongo::InsertWriteOperation::incrementalSize (this=0x14626a0) at
src/mongo/client/insert_write_operation.cpp:45
#1 0x000000000064da85 in mongo::CommandWriter::_fits (this=0xc3adc0,
builder=0x1960320, operation=<optimized out>) at
src/mongo/client/command_writer.cpp:116
#2 0x000000000064e5c5 in mongo::CommandWriter::write (this=0xc3adc0,
ns=..., write_operations=..., ordered=false, writeConcern=0xc13d60
<mongo::WriteConcern::acknowledged>, writeResult=0x7fffffffbe40)
at src/mongo/client/command_writer.cpp:53
#3 0x00000000005a8c7c in mongo::DBClientBase::_write (this=0xc47a00,
ns=..., writes=..., ordered=<optimized out>,
<mongo::WriteConcern::acknowledged>,
<mongo::WriteConcern::acknowledged>) at src/mongo/client/dbclient.cpp:2060
#4 0x00000000005a5cf5 in mongo::BulkOperationBuilder::execute
<mongo::WriteConcern::acknowledged>,
writeResult=0xc13d60 <mongo::WriteConcern::acknowledged>,
src/mongo/client/bulk_operation_builder.cpp:73
#5 0x0000000000506521 in copyData (mgr=..., fromTICollection=...,
fromTRCollection=..., toTICollection=..., toTRCollection=...)
at
/model/home/sdutta/build/oas/CLEAN_HEAD/cmo_oas_meth/dev/trancheinfo/tiutils.h:48
#6 0x0000000000507cfb in main (argc=<optimized out>, argv=0x7fffffffc788)
at ./ticopymongo.cpp:108
Thanks!
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+***@googlegroups.com.
To post to this group, send email to mongodb-***@googlegroups.com.
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/d4d6117b-d352-4fa0-b8a9-a138acf8cf3b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Andrew Morrow
2016-01-08 13:26:39 UTC
Permalink
Hi -

Thanks for providing the additional details. Could you please clarify
exactly what legacy driver version you are using? It will help us either
eliminate or bring into consideration bugs that were found or fixed in
various point releases. Also, when you say you are inserting "222k", do you
mean that you are inserting a total of 222kb of data, or 222k documents?
Finally, do you have any additional crash details beyond the stack trace
(e.g. the signal it died with, etc.)? Is the crash consistently
reproducible?

Thanks,
Andrew
Post by Sumanta Dutta
- The version of the driver you are trying to build (branch or tag).
- C++, *legacy-1.x*
- Host OS, version, and architecture.
- *Redhat Linux 5.2 64-bit x86*
- C++ Compiler and version.
- *GCC 4.9.1*
- Boost version.
- *boost 1.56*
- How boost was built or installed.
- *built from source*
- Most likely used standard instructions (./bootstrap.sh & ./b2
install). I don't have full detail at this time.
Thanks!
Post by Sumanta Dutta
Hi, I am trying to bulk insert large amount of documents (~222K). The
execute() crashes with the following. If size is the issue, what's the best
way of managing it?
#0 mongo::InsertWriteOperation::incrementalSize (this=0x14626a0) at
src/mongo/client/insert_write_operation.cpp:45
#1 0x000000000064da85 in mongo::CommandWriter::_fits (this=0xc3adc0,
builder=0x1960320, operation=<optimized out>) at
src/mongo/client/command_writer.cpp:116
#2 0x000000000064e5c5 in mongo::CommandWriter::write (this=0xc3adc0,
ns=..., write_operations=..., ordered=false, writeConcern=0xc13d60
<mongo::WriteConcern::acknowledged>, writeResult=0x7fffffffbe40)
at src/mongo/client/command_writer.cpp:53
#3 0x00000000005a8c7c in mongo::DBClientBase::_write (this=0xc47a00,
<mongo::WriteConcern::acknowledged>,
<mongo::WriteConcern::acknowledged>) at src/mongo/client/dbclient.cpp:2060
#4 0x00000000005a5cf5 in mongo::BulkOperationBuilder::execute
<mongo::WriteConcern::acknowledged>,
writeResult=0xc13d60 <mongo::WriteConcern::acknowledged>,
src/mongo/client/bulk_operation_builder.cpp:73
#5 0x0000000000506521 in copyData (mgr=..., fromTICollection=...,
fromTRCollection=..., toTICollection=..., toTRCollection=...)
at
/model/home/sdutta/build/oas/CLEAN_HEAD/cmo_oas_meth/dev/trancheinfo/tiutils.h:48
#6 0x0000000000507cfb in main (argc=<optimized out>,
argv=0x7fffffffc788) at ./ticopymongo.cpp:108
Thanks!
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups
"mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit
https://groups.google.com/d/msgid/mongodb-user/d4d6117b-d352-4fa0-b8a9-a138acf8cf3b%40googlegroups.com
<https://groups.google.com/d/msgid/mongodb-user/d4d6117b-d352-4fa0-b8a9-a138acf8cf3b%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+***@googlegroups.com.
To post to this group, send email to mongodb-***@googlegroups.com.
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/CAHX05qHoU9JppMWAQxu8m7n-7BBJPAO6-739CP9UGksiFHGpxw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Sumanta Dutta
2016-01-08 13:44:49 UTC
Permalink
Hi Andrew,

The best info I have is 1.0.5 which is in the mongo/version.h file of
client lib.

I am trying to insert ~222,000 documents. In fact, I tried 1000 documents
at a time and got the same for the very first batch. So I am not sure if
it's size related. The documents are large but we have no problem inserting
individually.

The issue is consistently reproducible.

I am not in front of my desk at present. I will send you another update
later today with any additional crash related info.

Thanks,
Sumanta
Post by Andrew Morrow
Hi -
Thanks for providing the additional details. Could you please clarify
exactly what legacy driver version you are using? It will help us either
eliminate or bring into consideration bugs that were found or fixed in
various point releases. Also, when you say you are inserting "222k", do you
mean that you are inserting a total of 222kb of data, or 222k documents?
Finally, do you have any additional crash details beyond the stack trace
(e.g. the signal it died with, etc.)? Is the crash consistently
reproducible?
Thanks,
Andrew
Post by Sumanta Dutta
- The version of the driver you are trying to build (branch or tag).
- C++, *legacy-1.x*
- Host OS, version, and architecture.
- *Redhat Linux 5.2 64-bit x86*
- C++ Compiler and version.
- *GCC 4.9.1*
- Boost version.
- *boost 1.56*
- How boost was built or installed.
- *built from source*
- Most likely used standard instructions (./bootstrap.sh & ./b2
install). I don't have full detail at this time.
Thanks!
Post by Sumanta Dutta
Hi, I am trying to bulk insert large amount of documents (~222K). The
execute() crashes with the following. If size is the issue, what's the best
way of managing it?
#0 mongo::InsertWriteOperation::incrementalSize (this=0x14626a0) at
src/mongo/client/insert_write_operation.cpp:45
#1 0x000000000064da85 in mongo::CommandWriter::_fits (this=0xc3adc0,
builder=0x1960320, operation=<optimized out>) at
src/mongo/client/command_writer.cpp:116
#2 0x000000000064e5c5 in mongo::CommandWriter::write (this=0xc3adc0,
ns=..., write_operations=..., ordered=false, writeConcern=0xc13d60
<mongo::WriteConcern::acknowledged>, writeResult=0x7fffffffbe40)
at src/mongo/client/command_writer.cpp:53
#3 0x00000000005a8c7c in mongo::DBClientBase::_write (this=0xc47a00,
<mongo::WriteConcern::acknowledged>,
<mongo::WriteConcern::acknowledged>) at src/mongo/client/dbclient.cpp:2060
#4 0x00000000005a5cf5 in mongo::BulkOperationBuilder::execute
<mongo::WriteConcern::acknowledged>,
writeResult=0xc13d60 <mongo::WriteConcern::acknowledged>,
src/mongo/client/bulk_operation_builder.cpp:73
#5 0x0000000000506521 in copyData (mgr=..., fromTICollection=...,
fromTRCollection=..., toTICollection=..., toTRCollection=...)
at
/model/home/sdutta/build/oas/CLEAN_HEAD/cmo_oas_meth/dev/trancheinfo/tiutils.h:48
#6 0x0000000000507cfb in main (argc=<optimized out>,
argv=0x7fffffffc788) at ./ticopymongo.cpp:108
Thanks!
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups
"mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit
https://groups.google.com/d/msgid/mongodb-user/d4d6117b-d352-4fa0-b8a9-a138acf8cf3b%40googlegroups.com
<https://groups.google.com/d/msgid/mongodb-user/d4d6117b-d352-4fa0-b8a9-a138acf8cf3b%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to a topic in the
Google Groups "mongodb-user" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/mongodb-user/3_bW4pOQraI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit
https://groups.google.com/d/msgid/mongodb-user/CAHX05qHoU9JppMWAQxu8m7n-7BBJPAO6-739CP9UGksiFHGpxw%40mail.gmail.com
<https://groups.google.com/d/msgid/mongodb-user/CAHX05qHoU9JppMWAQxu8m7n-7BBJPAO6-739CP9UGksiFHGpxw%40mail.gmail.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+***@googlegroups.com.
To post to this group, send email to mongodb-***@googlegroups.com.
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/CAGNFCEzQaApLJjLfS5x42BUpME%2Bvf8grE_xek62v1YfiJVO1bw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Sumanta Dutta
2016-01-08 14:59:27 UTC
Permalink
It's crashing due to SIGSEGV.

default[FATAL] 01-08-2016 09-57-29.009040 void
el::base::debug::logCrashReason(int, bool, el::Level, const char*) - CRASH
HANDLED; Application has crashed due to [SIGSEGV] signal
======= Backtrace: =========
[1] ./dev/OAS/bin/ticopymongo [0x535136]
[2] /lib64/libc.so.6 [0x3115a301b0]
[3]
./dev/OAS/bin/ticopymongo:mongo::InsertWriteOperation::incrementalSize()
const+0x4 [0x5e47c4]
[4]
./dev/OAS/bin/ticopymongo:mongo::CommandWriter::_fits(mongo::BSONArrayBuilder*,
mongo::WriteOperation*)+0x15 [0x64dad5]
[5]
./dev/OAS/bin/ticopymongo:mongo::CommandWriter::write(mongo::StringData
const&, std::vector<mongo::WriteOperation*,
std::allocator<mongo::WriteOperation*> > const&, bool, mongo::WriteConcern
const*, mongo::WriteResult*)+0x245 [0x64e615]
[6] ./dev/OAS/bin/ticopymongo:mongo::DBClientBase::_write(std::string
const&, std::vector<mongo::WriteOperation*,
std::allocator<mongo::WriteOperation*> > const&, bool, mongo::WriteConcern
const*, mongo::WriteResult*)+0x7c [0x5a8ccc]
[7] ./dev/OAS/bin/ticopymongo:copyData(MongoDataManagerImpl&,
std::string const&, std::string const&, std::string const&, std::string
const&)+0x155 [0x50653b]
[8] ./dev/OAS/bin/ticopymongo:main+0x1160 [0x507e34]
[9] /lib64/libc.so.6:__libc_start_main+0xf4 [0x3115a1d8b4]
[10] ./dev/OAS/bin/ticopymongo [0x5062a9]
Post by Sumanta Dutta
Hi, I am trying to bulk insert large amount of documents (~222K). The
execute() crashes with the following. If size is the issue, what's the best
way of managing it?
#0 mongo::InsertWriteOperation::incrementalSize (this=0x14626a0) at
src/mongo/client/insert_write_operation.cpp:45
#1 0x000000000064da85 in mongo::CommandWriter::_fits (this=0xc3adc0,
builder=0x1960320, operation=<optimized out>) at
src/mongo/client/command_writer.cpp:116
#2 0x000000000064e5c5 in mongo::CommandWriter::write (this=0xc3adc0,
ns=..., write_operations=..., ordered=false, writeConcern=0xc13d60
<mongo::WriteConcern::acknowledged>, writeResult=0x7fffffffbe40)
at src/mongo/client/command_writer.cpp:53
#3 0x00000000005a8c7c in mongo::DBClientBase::_write (this=0xc47a00,
ns=..., writes=..., ordered=<optimized out>,
<mongo::WriteConcern::acknowledged>,
<mongo::WriteConcern::acknowledged>) at src/mongo/client/dbclient.cpp:2060
#4 0x00000000005a5cf5 in mongo::BulkOperationBuilder::execute
<mongo::WriteConcern::acknowledged>,
writeResult=0xc13d60 <mongo::WriteConcern::acknowledged>,
src/mongo/client/bulk_operation_builder.cpp:73
#5 0x0000000000506521 in copyData (mgr=..., fromTICollection=...,
fromTRCollection=..., toTICollection=..., toTRCollection=...)
at
/model/home/sdutta/build/oas/CLEAN_HEAD/cmo_oas_meth/dev/trancheinfo/tiutils.h:48
#6 0x0000000000507cfb in main (argc=<optimized out>, argv=0x7fffffffc788)
at ./ticopymongo.cpp:108
Thanks!
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+***@googlegroups.com.
To post to this group, send email to mongodb-***@googlegroups.com.
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/a05c5df2-6c7d-42f3-8020-464558ca1ed9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Andrew Morrow
2016-01-08 17:43:56 UTC
Permalink
What happens if you run this workload under address sanitizer or valgrind?
Or, can you provide a small, compilable, reproducing example that we can
test with locally?

Thanks,
Andrew
Post by Sumanta Dutta
It's crashing due to SIGSEGV.
default[FATAL] 01-08-2016 09-57-29.009040 void
el::base::debug::logCrashReason(int, bool, el::Level, const char*) - CRASH
HANDLED; Application has crashed due to [SIGSEGV] signal
======= Backtrace: =========
[1] ./dev/OAS/bin/ticopymongo [0x535136]
[2] /lib64/libc.so.6 [0x3115a301b0]
[3]
./dev/OAS/bin/ticopymongo:mongo::InsertWriteOperation::incrementalSize()
const+0x4 [0x5e47c4]
[4]
./dev/OAS/bin/ticopymongo:mongo::CommandWriter::_fits(mongo::BSONArrayBuilder*,
mongo::WriteOperation*)+0x15 [0x64dad5]
[5]
./dev/OAS/bin/ticopymongo:mongo::CommandWriter::write(mongo::StringData
const&, std::vector<mongo::WriteOperation*,
std::allocator<mongo::WriteOperation*> > const&, bool, mongo::WriteConcern
const*, mongo::WriteResult*)+0x245 [0x64e615]
[6] ./dev/OAS/bin/ticopymongo:mongo::DBClientBase::_write(std::string
const&, std::vector<mongo::WriteOperation*,
std::allocator<mongo::WriteOperation*> > const&, bool, mongo::WriteConcern
const*, mongo::WriteResult*)+0x7c [0x5a8ccc]
[7] ./dev/OAS/bin/ticopymongo:copyData(MongoDataManagerImpl&,
std::string const&, std::string const&, std::string const&, std::string
const&)+0x155 [0x50653b]
[8] ./dev/OAS/bin/ticopymongo:main+0x1160 [0x507e34]
[9] /lib64/libc.so.6:__libc_start_main+0xf4 [0x3115a1d8b4]
[10] ./dev/OAS/bin/ticopymongo [0x5062a9]
Post by Sumanta Dutta
Hi, I am trying to bulk insert large amount of documents (~222K). The
execute() crashes with the following. If size is the issue, what's the best
way of managing it?
#0 mongo::InsertWriteOperation::incrementalSize (this=0x14626a0) at
src/mongo/client/insert_write_operation.cpp:45
#1 0x000000000064da85 in mongo::CommandWriter::_fits (this=0xc3adc0,
builder=0x1960320, operation=<optimized out>) at
src/mongo/client/command_writer.cpp:116
#2 0x000000000064e5c5 in mongo::CommandWriter::write (this=0xc3adc0,
ns=..., write_operations=..., ordered=false, writeConcern=0xc13d60
<mongo::WriteConcern::acknowledged>, writeResult=0x7fffffffbe40)
at src/mongo/client/command_writer.cpp:53
#3 0x00000000005a8c7c in mongo::DBClientBase::_write (this=0xc47a00,
<mongo::WriteConcern::acknowledged>,
<mongo::WriteConcern::acknowledged>) at src/mongo/client/dbclient.cpp:2060
#4 0x00000000005a5cf5 in mongo::BulkOperationBuilder::execute
<mongo::WriteConcern::acknowledged>,
writeResult=0xc13d60 <mongo::WriteConcern::acknowledged>,
src/mongo/client/bulk_operation_builder.cpp:73
#5 0x0000000000506521 in copyData (mgr=..., fromTICollection=...,
fromTRCollection=..., toTICollection=..., toTRCollection=...)
at
/model/home/sdutta/build/oas/CLEAN_HEAD/cmo_oas_meth/dev/trancheinfo/tiutils.h:48
#6 0x0000000000507cfb in main (argc=<optimized out>,
argv=0x7fffffffc788) at ./ticopymongo.cpp:108
Thanks!
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups
"mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit
https://groups.google.com/d/msgid/mongodb-user/a05c5df2-6c7d-42f3-8020-464558ca1ed9%40googlegroups.com
<https://groups.google.com/d/msgid/mongodb-user/a05c5df2-6c7d-42f3-8020-464558ca1ed9%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+***@googlegroups.com.
To post to this group, send email to mongodb-***@googlegroups.com.
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/CAHX05qGg5WckJDEfyTGWO737besz0DzJLoBiicyJ7t7mcU_z1Q%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Tim Hawkins
2016-01-08 18:02:58 UTC
Permalink
Hi sumanta

Re redhat 5.2.

That version of rh linux is 17 years old, and ended its extended support
update cycle in march 2010.

Its using a very old version of glibc amd many other components such as gcc
etc are well past thier sell by date.

Is there any reason why you are using such an old version of rh, as that
may be contributing to the issues you are experiencing. I belive its using
glibc 2.0.7 and gcc 2.7.2.3

See: http://distrowatch.com/table.php?distribution=redhat
Post by Sumanta Dutta
It's crashing due to SIGSEGV.
default[FATAL] 01-08-2016 09-57-29.009040 void
el::base::debug::logCrashReason(int, bool, el::Level, const char*) - CRASH
HANDLED; Application has crashed due to [SIGSEGV] signal
======= Backtrace: =========
[1] ./dev/OAS/bin/ticopymongo [0x535136]
[2] /lib64/libc.so.6 [0x3115a301b0]
[3]
./dev/OAS/bin/ticopymongo:mongo::InsertWriteOperation::incrementalSize()
const+0x4 [0x5e47c4]
[4]
./dev/OAS/bin/ticopymongo:mongo::CommandWriter::_fits(mongo::BSONArrayBuilder*,
mongo::WriteOperation*)+0x15 [0x64dad5]
[5]
./dev/OAS/bin/ticopymongo:mongo::CommandWriter::write(mongo::StringData
const&, std::vector<mongo::WriteOperation*,
std::allocator<mongo::WriteOperation*> > const&, bool, mongo::WriteConcern
const*, mongo::WriteResult*)+0x245 [0x64e615]
[6] ./dev/OAS/bin/ticopymongo:mongo::DBClientBase::_write(std::string
const&, std::vector<mongo::WriteOperation*,
std::allocator<mongo::WriteOperation*> > const&, bool, mongo::WriteConcern
const*, mongo::WriteResult*)+0x7c [0x5a8ccc]
[7] ./dev/OAS/bin/ticopymongo:copyData(MongoDataManagerImpl&,
std::string const&, std::string const&, std::string const&, std::string
const&)+0x155 [0x50653b]
[8] ./dev/OAS/bin/ticopymongo:main+0x1160 [0x507e34]
[9] /lib64/libc.so.6:__libc_start_main+0xf4 [0x3115a1d8b4]
[10] ./dev/OAS/bin/ticopymongo [0x5062a9]
Post by Sumanta Dutta
Hi, I am trying to bulk insert large amount of documents (~222K). The
execute() crashes with the following. If size is the issue, what's the best
way of managing it?
#0 mongo::InsertWriteOperation::incrementalSize (this=0x14626a0) at
src/mongo/client/insert_write_operation.cpp:45
#1 0x000000000064da85 in mongo::CommandWriter::_fits (this=0xc3adc0,
builder=0x1960320, operation=<optimized out>) at
src/mongo/client/command_writer.cpp:116
#2 0x000000000064e5c5 in mongo::CommandWriter::write (this=0xc3adc0,
ns=..., write_operations=..., ordered=false, writeConcern=0xc13d60
<mongo::WriteConcern::acknowledged>, writeResult=0x7fffffffbe40)
at src/mongo/client/command_writer.cpp:53
#3 0x00000000005a8c7c in mongo::DBClientBase::_write (this=0xc47a00,
<mongo::WriteConcern::acknowledged>,
<mongo::WriteConcern::acknowledged>) at src/mongo/client/dbclient.cpp:2060
#4 0x00000000005a5cf5 in mongo::BulkOperationBuilder::execute
<mongo::WriteConcern::acknowledged>,
writeResult=0xc13d60 <mongo::WriteConcern::acknowledged>,
src/mongo/client/bulk_operation_builder.cpp:73
#5 0x0000000000506521 in copyData (mgr=..., fromTICollection=...,
fromTRCollection=..., toTICollection=..., toTRCollection=...)
at
/model/home/sdutta/build/oas/CLEAN_HEAD/cmo_oas_meth/dev/trancheinfo/tiutils.h:48
#6 0x0000000000507cfb in main (argc=<optimized out>,
argv=0x7fffffffc788) at ./ticopymongo.cpp:108
Thanks!
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups
"mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit
https://groups.google.com/d/msgid/mongodb-user/a05c5df2-6c7d-42f3-8020-464558ca1ed9%40googlegroups.com
<https://groups.google.com/d/msgid/mongodb-user/a05c5df2-6c7d-42f3-8020-464558ca1ed9%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+***@googlegroups.com.
To post to this group, send email to mongodb-***@googlegroups.com.
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/CAEbsasQxya%2B8qH3o22vmRdxaVXgSj6Jy4oZYK%2BzwvCP%3DkgKtqw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Sumanta Dutta
2016-01-08 18:28:09 UTC
Permalink
Hi Guys, thanks for your pointers but the problem was in my code. I didn't
copy the BSONObj returned by the cursor before passing it
to mongo::BulkOperationBuilder for insert.

Regarding Redhat 5.2, we desperately want to upgrade but currently not
possible due to various internal constraints. We have built GCC 4.9.1 in
house and using C++14. So it's not that bad :)

Thanks again!
Sumanta
Post by Sumanta Dutta
Hi, I am trying to bulk insert large amount of documents (~222K). The
execute() crashes with the following. If size is the issue, what's the best
way of managing it?
#0 mongo::InsertWriteOperation::incrementalSize (this=0x14626a0) at
src/mongo/client/insert_write_operation.cpp:45
#1 0x000000000064da85 in mongo::CommandWriter::_fits (this=0xc3adc0,
builder=0x1960320, operation=<optimized out>) at
src/mongo/client/command_writer.cpp:116
#2 0x000000000064e5c5 in mongo::CommandWriter::write (this=0xc3adc0,
ns=..., write_operations=..., ordered=false, writeConcern=0xc13d60
<mongo::WriteConcern::acknowledged>, writeResult=0x7fffffffbe40)
at src/mongo/client/command_writer.cpp:53
#3 0x00000000005a8c7c in mongo::DBClientBase::_write (this=0xc47a00,
ns=..., writes=..., ordered=<optimized out>,
<mongo::WriteConcern::acknowledged>,
<mongo::WriteConcern::acknowledged>) at src/mongo/client/dbclient.cpp:2060
#4 0x00000000005a5cf5 in mongo::BulkOperationBuilder::execute
<mongo::WriteConcern::acknowledged>,
writeResult=0xc13d60 <mongo::WriteConcern::acknowledged>,
src/mongo/client/bulk_operation_builder.cpp:73
#5 0x0000000000506521 in copyData (mgr=..., fromTICollection=...,
fromTRCollection=..., toTICollection=..., toTRCollection=...)
at
/model/home/sdutta/build/oas/CLEAN_HEAD/cmo_oas_meth/dev/trancheinfo/tiutils.h:48
#6 0x0000000000507cfb in main (argc=<optimized out>, argv=0x7fffffffc788)
at ./ticopymongo.cpp:108
Thanks!
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+***@googlegroups.com.
To post to this group, send email to mongodb-***@googlegroups.com.
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/9aa522d8-cf0e-42a1-966d-454df3a6af2b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Andrew Morrow
2016-01-08 18:34:21 UTC
Permalink
Hi Sumanta -

Happy to hear you were able to get it working. In the future, valgrind or
ASAN will definitely help catch that sort of issue. Also, if you are using
C++14, you may be interested in the new C++11 driver that we are
developing. You can find a pre-release version of it on the master branch
of the C++ driver repository:
https://github.com/mongodb/mongo-cxx-driver/tree/master

We are hoping to issue the first release candidate for the new C++11 driver
within the next few days.

Thanks,
Andrew
Post by Sumanta Dutta
Hi Guys, thanks for your pointers but the problem was in my code. I didn't
copy the BSONObj returned by the cursor before passing it
to mongo::BulkOperationBuilder for insert.
Regarding Redhat 5.2, we desperately want to upgrade but currently not
possible due to various internal constraints. We have built GCC 4.9.1 in
house and using C++14. So it's not that bad :)
Thanks again!
Sumanta
Post by Sumanta Dutta
Hi, I am trying to bulk insert large amount of documents (~222K). The
execute() crashes with the following. If size is the issue, what's the best
way of managing it?
#0 mongo::InsertWriteOperation::incrementalSize (this=0x14626a0) at
src/mongo/client/insert_write_operation.cpp:45
#1 0x000000000064da85 in mongo::CommandWriter::_fits (this=0xc3adc0,
builder=0x1960320, operation=<optimized out>) at
src/mongo/client/command_writer.cpp:116
#2 0x000000000064e5c5 in mongo::CommandWriter::write (this=0xc3adc0,
ns=..., write_operations=..., ordered=false, writeConcern=0xc13d60
<mongo::WriteConcern::acknowledged>, writeResult=0x7fffffffbe40)
at src/mongo/client/command_writer.cpp:53
#3 0x00000000005a8c7c in mongo::DBClientBase::_write (this=0xc47a00,
<mongo::WriteConcern::acknowledged>,
<mongo::WriteConcern::acknowledged>) at src/mongo/client/dbclient.cpp:2060
#4 0x00000000005a5cf5 in mongo::BulkOperationBuilder::execute
<mongo::WriteConcern::acknowledged>,
writeResult=0xc13d60 <mongo::WriteConcern::acknowledged>,
src/mongo/client/bulk_operation_builder.cpp:73
#5 0x0000000000506521 in copyData (mgr=..., fromTICollection=...,
fromTRCollection=..., toTICollection=..., toTRCollection=...)
at
/model/home/sdutta/build/oas/CLEAN_HEAD/cmo_oas_meth/dev/trancheinfo/tiutils.h:48
#6 0x0000000000507cfb in main (argc=<optimized out>,
argv=0x7fffffffc788) at ./ticopymongo.cpp:108
Thanks!
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups
"mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit
https://groups.google.com/d/msgid/mongodb-user/9aa522d8-cf0e-42a1-966d-454df3a6af2b%40googlegroups.com
<https://groups.google.com/d/msgid/mongodb-user/9aa522d8-cf0e-42a1-966d-454df3a6af2b%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+***@googlegroups.com.
To post to this group, send email to mongodb-***@googlegroups.com.
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/CAHX05qG0Yyj7GAfD6yvWH7kP2raK3vf_3KaSQytW3J%2BjNqejoQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Loading...