Discussion:
mongo-hadoop hive insert error
Josh Morris
2014-06-27 18:19:10 UTC
Permalink
I get an error (see below) while trying to insert data into a hive table
that is connected to a Mongodb table. I'm able to create, drop and query
the table just fine e.g. if I insert data into the Mongo collection first
and perform a select from the Hive cli I see the Mongo data.

FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.mr.MapRedTask

I'm using HDP 2.1 set up across 3 ec2 instances, Mongodb 2.6 on another ec2
instance.

So far I haven't been able to find more details about the error. Any help
would be appreciated.

*Details*

hive> CREATE TABLE individuals
(
id INT,
name STRING,
age INT
)
STORED BY 'com.mongodb.hadoop.hive.MongoStorageHandler'
TBLPROPERTIES('mongo.uri'='mongodb://testuser:test-***@public.gmane.org:27017/test.persons2');
OK
Time taken: 0.294 seconds
hive> select * from individuals;
OK
Time taken: 0.351 seconds
hive> insert into table individuals select 1, 'name column', 5 from
some_table limit 1;
Query ID = hdfs_20140627134040_5a3db212-e6e1-42cf-a6cc-0a0078ef3bb3
Total jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
set mapreduce.job.reduces=<number>
Starting Job = job_1403731258511_0038, Tracking URL =
http://192.168.2.40:8088/proxy/application_1403731258511_0038/
Kill Command = /usr/lib/hadoop/bin/hadoop job -kill job_1403731258511_0038
Hadoop job information for Stage-0: number of mappers: 0; number of
reducers: 0
2014-06-27 13:40:16,335 Stage-0 map = 0%, reduce = 0%
Ended Job = job_1403731258511_0038 with errors
*Error during job, obtaining debugging information...*
*FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.mr.MapRedTask*
MapReduce Jobs Launched:
Job 0: HDFS Read: 0 HDFS Write: 0 FAIL
Total MapReduce CPU Time Spent: 0 msec
hive>
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
To post to this group, send email to mongodb-user-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/5e188766-e0a1-4cd1-a776-428f7ec3d20c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Justin Lee
2014-06-27 19:01:03 UTC
Permalink
There's nothing more in the logs?
Post by Josh Morris
I get an error (see below) while trying to insert data into a hive table
that is connected to a Mongodb table. I'm able to create, drop and query
the table just fine e.g. if I insert data into the Mongo collection first
and perform a select from the Hive cli I see the Mongo data.
FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.mr.MapRedTask
I'm using HDP 2.1 set up across 3 ec2 instances, Mongodb 2.6 on another
ec2 instance.
So far I haven't been able to find more details about the error. Any help
would be appreciated.
*Details*
hive> CREATE TABLE individuals
(
id INT,
name STRING,
age INT
)
STORED BY 'com.mongodb.hadoop.hive.MongoStorageHandler'
TBLPROPERTIES('mongo.uri'='mongodb://
OK
Time taken: 0.294 seconds
hive> select * from individuals;
OK
Time taken: 0.351 seconds
hive> insert into table individuals select 1, 'name column', 5 from
some_table limit 1;
Query ID = hdfs_20140627134040_5a3db212-e6e1-42cf-a6cc-0a0078ef3bb3
Total jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
set hive.exec.reducers.bytes.per.reducer=<number>
set hive.exec.reducers.max=<number>
set mapreduce.job.reduces=<number>
Starting Job = job_1403731258511_0038, Tracking URL =
http://192.168.2.40:8088/proxy/application_1403731258511_0038/
Kill Command = /usr/lib/hadoop/bin/hadoop job -kill job_1403731258511_0038
Hadoop job information for Stage-0: number of mappers: 0; number of
reducers: 0
2014-06-27 13:40:16,335 Stage-0 map = 0%, reduce = 0%
Ended Job = job_1403731258511_0038 with errors
*Error during job, obtaining debugging information...*
*FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.mr.MapRedTask*
Job 0: HDFS Read: 0 HDFS Write: 0 FAIL
Total MapReduce CPU Time Spent: 0 msec
hive>
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups
"mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit
https://groups.google.com/d/msgid/mongodb-user/5e188766-e0a1-4cd1-a776-428f7ec3d20c%40googlegroups.com
<https://groups.google.com/d/msgid/mongodb-user/5e188766-e0a1-4cd1-a776-428f7ec3d20c%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
To post to this group, send email to mongodb-user-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/CAH7JHGdTqGDAuahQg8o93nANg-oSCKLXG7XuS2oYhv9pyXug6w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Josh Morris
2014-06-27 19:11:39 UTC
Permalink
I've been trying to find more information, not sure if this is related to
the issue, it is related to the job that was run (let me know if you have
any suggestions on where I can look).


Log Length: 294

[Fatal Error] job.xml:623:51: Character reference "&#
log4j:WARN No appenders could be found for logger (org.apache.hadoop.mapreduce.v2.app.MRAppMaster).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.

Log Type: stdout

Log Length: 0

Log Type: syslog

Log Length: 2867

2014-06-27 14:03:03,668 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Created MRAppMaster for application appattempt_1403731258511_0041_000001
2014-06-27 14:03:04,380 FATAL [main] org.apache.hadoop.conf.Configuration: error parsing conf job.xml
org.xml.sax.SAXParseException; systemId: file:///hadoop/yarn/local/usercache/hdfs/appcache/application_1403731258511_0041/container_1403731258511_0041_01_000001/job.xml; lineNumber: 623; columnNumber: 51; Character reference "&#
at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:257)
at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:347)
at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:150)
at org.apache.hadoop.conf.Configuration.parse(Configuration.java:2173)
at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2242)
at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2195)
at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2102)
at org.apache.hadoop.conf.Configuration.get(Configuration.java:1068)
at org.apache.hadoop.mapreduce.v2.util.MRWebAppUtil.initialize(MRWebAppUtil.java:51)
at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1381)
2014-06-27 14:03:04,383 FATAL [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Error starting MRAppMaster
java.lang.RuntimeException: org.xml.sax.SAXParseException; systemId: file:///hadoop/yarn/local/usercache/hdfs/appcache/application_1403731258511_0041/container_1403731258511_0041_01_000001/job.xml; lineNumber: 623; columnNumber: 51; Character reference "&#
at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2338)
at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2195)
at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2102)
at org.apache.hadoop.conf.Configuration.get(Configuration.java:1068)
at org.apache.hadoop.mapreduce.v2.util.MRWebAppUtil.initialize(MRWebAppUtil.java:51)
at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1381)
Caused by: org.xml.sax.SAXParseException; systemId: file:///hadoop/yarn/local/usercache/hdfs/appcache/application_1403731258511_0041/container_1403731258511_0041_01_000001/job.xml; lineNumber: 623; columnNumber: 51; Character reference "&#
at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:257)
at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:347)
at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:150)
at org.apache.hadoop.conf.Configuration.parse(Configuration.java:2173)
at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2242)
... 5 more
2014-06-27 14:03:04,386 INFO [main] org.apache.hadoop.util.ExitUtil: Exiting with status 1
Post by Justin Lee
There's nothing more in the logs?
Post by Josh Morris
I get an error (see below) while trying to insert data into a hive table
that is connected to a Mongodb table. I'm able to create, drop and query
the table just fine e.g. if I insert data into the Mongo collection first
and perform a select from the Hive cli I see the Mongo data.
FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.mr.MapRedTask
I'm using HDP 2.1 set up across 3 ec2 instances, Mongodb 2.6 on another
ec2 instance.
So far I haven't been able to find more details about the error. Any help
would be appreciated.
*Details*
hive> CREATE TABLE individuals
(
id INT,
name STRING,
age INT
)
STORED BY 'com.mongodb.hadoop.hive.MongoStorageHandler'
TBLPROPERTIES('mongo.uri'='mongodb://
OK
Time taken: 0.294 seconds
hive> select * from individuals;
OK
Time taken: 0.351 seconds
hive> insert into table individuals select 1, 'name column', 5 from
some_table limit 1;
Query ID = hdfs_20140627134040_5a3db212-e6e1-42cf-a6cc-0a0078ef3bb3
Total jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
set hive.exec.reducers.bytes.per.reducer=<number>
set hive.exec.reducers.max=<number>
set mapreduce.job.reduces=<number>
Starting Job = job_1403731258511_0038, Tracking URL =
http://192.168.2.40:8088/proxy/application_1403731258511_0038/
Kill Command = /usr/lib/hadoop/bin/hadoop job -kill
job_1403731258511_0038
Hadoop job information for Stage-0: number of mappers: 0; number of
reducers: 0
2014-06-27 13:40:16,335 Stage-0 map = 0%, reduce = 0%
Ended Job = job_1403731258511_0038 with errors
*Error during job, obtaining debugging information...*
*FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.mr.MapRedTask*
Job 0: HDFS Read: 0 HDFS Write: 0 FAIL
Total MapReduce CPU Time Spent: 0 msec
hive>
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups
"mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an
<javascript:>.
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit
https://groups.google.com/d/msgid/mongodb-user/5e188766-e0a1-4cd1-a776-428f7ec3d20c%40googlegroups.com
<https://groups.google.com/d/msgid/mongodb-user/5e188766-e0a1-4cd1-a776-428f7ec3d20c%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
To post to this group, send email to mongodb-user-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/15c78506-5c78-41fb-bf6d-1fc051709748%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Justin Lee
2014-06-27 19:24:52 UTC
Permalink
I'd be curious what was wrong with that file to be sure.
Post by Josh Morris
I've been trying to find more information, not sure if this is related to
the issue, it is related to the job that was run (let me know if you have
any suggestions on where I can look).
Log Length: 294
[Fatal Error] job.xml:623:51: Character reference "&#
log4j:WARN No appenders could be found for logger (org.apache.hadoop.mapreduce.v2.app.MRAppMaster).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Log Type: stdout
Log Length: 0
Log Type: syslog
Log Length: 2867
2014-06-27 14:03:03,668 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Created MRAppMaster for application appattempt_1403731258511_0041_000001
2014-06-27 14:03:04,380 FATAL [main] org.apache.hadoop.conf.Configuration: error parsing conf job.xml
org.xml.sax.SAXParseException; systemId: file:///hadoop/yarn/local/usercache/hdfs/appcache/application_1403731258511_0041/container_1403731258511_0041_01_000001/job.xml; lineNumber: 623; columnNumber: 51; Character reference "&#
at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:257)
at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:347)
at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:150)
at org.apache.hadoop.conf.Configuration.parse(Configuration.java:2173)
at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2242)
at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2195)
at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2102)
at org.apache.hadoop.conf.Configuration.get(Configuration.java:1068)
at org.apache.hadoop.mapreduce.v2.util.MRWebAppUtil.initialize(MRWebAppUtil.java:51)
at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1381)
2014-06-27 14:03:04,383 FATAL [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Error starting MRAppMaster
java.lang.RuntimeException: org.xml.sax.SAXParseException; systemId: file:///hadoop/yarn/local/usercache/hdfs/appcache/application_1403731258511_0041/container_1403731258511_0041_01_000001/job.xml; lineNumber: 623; columnNumber: 51; Character reference "&#
at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2338)
at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2195)
at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2102)
at org.apache.hadoop.conf.Configuration.get(Configuration.java:1068)
at org.apache.hadoop.mapreduce.v2.util.MRWebAppUtil.initialize(MRWebAppUtil.java:51)
at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1381)
Caused by: org.xml.sax.SAXParseException; systemId: file:///hadoop/yarn/local/usercache/hdfs/appcache/application_1403731258511_0041/container_1403731258511_0041_01_000001/job.xml; lineNumber: 623; columnNumber: 51; Character reference "&#
at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:257)
at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:347)
at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:150)
at org.apache.hadoop.conf.Configuration.parse(Configuration.java:2173)
at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2242)
... 5 more
2014-06-27 14:03:04,386 INFO [main] org.apache.hadoop.util.ExitUtil: Exiting with status 1
Post by Justin Lee
There's nothing more in the logs?
Post by Josh Morris
I get an error (see below) while trying to insert data into a hive table
that is connected to a Mongodb table. I'm able to create, drop and query
the table just fine e.g. if I insert data into the Mongo collection first
and perform a select from the Hive cli I see the Mongo data.
FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.
exec.mr.MapRedTask
I'm using HDP 2.1 set up across 3 ec2 instances, Mongodb 2.6 on another
ec2 instance.
So far I haven't been able to find more details about the error. Any
help would be appreciated.
*Details*
hive> CREATE TABLE individuals
(
id INT,
name STRING,
age INT
)
STORED BY 'com.mongodb.hadoop.hive.MongoStorageHandler'
168.2.151:27017/test.persons2');
OK
Time taken: 0.294 seconds
hive> select * from individuals;
OK
Time taken: 0.351 seconds
hive> insert into table individuals select 1, 'name column', 5 from
some_table limit 1;
Query ID = hdfs_20140627134040_5a3db212-e6e1-42cf-a6cc-0a0078ef3bb3
Total jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
set hive.exec.reducers.bytes.per.reducer=<number>
set hive.exec.reducers.max=<number>
set mapreduce.job.reduces=<number>
Starting Job = job_1403731258511_0038, Tracking URL =
http://192.168.2.40:8088/proxy/application_1403731258511_0038/
Kill Command = /usr/lib/hadoop/bin/hadoop job -kill
job_1403731258511_0038
Hadoop job information for Stage-0: number of mappers: 0; number of
reducers: 0
2014-06-27 13:40:16,335 Stage-0 map = 0%, reduce = 0%
Ended Job = job_1403731258511_0038 with errors
*Error during job, obtaining debugging information...*
*FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.mr.MapRedTask*
Job 0: HDFS Read: 0 HDFS Write: 0 FAIL
Total MapReduce CPU Time Spent: 0 msec
hive>
--
You received this message because you are subscribed to the Google
Groups "mongodb-user"
group.
http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google
Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/
msgid/mongodb-user/5e188766-e0a1-4cd1-a776-428f7ec3d20c%
40googlegroups.com
<https://groups.google.com/d/msgid/mongodb-user/5e188766-e0a1-4cd1-a776-428f7ec3d20c%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups
"mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit
https://groups.google.com/d/msgid/mongodb-user/15c78506-5c78-41fb-bf6d-1fc051709748%40googlegroups.com
<https://groups.google.com/d/msgid/mongodb-user/15c78506-5c78-41fb-bf6d-1fc051709748%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
To post to this group, send email to mongodb-user-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/CAH7JHGfqLKEt4Am32QoYoHHVmreWtKJkyv5ndk_wCxgssSTZ%2BA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Josh Morris
2014-06-27 19:53:46 UTC
Permalink
Yeah, looks like the line below in the jobs.xml is causing the problem. Not
sure what process is putting it in there:

<property><name>columns.comments</name><value>&#0;&#0;</value><source>programatically</source></property>
Post by Justin Lee
I'd be curious what was wrong with that file to be sure.
Post by Josh Morris
I've been trying to find more information, not sure if this is related to
the issue, it is related to the job that was run (let me know if you have
any suggestions on where I can look).
Log Length: 294
[Fatal Error] job.xml:623:51: Character reference "&#
log4j:WARN No appenders could be found for logger (org.apache.hadoop.mapreduce.v2.app.MRAppMaster).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Log Type: stdout
Log Length: 0
Log Type: syslog
Log Length: 2867
2014-06-27 14:03:03,668 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Created MRAppMaster for application appattempt_1403731258511_0041_000001
2014-06-27 14:03:04,380 FATAL [main] org.apache.hadoop.conf.Configuration: error parsing conf job.xml
org.xml.sax.SAXParseException; systemId: file:///hadoop/yarn/local/usercache/hdfs/appcache/application_1403731258511_0041/container_1403731258511_0041_01_000001/job.xml; lineNumber: 623; columnNumber: 51; Character reference "&#
at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:257)
at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:347)
at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:150)
at org.apache.hadoop.conf.Configuration.parse(Configuration.java:2173)
at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2242)
at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2195)
at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2102)
at org.apache.hadoop.conf.Configuration.get(Configuration.java:1068)
at org.apache.hadoop.mapreduce.v2.util.MRWebAppUtil.initialize(MRWebAppUtil.java:51)
at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1381)
2014-06-27 14:03:04,383 FATAL [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Error starting MRAppMaster
java.lang.RuntimeException: org.xml.sax.SAXParseException; systemId: file:///hadoop/yarn/local/usercache/hdfs/appcache/application_1403731258511_0041/container_1403731258511_0041_01_000001/job.xml; lineNumber: 623; columnNumber: 51; Character reference "&#
at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2338)
at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2195)
at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2102)
at org.apache.hadoop.conf.Configuration.get(Configuration.java:1068)
at org.apache.hadoop.mapreduce.v2.util.MRWebAppUtil.initialize(MRWebAppUtil.java:51)
at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1381)
Caused by: org.xml.sax.SAXParseException; systemId: file:///hadoop/yarn/local/usercache/hdfs/appcache/application_1403731258511_0041/container_1403731258511_0041_01_000001/job.xml; lineNumber: 623; columnNumber: 51; Character reference "&#
at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:257)
at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:347)
at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:150)
at org.apache.hadoop.conf.Configuration.parse(Configuration.java:2173)
at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2242)
... 5 more
2014-06-27 14:03:04,386 INFO [main] org.apache.hadoop.util.ExitUtil: Exiting with status 1
Post by Justin Lee
There's nothing more in the logs?
Post by Josh Morris
I get an error (see below) while trying to insert data into a hive
table that is connected to a Mongodb table. I'm able to create, drop and
query the table just fine e.g. if I insert data into the Mongo collection
first and perform a select from the Hive cli I see the Mongo data.
FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.
exec.mr.MapRedTask
I'm using HDP 2.1 set up across 3 ec2 instances, Mongodb 2.6 on another
ec2 instance.
So far I haven't been able to find more details about the error. Any
help would be appreciated.
*Details*
hive> CREATE TABLE individuals
(
id INT,
name STRING,
age INT
)
STORED BY 'com.mongodb.hadoop.hive.MongoStorageHandler'
168.2.151:27017/test.persons2');
OK
Time taken: 0.294 seconds
hive> select * from individuals;
OK
Time taken: 0.351 seconds
hive> insert into table individuals select 1, 'name column', 5 from
some_table limit 1;
Query ID = hdfs_20140627134040_5a3db212-e6e1-42cf-a6cc-0a0078ef3bb3
Total jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
set hive.exec.reducers.bytes.per.reducer=<number>
set hive.exec.reducers.max=<number>
set mapreduce.job.reduces=<number>
Starting Job = job_1403731258511_0038, Tracking URL =
http://192.168.2.40:8088/proxy/application_1403731258511_0038/
Kill Command = /usr/lib/hadoop/bin/hadoop job -kill
job_1403731258511_0038
Hadoop job information for Stage-0: number of mappers: 0; number of
reducers: 0
2014-06-27 13:40:16,335 Stage-0 map = 0%, reduce = 0%
Ended Job = job_1403731258511_0038 with errors
*Error during job, obtaining debugging information...*
*FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.mr.MapRedTask*
Job 0: HDFS Read: 0 HDFS Write: 0 FAIL
Total MapReduce CPU Time Spent: 0 msec
hive>
--
You received this message because you are subscribed to the Google
Groups "mongodb-user"
group.
http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google
Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/
msgid/mongodb-user/5e188766-e0a1-4cd1-a776-428f7ec3d20c%
40googlegroups.com
<https://groups.google.com/d/msgid/mongodb-user/5e188766-e0a1-4cd1-a776-428f7ec3d20c%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups
"mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an
<javascript:>.
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit
https://groups.google.com/d/msgid/mongodb-user/15c78506-5c78-41fb-bf6d-1fc051709748%40googlegroups.com
<https://groups.google.com/d/msgid/mongodb-user/15c78506-5c78-41fb-bf6d-1fc051709748%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
To post to this group, send email to mongodb-user-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/5e9edb88-4fa4-49ff-9624-c159b5118bdd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Justin Lee
2014-06-27 20:01:56 UTC
Permalink
I'm guessing my the single quotes in the query. Try calling it just *name*
instead of *name column*
Post by Josh Morris
Yeah, looks like the line below in the jobs.xml is causing the problem.
<property><name>columns.comments</name><value>&#0;&#0;</value><source>programatically</source></property>
Post by Justin Lee
I'd be curious what was wrong with that file to be sure.
Post by Josh Morris
I've been trying to find more information, not sure if this is related
to the issue, it is related to the job that was run (let me know if you
have any suggestions on where I can look).
Log Length: 294
[Fatal Error] job.xml:623:51: Character reference "&#
log4j:WARN No appenders could be found for logger (org.apache.hadoop.mapreduce.v2.app.MRAppMaster).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Log Type: stdout
Log Length: 0
Log Type: syslog
Log Length: 2867
2014-06-27 14:03:03,668 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Created MRAppMaster for application appattempt_1403731258511_0041_000001
2014-06-27 14:03:04,380 FATAL [main] org.apache.hadoop.conf.Configuration: error parsing conf job.xml
org.xml.sax.SAXParseException; systemId: file:///hadoop/yarn/local/usercache/hdfs/appcache/application_1403731258511_0041/container_1403731258511_0041_01_000001/job.xml; lineNumber: 623; columnNumber: 51; Character reference "&#
at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:257)
at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:347)
at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:150)
at org.apache.hadoop.conf.Configuration.parse(Configuration.java:2173)
at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2242)
at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2195)
at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2102)
at org.apache.hadoop.conf.Configuration.get(Configuration.java:1068)
at org.apache.hadoop.mapreduce.v2.util.MRWebAppUtil.initialize(MRWebAppUtil.java:51)
at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1381)
2014-06-27 14:03:04,383 FATAL [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Error starting MRAppMaster
java.lang.RuntimeException: org.xml.sax.SAXParseException; systemId: file:///hadoop/yarn/local/usercache/hdfs/appcache/application_1403731258511_0041/container_1403731258511_0041_01_000001/job.xml; lineNumber: 623; columnNumber: 51; Character reference "&#
at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2338)
at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2195)
at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2102)
at org.apache.hadoop.conf.Configuration.get(Configuration.java:1068)
at org.apache.hadoop.mapreduce.v2.util.MRWebAppUtil.initialize(MRWebAppUtil.java:51)
at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1381)
Caused by: org.xml.sax.SAXParseException; systemId: file:///hadoop/yarn/local/usercache/hdfs/appcache/application_1403731258511_0041/container_1403731258511_0041_01_000001/job.xml; lineNumber: 623; columnNumber: 51; Character reference "&#
at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:257)
at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:347)
at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:150)
at org.apache.hadoop.conf.Configuration.parse(Configuration.java:2173)
at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2242)
... 5 more
2014-06-27 14:03:04,386 INFO [main] org.apache.hadoop.util.ExitUtil: Exiting with status 1
Post by Justin Lee
There's nothing more in the logs?
Post by Josh Morris
I get an error (see below) while trying to insert data into a hive
table that is connected to a Mongodb table. I'm able to create, drop and
query the table just fine e.g. if I insert data into the Mongo collection
first and perform a select from the Hive cli I see the Mongo data.
FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.
exec.mr.MapRedTask
I'm using HDP 2.1 set up across 3 ec2 instances, Mongodb 2.6 on
another ec2 instance.
So far I haven't been able to find more details about the error. Any
help would be appreciated.
*Details*
hive> CREATE TABLE individuals
(
id INT,
name STRING,
age INT
)
STORED BY 'com.mongodb.hadoop.hive.MongoStorageHandler'
2.151:27017/test.persons2');
OK
Time taken: 0.294 seconds
hive> select * from individuals;
OK
Time taken: 0.351 seconds
hive> insert into table individuals select 1, 'name column', 5 from
some_table limit 1;
Query ID = hdfs_20140627134040_5a3db212-e6e1-42cf-a6cc-0a0078ef3bb3
Total jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
set hive.exec.reducers.bytes.per.reducer=<number>
set hive.exec.reducers.max=<number>
set mapreduce.job.reduces=<number>
Starting Job = job_1403731258511_0038, Tracking URL =
http://192.168.2.40:8088/proxy/application_1403731258511_0038/
Kill Command = /usr/lib/hadoop/bin/hadoop job -kill
job_1403731258511_0038
Hadoop job information for Stage-0: number of mappers: 0; number of
reducers: 0
2014-06-27 13:40:16,335 Stage-0 map = 0%, reduce = 0%
Ended Job = job_1403731258511_0038 with errors
*Error during job, obtaining debugging information...*
*FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.mr.MapRedTask*
Job 0: HDFS Read: 0 HDFS Write: 0 FAIL
Total MapReduce CPU Time Spent: 0 msec
hive>
--
You received this message because you are subscribed to the Google
Groups "mongodb-user"
group.
http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google
Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/
msgid/mongodb-user/5e188766-e0a1-4cd1-a776-428f7ec3d20c%40goog
legroups.com
<https://groups.google.com/d/msgid/mongodb-user/5e188766-e0a1-4cd1-a776-428f7ec3d20c%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google
Groups "mongodb-user"
group.
http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google
Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/
msgid/mongodb-user/15c78506-5c78-41fb-bf6d-1fc051709748%
40googlegroups.com
<https://groups.google.com/d/msgid/mongodb-user/15c78506-5c78-41fb-bf6d-1fc051709748%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups
"mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit
https://groups.google.com/d/msgid/mongodb-user/5e9edb88-4fa4-49ff-9624-c159b5118bdd%40googlegroups.com
<https://groups.google.com/d/msgid/mongodb-user/5e9edb88-4fa4-49ff-9624-c159b5118bdd%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
To post to this group, send email to mongodb-user-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/CAH7JHGeGoeRzzA1CWf0qwP6F8Nxn2APSUCNyepTC6qfNf3xUag%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Josh Morris
2014-06-27 21:15:45 UTC
Permalink
The single quote change, and change from name column to just name did not
work. I'm currently trying to add COMMENT's to the column definition during
the Hive create table command, however so far that has not made a
difference.
Post by Justin Lee
I'm guessing my the single quotes in the query. Try calling it just
*name* instead of *name column*
Post by Josh Morris
Yeah, looks like the line below in the jobs.xml is causing the problem.
<property><name>columns.comments</name><value>&#0;&#0;</value><source>programatically</source></property>
Post by Justin Lee
I'd be curious what was wrong with that file to be sure.
Post by Josh Morris
I've been trying to find more information, not sure if this is related
to the issue, it is related to the job that was run (let me know if you
have any suggestions on where I can look).
Log Length: 294
[Fatal Error] job.xml:623:51: Character reference "&#
log4j:WARN No appenders could be found for logger (org.apache.hadoop.mapreduce.v2.app.MRAppMaster).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Log Type: stdout
Log Length: 0
Log Type: syslog
Log Length: 2867
2014-06-27 14:03:03,668 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Created MRAppMaster for application appattempt_1403731258511_0041_000001
2014-06-27 14:03:04,380 FATAL [main] org.apache.hadoop.conf.Configuration: error parsing conf job.xml
org.xml.sax.SAXParseException; systemId: file:///hadoop/yarn/local/usercache/hdfs/appcache/application_1403731258511_0041/container_1403731258511_0041_01_000001/job.xml; lineNumber: 623; columnNumber: 51; Character reference "&#
at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:257)
at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:347)
at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:150)
at org.apache.hadoop.conf.Configuration.parse(Configuration.java:2173)
at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2242)
at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2195)
at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2102)
at org.apache.hadoop.conf.Configuration.get(Configuration.java:1068)
at org.apache.hadoop.mapreduce.v2.util.MRWebAppUtil.initialize(MRWebAppUtil.java:51)
at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1381)
2014-06-27 14:03:04,383 FATAL [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Error starting MRAppMaster
java.lang.RuntimeException: org.xml.sax.SAXParseException; systemId: file:///hadoop/yarn/local/usercache/hdfs/appcache/application_1403731258511_0041/container_1403731258511_0041_01_000001/job.xml; lineNumber: 623; columnNumber: 51; Character reference "&#
at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2338)
at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2195)
at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2102)
at org.apache.hadoop.conf.Configuration.get(Configuration.java:1068)
at org.apache.hadoop.mapreduce.v2.util.MRWebAppUtil.initialize(MRWebAppUtil.java:51)
at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1381)
Caused by: org.xml.sax.SAXParseException; systemId: file:///hadoop/yarn/local/usercache/hdfs/appcache/application_1403731258511_0041/container_1403731258511_0041_01_000001/job.xml; lineNumber: 623; columnNumber: 51; Character reference "&#
at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:257)
at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:347)
at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:150)
at org.apache.hadoop.conf.Configuration.parse(Configuration.java:2173)
at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2242)
... 5 more
2014-06-27 14:03:04,386 INFO [main] org.apache.hadoop.util.ExitUtil: Exiting with status 1
Post by Justin Lee
There's nothing more in the logs?
Post by Josh Morris
I get an error (see below) while trying to insert data into a hive
table that is connected to a Mongodb table. I'm able to create, drop and
query the table just fine e.g. if I insert data into the Mongo collection
first and perform a select from the Hive cli I see the Mongo data.
FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.
exec.mr.MapRedTask
I'm using HDP 2.1 set up across 3 ec2 instances, Mongodb 2.6 on
another ec2 instance.
So far I haven't been able to find more details about the error. Any
help would be appreciated.
*Details*
hive> CREATE TABLE individuals
(
id INT,
name STRING,
age INT
)
STORED BY 'com.mongodb.hadoop.hive.MongoStorageHandler'
2.151:27017/test.persons2');
OK
Time taken: 0.294 seconds
hive> select * from individuals;
OK
Time taken: 0.351 seconds
hive> insert into table individuals select 1, 'name column', 5 from
some_table limit 1;
Query ID = hdfs_20140627134040_5a3db212-e6e1-42cf-a6cc-0a0078ef3bb3
Total jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
set hive.exec.reducers.bytes.per.reducer=<number>
set hive.exec.reducers.max=<number>
set mapreduce.job.reduces=<number>
Starting Job = job_1403731258511_0038, Tracking URL =
http://192.168.2.40:8088/proxy/application_1403731258511_0038/
Kill Command = /usr/lib/hadoop/bin/hadoop job -kill
job_1403731258511_0038
Hadoop job information for Stage-0: number of mappers: 0; number of
reducers: 0
2014-06-27 13:40:16,335 Stage-0 map = 0%, reduce = 0%
Ended Job = job_1403731258511_0038 with errors
*Error during job, obtaining debugging information...*
*FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.mr.MapRedTask*
Job 0: HDFS Read: 0 HDFS Write: 0 FAIL
Total MapReduce CPU Time Spent: 0 msec
hive>
--
You received this message because you are subscribed to the Google
Groups "mongodb-user"
group.
http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google
Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it,
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/
msgid/mongodb-user/5e188766-e0a1-4cd1-a776-428f7ec3d20c%40goog
legroups.com
<https://groups.google.com/d/msgid/mongodb-user/5e188766-e0a1-4cd1-a776-428f7ec3d20c%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google
Groups "mongodb-user"
group.
http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google
Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/
msgid/mongodb-user/15c78506-5c78-41fb-bf6d-1fc051709748%
40googlegroups.com
<https://groups.google.com/d/msgid/mongodb-user/15c78506-5c78-41fb-bf6d-1fc051709748%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups
"mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an
<javascript:>.
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit
https://groups.google.com/d/msgid/mongodb-user/5e9edb88-4fa4-49ff-9624-c159b5118bdd%40googlegroups.com
<https://groups.google.com/d/msgid/mongodb-user/5e9edb88-4fa4-49ff-9624-c159b5118bdd%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
To post to this group, send email to mongodb-user-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/051ef8b3-2bf6-4315-a3f1-fafbf14b7c4c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Josh Morris
2014-06-30 16:05:24 UTC
Permalink
The issue appears to be with this line:

https://github.com/apache/hive/blob/4fd26aa1bbd0d568c4fcc900fef42d46b16dbd22/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java#L942

The null character once in jobs.xml is an invalid XML character.

I changed this to be an empty space for testing, and it worked. I was able
to get past this issue and insert from Hive to Mongodb.

This code appears to have been added with the following
ticket https://issues.apache.org/jira/browse/HIVE-6681. I would be happy
to submit a pull request, though not sure if a empty space should be used
or some other valid XML character?
Post by Josh Morris
The single quote change, and change from name column to just name did not
work. I'm currently trying to add COMMENT's to the column definition during
the Hive create table command, however so far that has not made a
difference.
Post by Justin Lee
I'm guessing my the single quotes in the query. Try calling it just
*name* instead of *name column*
Post by Josh Morris
Yeah, looks like the line below in the jobs.xml is causing the problem.
<property><name>columns.comments</name><value>&#0;&#0;</value><source>programatically</source></property>
Post by Justin Lee
I'd be curious what was wrong with that file to be sure.
Post by Josh Morris
I've been trying to find more information, not sure if this is related
to the issue, it is related to the job that was run (let me know if you
have any suggestions on where I can look).
Log Length: 294
[Fatal Error] job.xml:623:51: Character reference "&#
log4j:WARN No appenders could be found for logger (org.apache.hadoop.mapreduce.v2.app.MRAppMaster).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Log Type: stdout
Log Length: 0
Log Type: syslog
Log Length: 2867
2014-06-27 14:03:03,668 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Created MRAppMaster for application appattempt_1403731258511_0041_000001
2014-06-27 14:03:04,380 FATAL [main] org.apache.hadoop.conf.Configuration: error parsing conf job.xml
org.xml.sax.SAXParseException; systemId: file:///hadoop/yarn/local/usercache/hdfs/appcache/application_1403731258511_0041/container_1403731258511_0041_01_000001/job.xml; lineNumber: 623; columnNumber: 51; Character reference "&#
at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:257)
at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:347)
at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:150)
at org.apache.hadoop.conf.Configuration.parse(Configuration.java:2173)
at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2242)
at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2195)
at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2102)
at org.apache.hadoop.conf.Configuration.get(Configuration.java:1068)
at org.apache.hadoop.mapreduce.v2.util.MRWebAppUtil.initialize(MRWebAppUtil.java:51)
at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1381)
2014-06-27 14:03:04,383 FATAL [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Error starting MRAppMaster
java.lang.RuntimeException: org.xml.sax.SAXParseException; systemId: file:///hadoop/yarn/local/usercache/hdfs/appcache/application_1403731258511_0041/container_1403731258511_0041_01_000001/job.xml; lineNumber: 623; columnNumber: 51; Character reference "&#
at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2338)
at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2195)
at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2102)
at org.apache.hadoop.conf.Configuration.get(Configuration.java:1068)
at org.apache.hadoop.mapreduce.v2.util.MRWebAppUtil.initialize(MRWebAppUtil.java:51)
at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1381)
Caused by: org.xml.sax.SAXParseException; systemId: file:///hadoop/yarn/local/usercache/hdfs/appcache/application_1403731258511_0041/container_1403731258511_0041_01_000001/job.xml; lineNumber: 623; columnNumber: 51; Character reference "&#
at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:257)
at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:347)
at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:150)
at org.apache.hadoop.conf.Configuration.parse(Configuration.java:2173)
at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2242)
... 5 more
2014-06-27 14:03:04,386 INFO [main] org.apache.hadoop.util.ExitUtil: Exiting with status 1
Post by Justin Lee
There's nothing more in the logs?
On Fri, Jun 27, 2014 at 2:19 PM, Josh Morris <
Post by Josh Morris
I get an error (see below) while trying to insert data into a hive
table that is connected to a Mongodb table. I'm able to create, drop and
query the table just fine e.g. if I insert data into the Mongo collection
first and perform a select from the Hive cli I see the Mongo data.
FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.mr.MapRedTask
I'm using HDP 2.1 set up across 3 ec2 instances, Mongodb 2.6 on
another ec2 instance.
So far I haven't been able to find more details about the error. Any
help would be appreciated.
*Details*
hive> CREATE TABLE individuals
(
id INT,
name STRING,
age INT
)
STORED BY 'com.mongodb.hadoop.hive.MongoStorageHandler'
2.151:27017/test.persons2');
OK
Time taken: 0.294 seconds
hive> select * from individuals;
OK
Time taken: 0.351 seconds
hive> insert into table individuals select 1, 'name column', 5 from
some_table limit 1;
Query ID = hdfs_20140627134040_5a3db212-e6e1-42cf-a6cc-0a0078ef3bb3
Total jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
set hive.exec.reducers.bytes.per.reducer=<number>
set hive.exec.reducers.max=<number>
set mapreduce.job.reduces=<number>
Starting Job = job_1403731258511_0038, Tracking URL =
http://192.168.2.40:8088/proxy/application_1403731258511_0038/
Kill Command = /usr/lib/hadoop/bin/hadoop job -kill
job_1403731258511_0038
Hadoop job information for Stage-0: number of mappers: 0; number of
reducers: 0
2014-06-27 13:40:16,335 Stage-0 map = 0%, reduce = 0%
Ended Job = job_1403731258511_0038 with errors
*Error during job, obtaining debugging information...*
*FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.mr.MapRedTask*
Job 0: HDFS Read: 0 HDFS Write: 0 FAIL
Total MapReduce CPU Time Spent: 0 msec
hive>
--
You received this message because you are subscribed to the Google
Groups "mongodb-user"
group.
http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google
Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it,
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit
https://groups.google.com/d/msgid/mongodb-user/5e188766-e0a1
-4cd1-a776-428f7ec3d20c%40googlegroups.com
<https://groups.google.com/d/msgid/mongodb-user/5e188766-e0a1-4cd1-a776-428f7ec3d20c%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google
Groups "mongodb-user"
group.
http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google
Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/
msgid/mongodb-user/15c78506-5c78-41fb-bf6d-1fc051709748%
40googlegroups.com
<https://groups.google.com/d/msgid/mongodb-user/15c78506-5c78-41fb-bf6d-1fc051709748%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google
Groups "mongodb-user"
group.
http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google
Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit
https://groups.google.com/d/msgid/mongodb-user/5e9edb88-4fa4-49ff-9624-c159b5118bdd%40googlegroups.com
<https://groups.google.com/d/msgid/mongodb-user/5e9edb88-4fa4-49ff-9624-c159b5118bdd%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
To post to this group, send email to mongodb-user-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/d3396e27-9630-4f8a-b09f-5b12dd69d15d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Justin Lee
2014-06-30 17:07:35 UTC
Permalink
Pull requests are always appreciated but in this case the bug isn't with
mongo-hadoop but with apache's hive project. You'll have to talk with them
about any fix.
Post by Josh Morris
https://github.com/apache/hive/blob/4fd26aa1bbd0d568c4fcc900fef42d46b16dbd22/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java#L942
The null character once in jobs.xml is an invalid XML character.
I changed this to be an empty space for testing, and it worked. I was able
to get past this issue and insert from Hive to Mongodb.
This code appears to have been added with the following ticket
https://issues.apache.org/jira/browse/HIVE-6681. I would be happy to
submit a pull request, though not sure if a empty space should be used or
some other valid XML character?
Post by Josh Morris
The single quote change, and change from name column to just name did not
work. I'm currently trying to add COMMENT's to the column definition during
the Hive create table command, however so far that has not made a
difference.
Post by Justin Lee
I'm guessing my the single quotes in the query. Try calling it just
*name* instead of *name column*
Post by Josh Morris
Yeah, looks like the line below in the jobs.xml is causing the problem.
<property><name>columns.comments</name><value>&#0;&#0;</value><source>
programatically</source></property>
Post by Justin Lee
I'd be curious what was wrong with that file to be sure.
Post by Josh Morris
I've been trying to find more information, not sure if this is
related to the issue, it is related to the job that was run (let me know if
you have any suggestions on where I can look).
Log Length: 294
[Fatal Error] job.xml:623:51: Character reference "&#
log4j:WARN No appenders could be found for logger (org.apache.hadoop.mapreduce.v2.app.MRAppMaster).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Log Type: stdout
Log Length: 0
Log Type: syslog
Log Length: 2867
2014-06-27 14:03:03,668 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Created MRAppMaster for application appattempt_1403731258511_0041_000001
2014-06-27 14:03:04,380 FATAL [main] org.apache.hadoop.conf.Configuration: error parsing conf job.xml
org.xml.sax.SAXParseException; systemId: file:///hadoop/yarn/local/usercache/hdfs/appcache/application_1403731258511_0041/container_1403731258511_0041_01_000001/job.xml; lineNumber: 623; columnNumber: 51; Character reference "&#
at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:257)
at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:347)
at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:150)
at org.apache.hadoop.conf.Configuration.parse(Configuration.java:2173)
at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2242)
at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2195)
at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2102)
at org.apache.hadoop.conf.Configuration.get(Configuration.java:1068)
at org.apache.hadoop.mapreduce.v2.util.MRWebAppUtil.initialize(MRWebAppUtil.java:51)
at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1381)
2014-06-27 14:03:04,383 FATAL [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Error starting MRAppMaster
java.lang.RuntimeException: org.xml.sax.SAXParseException; systemId: file:///hadoop/yarn/local/usercache/hdfs/appcache/application_1403731258511_0041/container_1403731258511_0041_01_000001/job.xml; lineNumber: 623; columnNumber: 51; Character reference "&#
at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2338)
at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2195)
at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2102)
at org.apache.hadoop.conf.Configuration.get(Configuration.java:1068)
at org.apache.hadoop.mapreduce.v2.util.MRWebAppUtil.initialize(MRWebAppUtil.java:51)
at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1381)
Caused by: org.xml.sax.SAXParseException; systemId: file:///hadoop/yarn/local/usercache/hdfs/appcache/application_1403731258511_0041/container_1403731258511_0041_01_000001/job.xml; lineNumber: 623; columnNumber: 51; Character reference "&#
at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:257)
at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:347)
at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:150)
at org.apache.hadoop.conf.Configuration.parse(Configuration.java:2173)
at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2242)
... 5 more
2014-06-27 14:03:04,386 INFO [main] org.apache.hadoop.util.ExitUtil: Exiting with status 1
Post by Justin Lee
There's nothing more in the logs?
On Fri, Jun 27, 2014 at 2:19 PM, Josh Morris <
Post by Josh Morris
I get an error (see below) while trying to insert data into a hive
table that is connected to a Mongodb table. I'm able to create, drop and
query the table just fine e.g. if I insert data into the Mongo collection
first and perform a select from the Hive cli I see the Mongo data.
FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.mr.MapRedTask
I'm using HDP 2.1 set up across 3 ec2 instances, Mongodb 2.6 on
another ec2 instance.
So far I haven't been able to find more details about the error.
Any help would be appreciated.
*Details*
hive> CREATE TABLE individuals
(
id INT,
name STRING,
age INT
)
STORED BY 'com.mongodb.hadoop.hive.MongoStorageHandler'
.151:27017/test.persons2');
OK
Time taken: 0.294 seconds
hive> select * from individuals;
OK
Time taken: 0.351 seconds
hive> insert into table individuals select 1, 'name column', 5 from
some_table limit 1;
Query ID = hdfs_20140627134040_5a3db212-e6e1-42cf-a6cc-0a0078ef3bb3
Total jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
set hive.exec.reducers.bytes.per.reducer=<number>
set hive.exec.reducers.max=<number>
set mapreduce.job.reduces=<number>
Starting Job = job_1403731258511_0038, Tracking URL =
http://192.168.2.40:8088/proxy/application_1403731258511_0038/
Kill Command = /usr/lib/hadoop/bin/hadoop job -kill
job_1403731258511_0038
Hadoop job information for Stage-0: number of mappers: 0; number of
reducers: 0
2014-06-27 13:40:16,335 Stage-0 map = 0%, reduce = 0%
Ended Job = job_1403731258511_0038 with errors
*Error during job, obtaining debugging information...*
*FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.mr.MapRedTask*
Job 0: HDFS Read: 0 HDFS Write: 0 FAIL
Total MapReduce CPU Time Spent: 0 msec
hive>
--
You received this message because you are subscribed to the Google
Groups "mongodb-user"
group.
http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google
Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it,
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit
https://groups.google.com/d/msgid/mongodb-user/5e188766-e0a1
-4cd1-a776-428f7ec3d20c%40googlegroups.com
<https://groups.google.com/d/msgid/mongodb-user/5e188766-e0a1-4cd1-a776-428f7ec3d20c%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google
Groups "mongodb-user"
group.
http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google
Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it,
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/
msgid/mongodb-user/15c78506-5c78-41fb-bf6d-1fc051709748%40goog
legroups.com
<https://groups.google.com/d/msgid/mongodb-user/15c78506-5c78-41fb-bf6d-1fc051709748%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google
Groups "mongodb-user"
group.
http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google
Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/
msgid/mongodb-user/5e9edb88-4fa4-49ff-9624-c159b5118bdd%
40googlegroups.com
<https://groups.google.com/d/msgid/mongodb-user/5e9edb88-4fa4-49ff-9624-c159b5118bdd%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups
"mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit
https://groups.google.com/d/msgid/mongodb-user/d3396e27-9630-4f8a-b09f-5b12dd69d15d%40googlegroups.com
<https://groups.google.com/d/msgid/mongodb-user/d3396e27-9630-4f8a-b09f-5b12dd69d15d%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
To post to this group, send email to mongodb-user-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/CAH7JHGfrovs-uHTOZxMtNC-Gg2mQ7kCYgJivNUmWjs8dkVBK0g%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Josh Morris
2014-06-30 17:29:08 UTC
Permalink
Good point, my bad. Thanks.
Post by Justin Lee
Pull requests are always appreciated but in this case the bug isn't with
mongo-hadoop but with apache's hive project. You'll have to talk with them
about any fix.
Post by Josh Morris
https://github.com/apache/hive/blob/4fd26aa1bbd0d568c4fcc900fef42d46b16dbd22/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java#L942
The null character once in jobs.xml is an invalid XML character.
I changed this to be an empty space for testing, and it worked. I was
able to get past this issue and insert from Hive to Mongodb.
This code appears to have been added with the following ticket
https://issues.apache.org/jira/browse/HIVE-6681. I would be happy to
submit a pull request, though not sure if a empty space should be used or
some other valid XML character?
Post by Josh Morris
The single quote change, and change from name column to just name did
not work. I'm currently trying to add COMMENT's to the column definition
during the Hive create table command, however so far that has not made a
difference.
Post by Justin Lee
I'm guessing my the single quotes in the query. Try calling it just
*name* instead of *name column*
Post by Josh Morris
Yeah, looks like the line below in the jobs.xml is causing the
<property><name>columns.comments</name><value>&#0;&#0;</value><source>
programatically</source></property>
Post by Justin Lee
I'd be curious what was wrong with that file to be sure.
On Fri, Jun 27, 2014 at 3:11 PM, Josh Morris <
Post by Josh Morris
I've been trying to find more information, not sure if this is
related to the issue, it is related to the job that was run (let me know if
you have any suggestions on where I can look).
Log Length: 294
[Fatal Error] job.xml:623:51: Character reference "&#
log4j:WARN No appenders could be found for logger (org.apache.hadoop.mapreduce.v2.app.MRAppMaster).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Log Type: stdout
Log Length: 0
Log Type: syslog
Log Length: 2867
2014-06-27 14:03:03,668 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Created MRAppMaster for application appattempt_1403731258511_0041_000001
2014-06-27 14:03:04,380 FATAL [main] org.apache.hadoop.conf.Configuration: error parsing conf job.xml
org.xml.sax.SAXParseException; systemId: file:///hadoop/yarn/local/usercache/hdfs/appcache/application_1403731258511_0041/container_1403731258511_0041_01_000001/job.xml; lineNumber: 623; columnNumber: 51; Character reference "&#
at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:257)
at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:347)
at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:150)
at org.apache.hadoop.conf.Configuration.parse(Configuration.java:2173)
at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2242)
at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2195)
at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2102)
at org.apache.hadoop.conf.Configuration.get(Configuration.java:1068)
at org.apache.hadoop.mapreduce.v2.util.MRWebAppUtil.initialize(MRWebAppUtil.java:51)
at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1381)
2014-06-27 14:03:04,383 FATAL [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Error starting MRAppMaster
java.lang.RuntimeException: org.xml.sax.SAXParseException; systemId: file:///hadoop/yarn/local/usercache/hdfs/appcache/application_1403731258511_0041/container_1403731258511_0041_01_000001/job.xml; lineNumber: 623; columnNumber: 51; Character reference "&#
at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2338)
at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2195)
at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2102)
at org.apache.hadoop.conf.Configuration.get(Configuration.java:1068)
at org.apache.hadoop.mapreduce.v2.util.MRWebAppUtil.initialize(MRWebAppUtil.java:51)
at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1381)
Caused by: org.xml.sax.SAXParseException; systemId: file:///hadoop/yarn/local/usercache/hdfs/appcache/application_1403731258511_0041/container_1403731258511_0041_01_000001/job.xml; lineNumber: 623; columnNumber: 51; Character reference "&#
at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:257)
at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:347)
at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:150)
at org.apache.hadoop.conf.Configuration.parse(Configuration.java:2173)
at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2242)
... 5 more
2014-06-27 14:03:04,386 INFO [main] org.apache.hadoop.util.ExitUtil: Exiting with status 1
Post by Justin Lee
There's nothing more in the logs?
On Fri, Jun 27, 2014 at 2:19 PM, Josh Morris <
Post by Josh Morris
I get an error (see below) while trying to insert data into a hive
table that is connected to a Mongodb table. I'm able to create, drop and
query the table just fine e.g. if I insert data into the Mongo collection
first and perform a select from the Hive cli I see the Mongo data.
FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.mr.MapRedTask
I'm using HDP 2.1 set up across 3 ec2 instances, Mongodb 2.6 on
another ec2 instance.
So far I haven't been able to find more details about the error.
Any help would be appreciated.
*Details*
hive> CREATE TABLE individuals
(
id INT,
name STRING,
age INT
)
STORED BY 'com.mongodb.hadoop.hive.MongoStorageHandler'
2.151:27017/test.persons2');
OK
Time taken: 0.294 seconds
hive> select * from individuals;
OK
Time taken: 0.351 seconds
hive> insert into table individuals select 1, 'name column', 5
from some_table limit 1;
Query ID = hdfs_20140627134040_5a3db212-e
6e1-42cf-a6cc-0a0078ef3bb3
Total jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
set hive.exec.reducers.bytes.per.reducer=<number>
set hive.exec.reducers.max=<number>
set mapreduce.job.reduces=<number>
Starting Job = job_1403731258511_0038, Tracking URL =
http://192.168.2.40:8088/proxy/application_1403731258511_0038/
Kill Command = /usr/lib/hadoop/bin/hadoop job -kill
job_1403731258511_0038
Hadoop job information for Stage-0: number of mappers: 0; number
of reducers: 0
2014-06-27 13:40:16,335 Stage-0 map = 0%, reduce = 0%
Ended Job = job_1403731258511_0038 with errors
*Error during job, obtaining debugging information...*
*FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.mr.MapRedTask*
Job 0: HDFS Read: 0 HDFS Write: 0 FAIL
Total MapReduce CPU Time Spent: 0 msec
hive>
--
You received this message because you are subscribed to the Google
Groups "mongodb-user"
group.
http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google
Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it,
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit
https://groups.google.com/d/msgid/mongodb-user/5e188766-e0a1
-4cd1-a776-428f7ec3d20c%40googlegroups.com
<https://groups.google.com/d/msgid/mongodb-user/5e188766-e0a1-4cd1-a776-428f7ec3d20c%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google
Groups "mongodb-user"
group.
http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google
Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it,
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit
https://groups.google.com/d/msgid/mongodb-user/15c78506-5c78
-41fb-bf6d-1fc051709748%40googlegroups.com
<https://groups.google.com/d/msgid/mongodb-user/15c78506-5c78-41fb-bf6d-1fc051709748%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google
Groups "mongodb-user"
group.
http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google
Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/
msgid/mongodb-user/5e9edb88-4fa4-49ff-9624-c159b5118bdd%
40googlegroups.com
<https://groups.google.com/d/msgid/mongodb-user/5e9edb88-4fa4-49ff-9624-c159b5118bdd%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups
"mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an
<javascript:>.
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit
https://groups.google.com/d/msgid/mongodb-user/d3396e27-9630-4f8a-b09f-5b12dd69d15d%40googlegroups.com
<https://groups.google.com/d/msgid/mongodb-user/d3396e27-9630-4f8a-b09f-5b12dd69d15d%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
To post to this group, send email to mongodb-user-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/8ff7f922-c056-471b-be3e-04d8c0efa392%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
LM Heah
2014-07-01 06:16:57 UTC
Permalink
Hi Josh Morris,

I am having the same issue.
Would you mind to share the JAR file for the fix?
Post by Josh Morris
https://github.com/apache/hive/blob/4fd26aa1bbd0d568c4fcc900fef42d46b16dbd22/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java#L942
The null character once in jobs.xml is an invalid XML character.
I changed this to be an empty space for testing, and it worked. I was able
to get past this issue and insert from Hive to Mongodb.
This code appears to have been added with the following ticket
https://issues.apache.org/jira/browse/HIVE-6681. I would be happy to
submit a pull request, though not sure if a empty space should be used or
some other valid XML character?
Post by Josh Morris
The single quote change, and change from name column to just name did not
work. I'm currently trying to add COMMENT's to the column definition during
the Hive create table command, however so far that has not made a
difference.
Post by Justin Lee
I'm guessing my the single quotes in the query. Try calling it just
*name* instead of *name column*
Post by Josh Morris
Yeah, looks like the line below in the jobs.xml is causing the problem.
<property><name>columns.comments</name><value>&#0;&#0;</value><source>programatically</source></property>
Post by Justin Lee
I'd be curious what was wrong with that file to be sure.
Post by Josh Morris
I've been trying to find more information, not sure if this is
related to the issue, it is related to the job that was run (let me know if
you have any suggestions on where I can look).
Log Length: 294
[Fatal Error] job.xml:623:51: Character reference "&#
log4j:WARN No appenders could be found for logger (org.apache.hadoop.mapreduce.v2.app.MRAppMaster).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Log Type: stdout
Log Length: 0
Log Type: syslog
Log Length: 2867
2014-06-27 14:03:03,668 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Created MRAppMaster for application appattempt_1403731258511_0041_000001
2014-06-27 14:03:04,380 FATAL [main] org.apache.hadoop.conf.Configuration: error parsing conf job.xml
org.xml.sax.SAXParseException; systemId: file:///hadoop/yarn/local/usercache/hdfs/appcache/application_1403731258511_0041/container_1403731258511_0041_01_000001/job.xml; lineNumber: 623; columnNumber: 51; Character reference "&#
at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:257)
at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:347)
at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:150)
at org.apache.hadoop.conf.Configuration.parse(Configuration.java:2173)
at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2242)
at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2195)
at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2102)
at org.apache.hadoop.conf.Configuration.get(Configuration.java:1068)
at org.apache.hadoop.mapreduce.v2.util.MRWebAppUtil.initialize(MRWebAppUtil.java:51)
at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1381)
2014-06-27 14:03:04,383 FATAL [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Error starting MRAppMaster
java.lang.RuntimeException: org.xml.sax.SAXParseException; systemId: file:///hadoop/yarn/local/usercache/hdfs/appcache/application_1403731258511_0041/container_1403731258511_0041_01_000001/job.xml; lineNumber: 623; columnNumber: 51; Character reference "&#
at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2338)
at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2195)
at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2102)
at org.apache.hadoop.conf.Configuration.get(Configuration.java:1068)
at org.apache.hadoop.mapreduce.v2.util.MRWebAppUtil.initialize(MRWebAppUtil.java:51)
at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1381)
Caused by: org.xml.sax.SAXParseException; systemId: file:///hadoop/yarn/local/usercache/hdfs/appcache/application_1403731258511_0041/container_1403731258511_0041_01_000001/job.xml; lineNumber: 623; columnNumber: 51; Character reference "&#
at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:257)
at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:347)
at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:150)
at org.apache.hadoop.conf.Configuration.parse(Configuration.java:2173)
at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2242)
... 5 more
2014-06-27 14:03:04,386 INFO [main] org.apache.hadoop.util.ExitUtil: Exiting with status 1
Post by Justin Lee
There's nothing more in the logs?
On Fri, Jun 27, 2014 at 2:19 PM, Josh Morris <
Post by Josh Morris
I get an error (see below) while trying to insert data into a hive
table that is connected to a Mongodb table. I'm able to create, drop and
query the table just fine e.g. if I insert data into the Mongo collection
first and perform a select from the Hive cli I see the Mongo data.
FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.mr.MapRedTask
I'm using HDP 2.1 set up across 3 ec2 instances, Mongodb 2.6 on
another ec2 instance.
So far I haven't been able to find more details about the error.
Any help would be appreciated.
*Details*
hive> CREATE TABLE individuals
(
id INT,
name STRING,
age INT
)
STORED BY 'com.mongodb.hadoop.hive.MongoStorageHandler'
2.151:27017/test.persons2');
OK
Time taken: 0.294 seconds
hive> select * from individuals;
OK
Time taken: 0.351 seconds
hive> insert into table individuals select 1, 'name column', 5 from
some_table limit 1;
Query ID = hdfs_20140627134040_5a3db212-e6e1-42cf-a6cc-0a0078ef3bb3
Total jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
set hive.exec.reducers.bytes.per.reducer=<number>
set hive.exec.reducers.max=<number>
set mapreduce.job.reduces=<number>
Starting Job = job_1403731258511_0038, Tracking URL =
http://192.168.2.40:8088/proxy/application_1403731258511_0038/
Kill Command = /usr/lib/hadoop/bin/hadoop job -kill
job_1403731258511_0038
Hadoop job information for Stage-0: number of mappers: 0; number of
reducers: 0
2014-06-27 13:40:16,335 Stage-0 map = 0%, reduce = 0%
Ended Job = job_1403731258511_0038 with errors
*Error during job, obtaining debugging information...*
*FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.mr.MapRedTask*
Job 0: HDFS Read: 0 HDFS Write: 0 FAIL
Total MapReduce CPU Time Spent: 0 msec
hive>
--
You received this message because you are subscribed to the Google
Groups "mongodb-user"
group.
http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google
Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it,
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit
https://groups.google.com/d/msgid/mongodb-user/5e188766-e0a1
-4cd1-a776-428f7ec3d20c%40googlegroups.com
<https://groups.google.com/d/msgid/mongodb-user/5e188766-e0a1-4cd1-a776-428f7ec3d20c%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google
Groups "mongodb-user"
group.
http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google
Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it,
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/
msgid/mongodb-user/15c78506-5c78-41fb-bf6d-1fc051709748%
40googlegroups.com
<https://groups.google.com/d/msgid/mongodb-user/15c78506-5c78-41fb-bf6d-1fc051709748%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google
Groups "mongodb-user"
group.
http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google
Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit
https://groups.google.com/d/msgid/mongodb-user/5e9edb88-4fa4-49ff-9624-c159b5118bdd%40googlegroups.com
<https://groups.google.com/d/msgid/mongodb-user/5e9edb88-4fa4-49ff-9624-c159b5118bdd%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
To post to this group, send email to mongodb-user-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/a99bc420-729f-426c-8841-7efafde13dd9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
mahesh kumar
2014-09-16 13:18:13 UTC
Permalink
Post by LM Heah
Hi Josh Morris,
Am also having same problem.Please
help to solve the issue.


Thanks

Mahesh,
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
To post to this group, send email to mongodb-user-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/5d171148-56f9-4259-b63f-2c4f97439f9a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Josh Morris
2014-09-16 13:47:32 UTC
Permalink
Hi Mahesh,

I can haven't had a chance to log a fix yet. I can try to help you out,
give me a bit of time and I can see what the easiest/quickest way to do
that will be.

Josh
Post by LM Heah
Hi Josh Morris,
Am also having same problem.Please
help to solve the issue.
Thanks
Mahesh,
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to a topic in the
Google Groups "mongodb-user" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/mongodb-user/lKbha0SzMP8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit
https://groups.google.com/d/msgid/mongodb-user/5d171148-56f9-4259-b63f-2c4f97439f9a%40googlegroups.com
<https://groups.google.com/d/msgid/mongodb-user/5d171148-56f9-4259-b63f-2c4f97439f9a%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
------------------------------
Joshua Morris
senior software engineer | Mastery*Connect*
josh-***@public.gmane.org | 801.736.0258

*Check out a quick video about MasteryConnect - http://mstr.it/WRWANw
<http://mstr.it/WRWANw>*
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
To post to this group, send email to mongodb-user-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/CAMSdV9BtA9-LGUxx0DDeLSdR5YBKa_J_xzvLAL4Z35-ZUzrKvQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
mahesh kumar
2014-09-17 03:58:09 UTC
Permalink
Hi Josh,
Thank you for your quick reply.i am also trying to
find the solution.If you find the solution means tell me as soon as
possible.

Thanks
Mahesh
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
To post to this group, send email to mongodb-user-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/3f1caa59-344d-4c33-82ee-6bbcde93472c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
mahesh kumar
2014-09-17 08:30:57 UTC
Permalink
Hi Josh,

Thank you for your quick reply.i am also trying to
find the solution.If you find the solution means tell me as soon as
possible.

Thanks
Mahesh
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
To post to this group, send email to mongodb-user-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/e0cca375-34b8-452b-bca8-5a286fa21fc8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
mahesh kumar
2014-09-17 08:34:44 UTC
Permalink
Post by mahesh kumar
Hi Josh,
I just changed my hive version into 0.12.0 and working
fine now..My previous hive version is 0.13.0.

Thanks
Mahesh
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
To post to this group, send email to mongodb-user-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/557df6bc-16bc-43e9-901b-a18cecca4cb5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
John De Goes
2014-09-17 20:39:04 UTC
Permalink
Hi Josh,

I don't know if you have to use Hive or not, but if you're just looking for
an easy way to query MongoDB, you might give SlamData a try.

SlamData is an open source project that lets you write ANSI SQL against
MongoDB. There's no need to "create" tables, you can instantly begin
querying data. Also you can create intermediate results in the front-end or
the REPL, e.g. `out1 := SELECT ...`, which will appear in MongoDB. Useful
when you have to do some data prep / transformation.

Regards,

John
Post by Josh Morris
I get an error (see below) while trying to insert data into a hive table
that is connected to a Mongodb table. I'm able to create, drop and query
the table just fine e.g. if I insert data into the Mongo collection first
and perform a select from the Hive cli I see the Mongo data.
FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.mr.MapRedTask
I'm using HDP 2.1 set up across 3 ec2 instances, Mongodb 2.6 on another
ec2 instance.
So far I haven't been able to find more details about the error. Any help
would be appreciated.
*Details*
hive> CREATE TABLE individuals
(
id INT,
name STRING,
age INT
)
STORED BY 'com.mongodb.hadoop.hive.MongoStorageHandler'
TBLPROPERTIES('mongo.uri'='mongodb://
OK
Time taken: 0.294 seconds
hive> select * from individuals;
OK
Time taken: 0.351 seconds
hive> insert into table individuals select 1, 'name column', 5 from
some_table limit 1;
Query ID = hdfs_20140627134040_5a3db212-e6e1-42cf-a6cc-0a0078ef3bb3
Total jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
set hive.exec.reducers.bytes.per.reducer=<number>
set hive.exec.reducers.max=<number>
set mapreduce.job.reduces=<number>
Starting Job = job_1403731258511_0038, Tracking URL =
http://192.168.2.40:8088/proxy/application_1403731258511_0038/
Kill Command = /usr/lib/hadoop/bin/hadoop job -kill job_1403731258511_0038
Hadoop job information for Stage-0: number of mappers: 0; number of
reducers: 0
2014-06-27 13:40:16,335 Stage-0 map = 0%, reduce = 0%
Ended Job = job_1403731258511_0038 with errors
*Error during job, obtaining debugging information...*
*FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.mr.MapRedTask*
Job 0: HDFS Read: 0 HDFS Write: 0 FAIL
Total MapReduce CPU Time Spent: 0 msec
hive>
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
To post to this group, send email to mongodb-user-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/799bc083-5d2b-44c8-8861-dd2ec6b9f9c0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
JOHN MILLER
2016-05-17 13:32:46 UTC
Permalink
*Hi JoshSeems like this is becoming a popular problem I am also having the
same issues when doing a Hive query or cascading-hivehive> select count(*)
from commoncrawl18;Query ID =
jmill383_20160517082328_d80b48c0-5382-4499-b574-ab6d0ab80fcdTotal jobs =
1Launching Job 1 out of 1Number of reduce tasks determined at compile time:
1In order to change the average load for a reducer (in bytes): set
hive.exec.reducers.bytes.per.reducer=<number>In order to limit the maximum
number of reducers: set hive.exec.reducers.max=<number>In order to set a
constant number of reducers: set mapreduce.job.reduces=<number>Starting
Job = job_1463423258231_0003, Tracking URL =
http://starchild.ltsnet.net:8088/proxy/application_1463423258231_0003/Kill
Command = /opt/hadoop/bin/hadoop job -kill
job_1463423258231_0003Interrupting... Be patient, this might take some
time.Press Ctrl+C again to kill JVMkilling job with:
job_1463423258231_0003Hadoop job information for Stage-1: number of
mappers: 0; number of reducers: 02016-05-17 08:48:08,833 Stage-1 map = 0%,
reduce = 0%Ended Job = job_1463423258231_0003 with errorsError during job,
obtaining debugging information...Job Tracking URL:
http://starchild.ltsnet.net:8088/cluster/app/application_1463423258231_0003FAILED:
Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.mr.MapRedTaskMapReduce Jobs Launched:
Stage-Stage-1: HDFS Read: 0 HDFS Write: 0 FAILTotal MapReduce CPU Time
Spent: 0 msechive> Please advise if u can assistJohn M*
Post by Josh Morris
I get an error (see below) while trying to insert data into a hive table
that is connected to a Mongodb table. I'm able to create, drop and query
the table just fine e.g. if I insert data into the Mongo collection first
and perform a select from the Hive cli I see the Mongo data.
FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.mr.MapRedTask
I'm using HDP 2.1 set up across 3 ec2 instances, Mongodb 2.6 on another
ec2 instance.
So far I haven't been able to find more details about the error. Any help
would be appreciated.
*Details*
hive> CREATE TABLE individuals
(
id INT,
name STRING,
age INT
)
STORED BY 'com.mongodb.hadoop.hive.MongoStorageHandler'
TBLPROPERTIES('mongo.uri'='mongodb://
OK
Time taken: 0.294 seconds
hive> select * from individuals;
OK
Time taken: 0.351 seconds
hive> insert into table individuals select 1, 'name column', 5 from
some_table limit 1;
Query ID = hdfs_20140627134040_5a3db212-e6e1-42cf-a6cc-0a0078ef3bb3
Total jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
set hive.exec.reducers.bytes.per.reducer=<number>
set hive.exec.reducers.max=<number>
set mapreduce.job.reduces=<number>
Starting Job = job_1403731258511_0038, Tracking URL =
http://192.168.2.40:8088/proxy/application_1403731258511_0038/
Kill Command = /usr/lib/hadoop/bin/hadoop job -kill job_1403731258511_0038
Hadoop job information for Stage-0: number of mappers: 0; number of
reducers: 0
2014-06-27 13:40:16,335 Stage-0 map = 0%, reduce = 0%
Ended Job = job_1403731258511_0038 with errors
*Error during job, obtaining debugging information...*
*FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.mr.MapRedTask*
Job 0: HDFS Read: 0 HDFS Write: 0 FAIL
Total MapReduce CPU Time Spent: 0 msec
hive>
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: https://docs.mongodb.org/manual/support/
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+***@googlegroups.com.
To post to this group, send email to mongodb-***@googlegroups.com.
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/9ec1fc5c-dbcb-42fd-ab81-38d1cf65eb33%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Josh Morris
2016-05-18 15:18:41 UTC
Permalink
John,

I haven't looked at this for some time so I might be a bit rusty. Do you
see the same SAXParseException with this error? I'm don't remember where I
looked to find that, however I do remember it came down to an invalid XML
character.
Post by JOHN MILLER
*Hi JoshSeems like this is becoming a popular problem I am also having
the same issues when doing a Hive query or cascading-hivehive> select
count(*) from commoncrawl18;Query ID =
jmill383_20160517082328_d80b48c0-5382-4499-b574-ab6d0ab80fcdTotal jobs =
1In order to change the average load for a reducer (in bytes): set
hive.exec.reducers.bytes.per.reducer=<number>In order to limit the maximum
number of reducers: set hive.exec.reducers.max=<number>In order to set a
constant number of reducers: set mapreduce.job.reduces=<number>Starting
Job = job_1463423258231_0003, Tracking URL =
http://starchild.ltsnet.net:8088/proxy/application_1463423258231_0003/
<http://starchild.ltsnet.net:8088/proxy/application_1463423258231_0003/>Kill
Command = /opt/hadoop/bin/hadoop job -kill
job_1463423258231_0003Interrupting... Be patient, this might take some
job_1463423258231_0003Hadoop job information for Stage-1: number of
mappers: 0; number of reducers: 02016-05-17 08:48:08,833 Stage-1 map = 0%,
reduce = 0%Ended Job = job_1463423258231_0003 with errorsError during job,
http://starchild.ltsnet.net:8088/cluster/app/application_1463423258231_0003
Execution Error, return code 2 from
Stage-Stage-1: HDFS Read: 0 HDFS Write: 0 FAILTotal MapReduce CPU Time
Spent: 0 msechive> Please advise if u can assistJohn M*
Post by Josh Morris
I get an error (see below) while trying to insert data into a hive table
that is connected to a Mongodb table. I'm able to create, drop and query
the table just fine e.g. if I insert data into the Mongo collection first
and perform a select from the Hive cli I see the Mongo data.
FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.mr.MapRedTask
I'm using HDP 2.1 set up across 3 ec2 instances, Mongodb 2.6 on another
ec2 instance.
So far I haven't been able to find more details about the error. Any help
would be appreciated.
*Details*
hive> CREATE TABLE individuals
(
id INT,
name STRING,
age INT
)
STORED BY 'com.mongodb.hadoop.hive.MongoStorageHandler'
TBLPROPERTIES('mongo.uri'='mongodb://
OK
Time taken: 0.294 seconds
hive> select * from individuals;
OK
Time taken: 0.351 seconds
hive> insert into table individuals select 1, 'name column', 5 from
some_table limit 1;
Query ID = hdfs_20140627134040_5a3db212-e6e1-42cf-a6cc-0a0078ef3bb3
Total jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
set hive.exec.reducers.bytes.per.reducer=<number>
set hive.exec.reducers.max=<number>
set mapreduce.job.reduces=<number>
Starting Job = job_1403731258511_0038, Tracking URL =
http://192.168.2.40:8088/proxy/application_1403731258511_0038/
Kill Command = /usr/lib/hadoop/bin/hadoop job -kill
job_1403731258511_0038
Hadoop job information for Stage-0: number of mappers: 0; number of
reducers: 0
2014-06-27 13:40:16,335 Stage-0 map = 0%, reduce = 0%
Ended Job = job_1403731258511_0038 with errors
*Error during job, obtaining debugging information...*
*FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.mr.MapRedTask*
Job 0: HDFS Read: 0 HDFS Write: 0 FAIL
Total MapReduce CPU Time Spent: 0 msec
hive>
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
https://docs.mongodb.org/manual/support/
---
You received this message because you are subscribed to a topic in the
Google Groups "mongodb-user" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/mongodb-user/lKbha0SzMP8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit
https://groups.google.com/d/msgid/mongodb-user/9ec1fc5c-dbcb-42fd-ab81-38d1cf65eb33%40googlegroups.com
<https://groups.google.com/d/msgid/mongodb-user/9ec1fc5c-dbcb-42fd-ab81-38d1cf65eb33%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
------------------------------
Joshua Morris
director of engineering | Mastery*Connect*
***@masteryconnect.com | 801.736.0258

*Check out a quick video about MasteryConnect - http://mstr.it/WRWANw
<http://mstr.it/WRWANw>*
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: https://docs.mongodb.org/manual/support/
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+***@googlegroups.com.
To post to this group, send email to mongodb-***@googlegroups.com.
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/CAMSdV9A2G6bTWt81nhJ2RuPZOwdVoBwnNNKek8w_VLvJMT%2B5jg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Luke Lovett
2016-05-18 15:19:13 UTC
Permalink
Hi John,

The output you posted just shows there was an error when executing a MR job
launched from Hive. Can you provide more detailed output? I'd be interested
in seeing what's in the Hive logs after this job failed.
Post by JOHN MILLER
*Hi JoshSeems like this is becoming a popular problem I am also having
the same issues when doing a Hive query or cascading-hivehive> select
count(*) from commoncrawl18;Query ID =
jmill383_20160517082328_d80b48c0-5382-4499-b574-ab6d0ab80fcdTotal jobs =
1In order to change the average load for a reducer (in bytes): set
hive.exec.reducers.bytes.per.reducer=<number>In order to limit the maximum
number of reducers: set hive.exec.reducers.max=<number>In order to set a
constant number of reducers: set mapreduce.job.reduces=<number>Starting
Job = job_1463423258231_0003, Tracking URL =
http://starchild.ltsnet.net:8088/proxy/application_1463423258231_0003/
<http://starchild.ltsnet.net:8088/proxy/application_1463423258231_0003/>Kill
Command = /opt/hadoop/bin/hadoop job -kill
job_1463423258231_0003Interrupting... Be patient, this might take some
job_1463423258231_0003Hadoop job information for Stage-1: number of
mappers: 0; number of reducers: 02016-05-17 08:48:08,833 Stage-1 map = 0%,
reduce = 0%Ended Job = job_1463423258231_0003 with errorsError during job,
http://starchild.ltsnet.net:8088/cluster/app/application_1463423258231_0003
Execution Error, return code 2 from
Stage-Stage-1: HDFS Read: 0 HDFS Write: 0 FAILTotal MapReduce CPU Time
Spent: 0 msechive> Please advise if u can assistJohn M*
Post by Josh Morris
I get an error (see below) while trying to insert data into a hive table
that is connected to a Mongodb table. I'm able to create, drop and query
the table just fine e.g. if I insert data into the Mongo collection first
and perform a select from the Hive cli I see the Mongo data.
FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.mr.MapRedTask
I'm using HDP 2.1 set up across 3 ec2 instances, Mongodb 2.6 on another
ec2 instance.
So far I haven't been able to find more details about the error. Any help
would be appreciated.
*Details*
hive> CREATE TABLE individuals
(
id INT,
name STRING,
age INT
)
STORED BY 'com.mongodb.hadoop.hive.MongoStorageHandler'
TBLPROPERTIES('mongo.uri'='mongodb://
OK
Time taken: 0.294 seconds
hive> select * from individuals;
OK
Time taken: 0.351 seconds
hive> insert into table individuals select 1, 'name column', 5 from
some_table limit 1;
Query ID = hdfs_20140627134040_5a3db212-e6e1-42cf-a6cc-0a0078ef3bb3
Total jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
set hive.exec.reducers.bytes.per.reducer=<number>
set hive.exec.reducers.max=<number>
set mapreduce.job.reduces=<number>
Starting Job = job_1403731258511_0038, Tracking URL =
http://192.168.2.40:8088/proxy/application_1403731258511_0038/
Kill Command = /usr/lib/hadoop/bin/hadoop job -kill
job_1403731258511_0038
Hadoop job information for Stage-0: number of mappers: 0; number of
reducers: 0
2014-06-27 13:40:16,335 Stage-0 map = 0%, reduce = 0%
Ended Job = job_1403731258511_0038 with errors
*Error during job, obtaining debugging information...*
*FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.mr.MapRedTask*
Job 0: HDFS Read: 0 HDFS Write: 0 FAIL
Total MapReduce CPU Time Spent: 0 msec
hive>
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: https://docs.mongodb.org/manual/support/
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+***@googlegroups.com.
To post to this group, send email to mongodb-***@googlegroups.com.
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/a1d03334-3366-416e-8d7e-73d7190aae29%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Continue reading on narkive:
Loading...