Siti Hajar Hanani
2018-11-28 08:09:25 UTC
Hi,
I'm having trouble setting up mongo-hadoop connector.. I try to load data
from mongodb to hadoop using Hive.
At first I got error while trying to create a table using below command:
add jar /usr/local/hadoop/lib/mongo-hadoop-core-2.0.2.jar;
[image: mongohive.PNG]
but then I try add a few others jar:
add jar /usr/local/hadoop/lib/hive-serde-1.2.1.jar;
add jar
/usr/local/Hive-JSON-Serde/json-serde/target/json-serde-1.3.9-SNAPSHOT-jar-with-dependencies.jar;
and I also change ROW FORMAT SERDE 'org.openx.data.jsonserde.JsonSerDe',
the error is gone and I'm* successfuly* create the table. However, the
table is empty. I try to run SELECT * FROM testing22_to_mongo; but got this
error.
[image: Screenshot from 2018-11-28 15-51-34.png]
I am using :-
java version "1.8.0_191"
Java(TM) SE Runtime Environment (build 1.8.0_191-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.191-b12, mixed mode)
MongoDB shell version v4.0.4
Hive 2.3.3
I not user if this is jar compatibility issue of something else. But if it
is, how to find the compatible jar version? I'm a newbie in all of this. I
just want to load the data from mongodb to hadoop, If have any other option
or any other way to do it, I would love to try the suggestion.
Thank you.
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
For other MongoDB technical support options, see: https://docs.mongodb.com/manual/support/
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+***@googlegroups.com.
To post to this group, send email to mongodb-***@googlegroups.com.
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/35146fc6-9618-4ebc-bc61-60fd9e644f09%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
I'm having trouble setting up mongo-hadoop connector.. I try to load data
from mongodb to hadoop using Hive.
At first I got error while trying to create a table using below command:
add jar /usr/local/hadoop/lib/mongo-hadoop-core-2.0.2.jar;
add jar /usr/local/hadoop/lib/mongo-hadoop-hive-2.0.2.jar;
add jar /usr/local/hadoop/lib/mongo-java-driver-3.2.1.jar;
CREATE TABLE testing22_to_mongo (add jar /usr/local/hadoop/lib/mongo-java-driver-3.2.1.jar;
id String,
age String,
gender String,
race String,
custState String,
purchaseDate String,
purchaseTime String,
foodname String,
restaurant String,
foodtype String,
quantity String,
totalPrice String,
orderType String,
rating String,
servingType String,
characteristic String,
restaurantType String,
restaurantState String,
priceRange String,
paymentMethod String,
tableBooking String,
onlineBooking String,
deliveryService String
)
ROW FORMAT SERDE 'com.mongodb.hadoop.hive.BSONSerDe' STORED BY
'com.mongodb.hadoop.hive.MongoStorageHandler'
WITH SERDEPROPERTIES ('mongo.columns.mapping'='{"id":"_id", "age":"age",age String,
gender String,
race String,
custState String,
purchaseDate String,
purchaseTime String,
foodname String,
restaurant String,
foodtype String,
quantity String,
totalPrice String,
orderType String,
rating String,
servingType String,
characteristic String,
restaurantType String,
restaurantState String,
priceRange String,
paymentMethod String,
tableBooking String,
onlineBooking String,
deliveryService String
)
ROW FORMAT SERDE 'com.mongodb.hadoop.hive.BSONSerDe' STORED BY
'com.mongodb.hadoop.hive.MongoStorageHandler'
"gender":"gender", "race":"race", "custState":"custState",
"purchaseDate":"purchaseDate", "purchaseTime":"purchaseTime",
"foodname":"foodname", "restaurant":"restaurant", "foodtype":"foodtype",
"quantity":"quantity", "totalPrice":"totalPrice", "orderType":"orderType",
"rating":"rating", "servingType":"servingType",
"characteristic":"characteristic", "restaurantType":"restaurantType",
"restaurantState":"restaurantState", "priceRange":"priceRange",
"paymentMethod":"paymentMethod", "tableBooking":"tableBooking",
"onlineBooking":"onlineBooking", "deliveryService":"deliveryService"}')
TBLPROPERTIES('mongo.uri'='mongodb://localhost:27017/database.dataset');
and I got this error"purchaseDate":"purchaseDate", "purchaseTime":"purchaseTime",
"foodname":"foodname", "restaurant":"restaurant", "foodtype":"foodtype",
"quantity":"quantity", "totalPrice":"totalPrice", "orderType":"orderType",
"rating":"rating", "servingType":"servingType",
"characteristic":"characteristic", "restaurantType":"restaurantType",
"restaurantState":"restaurantState", "priceRange":"priceRange",
"paymentMethod":"paymentMethod", "tableBooking":"tableBooking",
"onlineBooking":"onlineBooking", "deliveryService":"deliveryService"}')
TBLPROPERTIES('mongo.uri'='mongodb://localhost:27017/database.dataset');
[image: mongohive.PNG]
but then I try add a few others jar:
add jar /usr/local/hadoop/lib/hive-serde-1.2.1.jar;
add jar
/usr/local/Hive-JSON-Serde/json-serde/target/json-serde-1.3.9-SNAPSHOT-jar-with-dependencies.jar;
and I also change ROW FORMAT SERDE 'org.openx.data.jsonserde.JsonSerDe',
the error is gone and I'm* successfuly* create the table. However, the
table is empty. I try to run SELECT * FROM testing22_to_mongo; but got this
error.
[image: Screenshot from 2018-11-28 15-51-34.png]
I am using :-
java version "1.8.0_191"
Java(TM) SE Runtime Environment (build 1.8.0_191-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.191-b12, mixed mode)
MongoDB shell version v4.0.4
Hive 2.3.3
I not user if this is jar compatibility issue of something else. But if it
is, how to find the compatible jar version? I'm a newbie in all of this. I
just want to load the data from mongodb to hadoop, If have any other option
or any other way to do it, I would love to try the suggestion.
Thank you.
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
For other MongoDB technical support options, see: https://docs.mongodb.com/manual/support/
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+***@googlegroups.com.
To post to this group, send email to mongodb-***@googlegroups.com.
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/35146fc6-9618-4ebc-bc61-60fd9e644f09%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.