设为首页 加入收藏

TOP

HBase快速导入数据--BulkLoad(四)
2015-08-31 19:59:45 来源: 作者: 【 】 浏览:189
Tags:HBase 快速 导入 数据 --BulkLoad
ng up current regions for table temperature(表名)
15/06/14 14:31:07 INFO mapreduce.HFileOutputFormat2: Configuring 1 reduce partitions to match current region count
15/06/14 14:31:07 INFO mapreduce.HFileOutputFormat2: Writing partition information to /home/shaobo/hadoop/tmp/partitions_5d464f1e-d412-4dbe-bb98-367f8431bdc9
15/06/14 14:31:07 INFO zlib.ZlibFactory: Successfully loaded & initialized native-zlib library
15/06/14 14:31:07 INFO compress.CodecPool: Got brand-new compressor [.deflate]
15/06/14 14:31:08 INFO mapreduce.HFileOutputFormat2: Incremental table temperature(表名) output configured.
15/06/14 14:31:08 INFO client.RMProxy: Connecting to ResourceManager at localhost/127.0.0.1:8032
15/06/14 14:31:15 INFO input.FileInputFormat: Total input paths to process : 2
15/06/14 14:31:15 INFO mapreduce.JobSubmitter: number of splits:2
15/06/14 14:31:16 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1434262360688_0002
15/06/14 14:31:17 INFO impl.YarnClientImpl: Submitted application application_1434262360688_0002
15/06/14 14:31:17 INFO mapreduce.Job: The url to track the job: http://shaobo-ThinkPad-E420:8088/proxy/application_1434262360688_0002/
15/06/14 14:31:17 INFO mapreduce.Job: Running job: job_1434262360688_0002
15/06/14 14:31:28 INFO mapreduce.Job: Job job_1434262360688_0002 running in uber mode : false
15/06/14 14:31:28 INFO mapreduce.Job:? map 0% reduce 0%
15/06/14 14:32:24 INFO mapreduce.Job:? map 49% reduce 0%
15/06/14 14:32:37 INFO mapreduce.Job:? map 67% reduce 0%
15/06/14 14:32:43 INFO mapreduce.Job:? map 100% reduce 0%
15/06/14 14:33:39 INFO mapreduce.Job:? map 100% reduce 67%
15/06/14 14:33:42 INFO mapreduce.Job:? map 100% reduce 70%
15/06/14 14:33:45 INFO mapreduce.Job:? map 100% reduce 88%
15/06/14 14:33:48 INFO mapreduce.Job:? map 100% reduce 100%
15/06/14 14:33:52 INFO mapreduce.Job: Job job_1434262360688_0002 completed successfully
...
...
...
15/06/14 14:34:02 WARN mapreduce.LoadIncrementalHFiles: Skipping non-directory hdfs://localhost:9000/user/output/_SUCCESS
15/06/14 14:34:03 INFO hfile.CacheConfig: CacheConfig:disabled
15/06/14 14:34:03 INFO hfile.CacheConfig: CacheConfig:disabled
15/06/14 14:34:07 INFO mapreduce.LoadIncrementalHFiles: Trying to load hfile=hdfs://localhost:9000/user/output/date/c64cd2524fba48738bab26630d550b61 first=AQW00061705 last=USW00094910
15/06/14 14:34:07 INFO mapreduce.LoadIncrementalHFiles: Trying to load hfile=hdfs://localhost:9000/user/output/tempPerHour/43af29456913444795a820544691eb3d first=AQW00061705 last=USW00094910
Bulk Load Completed..
Job is successfully completed...


BulLoad过程的第三步也可以在用MapReduce作业生成HBase数据文件后在命令行中进行,不一定要与MapReduce过程写在一起。


$ hadoop jar hbase-server-VERSION.jar completebulkload [-c /path/to/hbase/config/hbase-site.xml] outputpath tablename1


若在提交作业是产生如下异常:


15/06/16 11:41:06 INFO mapreduce.Job: Job job_1434420992867_0003 failed with state FAILED due to: Application application_1434420992867_0003 failed 2 times due to AM Container for appattempt_1434420992867_0003_000002 exited with? exitCode: -1000
For more detailed output, check application tracking page:http://cdh1:8088/proxy/application_1434420992867_0003/Then, click on links to logs of each attempt.
Diagnostics: Rename cannot overwrite non empty destination directory /data/yarn/nm/usercache/hdfs/filecache/16
java.io.

首页 上一页 1 2 3 4 下一页 尾页 4/4/4
】【打印繁体】【投稿】【收藏】 【推荐】【举报】【评论】 【关闭】 【返回顶部
分享到: 
上一篇MySQL5.6新特性之crash-safe 下一篇Hive本地模式安装及遇到的问题和..

评论

帐  号: 密码: (新用户注册)
验 证 码:
表  情:
内  容:

·如何在 C 语言中管理 (2025-12-25 03:20:14)
·C语言和内存管理有什 (2025-12-25 03:20:11)
·为什么C语言从不被淘 (2025-12-25 03:20:08)
·常用meta整理 | 菜鸟 (2025-12-25 01:21:52)
·SQL HAVING 子句:深 (2025-12-25 01:21:47)