hadoop常用维护命令

列出正在运行的job

$ hadoop job -list

kill掉job（1234指jobid）

$ hadoop job -kill 1234

查看hdfs块状态

$ hadoop fsck /  
FSCK started by nange from /127.0.0.1 for path / at Mon Apr 21 22:26:29 CST 2014  
................................  
/hdfs/test/avg/out/part-r-00000:  Under replicated blk_958972218415296316_1141. Target Replicas is 3 but found 1 replica(s).  
.  
/hdfs/test/hdfsoper/2.f:  Under replicated blk_1024136243155007162_1126. Target Replicas is 3 but found 1 replica(s).

查看hdfs状态，并删除损坏块

$ hadoop fsck / -delete

检查dfs状态，以及datanode信息

    $ hadoop dfsadmin -report  
    Configured Capacity: 155277496320 (144.61 GB)  
    Present Capacity: 138432897024 (128.93 GB)  
    DFS Remaining: 138429386752 (128.92 GB)  
    DFS Used: 3510272 (3.35 MB)  
    DFS Used%: 0%  
    Under replicated blocks: 14  
    Blocks with corrupt replicas: 0  
    Missing blocks: 0  
    -------------------------------------------------  
    Datanodes available: 1 (1 total, 0 dead)  

    Name: 127.0.0.1:50010  
    Decommission Status : Normal  
    Configured Capacity: 155277496320 (144.61 GB)  
    DFS Used: 3510272 (3.35 MB)  
    Non DFS Used: 16844599296 (15.69 GB)  
    DFS Remaining: 138429386752(128.92 GB)  
    DFS Used%: 0%  
    DFS Remaining%: 89.15%  
    Last contact: Mon Apr 21 22:33:15 CST 2014

安全模式状态查询，进入，离开

hadoop dfsadmin -safemode [get|enter|leave]

hadoop 并行拷贝（将a节点数据拷贝到b节点）

hadoop distcp hdfs://a:9000/a hdfs://b:9000/b

平衡集群文件

$ start-balancer.sh   
starting balancer, logging to /home/nange/programs/hadoopWS/hadoop-1.2.1/libexec/../logs/hadoop-nange-balancer-ubuntu.out

喜欢 (1)or分享 (0)