Skip to main content

Posts

Recent posts

Run MapReduce Program Against Hadoop Cluster in Eclipse

For a long while, I used command line to run a mapreduce program against my hadoop cluster by  'hadoop jar ${jar name} ${class name} -libjars ${lib jars}". It's kind of terrible because I have to make the MapReduce program into jar ( I usually program it in IDE like Eclipse, who don't? ) and then switch to terminal to run the long long long 'hadoop jar ...' command. The reason I did so because I tried to run that Mapreduce in Eclipse, but I find it's always running locally, means LocalJobRunner is invoked. My hadoop cluster is not used ( and neither Yarn ). I searched online to find a way to solve this, but was only told that using Eclipse MapReduce program can only run in local mode. I was doing that way until yesterday I suddenly came up with an idea to configure Eclipse and run a MapReduce program against a cluster MYSELF. So here it is. The environment I'm using ( although I think that doesn't matter much): Hadoop Client Package: Apache Ha...