{"id":6,"date":"2014-06-09T13:22:00","date_gmt":"2014-06-09T17:22:00","guid":{"rendered":"https:\/\/mberlove.com\/blog\/?p=6"},"modified":"2014-10-08T11:23:54","modified_gmt":"2014-10-08T15:23:54","slug":"more-little-lessons-in-hadoop","status":"publish","type":"post","link":"https:\/\/mberlove.com\/blog\/more-little-lessons-in-hadoop\/","title":{"rendered":"More Little Lessons in Hadoop"},"content":{"rendered":"<p>Previously, I had posted on some common issues encountered when installing and running Hadoop for the first time.<\/p>\n<p>&nbsp;<\/p>\n<p>Now, in the joyful experience of attempting a multi-node setup on a simple virtual cluster, many more errors, unexpected consequences, and curios of server-related struggle have been forthcoming.<br \/>\nI hope that in relaying these attempts, failures, and (admittedly sparse) triumphs through this medium, I can allay some of the angst experienced by others in the future.<\/p>\n<p>&nbsp;<\/p>\n<p>Good luck, all Hadooper-troopers.<\/p>\n<p>&nbsp;<\/p>\n<p><span style=\"text-decoration: underline;\"><strong><span style=\"font-size: x-large;\">Can&#8217;t yum\/apt-get?\u00a0<\/span><\/strong><\/span><\/p>\n<p>&nbsp;<\/p>\n<p>Depending on OS version and pre-existing settings (especially when working on a stripped virtual image), your installer service may not be able to find certain packages.<\/p>\n<div class=\"separator\" style=\"clear: both; text-align: left;\"><a style=\"clear: left; float: left; margin-bottom: 1em; margin-right: 1em;\" href=\"http:\/\/1.bp.blogspot.com\/-SpGDB9lorC0\/U5CAuIobDYI\/AAAAAAAAAFk\/RRxw6D7b3gE\/s1600\/noyum.jpg\"><img loading=\"lazy\" decoding=\"async\" src=\"http:\/\/1.bp.blogspot.com\/-SpGDB9lorC0\/U5CAuIobDYI\/AAAAAAAAAFk\/RRxw6D7b3gE\/s1600\/noyum.jpg\" alt=\"\" width=\"640\" height=\"61\" border=\"0\" \/><\/a><\/div>\n<p>For a general inability to install, try the configurations\u00a0<a href=\"http:\/\/wiki.nginx.org\/Install\" target=\"_blank\">here<\/a>\u00a0for setting up the nginx repo.<br \/>\nA particular problem I ran across was installing the repository management on CentOS, either <b><span style=\"color: #cc0000;\">&#8220;python-software-properties&#8221;<\/span><\/b> or <b><span style=\"color: #cc0000;\">&#8220;software-properties-common&#8221; <\/span><\/b>depending on version.<\/p>\n<p>&nbsp;<\/p>\n<p>Turns out, you don&#8217;t need it, contrary to the results I found on several google searches.<br \/>\nSpecifically, if you&#8217;re following\u00a0<a href=\"http:\/\/www.michael-noll.com\/tutorials\/running-hadoop-on-ubuntu-linux-single-node-cluster\/\" target=\"_blank\">Michael Noll&#8217;s tutorial<\/a>, and you can&#8217;t install the software properties, try doing a yum search for Java, and you&#8217;ll probably get results.<br \/>\nHowever, if for some reason you don&#8217;t, here are some links that may be of use:<br \/>\n<a href=\"https:\/\/www.linuxquestions.org\/questions\/linux-software-2\/editing-yum-or-apt-repository-addreses-316205\/\" target=\"_blank\">Editing yum and apt-get repos.<\/a><br \/>\n<a href=\"http:\/\/wouldbetech.blogspot.com\/2013\/02\/the-basics-of-rpm-yum-repositories.html\" target=\"_blank\">Basics of yum repos.<\/a><\/p>\n<p>&nbsp;<\/p>\n<p><span style=\"text-decoration: underline;\"><strong><span style=\"font-size: x-large;\">Permission denied (publickey,gssapi-keyex,gssapi-with-mic).<\/span><\/strong><\/span><\/p>\n<p>&nbsp;<\/p>\n<p>This may be due to an incorrect permissions setting, which you can solve via <a href=\"http:\/\/ubuntuforums.org\/showthread.php?t=1932058\" target=\"_blank\">these directions<\/a>. However, especially if you&#8217;ve just sourced eucarc to get going, you may want to make sure you&#8217;re in your home directory&#8230;yes, I did make that mistake; it&#8217;s quite frustrating until it suddenly becomes embarrassing.<br \/>\nFor more general help on SSH through Ubuntu:<br \/>\n<a href=\"https:\/\/help.ubuntu.com\/community\/SSH\/OpenSSH\/Keys#Troubleshooting\" target=\"_blank\">https:\/\/help.ubuntu.com\/community\/SSH\/OpenSSH\/Keys#Troubleshooting<\/a><\/p>\n<p>&nbsp;<\/p>\n<p><b><span style=\"font-size: x-large;\"><span style=\"text-decoration: underline;\">\/usr\/lib\/jvm\/java&#8230;. Not a directory<\/span><\/span><\/b><\/p>\n<p>&nbsp;<\/p>\n<p>Most likely a mistyped or mixed-up Java directory, especially if you&#8217;ve switched Java versions recently or are setting up a new user.<br \/>\nCheck your <b><span style=\"color: #cc0000;\">hadoop-env.sh<\/span><\/b> file, and modify the path as necessary (probably ending with the jre directory).<\/p>\n<p>&nbsp;<\/p>\n<p><span style=\"text-decoration: underline;\"><strong><span style=\"font-size: x-large;\">java.net.UnknownHostException on\u00a0hadoop namenode -format<\/span><\/strong><\/span><\/p>\n<p>&nbsp;<\/p>\n<p><span style=\"font-family: inherit; line-height: 1.3;\"><a href=\"http:\/\/stackoverflow.com\/questions\/6307244\/hadoop-namenode-format-returns-a-java-net-unknownhostexception#8221\" target=\"_blank\">Follow these directions<\/a>.<\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Previously, I had posted on some common issues encountered when installing and running Hadoop for the first time. &nbsp; Now, in the joyful experience of attempting a multi-node setup on a simple virtual cluster, many more errors, unexpected consequences, and [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[20,21,22,16,23,24,32,26,27,28,29,30,31],"class_list":["post-6","post","type-post","status-publish","format-standard","hentry","category-uncategorized","tag-apt-get","tag-centos","tag-errors","tag-hadoop","tag-issues","tag-java","tag-linux","tag-permission-denied","tag-software-properties","tag-solution","tag-ubuntu","tag-unknownhostexception","tag-yum"],"_links":{"self":[{"href":"https:\/\/mberlove.com\/blog\/wp-json\/wp\/v2\/posts\/6","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mberlove.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/mberlove.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/mberlove.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/mberlove.com\/blog\/wp-json\/wp\/v2\/comments?post=6"}],"version-history":[{"count":6,"href":"https:\/\/mberlove.com\/blog\/wp-json\/wp\/v2\/posts\/6\/revisions"}],"predecessor-version":[{"id":53,"href":"https:\/\/mberlove.com\/blog\/wp-json\/wp\/v2\/posts\/6\/revisions\/53"}],"wp:attachment":[{"href":"https:\/\/mberlove.com\/blog\/wp-json\/wp\/v2\/media?parent=6"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/mberlove.com\/blog\/wp-json\/wp\/v2\/categories?post=6"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/mberlove.com\/blog\/wp-json\/wp\/v2\/tags?post=6"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}