From zero to HDFS in 60 min.

HDP Ambari dashboard

(Okay, so you can be up and running quicker if you have a better internet connection than me.)

Want to get your hands dirty with Hadoop related technologies but don’t have time to waste?  I’ve spent way too much time trying to get HBase, for example, running on my Macbook with Brew and wish I had just tried this Virtualbox approach before.

In this short post I show how easy it was for me to get an NFS share mounted on OSX – so I could transparently and simply copy files on HDFS without needing any special tools.   Here are the details…

Hortonworks Sandbox is an excellent virtual machine file, that’s primed and loaded with all the tools you would want to try out.  Best of all, it’s free and a simple (meh) 2.5GB download.  Virtualbox, VMWare and Hyper-V versions are all available.

Once I launched it, I had access to a simple web console, showing me all the varied apps I have running.

HDP main web console
HDP main web console

Here I enabled Ambari and was able to access that dashboard right away.  (Some tutorials say you have to reboot the VM, but I didn’t have to.)

HDP Ambari dashboard
HDP Ambari dashboard

The first thing I wanted to do was enable file access via NFS.  You read that right – I had heard earlier today that Hortonworks Data Platform (HDP) supported NFS right out of the box.  To be fair, I don’t know what others are doing, so maybe this is old news, but it was good news for me.

So using various configuration windows via Ambari, I went through a tutorial for configuring NFS capabilities.  However, to my joy, I found that all the configuration changes I needed to make were already made by default!  So, NFS by default sounded good to me.

After switching Virtualbox from NAT networking to Bridged mode, I then told OSX to mount the drive.  Et voila!

Mounting NFS share from HDP HDFS on OSX
Mounting NFS share from HDP HDFS on OSX

 

So I’m just getting started with using this Sandbox environment but it was definitely a timesaver for setting up a development environment.

I’ll want to jump into testing HBase with some other work I’m doing, but I may also get tempted into trying various other apps they’ve packaged neatly into this distribution.. when I have some spare time that is. 😉


© Tyler Mitchell / Spatialguru.com