Quantcast File System (QFS)

Anand Bisen bio photo By Anand Bisen Comment

In this post I will describe steps to prepare a node to be used as a build environment for compiling QFS binaries. With the pervasiveness of virtual machines I find it easier to spin up a new VM for performing tasks like these than to use an existing machine that is being used for some other use.

First spin up a new CentOS 6.4 virtual machine with the default “Minimal Config” profile. And the install the necessary tools for the build environment. Once the initial install is finished follow the steps below to get the system ready for compiling QFS.

1. Install Development Tools, EPEL Repository and some other required packages

$ yum groupinstall "Development tools"
$ rpm -Uvh http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
$ yum install -y boost boost-devel git xfsprogs xfsprogs-devel \
 	libuuid-devel cmake openssl-devel wget

2. Install and configure Sun/Oracle JAVA

I don’t know if it’s mandatory or optional, but I tend to stick with Oracle Java while playing with Hadoop as it minimizes the variables that I have to think about when I face some issue. There is a post on how to install / configure Sun Java properly on a CentOS host.

3. Install and configure Apache Maven

QFS build system relies on Apache Maven for which the required revision was not available on yum repository. So I just built my own. Below are the steps I took for the setup.::

$ wget http://apache.parentingamerica.com/maven/maven-3/3.0.5/binaries/apache-maven-3.0.5-bin.tar.gz
$ tar -zxvf apache-maven-3.0.5-bin.tar.gz -C /opt/
$ ln -s /opt/apache-maven-3.0.5 /opt/mvn

Create a file /etc/profile.d/maven.sh so that the proper environment variables are configured when one log’s in.


export M2_HOME=/opt/mvn
export PATH=$M2_HOME/bin:$PATH

4. Download and Build Binaries

Source code for latest version of QFS (Version 1.0.2 at the time of this post) could be downloaded from the following link on GitHub. The image below shows a screenshot… please make sure you are downloading 1.0.2 and not a different release or “master”

Unzip the content to a directory and before compiling I had to change the file src/cc/common/buildversgit.sh to ensure that the compiled jar’s had the proper version information as a suffix. The commented out lines are highlighted below

# Set official release version here.

# If git is present override the release version with git tag.
#if which git > /dev/null 2>&1; then
#    script_dir=`dirname "$0"`
#    qfs_release_version=`cd "$script_dir" >/dev/null 2>&1 && git describe --abbrev=0 --tags 2>/dev/null`
#    qfs_source_revision=`cd "$script_dir" >/dev/null 2>&1 && git log -n 1 --pretty=format:%H 2>/dev/null`

if [ $# -eq 1 -a x"$1" = x'-v' ]; then
   echo "$qfs_release_version"
   echo "$qfs_source_revision"
   exit 0

Once this is done just executing make; make tarball should compile the code successfully and the binary tarball should be placed in build/ subdirectory. This file is ready for distribution across your data-nodes.

comments powered by Disqus