Application Example - R

Overview

This section covers how to use the OASIS system to run a real application like R statistical package. For this example, we'll estimate the value of pi using a Monte Carlo method. We'll first run the program locally, then create a submit file, send it out to OSG-Connect, and collate our results.

Table of Contents

Background

Some background is useful here. We define a square inscribed by a unit circle. We randomly sample points, and calculate the ratio of the points outside of the circle to the points inside for the first quadrant. This ratio approaches pi/4.

This method converges extremely slowly, which makes it great for a CPU-intensive exercise (but bad for a real estimation!).

(Source: http://en.wikipedia.org/wiki/Monte_Carlo_method)

Accessing R on the submit host

First we'll need to create a working directory, you can either run tutorial R or type the following:

[user@login01 ~]$ mkdir osg-R; cd osg-R

Since R is installed into OASIS, it's not available in the normal system paths. We'll need to set up those paths so we can access R correctly. To do that we'll use PALMS:

[user@login01 osg-R]$ source /cvmfs/oasis.opensciencegrid.org/osg/palms/setup
[user@login01 osg-R]$ palmsdosetup R

Once we have the path set up, we can try to run R. Don't worry if you aren't an R expert, I'm not either.

[user@login01 osg-R]$ R

R version 3.0.1 (2013-05-16) -- "Good Sport"
Copyright (C) 2013 The R Foundation for Statistical Computing
Platform: x86_64-unknown-linux-gnu (64-bit)
R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.
  Natural language support but running in an English locale
R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.
Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

>

Great! R works. You can quit out with "q()".

> q()
Save workspace image? [y/n/c]: n
[user@login01 oc-R]$

Running R code

Now that we can run R, let's try using my Pi estimation code:

mcpi.R

montecarloPi <- function(trials) {
  count = 0
  for(i in 1:trials) {
    if((runif(1,0,1)^2 + runif(1,0,1)^2)<1) {
      count = count + 1
    }
  }
  return((count*4)/trials)
}
 
montecarloPi(10000000)

R normally runs as an interactive shell, but it's easy to run in batch mode too.

[user@login01 osg-R]$ Rscript --no-save < mcpi.R

 R version 2.15.2 (2012-10-26) -- "Trick or Treat"
Copyright (C) 2012 The R Foundation for Statistical Computing
ISBN 3-900051-07-0
Platform: x86_64-unknown-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> montecarloPi <- function(trials) {
+   count = 0
+   for(i in 1:trials) {
+     if((runif(1,0,1)^2 + runif(1,0,1)^2)<1) {
+       count = count + 1
+     }
+   }
+   return((count*4)/trials)
+ }
>
> montecarloPi(1000000)
[1] 3.141688

This should take few seconds to run. Now edit the file. Increasing the trials ten times (10000000) it will take little over a minute to run, but the estimation still isn't very good. Fortunately, this problem is pleasingly parallel since we're just sampling random points. So what do we need to do to run R on the campus grid?

Building the HTCondor job

The first thing we're going to need to do is create a wrapper for our R environment, based on the setup we did in previous sections.

R-wrapper.sh

#!/bin/bash

EXPECTED_ARGS=1

if [ $# -ne $EXPECTED_ARGS ]; then
  echo "Usage: R-wrapper.sh file.R"
  exit 1
else
  source /cvmfs/oasis.opensciencegrid.org/osg/palms/setup
  palmsdosetup R
  Rscript $1
fi

Notice here that I've changed to Rscript (equivalent to R --slave) instead of --no-save. It accepts the script as command line argument, it makes R much less verbose and it's easier to parse the output later.

Now that we've created a wrapper, let's build a Condor submit file around it.

R.submit

universe = vanilla
log = log/mcpi.log.$(Cluster).$(Process)
error = log/mcpi.err.$(Cluster).$(Process)
output = log/mcpi.out.$(Cluster).$(Process)

# Setup R path, run the mcpi.R script
executable = R-wrapper.sh
transfer_input_files = mcpi.R
arguments = mcpi.R

requirements = (HAS_CVMFS_oasis_opensciencegrid_org =?= TRUE)
+ProjectName="ConnectTrain"
queue 50

Notice the requirements line? You'll need to put HAS_CVMFS =?= TRUE any time you need software, such as R, from CVMFS. There's also one small gotcha here – make sure the "log" directory used in the submit file exists before you submit! Else Condor will fail because it has nowhere to write the logs.

Submit and analyze

Finally, submit the job to OSG-Connect!

[user@login01 osg-R]$ condor_submit R.submit
Submitting job(s)....................................................................................................
100 job(s) submitted to cluster 14027.
[user@login01 osg-R]$ condor_q user

-- Submitter: login01.osgconnect.net : <128.135.158.173:47839> : login01.osgconnect.net
 ID      OWNER            SUBMITTED     RUN_TIME ST PRI SIZE CMD
14027.0   marco           8/25 22:51   0+00:00:43 R  0   0.0  R-wrapper.sh mcpi.
14027.1   marco           8/25 22:51   0+00:00:43 R  0   0.0  R-wrapper.sh mcpi.
14027.2   marco           8/25 22:51   0+00:00:31 R  0   0.0  R-wrapper.sh mcpi.
14027.4   marco           8/25 22:51   0+00:00:41 R  0   0.0  R-wrapper.sh mcpi.
14027.5   marco           8/25 22:51   0+00:00:41 R  0   0.0  R-wrapper.sh mcpi.
...

Since our jobs just output their results to standard out, we can do the final analysis from the log files. Let's see what one looks like:

[user@login01 osg-R]$ cat log/mcpi.out.14027.1
[1] 3.141246

I'm just going to use a bit of awk magic to do the average for me.

[user@login01 osg-R]$ grep "[1]" log/mcpi.out.* | awk '{sum+=$2} END { print "Average = ", sum/NR}'
Average =  3.14151

That's pretty close!

What to do next?

The R.submit file may have included a few lines that you are unfamiliar with. For example, $(Cluster) and $(Process) are variables that will be replaced with the job's cluster and process id. This is useful when you have many jobs submitted in the same file. Each output and error file will be in a separate directory.

Also, did you notice the transfer_input_files line? This tells HTCondor what files to transfer with the job to the worker node. You don't have to tell it to transfer the executable, HTCondor is smart enough to know that the job will need that. But any extra files, such as our MonteCarlo R file, will need to be explicitly listed to be transferred with the job. You can use transfer_input_files for input data to the job, as shown in Transferring data with HTCondor. If you have larger data requirements, you may look into Transferring your Stash'd data with HTCondor.