CMSSW Analysis example

The following tutorial shows an analysis example that requires to run some CMSSW based analysis code to make flat trees from a group of ROOT files living in different directories and accessible via XRootD. So, for each directory, we want to be able to use all ROOT files on it to create a flat tree. This workflow would require:

  • Create a sandbox with a user-based CMSSW framework
    • Setup a CMSSW area
    • Compile some user-customized analysis code on it
    • Create a sandbox with it for the jobs to use
  • Create a job wrapper to use the previously created sandbox
  • Create a submission file to run this code using different input file directories from a list.

Creating a CMSSW sandbox

You can create a sandbox using the cmssw-sandbox tool. Assuming you already have your release area ready to use (e.g CMSSW_7_2_3), you just need to execute:

$ ls
$ CMSSW_7_2_3/
$ cmssw-sandbox create CMSSW_7_2_3
INFO:root:packing sandbox into ./sandbox-CMSSW_7_2_3-7b812c1.tar.bz2

You can find additional information here:.

Making your sandbox accessible via HTTP

Additionally, you can move your sandbox to your stash area, so you can wget the sandbox from your job script. Otherwise, you can transfer the sandbox with condor or any other mechanism like XRootD. Follow this link for more information.

Creating a job wrapper to use the sandbox

The following example shows how to use a function to setup the sandbox on the worker node:

runAnalysis.sh
#### FRAMEWORK SANDBOX SETUP ####
# Load cmssw_setup function
source cmssw_setup.sh

# Setup CMSSW Base
export VO_CMS_SW_DIR=/cvmfs/cms.cern.ch
source $VO_CMS_SW_DIR/cmsset_default.sh

# Download sandbox
wget --no-check-certificate "http://stash.osgconnect.net/+khurtado/sandbox-CMSSW_7_2_3-86c7ff0.tar.bz2"

# Setup framework from sandbox
cmssw_setup sandbox-CMSSW_7_2_3-86c7ff0.tar.bz2


cd $CMSSW_BASE
# Do some analysis...

The submission file

You can use condor to run your analysis code using different input files (in this case, different directories coming from XRootD). The following example shows how to do this:

analysis.submit
Universe = vanilla
Executable = runAnalysis.sh
Arguments = $(channel)
should_transfer_files = YES
transfer_input_files = /etc/ciconnect/templates/cmssw_setup.sh
# Note we will store files in a "log" directory
# that needs to be created beforehand
Error = log/job_$(Cluster)-$(Process).err
Output = log/job_$(Cluster)-$(Process).out
Log = log/job_$(Cluster).log
Queue channel from channels_list.txt

where channels_list.txt is:

root://cmseos.fnal.gov:1094//eos/uscms/store/user/kenai/v2/ttH125
root://cmseos.fnal.gov:1094//eos/uscms/store/user/kenai/v2/ttJets
root://cmseos.fnal.gov:1094//eos/uscms/store/user/kenai/v2/ttwJets
...

Submitting your file

The previous submission file example creates file on a "log" directory, so let's create that subdirectory and submit:

$ mkdir log
$ # Let's submit to all Sites available
$ source /etc/ciconnect/set_condor_sites.sh T*
$ condor_submit analysis.submit 
........
3 job(s) submitted to cluster 487067.
$


Tutorial example

This tutorial can be gotten via the tutorial command:

$ tutorial analysis
Installing analysis (master)...
Tutorial files installed in ./tutorial-analysis.
Running setup in ./tutorial-analysis...
$ cd tutorial-analysis$
$ condor_submit analysis.submit
........8 job(s) submitted to cluster 487067.