Submitting and overflowing jobs from a local condor cluster to CMS Connect

Enable jobs from a local SCHEDD to submit to CMS connect resources 

This guide is for Institutional Pools that for different reasons might want to submit condor jobs to Global Pool resources via CMS Connect from a local schedd rather than the CMS Connect submission host ( login-el7.uscms.org ). This is particularly useful when:

  • You have a schedd able to manage multiple VOs and you would prefer all your users to be able to submit from a centralized local submit host rather than connecting to different places.    
  • The local schedd has non-grid local resources already, but you would like to take advantage of the Global Pool and overflow your CMS jobs to it as well as a way to extend your own cluster resources.

Even though schedds can be authenticated with the Global Pool collector directly and flocking mechanism can be used to achieve this goal without depending on CMS Connect, schedds need to guarantee global user job accounting for fair share, advertise certain specific classAds to guarantee a better match performance, advertising schedd health, establish job-control rules in order not to overload and protect the Global Pool. Using CMS Connect as a gateway allows such protection and accounting.

Note: CMS Dashboard reporting (this is something CRAB does, but not cmsRun condor jobs) is enabled in CMS Connect, but it is currently broken for this mechanism, so it has been disabled  for this submission mechamism in the meantime.

Instructions

Both users and admins have to follow a few Instructions in order to make this happen.

Admins

  • The local condor pool needs to authenticate Global Pool resources. User CMS proxies will be used for this, the admin can choose to let any startd with a valid CMS proxy authenticate with the local collector or restrict the authentication to a specific group of users. Follow this guide for more information

Users