OSG Client Services

Richard Jones, University of Connecticut
last updated May 8, 2013

This document describes the daemons that must be running on a client machine in order to submit jobs from there to OSG resources through the globus and Condor-G remote job submission and monitoring tools. Provision is also made for gridftp access to OSG compute and storage elements so that users can install software and databases needed by jobs, and access output data saved by the jobs on OSG storage resources.

This document is a condensation of the globus firewall howto. That reference document describes how to configure the firewall on a client machine in order to enable full access to OSG resources, but it also explains what components of the stack are responsible for, and what takes place over the different ports and protocols that must be enabled in the firewall. What follows is a summary of the essential information from that article.

1. Job submission and management

The OSG virtual data toolkit (vdt) supports two mechanisms for job submission and management: Condor-G, and globus GRAM. Globus GRAM comes in two flavors, the web services (WS) interface (v.4) and the pre-WS interface (v.2). These two GRAM flavors are redundant, and only the pre-WS interface is mandatory for OSG sites, so at UConn we only support the pre-WS (v.2) interface. Nowadays the Condor-G mechanism is the preferred method for most OSG users, so the focus of this document is on that option, but GRAM is also very useful for issuing simple commands to remote servers and for testing the system.

2. File transfer and storage management

The OSG vdt provides a number of tools for file transfer to/from compute and storage elements on the grid. The most frequently used of these are gridftp user tools globus-url-copy and uberftp, and the SRM tool suite for managing grid storage. Normally jobs stage in their input files during job initiation and return them to the submission directory on the submit host upon completion. However there are cases where this mechanism is inappropriate, and there are files that should have a lifetime on the remote resource that is independent of the life cycle of any one job.

3. Final Comments

For simplicity, I have shared the same ephemeral port range between the Condor-G daemons, the globus GRAM components, and the gridftp tools. Each of these applications can take up 10-12 ephemeral ports per active request. A single user may have several requests active at once, and a single host may have several active users at once. The default setting above of 500 ephemeral ports should be sufficient, even for a client node hosting a dozen busy users.