Command Reference¶
cp¶
Options for this command are client-specific. Refer to the relevant client usage as shown below.
usage: cp [-options] <source> [<target>]
Source and target paths can be one of the following formats:
file://<local file on disk>
alien://<path in the Grid catalogue>
or without a prefix, in which case it's treated as a Grid path
options:
-g : treat the Grid path as a GUID
-S <SEs|QoS:count> : Storage target specifications for uploading, default is 'disk:2'
-t : no target needed, create a local temporary file instead, print out the local path at the end
-silent : execute command silently
-w : wait for all replicas to complete upload before returning (default false)
-W : do _not_ wait for all replicas to complete upload, return as soon as the first replica is available
-T : Use this many concurrent download threads (where possible) - default 1
-d : delete local file after a successful upload (i.e. move local to Grid)
-j <job ID> : the job ID that has created the file
-m : queue mirror operations to the missing SEs, in case of partial success. Forces '-w'
-q <SEs|QoS:count> : Queue async transfers to the specified targets
Command format is of the form of (with the strict order of arguments):
cp <options> src dst
or
cp <options> -input input_file
where src|dst are local files if prefixed with file:// or file: or grid files otherwise
and -input argument is a file with >src dst< pairs
after each src,dst can be added comma separated specifiers in the form of: @disk:N,SE1,SE2,!SE3
where disk selects the number of replicas and the following specifiers add (or remove) storage endpoints from the received list
%ALIEN alias have the special meaning of AliEn user home directory
options are the following :
-h : print help
-dryrun : just print the src,dst pairs that would have been transfered without actually doing so
-f : No longer used flag! md5 verification of already present destination is default; disable with -fastcheck
-fastcheck : When already present destination is check for validity, check only size not also md5
-S <aditional streams> : uses num additional parallel streams to do the transfer. (max = 15)
-chunks <nr chunks> : number of chunks that should be requested in parallel
-chunksz <bytes> : chunk size (bytes)
-T <nr_copy_jobs> : number of parralel copy jobs from a set (for recursive copy); defaults to 8 for downloads
-timeout <seconds> : the job will fail if did not finish in this nr of seconds
-retry <times> : retry N times the copy process if failed
-ratethreshold <bytes/s> : fail the job if the speed is lower than specified bytes/s
-noxrdzip: circumvent the XRootD mechanism of zip member copy and download the archive and locally extract the intended member.
N.B.!!! for recursive copy (all files) the same archive will be downloaded for each member.
If there are problems with native XRootD zip mechanism, download only the zip archive and locally extract the contents
For the recursive copy of directories the following options (of the find command) can be used:
-glob <globbing pattern> : this is the usual AliEn globbing format; N.B. this is NOT a REGEX!!! defaults to all "*"
-select <pattern> : select only these files to be copied; N.B. this is a REGEX applied to full path!!!
-name <pattern> : select only these files to be copied; N.B. this is a REGEX applied to a directory or file name!!!
-name <verb>_string : where verb = begin|contain|ends|ext and string is the text selection criteria.
verbs are aditive : -name begin_myf_contain_run1_ends_bla_ext_root
N.B. the text to be filtered cannont have underline <_> within!!!
-parent <parent depth> : in destination use this <parent depth> to add to destination ; defaults to 0
-a : copy also the hidden files .* (for recursive copy)
-j <queue_id> : select only the files created by the job with <queue_id> (for recursive copy)
-l <count> : copy only <count> nr of files (for recursive copy)
-o <offset> : skip first <offset> files found in the src directory (for recursive copy)
Further filtering of the files can be applied with the following options:
-mindepth/-maxdepth N : restrict results to N directories depth relative to the base/searched for directory.
N.B. for in directory globbing (/path1/path2/*.sh : the base directory is /path1/path2)
-minsize/-maxsize N : restrict results to at least/at most N bytes in size
-min-ctime/-max-ctime UNIX_TIME: restrict results to at least/at most this UNIX_TIME (ms, 13 decimals integer)
-user/-group string_name : restrict results to specified user/group
jsh: [alice] > cp file://example.file example.file
quota¶
Options for this command are client-specific. Refer to the relevant client usage as shown below.
Client-side implementation that make use of server's jquota and fquota (hidden by this implementation)
Command format: quota [user]
if [user] is not provided, it will be assumed the current user
cd¶
usage: cd [dir]
jsh: [alice] > cd /alice/cern.ch/user/o/odatskov
pwd¶
No help available for this command
jsh: [alice] > pwd
/alice/cern.ch/user/a/alienci/
mkdir¶
usage: mkdir [-options] <directory> [<directory>[,<directory>]]
options:
-p : create parents as needed
-silent : execute command silently
rmdir¶
usage: rmdir [<option>] <directory>
options:
--ignore-fail-on-non-empty : ignore each failure that is solely because a directory is non-empty
-p : --parents Remove DIRECTORY and its ancestors. E.g., 'rmdir -p a/b/c' is similar to 'rmdir a/b/c a/b a'.
-v : --verbose output a diagnostic for every directory processed
: --help display this help and exit
: --version output version information and exit
-silent : execute command silently
ls¶
usage: ls [-options] [<directory>]
options:
-l : long format
-a : show hidden .* files
-F : add trailing / to directory names
-b : print in guid format
-c : print canonical paths
-H : human readable file sizes (1024-based); implies '-l'
jsh: [alice] > ls -lFH example.file
-r-------- alienci alienci 40 B Oct 28 12:54 example.file
find¶
usage: find [flags] <path> <pattern>
options:
-a : show hidden .* files
-s : no sorting
-c : print the number of matching files
-x <target LFN> : create the indicated XML collection with the results of the find operation. Use '-' for screen output of the XML content.
-d : return also the directories
-w[h] : long format, optionally human readable file sizes
-j <queueid> : filter files created by a certain job ID
-l <count> : limit the number of returned entries to at most the indicated value
-o <offset> : skip over the first /offset/ results
-r : pattern is a regular expression
-f : return all LFN data as JSON fields (API flag only)
-y : (FOR THE OCDB) return only the biggest version of each file
-S <site name> : Sort the returned list by the distance to the given site
-e <pattern> : Exclude pattern
toXml¶
usage: toXml [-i] [-x xml_file_name] [-a] [-l list_from] [lfns]
options:
-i : ignore missing entries, continue even if some paths are not/no longer available
-x : write the XML content directly in this target AliEn file
-a : (requires -x) append to the respective collection
-l : read the list of LFNs from this file, one path per line
usage: toXml [-i] [-x xml_file_name] [-a] [-l list_from] [lfns]
options:
-i : ignore missing entries, continue even if some paths are not/no longer available
-x : write the XML content directly in this target AliEn file
-a : (requires -x) append to the respective collection
-l : read the list of LFNs from this file, one path per line
Additionally the client implements these options:
-local: specify that the target lfns are local files
for -x (output file) and -l (file with lfns) the file: and alien: represent the location of file
the inferred defaults are that the target files and the output files are of the same type
jsh: [alice] > toXml example.file
<?xml version="1.0"?>
<alien>
<collection name="tempCollection">
<event name="1">
<file name="example.file" aclId="" broken="0" ctime="2021-10-28 12:54:46" dir="233353357" entryId="306974419" expiretime="" gowner="alienci" guid="0f896750-37ee-11ec-8f15-024246e5e01d" guidtime="" jobid="" lfn="/alice/cern.ch/user/a/alienci/example.file" md5="3f8a7f1fa8fcfe1faeae60b6036de9de" owner="alienci" perm="400" replicated="0" size="40" turl="alien:///alice/cern.ch/user/a/alienci/example.file" type="f" />
</event>
<info command="example.file" creator="alienci" date="Wed Mar 22 17:02:54 UTC 2023" timestamp="1679504574533" />
</collection>
</alien>
cat¶
usage: cat [-options] [<filename>]
options:
-o : outputfilename
-n : number all output lines
-b : number nonblank output lines
-E : shows ends - display $ at end of each line number
-T : show tabs -display TAB characters as ^I
jsh: [alice] > cat example.file
Hello there! This is just a sample file
whereis¶
usage: whereis [-options] [<filename>]
options:
-g : use the lfn as guid
-r : resolve links (do not give back pointers to zip archives)
-l : lookup the LFN of the ZIP archive (slow and expensive IO operation, only use it sparingly!)
jsh: [alice] > whereis example.file
the file example.file is in
SE => ALICE::CERN::EOS pfn => username://eosalice.cern.ch:1094//02/03037/0f896750-37ee-11ec-8f15-024246e5e01d
SE => ALICE::FZK::SE pfn => username://alice-disk-se.gridka.de:1094//02/03037/0f896750-37ee-11ec-8f15-024246e5e01d
rm¶
usage: rm <LFN> [<LFN>[,<LFN>]]
options:
-f : ignore nonexistent files, never prompt
-r, -R : remove directories and their contents recursively
-i : prompt before every removal (for JSh clients)
mv¶
usage: mv <LFN> <newLFN>
touch¶
usage: touch <LFN> [<LFN>[,<LFN>]]
type¶
usage: type <lfn>
Print the LFN type (file / directory / collection)
jsh: [alice] > type example.file
file
lfn2guid¶
usage: lfn2guid <filename>
jsh: [alice] > lfn2guid example.file
/alice/cern.ch/user/a/alienci/example.file 0f896750-37ee-11ec-8f15-024246e5e01d
guid2lfn¶
usage: guid2lfn <GUID>
guidinfo¶
usage: guidinfo <uuid> ...
access¶
usage: access [options] <read|write> <lfn> [<specs>]
-s : for write requests, size of the file to be uploaded, when known
-m : for write requests, MD5 checksum of the file to be uploaded, when known
-j : for write requests, the job ID that created these files, when applicable
-f : for read requests, filter the SEs based on the given specs list
-u : for read requests, print http(s) URLs where available, and the envelopes in urlencoded format
commit¶
usage: commit API command only, you should not need to call this directly
chown¶
Usage: chown -R <user>[.<group>] <file>
Changes an owner or a group for a file
-R : do a recursive chown
chmod¶
Usage: chmod -R <octal mode> <path> [<path>...]
Changes the access mode for a catalogue path
-R : do a recursive chmod starting from the given path
jsh: [alice] > chmod 400 example.file
deleteMirror¶
Removes a replica of a file from the catalogue
Usage:
deleteMirror [-g] <lfn> <se> [<pfn>]
Options:
-g: the lfn is a guid
md5sum¶
usage: md5sum <filename1> [<or guid>] ...
jsh: [alice] > md5sum example.file
3f8a7f1fa8fcfe1faeae60b6036de9de example.file
mirror¶
mirror Copies/moves a file to one or more other SEs
Usage:
mirror [-g] [-try <number>] [-r SE] [-S [se[,se2[,!se3[,qos:count]]]]] <lfn> [<SE>]
-g: Use the lfn as a guid
-S: specifies the destination SEs/tags to be used
-r: remove this source replica after a successful transfer (a `move` operation)
-try <attempts> Specifies the number of attempts to try and mirror the file (default 5)
grep¶
usage: grep [-linux grep options] <pattern> [<filename>]+
options:
jsh: [alice] > grep Hello example.file
Hello there! This is just a sample file
changeDiff¶
Show changes between the current version of the file and the previous one (same file name with a '~' suffix)
usage: changeDiff [<filename>]
options:
listFilesFromCollection¶
usage: listFilesFromCollection [-options] collection
options:
-z : show size and other file details
-s : silent (API only)
packages¶
usage: packages list available packages
-platform : Platform name, default Linux-x86_64
-all : List packages on all platforms. Equivalent to '-p all'
listCEs¶
listCEs: print all (or a subset) of the defined CEs with their details
usage: listCEs [-s] [CE name] [CE name] ...
options:
-s : print summary information
-p : filter by partition names
-v : print verbose output (including partitions)
jobListMatch¶
jobListMatch: print all the CEs that can run a certain job
usage: jobListMatch [jobId]
options:
listpartitions¶
listpartitions: print all (or a subset) of the defined partitions
usage: listpartitions [-v] [Partition name] [Partition name] ...
options:
-v : print verbose output (including member CEs)
setCEstatus¶
setCEstatus: Sets the status of a set of Computing Elements
usage: setCEstatus [-status status] [CE name] [CE name] ...
options:
-status : Status to be set for the CEs (open / locked)
submit¶
usage: submit <URL>
<URL> => <LFN>
<URL> => file:///<local path>
ps¶
usage: ps [-options]
options:
-F l | -Fl | -L : long output format
-f <flags|status> : any number of (long or short) upper case job states, or 'a' for all, 'r' for running states, 'f' for failed, 'd' for done, 's' for queued
-u <userlist>
-s <sitelist>
-n <nodelist>
-m <masterjoblist>
-o <sortkey>
-j <jobidlist>
-l <query-limit>
-M : show only masterjobs
-X : active jobs in extended format
-A : select all owned jobs of you
-W : select all jobs which are waiting for execution of you
-E : select all jobs which are in error state of you
-a : select jobs of all users
-b : do only black-white output
-jdl <jobid> : display the job jdl
-trace <jobid> <tag>* : display the job trace information
-id : only list the matching job IDs, for batch processing (implies -b)
masterjob¶
usage: masterjob <jobIDs> [-options]
options:
-status <status> : display only the subjobs with that status
-id <id> : display only the subjobs with that id
-site <id> : display only the subjobs on that site
-printid : print also the id of all the subjobs
-printsite : split the number of jobs according to the execution site
kill¶
usage: kill <jobId> [<jobId>[,<jobId>]]
w¶
usage: w Show currently active users on the Grid
options:
-a : Sort by the number of active jobs
-w : Sort by the number of waiting jobs
-r : Reverse sorting order
uptime¶
usage: uptime
options:
jsh: [alice] > uptime
78225 running jobs, 62050 waiting jobs, 22 active users
resubmit¶
resubmit: resubmits a job or a group of jobs by IDs
Usage:
resubmit <jobid1> [<jobid2>....]
top¶
usage: top
registerOutput¶
usage: registerOutput <job ID>
options:
df¶
Shows free disk space
Usage: df
du¶
Gives the disk space usage of one or more directories
usage: du [-ncs] <path>
options:
-n : Print raw numbers in machine readable format
-c : Include collections in the summary information
-s : Print a summary of all parameters
jsh: [alice] > du /alice/cern.ch/user/o/odatskov
/alice/cern.ch/user/o/odatskov
Folders: 3
Files: 2 of an apparent size of 1.925 GB, of which:
- physical files: 2 files of 3.851 GB with 4 replicas (avg 2 replicas/file), size of one replica: 1.925 GB
fquota¶
fquota: Displays information about File Quotas.
usage: fquota list [-<options>]
Options:
-unit = B|K|M|G: unit of file size
jquota¶
usage: jquota Displays information about Job Quotas.
options:
list [username]* : get job quota information for the current account, or the indicated ones
set <user> <field> <value> : to set quota fileds (one of maxUnfinishedJobs, maxTotalCpuCost, maxTotalRunningTime)
listSEs¶
listSEs: print all (or a subset) of the defined SEs with their details
usage: listSEs [-qos filter,by,qos] [-s] [SE name] [SE name] ...
options:
-qos : filter the SEs by the given QoS classes. Comma separate entries for 'AND', pass multiple -qos options for an 'OR'
-s : print summary information
listSEDistance¶
listSEDistance: Returns the closest working SE for a particular site. Usage
options:
-site : site to base the results to, instead of using the default mapping of this client to a site
-read : use the read metrics, optionally with an LFN for which to sort the replicas. Default is to print the write metrics.
-qos : restrict the returned SEs to this particular tag
setSite¶
usage: setSite [site name]
options:
testSE¶
Test the functional status of Grid storage elements
Usage: testSE [options] <some SE names, numbers or @tags>
-v : verbose error messages even when the operation is expected to fail
-c : show full command line for each test
-t : time each operation
-a : test all SEs (obviously a very long operation)
listTransfer¶
listTransfer: returns all the transfers that are waiting in the system
Usage:
listTransfer [-status <status>] [-user <user>] [-id <queueId>] [-verbose] [-master] [-summary] [-all_status] [-jdl] [-destination <site>] [-list=<number(all transfers by default)>] [-desc]
uuid¶
usage: uuid <uuid|filename> [<uuid|filename> ...]
Decode v1 UUIDs and display the interesting bits
jsh: [alice] > uuid example.file
example.file : created on Thu Oct 28 12:53:34 UTC 2021 (1635425614405) by 02:42:46:e5:e0:1d
stat¶
usage: stat [-v] <filename1> [<or uuid>] ...
-v : More details on the status.
jsh: [alice] > stat example.file
File: /alice/cern.ch/user/a/alienci/example.file
Type: f
Owner: alienci:alienci
Permissions: 400
Last change: 2021-10-28 12:54:46.0 (1635425686000)
Size: 40 (40 B)
MD5: 3f8a7f1fa8fcfe1faeae60b6036de9de
GUID: 0f896750-37ee-11ec-8f15-024246e5e01d
GUID created on Thu Oct 28 12:53:34 UTC 2021 (1635425614405) by 02:42:46:e5:e0:1d
xrdstat¶
usage: xrdstat [-d [-i]] [-v] [-p PID,PID,...] [-s SE1,SE2,...] [-c] <filename1> [<or UUID>] ...
options:
-d : Check by physically downloading each replica and checking its content. Without this a stat (metadata) check is done only.
-i : When downloading each replica, ignore `stat` calls and directly try to fetch the content.
-s : Comma-separated list of SE names to restrict the checking to. Default is to check all replicas.
-c : Print the full command line in case of errors.
-v : More details on the status.
-p : Comma-separated list of job IDs to check the input data of
-o : Only show the online status (for files with tape replicas in particular)
-O : Request the file to be brought online
-4 : Force IPv4 usage on all commands
-6 : Force IPv6 usage on all commands
jsh: [alice] > xrdstat example.file
Checking the replicas of /alice/cern.ch/user/a/alienci/example.file
ALICE::CERN::EOS username://eosalice.cern.ch:1094//02/03037/0f896750-37ee-11ec-8f15-024246e5e01d [32mOK[0m
ALICE::FZK::SE username://alice-disk-se.gridka.de:1094//02/03037/0f896750-37ee-11ec-8f15-024246e5e01d [32mOK[0m
resyncLDAP¶
Usage: resyncLDAP
Synchronizes the DB with the updated values in LDAP
optimiserLogs¶
Usage: lastOptimiserLog [-l] [-v] [-f <frequency>] <classnames or metanames>
Gets the last log from the optimiser
-v : Verbose, displays the frequency, last run timestamp and log
-l : List the class names that match a query
-f <value> : Frequency, in seconds
showTagValue¶
usage: showtagValue [flags] <filename> [<filename>...]
options:
-t : restrict to this (comma separated) tag list only (default is to return all available tags)
-c : restrict to these (comma separated) list of attributes
-l : list available tags only
time¶
Usage: time <times> <command> [command_arguments]
Command format: time command arguments
timing¶
usage: timing [on|off]
options:
return server side timing information
commandlist¶
usage: commandlist
motd¶
usage: motd Message of the day/alice/cern.ch/user/a/alienci/alienci
usage: motd Message of the day
ping¶
usage: ping
-c : Number of iterations
-s : Sleep between iterations, default 1000 (milliseconds)
-n : Numeric IP addresses, don't try to resolve anything
ping <count>
where count is integer
jsh: [alice] > ping 5
Sending 3 messages with a pause of 1000 ms between them
reply from 137.138.99.147 (alice-jcentral.cern.ch / aliendb10.cern.ch): time=108.1 ms
reply from 137.138.99.147 (alice-jcentral.cern.ch / aliendb10.cern.ch): time=1.345 ms
reply from 137.138.99.147 (alice-jcentral.cern.ch / aliendb10.cern.ch): time=242.3 ms
3 packets transmitted, time 2354 ms
rtt min/avg/max/mdev = 1.345/117.2/242.3/98.57 ms
Central service endpoint information:
hostname : aliendb10.cern.ch
version¶
usage: version
Get information about the JAliEn version that answers your requests
alien.py version: 1.4.6
alien.py version date: 20230207_140652
alien.py version hash: 3ec8764
alien.py location: /persistent/sw/slc7_x86-64/xjalienfs/master-local48/lib/python/site-packages/alienpy/alien.py
script location: /persistent/sw/slc7_x86-64/xjalienfs/master-local48/bin/alien.py
Interpreter: /persistent/sw/slc7_x86-64/Python/v3.9.12-local6/bin/python3.9
Python version: 3.9.12 (main, Mar 17 2023, 13:53:57)
[GCC 7.3.0]
XRootD version: 5.5.3
XRootD path: /builds/jalien/jalien-ci/sw/slc7_x86-64/XRootD/v5.5.3-local3/lib/python/site-packages/XRootD/client/__init__.py
whoami¶
usage: whoami
options:
-v : verbose details of the current identity
jsh: [alice] > whoami
alienci
user¶
usage: user <user name>
Change effective role as specified.
whois¶
usage: whois [account name]
options:
-s : search for the given string(s) in usernames
-f : also search in full names
-e : search in email addresses too
-d : search in X509 DN (subject) fields
-a : search for the given string in all the above fields
groups¶
groups [<username>]
shows the groups current user is a member of.
jsh: [alice] > groups
User: alienci, main group: alienci
Member of groups: alienci webusers
token¶
usage: token [-options]
options:
-u <username> : switch to another role of yours
-v <validity (days)> : default depends on token type
-t <tokentype> : can be one of: job, jobagent, host, user (default)
-jobid <job DN extension> : expected to be present in a job token
-hostname <FQDN> : required for a host certificate
Print only command!!! Use >token-init< for token (re)generation, see below the arguments
usage: token [-options]
options:
-u <username> : switch to another role of yours
-v <validity (days)> : default depends on token type
-t <tokentype> : can be one of: job, jobagent, host, user (default)
-jobid <job DN extension> : expected to be present in a job token
-hostname <FQDN> : required for a host certificate
lfnexpiretime¶
usage: lfnexpiretime [-options] [<file>]
options:
-r : removes the expire time set for an LFN
-a : add a new expire time for the given LFN
-e : extends the current expire time for the given LFN
-d <number> : specifies the number of days in the expire time
-w <number> : specifies the number of weeks in the expire time
-m <number> : specifies the number of months in the expire time
-y <number> : specifies the number of years in the expire time