Token acquisition

FDP storage accepts WLCG bearer tokens (JWTs). The tools obtain them with htgettoken from the Fermilab vault, keep one per vault role, pick the role from the path you access, and fetch a token only when there is no valid one.

First login
Which role a path needs
How a token is chosen
Where tokens are kept
Inspect, renew and remove
Tokens from the environment
Where tokens are sent
MetaCat and Globus logins

First login

htgettoken (installed into venv/ by bootstrap.sh; bundled in the macOS app) talks to the vault htvaultprod.fnal.gov with issuer amsc. The first token for a role needs a CILogon / Fermilab SSO login in a browser; after that the vault token it stores lets htgettoken renew bearer tokens without a browser for as long as the vault token is valid, and Kerberos credentials are used when present.

source venv/bin/activate                         # htgettoken on PATH
fdp-token get --role duneread                    # opens the browser the first time
fdp-token get --role dunewrite --headless        # over ssh: prints a URL and code to open elsewhere
fdp-token get --for /amsc/dune/scratch -w        # role from the path (dunewrite)

You rarely need to do this by hand: any tool that needs a token runs htgettoken itself. --no-browser makes a tool fail instead of starting a browser login (Kerberos or a valid vault token still work), which is what scripts and cron jobs want.

Which role a path needs

The project is the first component below /amsc (/amsc/public/<project> for public paths). Reads use the project's read role, writes its write role, from the roles map in the configuration:

Project	Read role	Write role
`dune`	`duneread`	`dunewrite`
`mu2e`	`mu2eread`	`mu2ewrite`
`nova`	`novaread`	`novawrite`
`amsc`	`amscana`	`amscana`
any other `X`	`Xread`	`Xwrite`

# config/fdp.yaml or ~/.config/fdp/fdp.yaml
roles:
  dune:   {read: duneread, write: dunewrite}
  icarus: {read: icarusana, write: icaruspro}

fdp-ls --set roles.icarus.read=icarusana /amsc/icarus overrides one entry for one command; --role ROLE forces a role for every operation of a command.

How a token is chosen

--token-file FILE (token.file) is used as is.
Reads below /amsc/public over WebDAV are anonymous: no token is sent even if one is cached, because a token whose scopes do not cover the public area can make dCache refuse an otherwise public read.
$BEARER_TOKEN / $BEARER_TOKEN_FILE, if set and token.honor_env is true, unless a role was forced with --role.
The cached token of the mapped role, if its storage scopes cover the path.
Otherwise any token on disk of the configured issuer whose storage scopes cover the path (for example a novaendwrite token also grants storage.read:/amsc/novaend, where the map guesses novaendread).
Otherwise htgettoken runs for the mapped role when the cached token has less than token.min_lifetime (300 s) left.

fdp-token which PATH shows that decision, and why, without running htgettoken; the GUIs show the same as a tooltip on every path.

$ fdp-token which /amsc/novaend/run1
novaendwrite, 2h41m left, /tmp/bt_u503 (storage scope covers the path; map wants novaendread)

Over XRootD the FDP door rejects anonymous requests, so a token is always sent, also for public data (the project's read role, or amscana for paths without a project). XrdCl's ztn plugin reads it from $BEARER_TOKEN_FILE, which the tools point at the chosen token, on a connection per role.

Where tokens are kept

Each (issuer, role) has its own file, so read and write tokens never overwrite each other:

$XDG_RUNTIME_DIR/fdp_bt_u<uid>_amsc_<role>     # else the system temporary directory; token.dir overrides

Tokens written by a plain htgettoken run (bt_u<uid>, bt_u<uid>-*) are discovered too, with role and issuer taken from the token's wlcg.groups claim (/amsc/novaendwrite → issuer amsc, role novaendwrite). Only regular files owned by you are considered (not symlinks), and a token of another issuer is listed but never used.

Inspect, renew and remove

fdp-token list                         # roles map and the state of each role's token
fdp-token scan -v                      # every token found on disk, with scopes
fdp-token show --role duneread         # subject, expiry, scopes, groups (--raw: the JWT)
fdp-token refresh                      # renew every discovered role; keeps the old token if renewal fails
fdp-token get --role dunewrite --force # fetch a new one now
fdp-token destroy --role dunewrite     # delete the cached file
eval $(fdp-token env --role duneread)  # BEARER_TOKEN_FILE for curl, gfal, xrdcp ...
fdp-diag /amsc/dune/scratch            # endpoints, clock skew, which tokens cover read and write

Tokens from the environment

$BEARER_TOKEN or $BEARER_TOKEN_FILE (for example in a grid job) is used instead of the role tokens unless --role is given. --set token.honor_env=false ignores them completely; they are then neither used nor offered by discovery.

Where tokens are sent

Only to the configured endpoints, endpoints.webdav and endpoints.xrootd, with the same scheme, host and port. A URL naming another server, or http:// instead of https://, is accessed without a token. To use FDP tokens with another door make it the configured endpoint: --server https://other-door:2880.
MetaCat session tokens, the MetaCat login bearer, Globus tokens and third-party-copy credentials are dropped as soon as a redirect leaves the original scheme, host and port.
--set transfer.verbose=true prints the HTTP exchange with credential headers shown as <redacted>.
./config/fdp.yaml and ./.env found in the current directory are read only if they belong to you (they can name the htgettoken executable and the endpoints).

MetaCat and Globus logins

Declaring files needs a MetaCat session, obtained with a bearer token and kept in ~/.config/fdp/metacat_token (mode 600). Globus transfers need a Globus login with consent for the collections involved.

fdp-meta login                                                # MetaCat session from your amsc token
fdp-meta whoami
fdp-globus login -C fdp_public -C nersc --domain nersc.gov    # PKCE login in the browser
fdp-globus whoami