gnupg/agent/keyformat.txt

keyformat.txt (wk 2001-12-18)

Some notes on the format of the secret keys used with gpg-agent.

Location of keys

The secret keys[1] are stored on a per file basis in a directory below the ~/.gnupg home directory. This directory is named

private-keys-v1.d

and should have permissions 700.

The secret keys are stored in files with a name matching the hexadecimal representation of the keygrip[2] and suffixed with ".key".

Unprotected Private Key Format

The content of the file is an S-Expression like the ones used with Libgcrypt. Here is an example of an unprotected file:

(private-key (rsa (n #00e0ce9..[some bytes not shown]..51#) (e #010001#) (d #046129F..[some bytes not shown]..81#) (p #00e861b..[some bytes not shown]..f1#) (q #00f7a7c..[some bytes not shown]..61#) (u #304559a..[some bytes not shown]..9b#) ) (created-at timestamp) (uri http://foo.bar x-foo:whatever_you_want) (comment whatever) )

"comment", "created-at" and "uri" are optional. "comment" is currently used to keep track of ssh key comments. "created-at" is used to keep track of the creation time stamp used with OpenPGP keys; it is optional but required for some operations to calculate the fingerprint of the key. This timestamp should be a string with the number of seconds since Epoch or an ISO time string (yyyymmddThhmmss).

Actually this form should not be used for regular purposes and only accepted by gpg-agent with the configuration option: --allow-non-canonical-key-format. The regular way to represent the keys is in canonical representation[3]:

(private-key (rsa (n #00e0ce9..[some bytes not shown]..51#) (e #010001#) (d #046129F..[some bytes not shown]..81#) (p #00e861b..[some bytes not shown]..f1#) (q #00f7a7c..[some bytes not shown]..61#) (u #304559a..[some bytes not shown]..9b#) ) (uri http://foo.bar x-foo:whatever_you_want) )

Protected Private Key Format

A protected key is like this:

(protected-private-key (rsa (n #00e0ce9..[some bytes not shown]..51#) (e #010001#) (protected mode (parms) encrypted_octet_string) (protected-at ) ) (uri http://foo.bar x-foo:whatever_you_want) (comment whatever) )

In this scheme the encrypted_octet_string is encrypted according to the algorithm described after the keyword protected; most protection algorithms need some parameters, which are given in a list before the encrypted_octet_string. The result of the decryption process is a list of the secret key parameters. The protected-at expression is optional; the isotimestamp is 15 bytes long (e.g. "19610711T172000").

The currently defined protection modes are:

1. openpgp-s2k3-sha1-aes-cbc

This describes an algorithm using using AES in CBC mode for encryption, SHA-1 for integrity protection and the String to Key algorithm 3 from OpenPGP (rfc2440).

Example:

(protected openpgp-s2k3-sha1-aes-cbc ((sha1 16byte_salt no_of_iterations) 16byte_iv) encrypted_octet_string )

The encrypted_octet string should yield this S-Exp (in canonical representation) after decryption:

( ( (d #046129F..[some bytes not shown]..81#) (p #00e861b..[some bytes not shown]..f1#) (q #00f7a7c..[some bytes not shown]..61#) (u #304559a..[some bytes not shown]..9b#) ) (hash sha1 #...[hashvalue]...#) )

For padding reasons, random bytes are appended to this list - they can easily be stripped by looking for the end of the list.

The hash is calculated on the concatenation of the public key and secret key parameter lists: i.e it is required to hash the concatenation of these 6 canonical encoded lists for RSA, including the parenthesis, the algorithm keyword and (if used) the protected-at list.

(rsa (n #00e0ce9..[some bytes not shown]..51#) (e #010001#) (d #046129F..[some bytes not shown]..81#) (p #00e861b..[some bytes not shown]..f1#) (q #00f7a7c..[some bytes not shown]..61#) (u #304559a..[some bytes not shown]..9b#) (protected-at "18950523T000000") )

After decryption the hash must be recalculated and compared against the stored one - If they don't match the integrity of the key is not given.

2. openpgp-native

This is a wrapper around the OpenPGP Private Key Transport format which resembles the standard OpenPGP format and allows the use of an existing key without re-encrypting to the default protection format.

Example:

(protected openpgp-native (openpgp-private-key (version V) (algo PUBKEYALGO) (skey _ P1 _ P2 _ P3 ... e PN) (csum n) (protection PROTTYPE PROTALGO IV S2KMODE S2KHASH S2KSALT S2KCOUNT)))

Note that the public key paramaters in SKEY are duplicated and should be identical to their copies in the standard parameter elements. Here is an example of an entire protected private key using this format:

(protected-private-key (rsa (n #00e0ce9..[some bytes not shown]..51#) (e #010001#) (protected openpgp-native (openpgp-private-key (version 4) (algo rsa) (skey _ #00e0ce9..[some bytes not shown]..51# _ #010001# e #.........................#) (protection sha1 aes #aabbccddeeff00112233445566778899# 3 sha1 #2596f93e85f41e53# 3:190)))) (uri http://foo.bar x-foo:whatever_you_want) (comment whatever))

Shadowed Private Key Format

To keep track of keys stored on IC cards we use a third format for private kyes which are called shadow keys as they are only a reference to keys stored on a token:

(shadowed-private-key (rsa (n #00e0ce9..[some bytes not shown]..51#) (e #010001#) (shadowed protocol (info)) ) (uri http://foo.bar x-foo:whatever_you_want) (comment whatever) )

The currently used protocol is "ti-v1" (token info version 1). The second list with the information has this layout:

(card_serial_number id_string_of_key fixed_pin_length)

FIXED_PIN_LENGTH is optional. It can be used to store the length of the PIN; a value of 0 indicates that this information is not available. The rationale for this field is that some pinpad equipped readers don't allow passing a variable length PIN.

More items may be added to the list.

OpenPGP Private Key Transfer Format

This format is used to transfer keys between gpg and gpg-agent.

(openpgp-private-key (version V) (algo PUBKEYALGO) (curve CURVENAME) (skey _ P1 _ P2 _ P3 ... e PN) (csum n) (protection PROTTYPE PROTALGO IV S2KMODE S2KHASH S2KSALT S2KCOUNT))

• V is the packet version number (3 or 4).
• PUBKEYALGO is a Libgcrypt algo name
• CURVENAME is the name of the curve - only used with ECC.
• P1 .. PN are the parameters; the public parameters are never encrypted the secrect key parameters are encrypted if the "protection" list is given. To make this more explicit each parameter is preceded by a flag "_" for cleartext or "e" for encrypted text.
• CSUM is the deprecated 16 bit checksum as defined by OpenPGP. This is an optional element.
• If PROTTYPE is "sha1" the new style SHA1 checksum is used if it is "sum" the old 16 bit checksum (above) is used and if it is "none" no protection at all is used.
• PROTALGO is a Libgcrypt style cipher algorithm name
• IV is the initialization verctor.
• S2KMODE is the value from RFC-4880.
• S2KHASH is a a libgcrypt style hash algorithm identifier.
• S2KSALT is the 8 byte salt
• S2KCOUNT is the count value from RFC-4880.

Persistent Passphrase Format

To allow persistent storage of cached passphrases we use a scheme similar to the private-key storage format. This is a master passphrase format where each file may protect several secrets under one master passphrase. It is possible to have several of those files each protected by a dedicated master passphrase. Clear text keywords allow to list the available protected passphrases.

The name of the files with these protected secrets have this form: pw-.dat. STRING may be an arbitrary string, as a default name for the passphrase storage the name "pw-default.dat" is suggested.

(protected-shared-secret ((desc descriptive_text) (key [key_1] (keyword_1 keyword_2 keyword_n)) (key [key_2] (keyword_21 keyword_22 keyword_2n)) (key [key_n] (keyword_n1 keyword_n2 keyword_nn)) (protected mode (parms) encrypted_octet_string) (protected-at ) ) )

After decryption the encrypted_octet_string yields this S-expression:

( ( (value key_1 value_1) (value key_2 value_2) (value key_n value_n) ) (hash sha1 #...[hashvalue]...#) )

The "descriptive_text" is displayed with the prompt to enter the unprotection passphrase.

KEY_1 to KEY_N are unique identifiers for the shared secret, for example an URI. In case this information should be kept confidential as well, they may not appear in the unprotected part; however they are mandatory in the encrypted_octet_string. The list of keywords is optional. The oder of the "key" lists and the order of the "value" lists mut match, that is the first "key"-list is associated with the first "value" list in the encrypted_octet_string.

The protection mode etc. is indentical to the protection mode as decribed for the private key format.

list of the secret key parameters. The protected-at expression is optional; the isotimestamp is 15 bytes long (e.g. "19610711T172000").

The "hash" in the encrypted_octet_string is calculated on the concatenation of the key list and value lists: i.e it is required to hash the concatenation of all these lists, including the parenthesis and (if used) the protected-at list.

Example:

(protected-shared-secret ((desc "List of system passphrases") (key "uid-1002" ("Knuth" "Donald Ervin Knuth")) (key "uid-1001" ("Dijkstra" "Edsgar Wybe Dijkstra")) (key) (protected mode (parms) encrypted_octet_string) (protected-at "20100915T111722") ) )

with "encrypted_octet_string" decoding to:

( ( (value 4:1002 "signal flags at the lock") (value 4:1001 "taocp") (value 1:0 "premature optimization is the root of all evil") ) (hash sha1 #0102030405060708091011121314151617181920#) )

To compute the hash this S-expression (in canoncical format) was hashed:

((desc "List of system passphrases") (key "uid-1002" ("Knuth" "Donald Ervin Knuth")) (key "uid-1001" ("Dijkstra" "Edsgar Wybe Dijkstra")) (key) (value 4:1002 "signal flags at the lock") (value 4:1001 "taocp") (value 1:0 "premature optimization is the root of all evil") (protected-at "20100915T111722") )

Notes:

[1] I usually use the terms private and secret key exchangeable but prefer the term secret key because it can be visually be better distinguished from the term public key.

[2] The keygrip is a unique identifier for a key pair, it is independent of any protocol, so that the same key can be used with different protocols. PKCS-15 calls this a subjectKeyHash; it can be calculated using Libgcrypt's gcry_pk_get_keygrip ().

[3] Even when canonical representation are required we will show the S-expression here in a more readable representation.