Annex/Ssh.hs | 15 9 + 6 - 0 !
Assistant/Pairing/MakeRemote.hs | 4 2 + 2 - 0 !
Assistant/Ssh.hs | 11 7 + 4 - 0 !
Assistant/WebApp/Configurators/Ssh.hs | 44 22 + 22 - 0 !
CHANGELOG | 10 10 + 0 - 0 !
Remote/Ddar.hs | 10 5 + 5 - 0 !
Remote/GCrypt.hs | 6 4 + 2 - 0 !
Remote/Helper/Ssh.hs | 8 6 + 2 - 0 !
Remote/Rsync.hs | 4 3 + 1 - 0 !
Utility/SshHost.hs | 29 29 + 0 - 0 !
git-annex.cabal | 1 1 + 0 - 0 !
11 files changed, 98 insertions(+), 44 deletions(-)

 [patch] (stable) avoid the dashed ssh hostname class of security
 holes

Security fix: Disallow hostname starting with a dash, which would get
passed to ssh and be treated an option. This could be used by an attacker
who provides a crafted ssh url (for eg a git remote) to execute arbitrary
code via ssh -oProxyCommand.

No CVE has yet been assigned for this hole.
The same class of security hole recently affected git itself,
CVE-2017-1000117.

Method: Identified all places where ssh is run, by git grep '"ssh"'
Converted them all to use a SshHost, if they did not already, for
specifying the hostname.

SshHost was made a data type with a smart constructor, which rejects
hostnames starting with '-'.

Note that git-annex already contains extensive use of Utility.SafeCommand,
which fixes a similar class of problem where a filename starting with a
dash gets passed to a program which treats it as an option.

Annex/Content.hs | 13 2 + 11 - 0 !
Annex/Url.hs | 1 1 + 0 - 0 !
NEWS | 11 11 + 0 - 0 !
Types/GitConfig.hs | 12 10 + 2 - 0 !
Utility/Url.hs | 65 54 + 11 - 0 !
doc/git-annex.mdwn | 17 12 + 5 - 0 !
6 files changed, 90 insertions(+), 29 deletions(-)

 limit url downloads to whitelisted schemes

backported from 28720c795ff57a55b48e56d15f9b6bcb977f48d9

Security fix! Allowing any schemes, particularly file: and
possibly others like scp: allowed file exfiltration by anyone who had
write access to the git repository, since they could add an annexed file
using such an url, or using an url that redirected to such an url,
and wait for the victim to get it into their repository and send them a copy.

* Added annex.security.allowed-url-schemes setting, which defaults
  to only allowing http and https URLs. Note especially that file:/
  is no longer enabled by default.

* Removed annex.web-download-command, since its interface does not allow
  supporting annex.security.allowed-url-schemes across redirects.
  If you used this setting, you may want to instead use annex.web-options
  to pass options to curl.

With annex.web-download-command removed, nearly all url accesses in
git-annex are made via Utility.Url via http-client or curl. http-client
only supports http and https, so no problem there.
(Disabling one and not the other is not implemented.)

Used curl --proto to limit the allowed url schemes.
wget only supports http https ftp, so does not need any limiting.

Note that this will cause git annex fsck --from web to mark files using
a disallowed url scheme as not being present in the web. That seems
acceptable; fsck --from web also does that when a web server is not available.

quvi was not changed; it only supports a hardcoded set of urls, which
are http, not file urls.

This does not address any external special remotes that might download
an url themselves. Current thinking is all external special remotes will
need to be audited for this problem, although many of them will use
http libraries that only support http and not curl's menagarie.

The related problem of accessing private localhost and LAN urls is not
addressed by this commit.

This commit was sponsored by Brett Eisenberg on Patreon.

Annex/Quvi.hs | 11 9 + 2 - 0 !
Annex/Url.hs | 17 16 + 1 - 0 !
NEWS | 6 6 + 0 - 0 !
doc/git-annex.mdwn | 15 15 + 0 - 0 !
4 files changed, 46 insertions(+), 3 deletions(-)

 block url downloads by default

git-annex will refuse to download content from the web, to prevent
accidental exposure of data on private webservers on localhost and the
LAN. This can be overridden with the
annex.security.allowed-http-addresses setting.

This is the simplest possible fix for the security hole. A better fix
has been developed for newer versions of git-annex but would be a lot of

Test.hs | 10 8 + 2 - 0 !
1 file changed, 8 insertions(+), 2 deletions(-)

 update test suite for security fix

Remote/BitTorrent.hs | 2 2 + 0 - 0 !
Remote/Bup.hs | 3 3 + 0 - 0 !
Remote/Ddar.hs | 2 2 + 0 - 0 !
Remote/Directory.hs | 1 1 + 0 - 0 !
Remote/External.hs | 5 5 + 0 - 0 !
Remote/GCrypt.hs | 1 1 + 0 - 0 !
Remote/Git.hs | 1 1 + 0 - 0 !
Remote/Glacier.hs | 3 3 + 0 - 0 !
Remote/Helper/Special.hs | 8 8 + 0 - 0 !
Remote/Hook.hs | 3 3 + 0 - 0 !
Remote/P2P.hs | 1 1 + 0 - 0 !
Remote/Rsync.hs | 1 1 + 0 - 0 !
Remote/S3.hs | 3 3 + 0 - 0 !
Remote/Tahoe.hs | 2 2 + 0 - 0 !
Remote/Web.hs | 3 3 + 0 - 0 !
Remote/WebDAV.hs | 3 3 + 0 - 0 !
Types/Key.hs | 12 11 + 1 - 0 !
Types/Remote.hs | 31 30 + 1 - 0 !
18 files changed, 83 insertions(+), 2 deletions(-)

 add retrievalsecuritypolicy

This will be used to protect against CVE-2018-10859, where an encrypted
special remote is fed the wrong encrypted data, and so tricked into
decrypting something that the user encrypted with their gpg key and did
not store in git-annex.

It also protects against CVE-2018-10857, where a remote follows a http
redirect to a file:// url or to a local private web server. While that's
already been prevented in git-annex's own use of http, external special
remotes, hooks, etc use other http implementations and could still be
vulnerable.

The policy is not yet enforced, this commit only adds the appropriate
metadata to remotes.

This commit was sponsored by Boyd Stephen Smith Jr. on Patreon.

(cherry picked from commit 4315bb9e421f2c643e517d8982c6c35b1909c78b)

Annex/Content.hs | 68 51 + 17 - 0 !
Command/Get.hs | 2 1 + 1 - 0 !
Command/Move.hs | 2 1 + 1 - 0 !
Command/ReKey.hs | 2 1 + 1 - 0 !
Command/RecvKey.hs | 5 4 + 1 - 0 !
Command/Reinject.hs | 2 1 + 1 - 0 !
Command/SetKey.hs | 2 1 + 1 - 0 !
Command/TestRemote.hs | 6 3 + 3 - 0 !
Command/TransferKey.hs | 2 1 + 1 - 0 !
Command/TransferKeys.hs | 2 1 + 1 - 0 !
NEWS | 6 6 + 0 - 0 !
P2P/Annex.hs | 6 5 + 1 - 0 !
Remote.hs | 1 1 + 0 - 0 !
Remote/Git.hs | 3 2 + 1 - 0 !
Types/GitConfig.hs | 3 3 + 0 - 0 !
Types/Key.hs | 1 1 + 0 - 0 !
doc/git-annex.mdwn | 41 41 + 0 - 0 !
17 files changed, 124 insertions(+), 30 deletions(-)

 enforce retrievalsecuritypolicy

Leveraged the existing verification code by making it also check the
retrievalSecurityPolicy.

Also, prevented getViaTmp from running the download action at all when the
retrievalSecurityPolicy is going to prevent verifying and so storing it.

Added annex.security.allow-unverified-downloads. A per-remote version
would be nice to have too, but would need more plumbing, so KISS.
(Bill the Cat reference not too over the top I hope. The point is to
make this something the user reads the documentation for before using.)

A few calls to verifyKeyContent and getViaTmp, that don't
involve downloads from remotes, have RetrievalAllKeysSecure hard-coded.
It was also hard-coded for P2P.Annex and Command.RecvKey,
to match the values of the corresponding remotes.

A few things use retrieveKeyFile/retrieveKeyFileCheap without going
through getViaTmp.
* Command.Fsck when downloading content from a remote to verify it.
  That content does not get into the annex, so this is ok.
* Command.AddUrl when using a remote to download an url; this is new
  content being added, so this is ok.

This commit was sponsored by Fernando Jimenez on Patreon.

(cherry picked from commit b657242f5d946efae4cc77e8aef95dd2a306cd6b)

Remote/Glacier.hs | 6 4 + 2 - 0 !
1 file changed, 4 insertions(+), 2 deletions(-)

 don't assume boto will remain secure

On second thought, best to default to being secure even if boto changes
http libraries to one that happens to follow redirects.

(cherry picked from commit f1b29dbeb4277bf8a7febc795fe52e7b109c6d59)

Remote/Ddar.hs | 5 3 + 2 - 0 !
1 file changed, 3 insertions(+), 2 deletions(-)

 set ddar to retrievalallkeyssecure

Based on information from Robie Basak.

(cherry picked from commit 05ecee0db43919b3cfd8d2e2772022e16a804a04)