File: Transparent_compression_of_files.mdwn

package info (click to toggle)
git-annex 10.20230126-3
  • links: PTS, VCS
  • area: main
  • in suites: bookworm
  • size: 69,344 kB
  • sloc: haskell: 74,654; javascript: 9,103; sh: 1,304; makefile: 203; perl: 136; ansic: 44
file content (5 lines) | stat: -rw-r--r-- 853 bytes parent folder | download | duplicates (6)
1
2
3
4
5
Hi,

I have a lot of files which are around 80MB and can be easily compressed down to ~55MB. I did some tests with brotli and decompression was reasonable fast, at least fast enough that I would probably not notice given my current transfer speeds. In order to save disk space I would like to able to transparently compress my files. That means, a file is stored compressed in git-annex's blob store and decompressed when I `get` it.

I understand that gpg does compression, but I don't want to deal with encryption, all my repos are local. I've looked at the code and from what I could see the Hash-Backends are rather simple. However, that's probably not the right place. Is this a planned feature? Would it be hard to implement? Of course, ideally the compression algorithm should be configurable. E.g. by just doing a syscall to `brotli` or `gzip`.