Download a file with SSH/SCP, tar it inline and pipe it to openssl
I want to download a list of files from a SSH server, then put it in some kind of container (like a tar file) and finally encrypt it (e.g. with openssl).
The point of putting it into a archive is to keep the original filename, while the final encrypted file will have a different name.
So I am trying something like this:
This does not work; scp doesn't seem to pipe the file to tar as expected, and so the tar archive does not contain the downloaded file.
Is there a way to get this to work?
The scp command copies files, it doesn't care about stdin or stdout. Instead, perform the tar on the remote host, and encrypt the resulting stream:
Be careful if $filepath starts with / as it's (now) considered bad practice to try and create an archive with absolute paths. If this is the case you should consider using the -C flag to change directory to / and then using a relative path. For example, rather than this:
use this:
Or maybe
Also, generally it's considered better to compress a data stream before encrypting it. In this case (assuming gzip) you could add the z flag to tar like this:
If you have a standard non-GNU tar command, then these variants will work. Omit the line gzip | if you don't want compression:
Try running scp with the "-O" option:
Modern versions of scp use the SFTP protocol under the hood to do file transfers, and the SFTP support apparently tries to do file operations which fail on a pipe. "-O" tells scp to use the legacy SCP protocol, which ought to support writing to /dev/stdout. A quick demo on my system:
tar supports - to read or write stdin/out for the archive but not for a file to be archived/restored/listed. I don't know about others, but on my Ubuntu (with GNU tar) scp ... /dev/stdout | tar c[v]h /dev/stdin | ... works -- or -C/ dev/stdin to avoid a warning as per Chris Davies, or even -C/dev stdin. Of course this can't record the real filename in the archive, so you'll need to be careful when extracting although -C/dev stdin makes that a little easier/safer, and of course it doesn't preserve other metadata like modtime, permissions, and owner -- so what's really the point in making it a tar?