The GNU version of the tar archiving utility (and other old version of tar) can be use through network over ssh session.<\/p>\n
1. Tarred file transfer
\nScp is very inefficient when copying many small files because it sets up a separate transfer for each file. It is possible to solve this problem by creating a single archive containing all the files and piping it over SSH:<\/p>\n
$ tar zcf – tobearchived | \\
\nssh user@destination_server_ip \\
\n‘tar zxf -‘
\nThis will put \u2018tobearchived\u2019 in the server\u2019s home directory. It is possible to use the -C option to put the files somewhere else. (The \u2018z\u2019 tells tar to use gzip compression. To use bzip2 compressio, replace \u2018z\u2019 with \u2018j\u2019).<\/p>\n
Copying from the server is just like the above, but in reverse:<\/p>\n
$ ssh user@source_server_ip \\
\n‘tar zcf – tobearchived’ | \\
\ntar zxf –
\n2. Offsite backups
\nThis is pretty much the same as above, except we want to transfer a bunch of files and leave them as a tarball on the server rather than as a bunch of files.<\/p>\n
$ tar zcf – tobearchived | \\
\nssh user@destination_server_ip \\
\n‘cat – > tobearchived.tar.gz’
\nIt is possible to encrypt the tarball (it GPG keyring is set up):<\/p>\n
$ tar zcf – tobearchived | \\
\ngpg -e | \\
\nssh user@destination_server_ip \\
\n‘cat – > tobearchived.tar.gz.gpg’
\nIt is also possible to use a symmetric cipher:<\/p>\n
$ tar zcf – tobearchived | \\
\nopenssl enc -rc4 | \\
\nssh user@destination_server_ip \\
\n‘cat – > tobearchived.tar.gz.rc4’
\nIt is also possible to choose a different cipher:<\/p>\n
$ ssh user@destination_server_ip \\
\n‘cat tobearchived.tar.gz.rc4’ | \\
\nopenssl enc -rc4 -d -out tobearchived.tar.gz
\n3. Hard drive backup\/mirror
\nThis will copy the entire drive into a file on the remote machine:<\/p>\n
$ dd if=\/dev\/sdX | \\
\nssh user@destination_server_ip \\
\n‘dd of=sdX.img’
\nTo restore a local drive from the image on the server, reverse the command:<\/p>\n
$ ssh user@source_server_ip \\
\n‘dd if=sdX.img’ | \\
\ndd of=\/dev\/sdX
\nNote that to read or write block devices requires you to be root. Be very careful with dd as it can be very \u2018deadly\u2019 if used carelessly.<\/p>\n
4. Run a local script remotely
\nThis command will run a local file script.sh on the remote server and display any output locally:<\/p>\n
$ ssh user@destination_server_ip \\
\n‘bash -s’ < script.sh<\/p>\n
<\/p>\n
<\/p>\n
<\/p>\n
<\/p>\n
Using scp<\/p>\n
The basic syntax of scp is very simple to memorize. It looks like this<\/p>\n
$ scp source_file_path destination_file_path
\nDepending on the host, the file path should include the full host address, port number, username and password along with the directory path.<\/p>\n
So if you are “sending” file from your local machine to a remote machine (uploading) the syntax would look like this<\/p>\n
$ scp ~\/my_local_file.txt user@remote_host.com:\/some\/remote\/directory
\nWhen copying file from remote host to local host (downloading), its looks just the reverse<\/p>\n
$ scp user@remote_host.com:\/some\/remote\/directory ~\/my_local_file.txt<\/p>\n
# just download the file
\n$ scp user@192.168.1.3:\/some\/path\/file.txt .
\nThat is pretty much about using scp for regular tasks. Apart from it, there are a couple of extra options and functions that scp supports. Lets take a quick overview of those.<\/p>\n
And yes, by default scp will always overwrite files on the destination. If you need to avoid that, use a more powerful tool called rsync.<\/p>\n
1. Verbose output
\nWith verbose output, the scp program would output lots of information about what it does in the background. This is often useful when the program fails or is unable to complete the request. The verbose output would then indicate the exact point where the program ran into issues.<\/p>\n
$ scp -v ~\/test.txt root@192.168.1.3:\/root\/help2356.txt
\nExecuting: program \/usr\/bin\/ssh host 192.168.1.3, user root, command scp -v -t \/root\/help2356.txt
\nOpenSSH_6.2p2 Ubuntu-6ubuntu0.1, OpenSSL 1.0.1e 11 Feb 2013
\ndebug1: Reading configuration data \/home\/enlightened\/.ssh\/config
\ndebug1: Reading configuration data \/etc\/ssh\/ssh_config
\ndebug1: \/etc\/ssh\/ssh_config line 19: Applying options for *
\ndebug1: Connecting to 192.168.1.3 [192.168.1.3] port 22.
\ndebug1: Connection established.
\n….. OUTPUT TRUNCATED<\/p>\n
2. Transfer multiple files
\nMultiple files can be specified separated by a space like this<\/p>\n
$ scp foo.txt bar.txt username@remotehost:\/path\/directory\/
\nTo copy multiple files from remote host to current local directory<\/p>\n
$ scp username@remotehost:\/path\/directory\/\\{foo.txt,bar.txt\\} .<\/p>\n
$ scp root@192.168.1.3:~\/\\{abc.log,cde.txt\\} .
\n3. Copy entire directory (recursively)
\nTo copy an entire directory from one host to another use the r switch and specify the directory<\/p>\n
$ scp -v -r ~\/Downloads root@192.168.1.3:\/root\/Downloads
\n4. Copy files across 2 remote hosts
\nScp can copy files from 1 remote host to another remote host as well.<\/p>\n
$ scp user1@remotehost1:\/some\/remote\/dir\/foobar.txt user2@remotehost2:\/some\/remote\/dir\/
\n5. Speed up the transfer with compression
\nA super cool option to speed up the transfer to save time and bandwidth. All you need to do is use the C option to enable compression. The files are compressed on the fly and decompressed on the destination.<\/p>\n
$ scp -vrC ~\/Downloads root@192.168.1.3:\/root\/Downloads
\nIn the above example we moved the entire directory with compression enabled. The speed gain would depend on how much the files could be compressed.<\/p>\n
6. Limit the bandwidth usage
\nIf you do not want scp to take up the entire available bandwidth, then use the l option to limit the maximum speed in Kbit\/s.<\/p>\n
$ scp -vrC -l 400 ~\/Downloads root@192.168.1.3:\/root\/Downloads
\n7. Connect to a different port number on remote host
\nIf the remote server has ssh daemon running on a different port (default is 22), then you need to tell scp to use that particular port number using the ‘-P’ option.<\/p>\n
$ scp -vC -P 2200 ~\/test.txt root@192.168.1.3:\/some\/path\/test.txt
\n8. Preserve file attributes
\nThe ‘-p’ option (smallcase), would preserve modification times, access times, and modes from the original file.<\/p>\n
$ scp -C -p ~\/test.txt root@192.168.1.3:\/some\/path\/test.txt
\n9. Quiet mode
\nIn quiet mode ( ‘-q’ option ), the scp output would get suppressed, and would disable the progress meter as well as warning and diagnostic messages.<\/p>\n
$ scp -vCq ~\/test.txt root@192.168.1.3:\/some\/path\/test.txt
\n10. Specify identity file
\nWhen using key based (passwordless) authentication, you would need to specify the identity file which contains the private key. This option is directly passed to the ssh command and works the same way.<\/p>\n
$ scp -vCq -i private_key.pem ~\/test.txt root@192.168.1.3:\/some\/path\/test.txt
\n11. Use a different ssh_config file
\nUse the ‘-F’ option to specify a different ssh_config file.<\/p>\n
$ scp -vC -F \/home\/user\/my_ssh_config ~\/test.txt root@192.168.1.3:\/some\/path\/test.txt
\n12. Use different cipher
\nScp by default uses the AES cipher\/encryption. Sometimes you might want to use a different cipher. Using a different cipher can speed up the transfer process. For example blowfish and arcfour are known to be faster than AES (but less secure).<\/p>\n
$ scp -c blowfish -C ~\/local_file.txt username@remotehost:\/remote\/path\/file.txt
\nIn the above example we use the blowfish cipher along with compression. This can give significant speed boost depending on available bandwidth.<\/p>\n","protected":false},"excerpt":{"rendered":"
The GNU version of the tar archiving utility (and other old version of tar) can be use through network over ssh session.<\/p>\n
1. Tarred file transfer Scp is very inefficient when copying many small files because it sets up a separate transfer for each file. It is possible to solve this problem by creating a […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[17],"tags":[],"_links":{"self":[{"href":"https:\/\/mohan.sg\/index.php?rest_route=\/wp\/v2\/posts\/3498"}],"collection":[{"href":"https:\/\/mohan.sg\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/mohan.sg\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/mohan.sg\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/mohan.sg\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=3498"}],"version-history":[{"count":3,"href":"https:\/\/mohan.sg\/index.php?rest_route=\/wp\/v2\/posts\/3498\/revisions"}],"predecessor-version":[{"id":3501,"href":"https:\/\/mohan.sg\/index.php?rest_route=\/wp\/v2\/posts\/3498\/revisions\/3501"}],"wp:attachment":[{"href":"https:\/\/mohan.sg\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=3498"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/mohan.sg\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=3498"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/mohan.sg\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=3498"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}