The GNU version of the tar archiving utility (and other old version of tar) can be use through network over ssh session.
1. Tarred file transfer
Scp is very inefficient when copying many small files because it sets up a separate transfer for each file. It is possible to solve this problem by creating a single archive containing all the files and piping it over SSH:
$ tar zcf – tobearchived | \
ssh user@destination_server_ip \
‘tar zxf -‘
This will put ‘tobearchived’ in the server’s home directory. It is possible to use the -C option to put the files somewhere else. (The ‘z’ tells tar to use gzip compression. To use bzip2 compressio, replace ‘z’ with ‘j’).
Copying from the server is just like the above, but in reverse:
$ ssh user@source_server_ip \
‘tar zcf – tobearchived’ | \
tar zxf –
2. Offsite backups
This is pretty much the same as above, except we want to transfer a bunch of files and leave them as a tarball on the server rather than as a bunch of files.
$ tar zcf – tobearchived | \
ssh user@destination_server_ip \
‘cat – > tobearchived.tar.gz’
It is possible to encrypt the tarball (it GPG keyring is set up):
$ tar zcf – tobearchived | \
gpg -e | \
ssh user@destination_server_ip \
‘cat – > tobearchived.tar.gz.gpg’
It is also possible to use a symmetric cipher:
$ tar zcf – tobearchived | \
openssl enc -rc4 | \
ssh user@destination_server_ip \
‘cat – > tobearchived.tar.gz.rc4’
It is also possible to choose a different cipher:
$ ssh user@destination_server_ip \
‘cat tobearchived.tar.gz.rc4’ | \
openssl enc -rc4 -d -out tobearchived.tar.gz
3. Hard drive backup/mirror
This will copy the entire drive into a file on the remote machine:
$ dd if=/dev/sdX | \
ssh user@destination_server_ip \
‘dd of=sdX.img’
To restore a local drive from the image on the server, reverse the command:
$ ssh user@source_server_ip \
‘dd if=sdX.img’ | \
dd of=/dev/sdX
Note that to read or write block devices requires you to be root. Be very careful with dd as it can be very ‘deadly’ if used carelessly.
4. Run a local script remotely
This command will run a local file script.sh on the remote server and display any output locally:
$ ssh user@destination_server_ip \
‘bash -s’ < script.sh
Using scp
The basic syntax of scp is very simple to memorize. It looks like this
$ scp source_file_path destination_file_path
Depending on the host, the file path should include the full host address, port number, username and password along with the directory path.
So if you are “sending” file from your local machine to a remote machine (uploading) the syntax would look like this
$ scp ~/my_local_file.txt user@remote_host.com:/some/remote/directory
When copying file from remote host to local host (downloading), its looks just the reverse
$ scp user@remote_host.com:/some/remote/directory ~/my_local_file.txt
# just download the file
$ scp user@192.168.1.3:/some/path/file.txt .
That is pretty much about using scp for regular tasks. Apart from it, there are a couple of extra options and functions that scp supports. Lets take a quick overview of those.
And yes, by default scp will always overwrite files on the destination. If you need to avoid that, use a more powerful tool called rsync.
1. Verbose output
With verbose output, the scp program would output lots of information about what it does in the background. This is often useful when the program fails or is unable to complete the request. The verbose output would then indicate the exact point where the program ran into issues.
$ scp -v ~/test.txt root@192.168.1.3:/root/help2356.txt
Executing: program /usr/bin/ssh host 192.168.1.3, user root, command scp -v -t /root/help2356.txt
OpenSSH_6.2p2 Ubuntu-6ubuntu0.1, OpenSSL 1.0.1e 11 Feb 2013
debug1: Reading configuration data /home/enlightened/.ssh/config
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug1: Connecting to 192.168.1.3 [192.168.1.3] port 22.
debug1: Connection established.
….. OUTPUT TRUNCATED
2. Transfer multiple files
Multiple files can be specified separated by a space like this
$ scp foo.txt bar.txt username@remotehost:/path/directory/
To copy multiple files from remote host to current local directory
$ scp username@remotehost:/path/directory/\{foo.txt,bar.txt\} .
$ scp root@192.168.1.3:~/\{abc.log,cde.txt\} .
3. Copy entire directory (recursively)
To copy an entire directory from one host to another use the r switch and specify the directory
$ scp -v -r ~/Downloads root@192.168.1.3:/root/Downloads
4. Copy files across 2 remote hosts
Scp can copy files from 1 remote host to another remote host as well.
$ scp user1@remotehost1:/some/remote/dir/foobar.txt user2@remotehost2:/some/remote/dir/
5. Speed up the transfer with compression
A super cool option to speed up the transfer to save time and bandwidth. All you need to do is use the C option to enable compression. The files are compressed on the fly and decompressed on the destination.
$ scp -vrC ~/Downloads root@192.168.1.3:/root/Downloads
In the above example we moved the entire directory with compression enabled. The speed gain would depend on how much the files could be compressed.
6. Limit the bandwidth usage
If you do not want scp to take up the entire available bandwidth, then use the l option to limit the maximum speed in Kbit/s.
$ scp -vrC -l 400 ~/Downloads root@192.168.1.3:/root/Downloads
7. Connect to a different port number on remote host
If the remote server has ssh daemon running on a different port (default is 22), then you need to tell scp to use that particular port number using the ‘-P’ option.
$ scp -vC -P 2200 ~/test.txt root@192.168.1.3:/some/path/test.txt
8. Preserve file attributes
The ‘-p’ option (smallcase), would preserve modification times, access times, and modes from the original file.
$ scp -C -p ~/test.txt root@192.168.1.3:/some/path/test.txt
9. Quiet mode
In quiet mode ( ‘-q’ option ), the scp output would get suppressed, and would disable the progress meter as well as warning and diagnostic messages.
$ scp -vCq ~/test.txt root@192.168.1.3:/some/path/test.txt
10. Specify identity file
When using key based (passwordless) authentication, you would need to specify the identity file which contains the private key. This option is directly passed to the ssh command and works the same way.
$ scp -vCq -i private_key.pem ~/test.txt root@192.168.1.3:/some/path/test.txt
11. Use a different ssh_config file
Use the ‘-F’ option to specify a different ssh_config file.
$ scp -vC -F /home/user/my_ssh_config ~/test.txt root@192.168.1.3:/some/path/test.txt
12. Use different cipher
Scp by default uses the AES cipher/encryption. Sometimes you might want to use a different cipher. Using a different cipher can speed up the transfer process. For example blowfish and arcfour are known to be faster than AES (but less secure).
$ scp -c blowfish -C ~/local_file.txt username@remotehost:/remote/path/file.txt
In the above example we use the blowfish cipher along with compression. This can give significant speed boost depending on available bandwidth.
Recent Comments