October 2025
M T W T F S S
 12345
6789101112
13141516171819
20212223242526
2728293031  

Categories

October 2025
M T W T F S S
 12345
6789101112
13141516171819
20212223242526
2728293031  

GZIP COMMAND

Gzip is one of the frequent command used in linux . gzip command is use do to compression of a file for reducing size of file. This will saves the bandwidth if the file is transferring between different systems.Moreover the reduced size depends on the the content of the file, if the content is text, it will reduce 60% and for image, it should be 80%.

gzip linux command usage

if we want to copy the mulitple files,the files should becompressed, so that bandwidth of file is reduced.Gzip abbrivated as GUN zip

Gzip command examples:-

1.Compression the file with deleting original file.

Gzip linuxFileName

This will replace a linuxFileName.gz which has size of 80% of linuxFileName in the current directory. The filename size is reduced by this command. Once gz file is created, linuxFileName should be deleted

2.Compression the file with keeping original file.

Gzip –c  linuxFileName

This command will behave same expect deleting the original file. So original file should be kept as it is.
2. Uncompress/decompress the gz file

Gunzip fileName.gz

This will unzip the filename.gz and get the original file before using gzip command

3.Compression multiple files in a directory

Gzip -r directoryname

using -r option, recursively traverse all the files, meaning all the files in current directory including all the files subdirectory and create a directoryname.gz which contains all the files in the current directory and subdirectory
After compression, total size of the files is approximately 20% less gz file.

3. Uncompress/decompress the gz file into multiple files

Gunzip -r fileName.gz

This will unzip the filename.gz into the the multiple original files before using gzip -r command

4.Compression files fastly:-

Gzip -1 filename.txt
Gzip –fast filename.txt

The both above options compress filename.txt very fast and create filename.txt.gz folder

5.Compression files fastly:-

Gzip -9 filename.txt
Gzip –best filename.txt

The both above options compress filename.txt files slowly and create filename.txt.gz folder

Advanced gzip examples:-

6.zip each file in the current directory and creating separate gz

for filename in *.txt; do gzip -c “$filename” > “$filename.gz”;

let us say we have file1.txt,file2.txt,file3.txt in the current directory /tmp/directory. To do this, we have to iterate each file and do the gzip command redirect(>) the output as gz file
The above command create file1.txt.gz,file2.txt.gz,file3.txt.gz
-c option keep all the original files (file1.txt,file2.txt,file3.txt) and give the file to stdout console.
if we don’t specified any option, it will remove all the files, and create a gz file

Hope you got basic start for gzip with examples.

MySQL Commands for reference

The following MySQL Commands were originally split into several smaller blog posts that I had built up over the years, I have now consolidated the articles into a single post (feel free to link to this resource from your site).

Please note this article contains commands & examples for the mysql command line client, it does not contain information for phpMyadmin or similar GUI based software.
MySQL Set Root Password

By default MySQL has no password set, this might be fine for a private development environment but unacceptable for production servers. You can set the mysql root password various ways but below is a nice simple method that works:

1

    

mysqladmin -u root password YOURNEWPASSWORD

Set / Change MySQL Users Passwords from the Linux Shell

    

mysqladmin -u username -h your-mysql-host -p password ‘newpassword’

You should now be able to restart MySQL and login with your new root password.
How To Connect to MySQL

To connect to your local MySQL server from the command line enter:

    

mysql -u root -p

If you need to login to a remote MySQL server, you cn either SSH to the server and login or use the following commnd (if the server allows external connections):

    

mysql -h hostname -u root -p

MySQL Create Database

The following command will create a new MySQL database:

    

create database example_db;

Backup a MySQL Database using mysqldump

Backing up a MySQL database to a flat file is refered to as “dumping the database”, there are several ways to acomplish this taske here are a few of the methods I use.

Basic mysqldump to a .sql file:

    

mysqldump -u root -p database-name > /tmp/database-backup.sql

You can also dump the database and compress on the fly by piping it through gzip:

    

mysqldump -u root -p database-name | gzip -v > database-backup.sql.gz

Mysqldump a remote database & transfer over SSH using gzip compression

Note you should execute the following command on the remote server that is currently serving the database, so you are affectivly pushing the db to your local machine.

Mysqldump a remote mysql database to your local machine using SSH & gzip compression (a fast way of taking a backup of a remote database). :

    

mysqldump -u root -p database-name | gzip -c | ssh user@your-local-machine ‘cat > /tmp/database-backup.sql.gz’

Dump all MySQL Databases on a server

If you wish to dump all databses on a server to a single dump file enter:

    

mysqldump -u root -p your-root-password –opt >/tmp/databases.sql

Mysqldump & Skip Table(s)

While carrying out a nasty phpBB migration I was faced with the task of dumping a MyISAM databse with some broken tables, you will get an error “mysqldump: Error 1194” or something similar to:

    

mysqldump: Error 1194: Table ‘phpbb_sessions’ is marked as crashed and should be repaired when dumping table `phpbb_sessions` at row: 37 71.0%

The best option you have if you need to take a backup in it’s current state is to tell mysqldump to skip the tables with:

    

mysqldump -u username -p your-database –ignore-table=your-database.broken-table > your-database.sql

If you need to skip more than one table you can just add multiple, example below:

    

mysqldump -u username -p your-database –ignore-table=your-database.broken-table –ignore-table=your-database.broken-table2 > your-database.sql

Once you have a backup I would recommend repairing the tables.
Dump a specific table from a mysql database

    

mysqldump -c -u username -p your-pass database-name table-name > /tmp/db-name.table-name.sql

Import a MySQL Database

Simple mysql db import from a .sql file:

    

mysql -u username -p -h localhost database-name < database-backup.sql

Import a mysql database from .sql.gz

    

zcat database-backup.sql.gz | mysql -u root -p database-name

Import a .sql file from the mysql command line (you can se the output on the console as it imports, handy if your getting an import error from mysql), first select the mysql database you wish to import into and run:

    

source ./db-backup.sql

Select a Database in MySQL

How to select a database in mysql:

    

user database-name;

Show Tabels in a Database

First select the databse you wish to use and run the following to show tables in a mysql databse:

    

show tables;

Create MySQL User

The following example creates a MySQL user called “jesus” with the password “jedimaster”:

    

grant usage on *.* to jesus@localhost identified by ‘jedimaster’;

Next you need to grant the user permission to access your database:

    

grant all privileges on heaven_db.* to jesus@localhost

The above will allow permission for the user “jesus” on the database “heaven_db”.

If you want “jesus” to have access to all databases on the server you would enter:

    

grant all privileges on *.* to jesus@localhost;

Show MySQL Database Size

The simple way is to use the filesystem to show the mysql database size on the disk with:

    

cd /var/lib/mysql && ls -lh

If you need to find out the size of the mysql database from within mysql you could use:

    

SELECT table_schema “Database-Name”, SUM( data_length + index_length) / 1024 / 1024 “Data Base Size in MB” FROM information_schema.TABLES GROUP BY table_schema;

List MySQL Databases

The following will list all mysql databases on a server:

    

show databases;

This will give you an output similar to:

    

+——————–+
| Database           |
+——————–+
| mysql              |
| snort_log          |
| squirrelmail       |
| ssweb              |
| test               |
| wikidb             |
+——————–+
13 rows in set (0.07 sec)

Drop A MySQL Database (deletes a db)

The following will drop a databases, when you drop a database you are deleting it. Be careful with this command…

    

drop database db-name;

Drop a MySQL Table

The following will delete (drop) a mysql table, you need to select the database you wish to use first.

    

drop table table-name;

How To Reset the MySQL root password

The following proccess will allow you to reset the mysql root password:

Stop mysql:

/etc/init.d/mysqld stop

Start mysql in safe mode:

mysqld_safe –skip-grant-tables &

Login as root:

 

mysql -u root

Set the mysql root password:

use mysql;
update user set password=PASSWORD(“new-root-passwd”) where user=’root’;
flush privileges;
quit

Restart the mysql service and you can login with your new password:

/etc/init.d/mysql restart

Create a MySQL table

Here is the basic create table syntax for mysql:

CREATE TABLE example (
id INT,
data VARCHAR(100)
);

Here is a more complex example:

CREATE TABLE table-name (firstname VARCHAR(20), middleinitial VARCHAR(3), lastname VARCHAR(35),suffix VARCHAR(3),officeid VARCHAR(10),userid VARCHAR(15),username VARCHAR(8),email VARCHAR(35),phone VARCHAR(25), groups VARCHAR(15),datestamp DATE,timestamp time,pgpemail VARCHAR(255));

Create an INNODB Table in MySQL

The following will create an innodb table:

 CREATE TABLE your_table_name_innodb (
 id INT,
 data VARCHAR(100)
 ) TYPE=innodb;

Convert MyISAM to INNODB

It goes without saying, backup up your db first before running such a task, but here is the mysql syntax to convert a MyISAM table to INNODB:

    

ALTER TABLE ENGINE=INNODB;

Repair Broken Table(s) in MySQL

If you have a a corrupt / broken table (pretty common with MyiSAM) then take a dump (see the skip broken table with mysqldump instructions above) and then run:

    

repair table broke_table_name;

Show MySQL Database Fields & Field Formats

describe table-name;

MySQL Show Table Data (Displays the contents of a table)

SELECT * FROM table-name;

Show Columns in a MySQL Table

show columns from table-name;

Add a new column in MySQL

The following is an example of how to add a new column in mysql:

    

alter table table-name add column new-column varchar (20);

Delete a Column in MySQL

The following is an example of how to delete (drop) a column in mysql:

    

alter table table-name drop column column-name;

Delete a Row from a field

How to delete a row:

    

DELETE from table-name where field-name = ‘darth-vader’;

Show How Many Rows in a MySQL Table

SELECT COUNT(*) FROM table-name;

MySQL Join Tables

How to join tables in MySQL:

    

SELECT column_names FROM table-1, table-2 WHERE (table-1.column = table-2.column);

MySQL SUM Column Example

SELECT SUM(*) FROM table-name;

Show MySQL & List in Descending Order (DESC)

Show records from col6 and col5 and sort in a descending order using col6:

SELECT col6,col5 FROM table-name ORDER BY col6 DESC;

MySQL Show Records & List in Ascending Order (MySQL ASC)

Show records from col6 and col5 and sort in a ascending order using col6:

SELECT col6,col5 FROM table-name ORDER BY col6 ASC;

MySQL Show Unique Records

Shows all unique records from a mysql table:

    

SELECT DISTINCT column-name FROM table-name;

Search MySQL Records using a Regular Expression

This regular expression example will show you how to search for MySQL records using regular expressions and the REGXP Binary, the following example will return all results beging with the lower case letter z.

    

SELECT * FROM table-name WHERE rec RLIKE “^z”;

Show Rows Containing a Value

This example will show all rows containing “jesus”:

    

SELECT * FROM table-name WHERE field-name = “jesus”;

MySQL Search for a Record Matching (Various Examples)

Search for records with the name “Jesus” born in “1984”:

    

SELECT * FROM table-name WHERE name = “Jesus” AND year = ‘1984’;

Search for anyone called “Jesus” with the phone number “911”

    

SELECT * FROM table-name WHERE name = “Jesus” AND year = ‘1984’;

Search MySQL for any records matching the name “Jesus” with the phone number “911” and sort by phone number:

    

SELECT * FROM table-name WHERE name != “Jesus” AND phone_number = ‘911’ order by phone_number;

Show all records starting with “Jesus” and the phone number “911”:

  

SELECT * FROM table-name WHERE name like “Jesus%” AND phone_number = ‘911’;

Do the same as about but only show records 1 to 10:

  

SELECT * FROM [table name] WHERE name like Dave%” AND phone_number = ‘911’ limit 1,10;

Feel free to link to this resource from your blog, if you have any suggestions for additional commands please drop me a comment below and I will amend the post.

Centos harden Steps

When it comes to having a Linux server hosted in a data center or it is not behind any kind of Firewall or NAT device there are a number of security requirements that need to be addressed. Linux servers generally come with no protection configured by default and depending on the hosting company or distro can come preconfigured with many services installed that are not required, including Web Servers, FTP Servers, Mail Servers and SSH Remote Access.

The following is a compilation of various settings and techniques you can employ to harden the security of your vulnerable Linux systems. While I have tried to put them in order of the most important features first I would recommend all of these options be used on your critical production servers.

TIP #1 – Strong Passwords

Always create long passwords that contain upper and lower case letters, numbers and non alpha-numeric characters. Enforce password ageing so users need to change their passwords regularly. Lock user accounts after a certain number of failed login attempts.

TIP #2 – Use Public/Private Keys

Make use of Public/Private SSH keys for login of remote users instead of passwords, this provides the benefit of turning off password authentication in SSH so that your server can’t be Brute-Force cracked. However this does introduce a new problem whereby a malicious person could compromise a user’s computer or steal their laptop and then have access to the server. This can be overcome by using a password on the client certificate which must be entered before connecting, a kind of two factor authentication.

TIP #3 – Disable Root Login

Disable the Root user from being able to login either via the console or remote SSH connections. Instead have users use Sudo to run programs that require root privileges, or use sudo su to change to the Root user once logged in. This provides an audit path to show which user installed a piece of software or ran a program.

TIP #4 – Use Encrypted Traffic

Always use the encrypted equivalent protocol when transferring critical and sensitive data such as passwords and confidential material. Remove RSH and always use SSH for remote access. Instead of using FTP for file transfer, consider using SFTP or FTP/S (FTP over SSL) or RSYNC. Instead of having remote access open to the internet i.e. SSH or VNC setup an OpenVPN SSL VPN Server to connect to first.

TIP #6 – Use Centralized Password Server

Consider implementing either a LDAP or Kerebos server to perform password authentication. This allows for a central database to maintain user’s passwords between multiple servers for easy management. This prevents user account and password data from becoming inconsistent and out of date, and prevents user accounts that should have been deleted on all servers being left behind on one server.

TIP #7 – Use IPTABLES Firewall/TCP Wrapper

Implementing a secure IPTABLES firewall will limit your exposure to network threats such as DOS and Port Scanning attacks. You can lock down any ports that don’t require access from external networks. For instance you can use the following command to only allow SSH access to the server from the local network.

# iptables –A INPUT –s 192.168.0.0/24 –p tcp –dport 22 –j ACCEPT

You can install a TCP Wrapper named libwrap which will give information like who connected, when and from where and even which services they connected to. It can also be used for locking down access to ports and services for certain hosts or IP’s.

TIP #8 – Use Intrusion Detection Systems

Consider installing both a Network IDS (NIDS) and a Host Based IDS (HIDS). NIDS’s are used to protect against malicious threats such as DOS and Port Scan Attacks. HIDS’s such as AIDE are used to monitor file system changes such as an intruder replacing core system files like ls or ps with malicious ones that hide their Trojan from file or process lists. It will produce a report that tells you what files have been modified so you can repair or replace them.

TIP #9 – Users Assigned Least Privileges

Disable Shell access to users that don’t need it (ftp, mail users etc) by changing to /bin/noshell in the /etc/passwd file. Setup a group for standard users and remove permissions to tools that can be used to download malicious software like wget, lynx, ftp etc. Consider chrooting users to their home directories to stop them from modifying critical system files.

TIP #10 – Minimize Software

Only install software that is actually needed, some systems come preconfigured with many software packages that you may never need or use. When installing always choose the Minimal Installation or Manual Installation option if they exist. Then simply install the software that you actually need.

TIP #11 – Keep Software Updated

Always try to keep your software packages up to date, such as ensuring the latest version of Apache, MySQL and PHP on a standard LAMP setup will protect you against any vulnerabilities that have been discovered in previous versions.

TIP #12 – Disable Unwanted Services

Your servers will most likely have many background services (Daemons) running which are not required and some may be configured to run on start-up. The following command (Red Hat, Cent OS only) can be used to show all services that will start on boot.

# chkconfig --list | grep : on

Or just use the following command to view services which are turned on only for Run Level 3.

# chkconfig --list | grep 3:on

You would then use a command like this to remove the service from start-up.

# chkconfig --del ‘service-name’

TIP #13 – Remove X Windows

Consider completely removing X Windows from the system and just using the command line for management. There isn’t anything that you can do in the GUI that you can’t do using the command line and removing it will not only enhance security but also performance because no system resources are wasted displaying the GUI.

TIP #14 – Secure Linux Kernel

You can secure your Linux Kernel by modifying the /etc/sysctl.conf file, this file is read by the Kernel at boot time and can be edited with the following settings to add extra security.

# Turn on execshield
kernel.exec-shield = 1
kernel.randomize_va_space = 1
# Don't reply to broadcasts. Prevents joining a smurf attack
net.ipv4.icmp_echo_ignore_broadcasts = 1
# Enable protection for bad icmp error messages
net.ipv4.icmp_ignore_bogus_error_responses = 1
# Enable syncookies for SYN flood attack protection
net.ipv4.tcp_syncookies = 1
# Enable IP spoofing protection
net.ipv4.conf.all.rp_filter = 1
net.ipv4.conf.default.rp_filter = 1
# Log spoofed, source routed, and redirect packets
net.ipv4.conf.all.log_martians = 1
net.ipv4.conf.default.log_martians = 1
# Don't allow source routed packets
net.ipv4.conf.all.accept_source_route = 0
net.ipv4.conf.default.accept_source_route = 0
# Don't allow outsiders to alter the routing tables
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.default.accept_redirects = 0
net.ipv4.conf.all.secure_redirects = 0
net.ipv4.conf.default.secure_redirects = 0
# Don't pass traffic between networks or act as a router
net.ipv4.ip_forward = 0
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.default.send_redirects = 0

TIP #15 – Install Linux Kernel Patches

You should have a written security policy for handling Linux Kernel Patches, which should include which Linux security notices have been received, which updates have been tested to ensure problems don’t arise and which patches have been installed on the system. Always ensure Production servers are updated regularly to avoid any potential known vulnerability from being exploited on your system.

TIP #16 – Separate Partitions

You should create separate partitions for user modifiable directories and block write and execute access to unneeded partitions. You should consider placing the following file systems on different partitions.
/usr
/home
/var and /var/tmp
/tmp

Then you can edit the /etc/fstab file to prevent execution of binary files, disable block devices on the partition and prevent the SUID/SGID from being set on files. Here is a common fstab file entry to limit user access to the ftpdata directory.

/dev/sda5  /ftpdata  ext3    defaults,noexec,nodev,nosuid 1 2

TIP #17 – Use Linux Security Extensions

Make use of software like SELinux, AppArmor or GRSecurity to provide additional hardening to your Linux Kernel. These products provide additional policies to restrict processes and services based on Access Control Lists.

TIP #18 – Separate Servers for Services

Consider setting up different physical or virtual servers for different roles, i.e. separate your Mail server and your Webserver, or your Database server and your Application server. This ensures that if one particular service is compromised it is contained to just one server.

TIP #19 – Physical Server Security

You can secure your server as much as possible from remote attacks, but if you don’t do anything to protect the physical hardware it is pointless. If someone has access to your physical server they can remove your hard drive and read your confidential data or boot from a CD and access your data. Consider creating a BIOS password and disabling booting from CD or USB. Also you should password protect your boot loader (GRUB, LILO, etc) to prevent users from accessing Single User Mode or Recovery Environments where passwords are not required.

TIP #20 – Setup NTP

Having an accurate system clock is important for reviewing log files and determining when an event occurred. Often system clocks can become out of sync or be reset to an older date and this can cause havoc with tracking of errors. Consider creating a Cron job rather than running ntpd (See Tip #12) to update the time daily or hourly with a common source for all servers.

TIP #21 – Monitor All Logs

Setup logging and auditing software to track errors and changes to your servers, such as Auditd and Logwatch/Logcheck. Consider configuring a remote logging server that is updated regularly to protect against an intruder compromising your log files without your knowledge.

TIP #22 – Disable IPv6

IPv6 is very rarely needed at this stage as most traffic only utilizes IPv4 and having IPV6 enabled is just another network you need to monitor and protect. Disabling IPv6 is the easiest option but if for some reason you do require it then you should configure an IPv6 Firewall.

TIP #23 – Remove SUID and SGID from Files

After you have setup and configured your system and software you should run the following commands to search for all file and folders with either the SUID, SGID bit set or world writeable folders.
To find all SUID files:

# find / -xdev -type f -perm +u=s –print

To find all SGID files:

# find / -xdev -type f -perm +g=s -print

To find all World Writeable Dirs:

# find / -xdev -perm +o=w ! \( -type d -perm +o=t \) ! -type l -print

You should then inspect each file and folder to determine if they have the correct settings and if not use the chmod command to make changes to them.

TIP #24 – Encrypt Confidential Data

Your data is usually stored on a hard drive in an unencrypted format so any user that has access to the server can remove the hard drive and install it in another system and read all your data. You should consider configuring Linux disk or folder encryption on either your home directories or your sensitive folders (i.e. Database Files, Emails, etc). While you could encrypt your entire drive this is a lot of work and may not be worth the hassle.

TIP #25 – Harden Your Software

It is great to have a highly secure Linux server but your system is only secure as the software you run on it. You should always install the latest versions of software and ensure they stay up to date. Also most programs have ways to make them more secure by editing their configuration files and disabling unnecessary parts of the software. The following is an example for hardening your OpenSSH Server settings, simply add the following  to your OpenSSH config file.

# Use only SSH Protocol Ver 2
Protocol 2
# Only allow the following users SSH Access
AllowUsers User1 User2 etc
# Deny access to the following users
DenyUsers admin etc
# Set the timeout period for idle sessions (in seconds)
ClientAliveInterval 300
ClientAliveCountMax 0
# Disable .rhosts files
IgnoreRhosts yes
# Disable Host-Based Authentication
HostbasedAuthentication no
# Remove ability to login as Root
PermitRootLogin no
# Change the default SSH Port (Not essential but can help uncomment if you want)
#Port 22
#ListenAddress 192.168.1.1
# Consider CHRooting users to their own directories.
# Subsystem sftp internal-sftp
#Match group sftponly
#         ChrootDirectory /home/%u
#         X11Forwarding no
#         AllowTcpForwarding no
#         ForceCommand internal-sftp
# Disable empty passwords from login
PermitEmptyPasswords no
# Set your required Log Level (Either INFO or DEBUG)
LogLevel INFO
#  Turn on privilege separation
UsePrivilegeSeparation yes
# Prevent the use of insecure home directory and key file permissions
StrictModes yes
# Turn on  reverse name checking
VerifyReverseMapping yes
# Do you need port forwarding?
AllowTcpForwarding no
X11Forwarding no
#  Specifies whether password authentication is allowed.  The default is yes.
PasswordAuthentication no

tar

tar [-cxtzjvfpPN] file and directory ….
parameters:

-c: create a compressed file parameters command (create mean);
-x: parameter instructions to unlock a compressed file!
-T: View file tarfile inside!
Special attention to the parameters issued, c / x / t only the existence of a! Not exist!
Because it is not possible at the same time compression and decompression.
-Z: whether gzip attributes? That is, the need to use gzip compression?
-J: whether bzip2 attributes? That is, the need to use the bzip2 compression?
-V: compressed file! This commonly used, but is not recommended for use in the background during the execution!
-F: use the file name, please pay attention to, and then to immediately after the f file name Oh! Do not plus Parameters!
???For example, the use of “tar-zcvfP tfile sfile” is the wrong wording to be written as
???”tar-zcvPf tfile sfile” fishes Oh!
-P: the original with the original file attributes (based on user attributes are not changed)
-P: You can use an absolute path compression!
-N: than followed by the date (yyyy / mm / dd) also will be packaged into a new file!
– Exclude FILE: In the process of compression, not to FILE packaging!
Example:
Example: all packaged into the file under the / etc directory / tmp / etc.tar
[root@rmohan.com to] # tar -cvf /tmp /etc.tar / etc <== only packaged not compressed!
[Root@rmohan.com ~] # tar-zcvf /tmp /etc.tar.gz / etc <== packed gzip compression
[root@rmohan.com ~] # tar-jcvf /tmp/bzip2 compression
# etc.tar.bz2 / etc <== packed special attention to, in the name of the file, after the parameter f taken their own, we are accustomed to using. tar as identification.
#
plus j parameters by. tar.bz2 as an attached file name ah plus z parameters by. tar.gz or. tgz to represent gzip compressed tar file ~~ # ~
# in the implementation of the above command when will display a warning message:
# tar: Removing leading `/” from member names “that a special set absolute path.

Example: inspection which files the above / tmp / etc.tar.gz file?
The [root@rmohan.com] # tar-ztvf / tmp / etc.tar.gz
# Since we use gzip compression, so you want to access the file within the tar file,
# you have to add z this parameter ! This is very important!

Example: / tmp / etc.tar.gz file decompression beneath / usr / local / src
[root@rmohan.com ~~] # cd / usr / local / src
[root@rmohan.com src ] # tar-zxvf / tmp / etc.tar.gz
# In the case of default, we can unlock the archive anywhere!
# My working directory first conversion to / usr / local / src underneath this example and untied / tmp / etc.tar.gz, and
# then unlock directory in / usr / local / src / etc too! Also, if you enter the / usr / local / src in / etc
# is found, the file attributes in the directory / etc / may be different Oh!

Example: / tmp under, I only want to / tmp / etc / passwd etc.tar.gz within untie.
[root@rmohan.com ~~] # cd the / tmp
[root@www.linuxidc. com tmp] # tar-zxvf / tmp / etc.tar.gz etc / passwd
# I can through tar-ztvf to access the file name in the tarfile if single for so long as a file
# can through issued Noticed! the root within etc.tar.gz catalog / is removed!

Example: backup all the files in the / etc / down, and save their permission!
[Root@rmohan.com to] # tar-zxvpf / tmp / etc.tar.gz / etc
#-p attribute is very important, especially when you want to keep the original file attributes!

EXAMPLE 6: in / home them than 2005/06/01 new file backup the
[root@rmohan.com] # tar-N “2005/06/01”-zcvf home.tar.gz / home

Example 7: I want to back up / home, / etc., but do not / home / dmtsai in
root@rmohan.com ~~] # tar – exclude / home / dmtsai-zcvf myfile.tar.gz / home / * / etc

Example 8: / etc / packaged under the / tmp, without producing documents direct unlock!
[Root@rmohan.com] # cd / tmp
[root@rmohan.com tmp] # tar-cvf – / etc | tar-xvf –
# This action is a bit like cp-r / etc / tmp! ~ still has its uses!
# To note is that the output file into the – input file has become – have a | ~
# represent the standard output, standard input and pipeline command exists friends!

MySQL ERROR 1045 Access denied for ‘user’@’localhost’

The Problem

The logfile of mysqld, /var/log/upstart/mysql.log, reported yet another error:

?120618 14:07:31 [Note] /usr/sbin/mysqld: ready for connections.

Version: ‘5.5.24-0ubuntu0.12.04.1’ socket: ‘/var/run/mysqld/mysqld.sock’ port: 3306 (Ubuntu)

mysqld is alive

Checking for tables which need an upgrade, are corrupt or were

not closed cleanly.

120618 14:07:36 [ERROR] Cannot find or open table nova/projects from

the internal data dictionary of InnoDB though the .frm file for the

table exists. Maybe you have deleted and recreated InnoDB data

This looked like a DB corruption. Assuming healing it will solve the problem I wasted a few hours on that, in vain. Finally it turned this is a harmless alert that has nothing to do with the Access Denied issue (but is probably a nova bug).

Some posts on the subject suggested that the socket permissions prevented local access. In my installation:

# ll /var/run/mysqld/mysqld.sock
srwxrwxrwx 1 mysql mysql 0 Jun 18 17:34 /var/run/mysqld/mysqld.sock

Which is ok.

Back to mysql’s user accounts.

User ‘glance’ was properly defined in mysql, as well as ‘keystone’ and ‘nova’. Permissions and grants looked ok – and let’s recall openstack worked (!). Connecting from any remote host (same command as above, with –host of course) worked fine. The glance daemons glance-api and glance-registry weren’t the only services imapcted by error 1045: keystone and nova had the same issue, and their respective log files (under /var/log/upstart) had thousands of lines of the OperationalError quoted above (which is due to the fact openstack’s upstart jobs have a very “slim” logic).

All that suggested that there was something special about ‘localhost’ access to mysql.

I use ‘etckeeper’ to keep a log and trace of what’s getting installed and modified. Comparing log timestamps and git changes, I concluded that the mess was caused by a simple modification to /etc/hosts: the yellow line below, which is a step in the installation procedure, created the havoc:

?127.0.0.1 localhost

127.0.1.1 ostk-controller1

10.0.0.40 ostk-controller1

10.0.0.41 ostk-nova1

A Poor Workaround

Putting this yellow line in comment provided a poor workaround: it solved mysql ERROR 1045, but the controller must have its hostname resolved in /etc/hosts so we’re not satisfied.

Further reading suggested that the error has something to do with the way mysql interprets ‘%’ in statements such as:

GRANT USAGE ON *.* TO ‘glance’@’%’ IDENTIFIED BY PASSWORD(‘openstack’);

Is it possible that ‘%’ stands for “all hosts except localhost”? i read this more than once (see References below) but could hardly belive. Interestingly, the MySQL documentation isn’t clear about this question and that’s the reason, i guess, there’s so much confusion and so many posts related to ERROR 1045.

It’s worth noting that the hint to the final answer was found in a post with the title “Any way to make anyhost ‘%’ include localhost”…

Understanding MySQL Access

I’ve set up a separate VM to explore that, and here are my findings.

After installing mysql-server (latest for Precise is 5.5.24), the USER table and GRANTS get the default settings listed below (for clarity i’ve cut the right side of the output so it doesn’t look exactly as on screen):

[14:57:22]root@mysqltests[~]
# mysql -u root -p
. . .
Server version: 5.5.24-0ubuntu0.12.04.1 (Ubuntu)
. . .

mysql> SELECT user,host,password FROM mysql.user;
+——————+————+————————-+
| user | host | password |
+——————+————+————————-+
| root | localhost | *77B48D6366D102139D3719 |
| root | mysqltests | *77B48D6366D102139D3719 |
| root | 127.0.0.1 | *77B48D6366D102139D3719 |
| root | ::1 | *77B48D6366D102139D3719 |
| | localhost | |
| | mysqltests | |
| debian-sys-maint | localhost | *04D30B480932109EFD77E1 |
+——————+————+————————-+
7 rows in set (0.00 sec)

mysql> show grants;
+———————————————————+
| Grants for root@localhost |
+———————————————————+
| GRANT ALL PRIVILEGES ON *.* TO ‘root’@’localhost’ |

| IDENTIFIED BY PASSWORD ‘*77B48D6366D102139D3719’ |

| WITH GRANT OPTION |
| GRANT PROXY ON ”@” TO ‘root’@’localhost’ WITH GRANT |

| OPTION |
+———————————————————+
2 rows in set (0.00 sec)

The mysql.user Table

At first glance, we have 2 users (root and debian-sys-maint). That’s wrong, because mysql’s “user” is a ‘user’@’host’ pair association. So we have 7 in total: ‘root’ is defined (with the same password) for any combination of ‘localhost’ (the first 4 lines), then we have 2 strange lines with empty username, and finally the debian backdoor ‘debian-sys-maint’.

The grants

The ‘show grants’ above shows only grants for ‘root’. But if we run the next staement, we see what access is provided to any user connecting from ‘localhost’:

mysql> show grants for ”@’localhost’;
+————————————–+
| Grants for @localhost |
+————————————–+
| GRANT USAGE ON *.* TO ”@’localhost’ |
+————————————–+

Which (indirectly) explains why running this command (as Linux user ‘ori’) doesn’t require a password:

[16:16:57]ori@mysqltests[~]

$ mysqladmin ping
mysqld is alive

Where this one fails:

[16:14:59]ori@mysqltests[~]
$ mysqladmin -uroot ping
mysqladmin: connect to server at ‘localhost’ failed
error: ‘Access denied for user ‘root’@’localhost’ (using password: NO)’

Honestly, in the beginning i thought there’s some balck magic here related to the user (‘ori’, in this case) defined during ubuntu installation, or a special Linux group memebership, or some apparmor profile or god-knows what else.

But there’s no black magic after all, and it’s all inside mysql:

The first thing to bear in mind is that the empty USER field ” is a wildcard, same as ‘%’ for host.

The second is that mysql prefers the explicit match over the wildcard. For example, user ‘root’ can match either [1] the explicit ‘root’@localhost’ row or [2] the wildcard ”@’localhost’ row. Since there’s an explicit entry for [1] in the table mysql.user, it’ll be used. This in turn requires a password so when i try to connect as ‘root’ without a password i’m rejected.

When i connect as ‘ori’ – which isn’t even a mysql user, there’s only one possible match – ”@’localhost’ and this line in the table doesn’t have a password.

This nicely explains why the above mysqladmin command works for ‘ori’ and fails for ‘root’.

To sum it up: mysql controls access (or connection request) based on the USER table. Which user, from which host and whether a password is required.

Once connected, the GRANTS determine what the user is allowed to do. When connected as ‘ori’ i’m limited to “USAGE” (e.g. check if server is up, what version and the like of inoffensive commands).

So far so good – but why ‘glance’@’localhost’ is denied access on the OpenStack controller?

When the static IP address of the conroller wasn’t in /etc/hosts (or after it was commented-out), there was only one match for ‘glance’ = ‘glance’@’%’

This, in turn, comes from the connection string (in /etc/glance/glance-registry.conf) which is:

sql_connection = mysql://glance:openstack@10.0.0.40/glance

It specifies user, password and host.

The line I’ve added for 10.0.0.40 in /etc/hosts, told mysql (indirectly) that host ‘ostk-controller1’ is actually ‘localhsot’. From now on, there are 2 possible matches for ‘glance’, and the one picked by mysql is ”@’localhost’. This row, however, doesn’t require a password – which the sql_connection string provide.

And that’s why all OpenStack services couldn’t connect to mysql.

Check against the USER table below, this was taken from ostk-controller (not the test VM):

mysql> SELECT user,host,password FROM mysql.user;
+——————+——————+————————-+
| user | host | password |

+——————+——————+————————-+
| root | localhost | *3A4A03AC22526F6B591010 |

| root | ostk-controller1 | *3A4A03AC22526F6B591010 |

| root | 127.0.0.1 | *3A4A03AC22526F6B591010 |

| root | ::1 | *3A4A03AC22526F6B591010 |
| | localhost | |

| | ostk-controller1 | |
| debian-sys-maint | localhost | *F714636CE8A7836873F7C8 |
| nova | % | *3A4A03AC22526F6B591010 |
| glance | % | *3A4A03AC22526F6B591010 |
| keystone | % | *3A4A03AC22526F6B591010 |
+——————+——————+————————-+
10 rows in set (0.00 sec)

Solution for ERROR 1045

After understanding why, let’s improve on the poor workaround.

I’d like to credit an answer by Paul DuBois from 2004 for this solution(it’s worth noting that the subject was “Re: Any way to make anyhost ‘%’ include localhost”).

Borrowing from there, here’s the remedy:

in MySQL:

mysql -uroot -p

DELETE FROM mysql.user WHERE Host=’localhost’ AND User=”;

DELETE FROM mysql.user WHERE Host=’ostk-controller1′ AND User=”;

FLUSH PRIVILEGES;

in /etc/hosts:

Replace the line

127.0.1.1 ostk-controller1

by this one:

10.0.0.40 ostk-controller1

Quoting from Debian’s reference manual:

For a system with a permanent IP address, that permanent IP address should be used here instead of 127.0.1.1

finally restart networking and mysqld – or simply reboot.

A Second Solution

Months after going through the above study, i found out why some OpenStack installations don’t hit this issue; The keystone installation instructions (from Ubuntu, for Essex, can be found here) create each OSTK user in mysql twice, as in:

mysql> CREATE DATABASE keystone;
CREATE USER ‘keystone’@’localhost’ IDENTIFIED BY ‘Secret_pass’;
GRANT ALL PRIVILEGES ON keystone.* TO ‘keystone’@’localhost’
WITH GRANT OPTION;
CREATE USER ‘keystone’@’%’ IDENTIFIED BY ‘Secret_pass’;
GRANT ALL PRIVILEGES ON keystone.* TO ‘keystone’@’%’
IDENTIFIED BY ‘Secret_pass’;
FLUSH PRIVILEGES;

mysql -u root -p

CREATE USER ‘bill’@’%’ IDENTIFIED BY ‘passpass’;

grant all privileges on *.* to ‘bill’@’%’ with grant option;

mysql -u bill -p

ERROR 1045 (28000): Access denied for user ‘bill’@’localhost’ (using password: YES)

CREATE USER bill@localhost IDENTIFIED BY ‘passpass’;
grant all privileges on *.* to bill@localhost with grant option;

If you want to connect remotely, you must specify either the DNS name, the public IP, or 127.0.0.1 using TCP/IP:

mysql -u bill -p -hmydb@mydomain.com
mysql -u bill -p -h10.1.2.30
mysql -u bill -p -h127.0.0.1 –protocol=TCP

SELECT USER(),CURRENT_USER();

mysql> select user,host from mysql.user;
+———+———–+
| user | host |
+———+———–+
| lwdba | % |
| mywife | % |
| lwdba | 127.0.0.1 |
| root | 127.0.0.1 |
| lwdba | localhost |
| root | localhost |
| vanilla | localhost |
+———+———–+
7 rows in set (0.00 sec)

mysql> grant all on *.* to x@’%’;
Query OK, 0 rows affected (0.02 sec)

mysql> select user,host from mysql.user;
+———+———–+
| user | host |
+———+———–+
| lwdba | % |
| mywife | % |
| x | % |
| lwdba | 127.0.0.1 |
| root | 127.0.0.1 |
| lwdba | localhost |
| root | localhost |
| vanilla | localhost |
+———+———–+
8 rows in set (0.00 sec)

mysql> update mysql.user set user=” where user=’x’;
Query OK, 1 row affected (0.00 sec)
Rows matched: 1 Changed: 1 Warnings: 0

mysql> flush privileges;
Query OK, 0 rows affected (0.01 sec)

mysql> select user,host from mysql.user;
+———+———–+
| user | host |
+———+———–+
| | % |
| lwdba | % |
| mywife | % |
| lwdba | 127.0.0.1 |
| root | 127.0.0.1 |
| lwdba | localhost |
| root | localhost |
| vanilla | localhost |
+———+———–+
8 rows in set (0.00 sec)

mysql>

~$ mysql -u root -p
Enter Password:

mysql> grant all privileges on *.* to bill@localhost identified by ‘pass’ with grant option;

root@myhost:/home/mysql-5.5.16-linux2.6-x86_64# ./mysql -ubill -ppass –socket=/tmp/mysql-5.5.sock
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 7
Server version: 5.5.16 MySQL Community Server (GPL)

Type ‘help;’ or ‘\h’ for help. Type ‘\c’ to clear the buffer.

mysql> SELECT user, host FROM mysql.user;
+——+———–+
| user | host |
+——+———–+
| bill | % |
| root | 127.0.0.1 |
| root | ::1 |
| root | localhost |
+——+———–+
4 rows in set (0.00 sec)

mysql> SELECT USER(), CURRENT_USER();
+—————-+—————-+
| USER() | CURRENT_USER() |
+—————-+—————-+
| bill@localhost | bill@% |
+—————-+—————-+
1 row in set (0.02 sec)

mysql> SHOW VARIABLES LIKE ‘skip_networking’;
+—————–+——-+
| Variable_name | Value |
+—————–+——-+
| skip_networking | ON |
+—————–+——-+
1 row in set (0.00 sec)

mysql>

Optimizing my.cnf file for MySQL

[mysqld]
socket=/path/to/mysql.sock
datadir=/var/lib/mysql
skip-locking
skip-innodb
# MySQL 4.x has query caching available.
# Enable it for vast improvement and it may be all you need to tweak.
query_cache_type=1
query_cache_limit=1M
query_cache_size=32M
# max_connections=500
# Reduced to 200 as memory will not be enough for 500 connections.
# memory=key_buffer+(sort_buffer_size+read_buffer_size)*max_connections
# which is now: 64 + (1 + 1) * 200 = 464 MB
# max_connections = approx. MaxClients setting in httpd.conf file
# Default set to 100.
#max_connections=200
#interactive_timeout=180
interactive_timeout=100
#wait_timeout=180
#wait_timeout=100
# Reduced wait_timeout to prevent idle clients holding connections.
#wait_timeout=30
wait_timeout=15
connect_timeout=10
# max_connect_errors is set to 10 by default
#max_connect_errors=10
#table_cache=256
#table_cache=1024
# Checked opened tables and adjusted accordingly after running for a while.
table_cache=512
#tmp_table_size=32M by default
#thread_cache=128
# Reduced it to 32 to prevent memory hogging. Also, see notes below.
thread_cache=32
# key_buffer=258M
# Reduced it by checking current size of *.MYI files, see notes below.
key_buffer=128M
# Commented out the buffer sizes and keeping the default.
# sort_buffer_size=2M by default.
#sort_buffer_size=1M
# read_buffer_size=128K by default.
#read_buffer_size=1M
# 1Mb of read_rnd_buffer_size for 1GB RAM — see notes below.
# read_rnd_buffer_size=256K by default.
#read_rnd_buffer_size=1M
# myisam_sort_buffer_size used for ALTER, OPTIMIZE, REPAIR TABLE commands.
# myisam_sort_buffer_size=8M by default.
#myisam_sort_buffer_size=64M
# thread_concurrency = 2 * (no. of CPU)
thread_concurrency=2
# log slow queries is a must. Many queries that take more than 2 seconds.
# If so, then your tables need enhancement.
log_slow_queries=/var/log/mysqld.slow.log
long_query_time=2

[mysql.server]
user=mysql
basedir=/var/lib

[safe_mysqld]
err-log=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
open_files_limit=8192

[mysqldump]
quick
max_allowed_packet=16M

[mysql]
no-auto-rehash
# Remove the next comment character if you are not familiar with SQL
#safe-updates

[isamchk]
key_buffer=64M
sort_buffer=64M
read_buffer=16M
write_buffer=16M

[myisamchk]
key_buffer=64M
sort_buffer=64M
read_buffer=16M
write_buffer=16M

[mysqlhotcopy]
interactive-timeout

[client]
socket=/path/to/mysql.sock

Below are notes on some of the important variables, I took down while tuning the config file.

query_cache_size:
MySQL 4 provides one feature that can prove very handy – a query cache. In a situation where the database has to repeatedly run the same queries on the same data set, returning the same results each time, MySQL can cache the result set, avoiding the overhead of running through the data over and over and is extremely helpful on busy servers.
key_buffer_size:
The value of key_buffer_size is the size of the buffer used with indexes. The larger the buffer, the faster the SQL command will finish and a result will be returned. The rule-of-thumb is to set the key_buffer_size to at least a quarter, but no more than half, of the total amount of memory on the server. Ideally, it will be large enough to contain all the indexes (the total size of all .MYI files on the server).
A simple way to check the actual performance of the buffer is to examine four additional variables: key_read_requests, key_reads, key_write_requests, and key_writes.
If you divide the value of key_read by the value of key_reads_requests, the result should be less than 0.01. Also, if you divide the value of key_write by the value of key_writes_requests, the result should be less than 1.
table_cache:
The default is 64. Each time MySQL accesses a table, it places it in the cache. If the system accesses many tables, it is faster to have these in the cache. MySQL, being multi-threaded, may be running many queries on the table at one time, and each of these will open a table. Examine the value of open_tables at peak times. If you find it stays at the same value as your table_cache value, and then the number of opened_tables starts rapidly increasing, you should increase the table_cache if you have enough memory.
sort_buffer:
The sort_buffer is very useful for speeding up myisamchk operations (which is why it is set much higher for that purpose in the default configuration files), but it can also be useful everyday when performing large numbers of sorts.
read_rnd_buffer_size:
The read_rnd_buffer_size is used after a sort, when reading rows in sorted order. If you use many queries with ORDER BY, upping this can improve performance. Remember that, unlike key_buffer_size and table_cache, this buffer is allocated for each thread. This variable was renamed from record_rnd_buffer in MySQL 4.0.3. It defaults to the same size as the read_buffer_size. A rule-of-thumb is to allocate 1KB for each 1MB of memory on the server, for example 1MB on a machine with 1GB memory.
thread_cache:
If you have a busy server that’s getting a lot of quick connections, set your thread cache high enough that the Threads_created value in SHOW STATUS stops increasing. This should take some of the load off of the CPU.
tmp_table_size:
“Created_tmp_disk_tables” are the number of implicit temporary tables on disk created while executing statements and “created_tmp_tables” are memory-based. Obviously it is bad if you have to go to disk instead of memory all the time.

DjbDNS DNS Server On CentOS

 djbDNS DNS Server On CentOS

What is djbDNS? And why do we use djbDNS? There is a new point of view to serve the dns service – each of the dns server functionalities is a separate service, like authority, cache, forward and so on.

The other difference is the daemon-tools which will rapidly restart services to prevent zombies.

Log in as root.

# yum update

# yum install gcc

# mkdir pkg

# cd pkg

The first step is to install the daemon-tools:

# cd ~/pkg

# wget http://cr.yp.to/daemontools/daemontools-0.76.tar.gz

# gunzip daemontools-0.76.tar

# tar -xpf daemontools-0.76.tar

# rm -f daemontools-0.76.tar

# cd admin/daemontools-0.76

# vi src/conf-cc

Append the following line at the end of the gcc line:

-include /usr/include/errno.h
# ./package/install

One other package we need to prepare for djbdns to be functional is ucspi:

# cd ~/pkg

# wget http://cr.yp.to/ucspi-tcp/ucspi-tcp-0.88.tar.gz

# gunzip ucspi-tcp-0.88.tar

# tar -xf ucspi-tcp-0.88.tar

# cd ucspi-tcp-0.88

# vi src/conf-cc

Append the following line at the end of the gcc line:

-include /usr/include/errno.h
# make

# make setup check

The next step is the document publication:

# cd ~/pkg

# wget http://cr.yp.to/djbdns/doc.tar.gz

Next we will unzip docs under /doc:

# gunzip < doc.tar.gz | (cd /; tar -xf -)

Use the following script to merge in system docs:

#!/bin/sh
for i in packages commands cfunctions fileformats
do
  sort -f /dev/null `find /doc/merge -name $i.html` > /doc/$i.new
  mv /doc/$i.new /doc/$i.html
done

Save script into a file: script.sh

# chmod +x script.sh

# ./script.sh

# cd ~/pkg

# wget http://cr.yp.to/djbdns/djbdns-1.05.tar.gz

# gunzip djbdns-1.05.tar

 tar -xf djbdns-1.05.tar

# cd djbdns-1.05

# vi src/conf-cc

Append the following line at the end of the gcc line:

-include /usr/include/errno.h

# make

# make setup check

All compiling gets done.

The next step is the dns server configuration.

 

DNSCACHE
Create two system user accounts:

# useradd -d /var/dnscache -s /bin/false dnscache

# useradd -d /var/dnscache -s /bin/false dnslog

Configure the cache:

# dnscache-conf dnscache dnslog /var/dnscache/dnscache <listen-IP>

Example: dnscache-conf dnscache dnslog /var/dnscache/dnscache 192.168.20.1

Allow the rest of your network to query dnscache:

# touch /var/dnscache/dnscache/root/ip/<Net-ID>

Example: touch /var/dnscache/dnscache/root/ip/192.168

Add dnscache to the list of services to be monitored by svscan:

# ln -sf /var/dnscache/dnscache /service/

If you like ms-windows… you can make a reboot to be sure that all the world is in place.

There is a point here that your are still not able to query from your cache server, because your clients are now able to be resolved in reverse mode.

As mentioned before it’s one of the dns-cache security features to reverse-check clients. So in the next step we will work on tinydns to act for us.

 

DNSTINY
Once again, we need two system user accounts:

# useradd -d /var/dnscache -s /bin/false tinydns

# useradd -d /var/dnscache -s /bin/false tinylog

# tinydns-conf tinydns tinylog /var/dnscache/tinydns 127.0.0.1

Now it’s time to add nodes into dns database:

# cd /var/dnscache/tinydns/root

# ./add-ns rmohan.com 192.168.2.1

# ./add-ns 2.168.192.in-addr.arpa 192.168.2.1

# ./add-mx rmohan.com 192.168.2.2

# ./add-host ns1.rmohan.com 192.168.2.1

# ./add-host mail.rmohan.com 192.168.2.2

# ./add-alias test.rmohan.com 192.168.2.2

# make

These nodes with go to the database file /var/dnscache/tinydns/root/data that you are able to edit manually.

The last step is the dns service startup:

# ln -sf /var/dnscache/tinydns /service

ProxyPassReverse

am using mod rewrite to mask the context root of my application. For example,

RewriteRule ^/directory/(.*) balancer://appcluster/directory/$1 [P] 

The appcluster looks like this:

<Proxy balancer://appcluster> BalancerMember http://localhost:8080/App route=app_01 keepalive=On loadfactor=1 ttl=300 min=3 smax=5 max=15 ProxySet lbmethod=byrequests stickysession=JSESSIONID|jsessionid timeout=120 nofailover=On </Proxy> 

Do I need to use ProxyPassReverse at all? I used to use it because my old webserver code looked like this:

ProxyPass /App balancer://appcluster lbmethod=byrequests stickysession=JSESSIONID|jsessionid timeout=120 nofailover=On ProxyPassReverse /App http://localhost:9013/App 



The ProxyPassReverse is used to change the headers sent by the app (appcluster) to Apache, before Apache sends it the browser. For example, if the app sits at http://localhost:9013/, and it tries to redirect the browser to, say, /new_location/, then it will respond with a redirect and location header of http://localhost:9013/new_location/, and Apache will take this and send it off to the browser. Problem is, the browser (assuming it’s somewhere else) then tries to send a request to http://localhost:9013/new_location/, and gets an error.

What ProxyPassReverse does is intercepts those headers, and rewrites them so that they match what the Apache server that’s doing the proxying looks like. So if my apache server is hosting http://myhost.com/ and I have a ProxyPass that points / to http://localhost:9013/App, if the application sitting at localhost:9013 returns a redirect to http://localhost:9013/App/new_location/, I’ll need to use ProxyPassReverse so that it gets rewritten to http://myhost.com/new_location/ by Apache before sending the request back to the browser.

If you aren’t issuing redirects, it’s not going to be an issue, but it doesn’t hurt to have it there in case a 301/302 redirect is returned. As far as mod_rewrite, the RewriteRule applies to the request going to the App, and not the response coming from the App. So they are mutually exclusive events.



SSH login without password

we  need an automatic login from host A / user a to Host B / user b. You don’t want to enter any passwords, because you want to call sshfrom a within a shell script.

How to do it

First log in on A as user a and generate a pair of authentication keys. Do not enter a passphrase:

a@A:~> ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/a/.ssh/id_rsa): 
Created directory '/home/a/.ssh'.
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /home/a/.ssh/id_rsa.
Your public key has been saved in /home/a/.ssh/id_rsa.pub.
The key fingerprint is:
3e:4f:05:79:3a:9f:96:7c:3b:ad:e9:58:37:bc:37:e4 a@A

Now use ssh to create a directory ~/.ssh as user b on B. (The directory may already exist, which is fine):

a@A:~> ssh b@B mkdir -p .ssh
b@B's password: 

Finally append a’s new public key to b@B:.ssh/authorized_keys and enter b’s password one last time:

a@A:~> cat .ssh/id_rsa.pub | ssh b@B 'cat >> .ssh/authorized_keys'
b@B's password: 

From now on you can log into B as b from A as a without password:

a@A:~> ssh b@B hostname
B

A note from one of our readers: Depending on your version of SSH you might also have to do the following changes:

  • Put the public key in .ssh/authorized_keys2
  • Change the permissions of .ssh to 700
  • Change the permissions of .ssh/authorized_keys2 to 640

SSH Keys

Introduction

Ssh keys (key biased authentication) can be used as an alternate to using your user’s login password (password authentication) to access a ssh server. Keys can be used with or without a password (not to be confused with the login password).

This document is intended as an introduction to using ssh keys to log into a ssh server. I will review the basics of generating and implementing ssh keys and it is assumed you already have a ssh server installed and configured to accept remote connections.

PAM links (note PAM modules vary slightly across Linux distributions):

PLinux PAM configuration that allows or deny login via the sshd server
Pulling The Covers Off Linux PAM
The Linux-PAM System Administrators’ Guide

Generate keys

Keys come in pairs, a private key and a public key (see below). The public key is placed on the server and the private key is used by the client to log onto the server.

In general, I advise a unique key for each user on each server, however, you can use a pair of keys for as many users/clients/servers as you wish.

Using the command line

Keys can be generated on the command line using “ssh-keygen”. The general syntax to generate a key pair is :

ssh-keygen

Enter your desired password when prompted (you will not see anything on the screen as you type). To generate a key with no password, simply hit the enter key.

This command will generate two files, id_rsa and id_rsa.pub

  • id_rsa is the “private” file and is used to by the client to log into the ssh server(s).
  • id_rsa.pub is the “public” key and is placed on the ssh server(s).

You may specify a different name for your key pair with the -f option. For example to make a key named “foo”, use -f foo :

ssh-keygen -f foo

This will generate two files:

  • foo – this is the private key.
  • foo.pub – this is the public key.

You may use most any name you desire for you key.

Using graphical tools (seahorse)

Seahorse is a graphical tool to manage ssh keys. It has a number of features including key generation and managing keys. Seahorse will generate ssh keys and transfer your key to the ssh server (as long as passwords authentication is allowed).

There is a nice write up on how to use seahorse here (Debian Admin): SSH Key Authentication Using seahorse (GUI)

Changing the key password

You can change your key’s password either on the command line ( with ssh-keygen -p ) or from seahorse (graphical).

From the command line, use ssh-keygen with the -p option:

ssh-keygen -p -f ~/.ssh/id_rsa

Enter the old PW, then the new.

If you forgot the old password, you are out of luck and will need to generate a new key.

Transfer the key to the ssh server

You may do this via the easy way or the hard way.

Public keys are stored by default in ~/.ssh , one key per line (a user may have multiple keys). You can change the location by editing /etc/ssh/sshd_config (see man sshd_config ).

Easy way – use ssh-cop-id

ssh-copy-id is a command that automates transfer of your public key to the server. To perform the transfer you will need to log in, so do not disable password authentication until after you confirm the key is working.

ssh-copy-id -i key_name user@server

ssh-copy-id -i id_rsa bodhi@ssh.server.com

ssh-copy-id -i foo bodhi@ssh.server.com

Hard (Manual) way

You can manually transfer the key as outlined below.

First, using any method, transfer the .pub key to the server. You can do this using a network protocol (scp) or sneakerware (flash drive).

You then copy or add the .pub key to ~/.ssh/authorized keys.

Run these commands on the server (substitute the name of your key as needed).

cat id_rsa.pub >> ~/.ssh/authorized_keys

cat foo.pub >> ~/.ssh/autorized_keys

Using the key to login to a ssh server

By default, the private key should be stored in ~/.ssh , although you can keep it anywhere you wish.

Use the “-i” option with ssh to specify the private key you wish to use to log into the server.

ssh -i ~/.ssh/id_rsa bodhi@ssh.server.com

ssh -i ~/.ssh/foo bodhi@ssh.server.com

If you are using PuTTY, you specify the private, .ppk key (Under SSH -> Auth in the menu on the left).

You may use one key per client or you may use the same key on many clients. You may carry the private key with you on a flash drive.

Note on permissions :

Your ssh keys and the authorized_keys file must have restrictive permissions.

ssh keys can not be “world readable” Permissions must be 400 or 600 :

Otherwise you will see this error message :

@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@ WARNING: UNPROTECTED PRIVATE KEY FILE! @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@
Permissions 0440 for ‘./id_rsa’ are too open.
It is recommended that your private key files are NOT accessible by others.
This private key will be ignored.

To fix this message, use chmod:

chmod 600 id_rsa

authorized_keys can be world readable, but only writable by the owner.

You will not get an error message here, the key simply will not work.

To fix this, use chomd.

chmod 600 authorized_keys

You can use permissions of 400, 440, 444, 600, 640, or 644 .

For additional information on Linux permissions see : Linux permissions

PuTTY

PuTTY is a ssh client and runs on both Windows and Linux. You can not use openssh keys directly with PuTTY (PuTTY uses it’s own key, a .ppk or PuTTY Private Key). You can import your openssh key with PuTTYgen and save it as a PuTTY ppk key.

For a detailed walk through, see : How to import ssh keys to putty using PuTTYgen

Note : Be sure to save the key with the ssh2 protocol.

PuTTY will save the private keys ending in “.ppk” (Putty Private Key). If needed, save the public key as a .pub (same as openssh key pairs). If needed, you can import the PuTTY .pub keys to a server, similar to above (copy the PuTTY .pub key to ~/.ssh/authorized_keys).

id_rsa.ppk or foo.ppk

You may import public PuTTY keys to a ssh server as above.

Disable password authentication

Before disabling password authentication, first make sure you can log in to your server with a key

If you disable password authentication you will not be able to log into the server (remotely) without a working key. Do not disable password authentication without either a working key or physical access to the server.

Using any editor, edit /etc/ssh/sshd_config

nano /etc/ssh/sshd_config

Change

PasswordAuthentication yes

to

PasswordAuthentication no

Save your changes and restart (or reload) the ssh server.

sudo service ssh reload

If you attempt to log in without a key you will now get an error message :

Permission denied (publickey).

Using keys without entering a password

Rather then generating a key without a password, you can use ssh agent (command line tool) to load your ssh keys into memory. Once a key is loaded into memory, you can ssh into the server without manually entering the password.

In theory (and on some web pages) seahorse *should* automatically load your ssh keys when you log in. In practice, in my experience, the keys often fail to load. In that event you would need to fall back to the command line (ssh-add).

To use ssh-agent, first load the key:

ssh-add -i ~/.ssh/id_rsa

Enter your password when prompted. You can the ssh into the server without entering a password.

ssh bodhi@ssh.server.com

Notice you do not need to specify the key and ssh connects without asking a password

ssh-add is capable of loading multiple keys and has a few additional options including -x to lock and -X to unlock the agent.

SSH Agent without X

ssh-agent can be run without X, from a console, by starting a new shell. You may want to run a new session in “screen”.

ssh-agent bash
ssh-agent zsh #for those who prefer zsh to bash

This starts a new {bash,zsh} shell and you may now add keys with ssh-add as above. This option works well with screen.