October 2025
M T W T F S S
 12345
6789101112
13141516171819
20212223242526
2728293031  

Categories

October 2025
M T W T F S S
 12345
6789101112
13141516171819
20212223242526
2728293031  

FIND Command

FIND Command

Find is a versatile tool which can be used to locate files and directories satisfying different user criteria. But the sheer number of options for this command line tool makes it at the same time both powerful and encumbering for the user. Here I will list a few combinations which one can use to get useful results using find command.

f – file
d – directory
l – symbolic link
c – character
p – named pipe (FIFO)
s – socket
b – block device

Find all HTML files starting with letter ‘a’ in your current directory (Case sensitive)
find . -name a\*.html

Same as above but case insensitive search.
find . -iname a\*.html

Find files which are larger than 5 MB in size.
find . -size +5000k -type f

Here the ‘+’ in ‘+5000k’ indicates greater than and k is kilobytes. And the dot ‘.’ indicates the current directory. The -type option can take any of the following values:

… Which is all files with 0 bytes size. The option -size can take the following:

c – bytes
w – 2 byte words
k – kilo bytes
b – 512 byte blocks

Note: The above command can also take the -empty parameter.

Find is very powerful in that you can combine it with other commands. For example, to find all empty files in the current directory and delete them, do the following:
find . -empty -maxdepth 1 -exec rm {} \;

To search for a html file having the text ‘Web sites’ in it, you can combine find with grep as follows:
find . -type f -iname \*.html -exec grep -s “Web sites” {} \;

… the -s option in grep suppresses errors about non-existent or unreadable files. And {} is a placeholder for the files found. The semicolon ‘;’ is escaped using backslash so as not to be interpreted by bash shell.

Note: You can use the -exec option to combine any command in Linux with the find command. Some of the useful things you can do with it are as follows:

Compress log files on an individual basis
find /var -iname \*.log -exec bzip {} \;

Find all files which belong to user lal and change its ownership to ravi
find / -user lal -exec chown ravi {} \;

Note: You can also use xargs command instead of the -exec option as follows:
find /var -iname \*.log | xargs bzip –

Find all files which do not belong to any user:
find . -nouser

Find files which have permissions rwx for user and rw for group and others :
find . -perm 766

… and then list them.

find . -perm 766 -exec ls -l {} \;

Find all directories with name music_files
find . -type d -iname \*music_files\*

Suppose you want to find files of size between 700k and 1000k, do the following:
find . \( -size +700k -and -size -1000k \)

And how about getting a formatted output of the above command with the size of each file listed ?
find . \( -size +700k -and -size -1000k \) -exec du -Hs {} \; 2>/dev/null

… here, the ‘2>/dev/null’ means all the error messages are discarded or suppressed.

You can also limit your search by file system type. For example, to restrict search to files residing only in the NTFS and VFAT filesystem, do the following:
find / -maxdepth 2 \( -fstype vfat -or -fstype ntfs \) 2> /dev/null

To View Or List Only Directories In Linux?
Do you like www.linuxnix.com ? Please consider supporting us by becoming a subscriber and get a Linux basics e-book for free.

How to view/list only directories in Linux?
Ans : This can be achieved in two ways
1. Through ls command
2. Through find command

With ls we have to use grep to get the directory listings.
Ls –l grep ^d

Example :
[root@test rmohan_a]# ls -l grep ^d
d——— 2 rmohan_a rmohan_a 4096 Sep 8 09:54 HTWFAIP
drwxrwxr-x 2 rmohan_a root 4096 Nov 27 12:30 LinuxCBT – RHEL5
drwxrwxr-x 2 rmohan_a root 4096 Oct 12 16:40 Software
[root@test rmohan_a]#

With find we can have more controle on how to display only directories.

A. To display all the directories and sub-directories in present directory
#find . -type d

B. Displaying only directories in present directory
#find /root/ -type d –maxdepth 1

C. Displaying just directories in present directry and its sub-directories
#find /root/ -type d –maxdepth 2

* find top 10 largest files in /var:

$ find /var -type f -ls | sort -k 7 -r -n | head -10

* find all files having size more than 5 GB in /var/log/:

$ find /var/log/ -type f -size +5120M -exec ls -lh {} \;

* find all today’s files and copy them to another directory:

$ find /home/me/files -ctime 0 -print -exec cp {} /mnt/backup/{} \;

* find all temp files older than a week and delete:

$ find /temp/ -mtime +7-type f | xargs /bin/rm -f

* find and rename all mp3 files by changing their uppercase names to lowercase:

$ find /home/me/music/ -type f -name *.mp3 -exec rename ‘y/[A-Z]/[a-z]/’ ‘{}’ \;

find mtime

find . -mtime 0 # find files modified between now and 1 day ago
# (i.e., within the past 24 hours)
find . -mtime -1 # find files modified less than 1 day ago
# (i.e., within the past 24 hours, as before)
find . -mtime 1 # find files modified between 24 and 48 hours ago
find . -mtime +1 # find files modified more than 48 hours ago

find . -mmin +5 -mmin -10 # find files modified between
# 6 and 9 minutes ago

Find Parameters

-daystart This flag starts at the beginning of the day.
-atime The time the file was last accessed — in number of days.
-ctime The time the file’s status last changed — in number of days.
-mtime The time the file was last modified — in number of days.
-amin The time the file was last accessed — in number of minutes. (It is not available on all implementations.)
-cmin The time the file’s status last changed — in number of minutes. (It is not available on all implementations.)
-mmin The time the file was last modified — in number of minutes. (It is not available on all implementations.)
-type This flag describes the type of file, such as d for directories.
-userX Files belonging to user X.
-groupX Files belonging to group X.
-newerX Files that are newer than file X.

Here’s how to list all the files in your home directory tree that were modified exactly one hour ago:$ find ~ -mmin 60 \! -type d

Giving a negative value for a flag means to match that number or sooner. For example, here’s how to list all the files in your
home directory tree that were modified exactly one hour ago or any time since

find ~ -mmin -60 \! -type d

$ date
Mon Oct 23 09:42:42 EDT 2006
$ touch -t 10230842 temp
$ ls -l temp
-rw-r–r– 1 joe joe 0 Oct 23 08:42 temp
$ find ~ -newer temp \! -type d

find / -user `whoami` -daystart -atime -1 \! -type d

Give different values for the various time flags to change the search times. You can also combine flags. For instance,
you can list all the files in your home directory tree that were both accessed and modified between now and seven days ago:
find ~ -daystart -atime -7 -mtime -7 \! -type d

find /home/$1/mail/*/mail/.spam/cur -type f -mtime +7 -exec rm {} \;
find /home/$1/mail/*/mail/.spam/new -type f -mtime +7 -exec rm {} \;
find . -type f -exec grep ‘NMX_FXNG_AND_CONTRACT_DBF’ {} \;

Delete Empty Directories
# find folder/ -type d -empty | xargs -i -t rm -rf {}
or
# find folder/ -type d -empty -delete

DISK SPACE COMMAND

Find files based and sorted on Size
# find / -type f -size +20000k -exec ls -lh {} \; 2> /dev/null | awk ‘{ print $NF “: ” $5 }’ | sort -nrk 2,2

MYSQL Back On Unix

MYSQL Back On Unix

Shell script is a script where we are writing different types of commands and executing those commands from a single file. We can execute that command manually, by entering command one by one. But if we use shell script we have to write those commands into a text file for the first time and then we can run those commands as many times as required.

In this article first I will show you, Complete Script. Later on, you will get a description of that script. I assumed that you know about shell scripting, mysqldump and crontab.

Operating System: Any Linux or UNIX.

Main Script (for backup your mysql database):

This shell script will make the backup process of a database automatic. Just copy and paste this script in a text editor, put database user name, password, and database name. I will use mysqlump command to backup the database. Later on you will get description of each line.
1. Make a directory name “backup” and “oldbackup”
1
2

mkdir /backup
mkdir /oldbackup
2. Now make file name “backup.sh” and edit with any editor you like

I’m using vi here-
1

# vi /backup/backup.sh

Now write these lines into backup.sh file:
#!bin/bash
cd /backup
echo “You are In Backup Directory”
mv backup* /oldbackup
echo “Old Databases are Moved to oldbackup folder”
Now=$(date +”%d-%m-%Y–%H:%M:%S”)
File=backup-$Now.sql
mysqldump –u user-name –p ‘password’ database-name > $File
echo “Your Database Backup Successfully Completed”

Script Description:

Remember, in line number 8: Put your Database username, Password, database name after mysqldump command.

When we run this script, first it goes to a /backup directory. Then it will move old database backup files to /oldbackup folder. Next it generates a file name from system date and time. And finally mysqldump command will generate a database backup file with “.sql” format
3. Set permission to backup.sh file executable
1

# chmod +x /backup/backup.sh
4. Running the Script
1

#./backup.sh

You will get following output after executing the script.
root@Server1:/download#./backup.sh
You are in Download Directory
Old Backup Database is Moved to oldbackup folder
database backup successful completed
root@Server1:/download#

NB: first time when you run this script you will get a message “no such file”, because you don’t have old backup file. No problem just runs again this script, problem will be solved.
5. Schedule the Backup using cron

Using Cron job you can execute this script in a certain time, and all works will be done automatically. Use crontab command to edit editing cron schedules.

#crontab –e

Just add below line in the editor and save it.
0 13 * * * * /backup/backup.sh

In this way every day 1PM your database will back up in your desired folder. Please visit crontab manuals for more details about setting the cron jobs.

This is a very basic script for the beginners. Hope you can use the same idea for more complex backup. We will try to come up with new scripts to automate further. Please ask any question you have. We will try our best to address your questions. Thanks for staying with us.

No related posts.

IPTABLES Rules

Limiting Spam and Attacks
Security – Training

You can use a bridge to effectively limit spam and attacks by managing the IP Ranges per Country.The basis behind the thought here is that these IP Address Ranges probably do not need access to your network in any way, unless you are an International business. By blocking these country ranges you may be reducing SPAM and Malware by up to 25%. In addition, in the event of a catastrophic virus outbreak you may create a window of time to secure your server by blocking these IP Ranges. The following websites keep track of network subnets that are related to each country.

Lesson 9 / Lesson 11

These websites provide the subnets for each country.

http://www.countryipblocks.net/country-blocks/cidr/

http://ip.ludost.net

Why limit IP subnets?
Some may say, “if you want a global business you need to allow access to your server from anywhere.” If you have ever run a mail server and see that 70-85% of all email is Spam you may reconsider that. If you have ever run a web server and see scripting attacks from locations you cannot pronounce let allow speak their language, you may reconsider. The fact is, there are a lot of attacks on your infrastructure and if you do not take steps to protect it you will lose it. Blocking country subnets may not stop those who use proxies and it will certainly not stop the guy down the street on your subnet….but it will make as difference and you will notice it within the hour.

Implementing these restrictions will require you to add statements to your iptables in order to specifically drop subnets. The good thing about doing this from a bridge firewall is that you will do this once for the whole network. From the command line you will need to add a line to indicate the subnet source that you want to drop on the INPUT table. Here is an example that drops the subnet at 201.0.0.0/8. Remember that the bridge is only using the FORWARD so this must be reflecting in your rules.

iptables -A FORWARD -s 201.0.0.0/8 -j DROP

As an alternative you may want to only limit access to countries via port 80. This line will drop all attempts from the subnet at 201.0.0.0/8 in reaching any port except port 80.
iptables -A FORWARD -s 201.0.0.0/8 -p tcp –dport ! 80 -j DROP

Add A Script

When you view the number of subnets to work with you will realize that writing rules will get to be a lot of work. What you can do is create a file called banned and place it in your /etc/ directory and then add this script to your firewall to access the “banned” file.

##########################################
# BLOCK COUNTRY ATTACKS
BADIP=/etc/banned
BANNED=$( grep -v -E “^#” $BADIP )
for ip in $BANNED
do
iptables -A INPUT -p tcp -s $ip -j DROP
iptables -A FORWARD -p tcp -s $ip -j DROP
done

The /etc/banned file will look like this:

24.190.78.101
58.0.0.0/8
59.32.0.0/13
59.40.0.0/15
59.42.0.0/16
59.43.0.0/16
59.44.0.0/14
59.48.0.0/16
59.49.0.0/17

Prevent synchronization packet flooding (Sync Flood)
# Iptables-A FORWARD-p tcp – syn-m limit – limit 1 / s-j ACCEPT
Also was writing
# The iptables-A INPUT-p tcp – syn-m limit – limit 1 / s-j ACCEPT
– Limit 1 / s limit syn complicated by the number of times per second can be modified according to their needs
Prevent all forms of port scans
# Iptables-A FORWARD-p tcp – tcp-flags SYN, ACK, FIN, RST RST-m limit – limit 1 / s -j ACCEPT
Ping flood attacks (Ping of Death)
# Iptables-A FORWARD-p icmp – icmp-type echo-request-m limit – limit 1 / s -j ACCEPT

Linux IPTables: Incoming and Outgoing Rule Examples (SSH and HTTP)

# 2. Set default chain policies
iptables -P INPUT DROP
iptables -P FORWARD DROP
iptables -P OUTPUT DROP

# 3. Allow incoming SSH
iptables -A INPUT -i eth0 -p tcp –dport 22 -m state –state NEW,ESTABLISHED -j ACCEPT
iptables -A OUTPUT -o eth0 -p tcp –sport 22 -m state –state ESTABLISHED -j ACCEPT

# 4. Allow incoming HTTP
iptables -A INPUT -i eth0 -p tcp –dport 80 -m state –state NEW,ESTABLISHED -j ACCEPT
iptables -A OUTPUT -o eth0 -p tcp –sport 80 -m state –state ESTABLISHED -j ACCEPT

# 5. Allow outgoing SSH
iptables -A OUTPUT -o eth0 -p tcp –dport 22 -m state –state NEW,ESTABLISHED -j ACCEPT
iptables -A INPUT -i eth0 -p tcp –sport 22 -m state –state ESTABLISHED -j ACCEPT

iptables -A INPUT -i eth0 -p tcp –dport 443 -m state –state NEW,ESTABLISHED -j ACCEPT
iptables -A OUTPUT -o eth0 -p tcp –sport 443 -m state –state ESTABLISHED -j ACCEPT

iptables -A INPUT -i eth0 -p tcp -m multiport –dports 22,80,443 -m state –state NEW,ESTABLISHED -j ACCEPT
iptables -A OUTPUT -o eth0 -p tcp -m multiport –sports 22,80,443 -m state –state ESTABLISHED -j ACCEPT

Load Balance Incoming Web Traffic iptables

You can also load balance your incoming web traffic using iptables firewall rules.
This uses the iptables nth extension. The following example load balances the HTTPS traffic to three different ip-address. For every 3th packet, it is load balanced to the appropriate server (using the counter 0).

iptables -A PREROUTING -i eth0 -p tcp –dport 443 -m state –state NEW -m nth –counter 0 –every 3 –packet 0 -j DNAT –to-destination 192.168.1.101:443
iptables -A PREROUTING -i eth0 -p tcp –dport 443 -m state –state NEW -m nth –counter 0 –every 3 –packet 1 -j DNAT –to-destination 192.168.1.102:443
iptables -A PREROUTING -i eth0 -p tcp –dport 443 -m state –state NEW -m nth –counter 0 –every 3 –packet 2 -j DNAT –to-destination 192.168.1.103:443

12. Allow Ping from Outside to Inside
The following rules allow outside users to be able to ping your servers.

iptables -A INPUT -p icmp –icmp-type echo-request -j ACCEPT
iptables -A OUTPUT -p icmp –icmp-type echo-reply -j ACCEPT

13. Allow Ping from Inside to Outside
The following rules allow you to ping from inside to any of the outside servers.

iptables -A OUTPUT -p icmp –icmp-type echo-request -j ACCEPT
iptables -A INPUT -p icmp –icmp-type echo-reply -j ACCEPT

14. Allow Loopback Access
You should allow full loopback access on your servers. i.e access using 127.0.0.1

iptables -A INPUT -i lo -j ACCEPT
iptables -A OUTPUT -o lo -j ACCEPT

16. Allow outbound DNS
The following rules allow outgoing DNS connections.

iptables -A OUTPUT -p udp -o eth0 –dport 53 -j ACCEPT
iptables -A INPUT -p udp -i eth0 –sport 53 -j ACCEPT

Allow Rsync From a Specific Network
The following rules allows rsync only from a specific network.

iptables -A INPUT -i eth0 -p tcp -s 192.168.101.0/24 –dport 873 -m state –state NEW,ESTABLISHED -j ACCEPT
iptables -A OUTPUT -o eth0 -p tcp –sport 873 -m state –state ESTABLISHED -j ACCEPT

19. Allow MySQL connection only from a specific network
If you are running MySQL, typically you don’t want to allow direct connection from outside. In most cases, you might have web server running on the same server where the MySQL database runs.
However DBA and developers might need to login directly to the MySQL from their laptop and desktop using MySQL client. In those case, you might want to allow your internal network to talk to the MySQL directly as shown below.

iptables -A INPUT -i eth0 -p tcp -s 192.168.100.0/24 –dport 3306 -m state –state NEW,ESTABLISHED -j ACCEPT
iptables -A OUTPUT -o eth0 -p tcp –sport 3306 -m state –state ESTABLISHED -j ACCEPT

Prevent DoS Attack

The following iptables rule will help you prevent the Denial of Service (DoS) attack on your webserver.

iptables -A INPUT -p tcp –dport 80 -m limit –limit 25/minute –limit-burst 100 -j ACCEPT

Force SYN packets check

Make sure NEW incoming tcp connections are SYN packets; otherwise we need to drop them:

iptables -A INPUT -p tcp ! –syn -m state –state NEW -j DROP

Force Fragments packets check

Packets with incoming fragments drop them. This attack result into Linux server panic such data loss.

iptables -A INPUT -f -j DROP


XMAS packets

Incoming malformed XMAS packets drop them:

iptables -A INPUT -p tcp –tcp-flags ALL ALL -j DROP

Drop all NULL packets

Incoming malformed NULL packets:

iptables -A INPIT -p tcp –tcp-flags ALL NONE -j DROP

Bind Configuration in Chroot Environment

Bind Configuration in Chroot Environment
Wriiten by Babar Zahoor

Dated:12-01-2010

Pupose: Configuration of DNS (Bind) server in chroot environment.

OS CentOS 5.4 X86_64

————————————-
Please Install the bind packages
————————————-

[root@ns1 ~]# yum install bind bind-utils bind-*
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* addons: virror.hanoilug.org
* extras: ftp.hostrino.com
* updates: ftp.hostrino.com
addons

| 951 B 00:00
extras

| 1.1 kB 00:00
ftp

| 2.1 kB 00:00
updates

| 1.9 kB 00:00
updates/primary_db

| 444 kB 00:00
Setting up Install Process
Package 30:bind-9.3.6-4.P1.el5_4.1.x86_64 already installed and latest version
Package 30:bind-utils-9.3.6-4.P1.el5_4.1.x86_64 already installed and latest version
Package 30:bind-sdb-9.3.6-4.P1.el5_4.1.x86_64 already installed and latest version
Package 30:bind-chroot-9.3.6-4.P1.el5_4.1.x86_64 already installed and latest version
Package 30:bind-devel-9.3.6-4.P1.el5_4.1.x86_64 already installed and latest version
Package 30:bind-devel-9.3.6-4.P1.el5_4.1.i386 already installed and latest version
Package 30:bind-libs-9.3.6-4.P1.el5_4.1.x86_64 already installed and latest version
Package 30:bind-libs-9.3.6-4.P1.el5_4.1.i386 already installed and latest version
Package 30:bind-9.3.6-4.P1.el5_4.1.x86_64 already installed and latest version
Package 30:bind-utils-9.3.6-4.P1.el5_4.1.x86_64 already installed and latest version
Package 30:bind-libbind-devel-9.3.6-4.P1.el5_4.1.x86_64 already installed and latest version
Package 30:bind-libbind-devel-9.3.6-4.P1.el5_4.1.i386 already installed and latest version
Nothing to do

———————————————-
Please Configure Static IP and Default Gateway
———————————————-

[root@ns1 ~]# vi /etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE=eth0
BOOTPROTO=static
IPADDR=192.168.1.100
NETMASK=255.255.255.0
ONBOOT=yes
HWADDR=00:16:36:73:7e:4f

wq!

[root@ns1 ~]# ifconfig
eth0 Link encap:Ethernet HWaddr 00:16:36:73:7E:4F
inet addr:192.168.1.100 Bcast:192.168.1.255 Mask:255.255.255.0
inet6 addr: fe80::216:36ff:fe73:7e4f/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:1641 errors:0 dropped:0 overruns:0 frame:0
TX packets:950 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:192907 (188.3 KiB) TX bytes:117111 (114.3 KiB)

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:105 errors:0 dropped:0 overruns:0 frame:0
TX packets:105 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:10213 (9.9 KiB) TX bytes:10213 (9.9 KiB)

[root@ns1 ~]#
[root@ns1 ~]# vi /etc/sysconfig/network
NETWORKING=yes
NETWORKING_IPV6=no
HOSTNAME=dns.companydns.org
GATEWAY=192.168.1.1

wq!

——————————————————————————————————————–
Now we are going to configure the named service please copy the files content and modify with your network settings
——————————————————————————————————————–

[root@ns1 ~]#
[root@ns1 ~]# cd /var/named/chroot/
[root@ns1 chroot]# ll
total 24
drwxr-x— 2 root named 4096 Dec 1 00:00 dev
drwxr-x— 2 root named 4096 Jan 4 04:42 etc
dr-xr-xr-x 85 root root 0 Jan 11 22:41 proc
drwxr-x— 6 root named 4096 Dec 1 00:00 var
[root@ns1 chroot]#

——————————-
Now create zone file named.conf
——————————-

[root@ns1 chroot]# vi etc/named.conf

options
{
directory “/var/named”; // the default
dump-file “data/cache_dump.db”;
statistics-file “data/named_stats.txt”;
memstatistics-file “data/named_mem_stats.txt”;

};

zone “.” IN {
type hint;
file “named.root”;
};

zone “localhost” IN {
type master;
file “localhost.fwd”;
allow-update { none; };
};

zone “0.0.127.in-addr.arpa” IN {
type master;
file “localhost.rev”;
allow-update { none; };
};

zone “companydns.org” IN {
type master;
file “companydns.org.fwd”;
allow-update { none; };
};

zone “1.168.192.in-addr.arpa” IN {
type master;
file “companydns.org.rev”;
allow-update { none; };
};

wq!

[root@ns1 chroot]# cd var/named

[root@ns1 named]#

————————–
Now create named.root file
————————–

[root@ns1 named]#

First We confiure named.root file for root dns

[root@ns1 named]# vi named.root
. 6D IN NS A.ROOT-SERVERS.NET.
. 6D IN NS B.ROOT-SERVERS.NET.
. 6D IN NS C.ROOT-SERVERS.NET.
. 6D IN NS D.ROOT-SERVERS.NET.
. 6D IN NS E.ROOT-SERVERS.NET.
. 6D IN NS F.ROOT-SERVERS.NET.
. 6D IN NS G.ROOT-SERVERS.NET.
. 6D IN NS H.ROOT-SERVERS.NET.
. 6D IN NS I.ROOT-SERVERS.NET.
. 6D IN NS J.ROOT-SERVERS.NET.
. 6D IN NS K.ROOT-SERVERS.NET.
. 6D IN NS L.ROOT-SERVERS.NET.
. 6D IN NS M.ROOT-SERVERS.NET.
A.ROOT-SERVERS.NET. 6D IN A 198.41.0.4
B.ROOT-SERVERS.NET. 6D IN A 192.228.79.201
C.ROOT-SERVERS.NET. 6D IN A 192.33.4.12
D.ROOT-SERVERS.NET. 6D IN A 128.8.10.90
E.ROOT-SERVERS.NET. 6D IN A 192.203.230.10
F.ROOT-SERVERS.NET. 6D IN A 192.5.5.241
G.ROOT-SERVERS.NET. 6D IN A 192.112.36.4
H.ROOT-SERVERS.NET. 6D IN A 128.63.2.53
I.ROOT-SERVERS.NET. 6D IN A 192.36.148.17
J.ROOT-SERVERS.NET. 6D IN A 192.58.128.30
K.ROOT-SERVERS.NET. 6D IN A 193.0.14.129
L.ROOT-SERVERS.NET. 6D IN A 199.7.83.42
M.ROOT-SERVERS.NET. 6D IN A 202.12.27.33

wq!

———————————————————————————————————————————-
Now create zone db files one by one localhost.fwd and the localhost.rev are must then your network zone files forward and reverse
———————————————————————————————————————————-

[root@ns1 named]# vi localhost.fwd
$ORIGIN localhost.
$TTL 86400
@ IN SOA ns1.companydns.org. hostmaster.companydns.org. (
20100104 ; Serial number
3H ; Refresh 1 day
15M ; Retry 2 hours
1W ; Expire 41.67 days
1D ) ; Minimum TTL 2 days

@ IN NS dns.companydns.org.

localhost. IN A 127.0.0.1

wq! ##### Save the file after copying the content from here. #####

[root@ns1 named]# vi localhost.rev
$ORIGIN 0.0.127.in-addr.arpa.
$TTL 86400
@ IN SOA ns1.companydns.org. hostmaster.companydns.org. (
20100104 ; Serial number
3H ; Refresh 1 day
15M ; Retry 2 hours
1W ; Expire 41.67 days
1D ) ; Minimum TTL 2 days

@ IN NS ns1.companydns.org.

1.0.0.127.in-addr.arpa. IN PTR localhost.

wq!

[root@ns1 named]# vi companydns.org.fwd
$ORIGIN companydns.org.
$TTL 86400
@ IN SOA ns1.companydns.org. hostmaster.companydns.org. (
20100104 ; Serial number
3H ; Refresh 1 day
15M ; Retry 2 hours
1W ; Expire 41.67 days
1D ) ; Minimum TTL 2 days

@ IN NS ns1.companydns.org.

ns1.companydns.org. IN A 192.168.1.100
ftp.companydns.org. IN A 192.168.1.101
www.companydns.org. IN A 192.168.1.102
client3.companydns.org. IN A 192.168.1.103
client4.companydns.org. IN A 192.168.1.104

wq!

[root@ns1 named]# vi companydns.org.rev
$ORIGIN 1.168.192.in-addr.arpa.
$TTL 86400
@ IN SOA ns1.companydns.org. root.companydns.org. (
20100104 ; Serial number
3H ; Refresh 1 day
15M ; Retry 2 hours
1W ; Expire 41.67 days
1D ) ; Minimum TTL 2 days

@ IN NS ns1.companydns.org.

100.1.168.192.in-addr.arpa. IN PTR ns1.companydns.org.
101.1.168.192.in-addr.arpa. IN PTR ftp.companydns.org.
102.1.168.192.in-addr.arpa. IN PTR www.companydns.org.
103.1.168.192.in-addr.arpa. IN PTR client1.companydns.org.
104.1.168.192.in-addr.arpa. IN PTR clinet2.companydns.org.

wq!

[root@ns1 ~]# vi /etc/resolv.conf
search companydns.org
nameserver 192.168.1.100

wq!

—————————————————————–
Configuration has been done now start “/etc/init.d/named” service
—————————————————————–

[root@ns1 ~]# /etc/init.d/named start
Starting named: [ OK ]
[root@ns1 ~]# dig yahoo.com

; <<>> DiG 9.3.6-P1-RedHat-9.3.6-4.P1.el5_4.1 <<>> yahoo.com
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 46559 ;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 7, ADDITIONAL: 2 ;; QUESTION SECTION: ;yahoo.com. IN A ;; ANSWER SECTION: yahoo.com. 21600 IN A 209.191.93.53 yahoo.com. 21600 IN A 69.147.114.224 yahoo.com. 21600 IN A 209.131.36.159 ;; AUTHORITY SECTION: yahoo.com. 172800 IN NS ns1.yahoo.com. yahoo.com. 172800 IN NS ns2.yahoo.com. yahoo.com. 172800 IN NS ns3.yahoo.com. yahoo.com. 172800 IN NS ns4.yahoo.com. yahoo.com. 172800 IN NS ns5.yahoo.com. yahoo.com. 172800 IN NS ns6.yahoo.com. yahoo.com. 172800 IN NS ns8.yahoo.com. ;; ADDITIONAL SECTION: ns6.yahoo.com. 172800 IN A 202.43.223.170 ns8.yahoo.com. 172800 IN A 202.165.104.22 ;; Query time: 643 msec ;; SERVER: 192.168.1.100#53(192.168.1.100) ;; WHEN: Tue Jan 12 03:01:01 2010 ;; MSG SIZE rcvd: 233 [root@ns1 ~]# -------------------------------------------------- Now please open ports for named server for network -------------------------------------------------- [root@ns1 ~]# iptables -A INPUT -p tcp -m multiport --dport 53,953 -j ACCEPT [root@ns1 ~]# iptables -A INPUT -p udp -m multiport --dport 53,953 -j ACCEPT [root@ns1 ~]# [root@ns1 ~]# /etc/init.d/iptables save Saving firewall rules to /etc/sysconfig/iptables: [ OK ] [root@ns1 ~]# [root@ns1 ~]# dig ns1.companydns.org ; <<>> DiG 9.3.6-P1-RedHat-9.3.6-4.P1.el5_4.1 <<>> ns1.companydns.org
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 29732 ;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 1, ADDITIONAL: 0 ;; QUESTION SECTION: ;ns1.companydns.org. IN A ;; ANSWER SECTION: ns1.companydns.org. 86400 IN A 192.168.1.100 ;; AUTHORITY SECTION: companydns.org. 86400 IN NS ns1.companydns.org. ;; Query time: 1 msec ;; SERVER: 192.168.1.100#53(192.168.1.100) ;; WHEN: Tue Jan 12 03:13:33 2010 ;; MSG SIZE rcvd: 66 [root@ns1 ~]# [root@ns1 ~]# dig www.companydns.org ; <<>> DiG 9.3.6-P1-RedHat-9.3.6-4.P1.el5_4.1 <<>> www.companydns.org
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 10800 ;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 1, ADDITIONAL: 1 ;; QUESTION SECTION: ;www.companydns.org. IN A ;; ANSWER SECTION: www.companydns.org. 86400 IN A 192.168.1.102 ;; AUTHORITY SECTION: companydns.org. 86400 IN NS ns1.companydns.org. ;; ADDITIONAL SECTION: ns1.companydns.org. 86400 IN A 192.168.1.100 ;; Query time: 1 msec ;; SERVER: 192.168.1.100#53(192.168.1.100) ;; WHEN: Tue Jan 12 03:14:09 2010 ;; MSG SIZE rcvd: 86 [root@ns1 ~]#

VSFTP

CentOS 6

vsftpd 2.2.2

su – root

yum install vsftpd

cd /etc/vsftpd/

vi config

anonymous_enable=NO This is set to YES by default.

local_enable=YES This is set to NO by default and change when you want the local users to have ftp access.

xferlog_enable=Yes This is set to NO by default. Your logs will be written to /var/log/xferlog.

Most Linux’s have SELinux installed by default and this gives an error when the installer does not take care of the Selinux policy’s. The error is as follows:

500 OOPS: cannot change directory:/home/someuser

vi /etc/selinux/config

SELINUX=disabled

Setting SELinux for ftp access:

getsebool -a | grep ftp

setsebool -P ftp_home_dir on

chkconfig –levels 345 vsftpd on

service vsftpd start

The virtual users home folders will be under /var/ftp/. You need to have either ‘su’ permissions or ‘root’ access or ‘sudo’ access.

As authentication will be required pam_userdb is a good option and is installed by default. Check with:

yum info db4-utils

yum install db4-utils as necessary

Now cd to /etc/vsftpd and prepare the .txt user file with the usernames and passwords.
This file will have a username in single line and the password in the next as shown. It is good practice to put these in a separate folder.

cd /etc/vsftpd
mkdir vuser
cd vuser
vim vuser_list

sudhakar
password1
bellamkonda
password2

db_load -T -t hash /etc/vsftpd/vuser/vuser_list /etc/vsftpd/vuser/vuser_db.db

vi /etc/pam.d/vsftpd

cd /etc/pam.d/
vi vsftpd

auth sufficient pam_userdb.so db=/etc/vsftpd/vuser/vuser_db
account sufficient pam_userdb.so db=/etc/vsftpd/vuser/vuser_db

vi /etc/vsftpd/vsftpd.conf

guest_enable=YES # activate the virtual users
virtual_use_local_privs=YES # virtual users have local priveleges
user_sub_token=$USER
local_root=/var/ftp/vuser/$USER # specifies a home directory for each virtual user
chroot_local_user=YES # Restricting the user to the FTP area and HOME dir’s only

Create the Virtual User Folders

cd /var/ftp
mkdir vuser
mkdir vuser/sudhakar
mkdir vuser/bellamkonda
chown -R ftp:ftp /etc/ftp/vuser/

/var/ftp/vuser/

mkdir yourlocaluser
chown ftp:ftp yourlocaluser

ln -s /var/ftp/vuser/yourlocaluser /home/yourlocaluser/ftphome

service vsftpd start
service vsftpd restart

cd /etc/vsftpd
mkdir vuser

vuserchk – checks the necessary files and folders necessary for these scripts
vuser.conf – the file containing configuration parameters for these scripts
vuseradd – adds a virtual user
vuserdel – delets a virtual user
vuserres – restores a deleted user
vuserpas – changes a virtual user password
vusersho – displays the user password

vsftpd SSL

yum install vsftpd

openssl req -x509 -nodes -days 365 -newkey rsa:1024 \
-keyout /etc/vsftpd/vsftpd.pem \
-out /etc/vsftpd/vsftpd.pem

Configure vsftpd

To configure vsftpd you edit the file /etc/vsftpd/vsftpd.conf and add the following lines:

ssl_enable=YES
allow_anon_ssl=NO
force_local_data_ssl=NO
force_local_logins_ssl=NO
ssl_tlsv1=YES
ssl_sslv2=NO
ssl_sslv3=NO
rsa_cert_file=/etc/vsftpd/vsftpd.pem

/etc/rc.d/init.d/vsftpd restart

FTP Security – Chroot / Jail user (limiting user to own their home directory only)

Step1: Editing /etc/vsftpd/vsftpd.conf.

Option A: chroot all local user

By default, if you are adding in chroot_local_user=YES .All the local users are’ chroot()’ /jailed to their /home/user direcory. Go to last line adding in the line
vim /etc/vsftpd/vsftpd.conf

chroot_local_user=YES

Option B: chroot only selected users

If you want only selected ftp user restricted to their home directory, uncomment/delete the # sign at line 94 and 96. If chroot_local_user=YES was previously added , make sure that chroot_local_user=YES is removed from your vsftpd.conf file.
vim /etc/vsftpd/vsftpd.conf

91 # You may specify an explicit list of local users to chroot() to their home
92 # directory. If chroot_local_user is YES, then this list becomes a list of
93 # users to NOT chroot().
94 chroot_list_enable=YES
95 # (default follows)
96 chroot_list_file=/etc/vsftpd/chroot_list

CentOS Linux FTP Server

FTP Security – Chroot / Jail user (limiting user to own their home directory only)

Local account ftp user has the rights to change to any directory outside from their /home/user by default. Therefore, they can browse any files in any directory in FTP servers. Let’s have a close look at the example below. The user james is browsing the /etc/sysconfig/networking directory and he knows that there are two directories which is devices and profiles. If james has rights on the file outside his /home directory(such as group rights), he can just download these files.
>C:\>ftp 192.168.13.145
Connected to 192.168.13.145.
220 (vsFTPd 2.0.5)
User (192.168.13.145:(none)): james
331 Please specify the password.
Password:
230 Login successful.
ftp> pwd
257 “/home/james”
ftp> cd /etc/sysconfig/networking
250 Directory successfully changed.
ftp> pwd
257 “/etc/sysconfig/networking”
ftp> ls
200 PORT command successful. Consider using PASV.
150 Here comes the directory listing.
devices
profiles
226 Directory send OK.
ftp: 19 bytes received in 0.00Seconds 19.00Kbytes/sec.
ftp> bin
200 Switching to Binary mode.
ftp> cd devices
250 Directory successfully changed.
ftp> ls
200 PORT command successful. Consider using PASV.
150 Here comes the directory listing.
ifcfg-eth0
ifcfg-eth0.bak
ifcfg-eth1
ifcfg-eth1.bak
226 Directory send OK.
ftp: 56 bytes received in 0.00Seconds 28.00Kbytes/sec.
ftp> get ifcfg-eth0
200 PORT command successful. Consider using PASV.
150 Opening BINARY mode data connection for ifcfg-eth0 (117 bytes).
226 File send OK.
ftp: 117 bytes received in 0.00Seconds 117.00Kbytes/sec.

Thus, its always recommended to jail/ restrict FTP user access only to their /home/user direcotory.

Step1: Editing /etc/vsftpd/vsftpd.conf.

Option A: chroot all local user

By default, if you are adding in chroot_local_user=YES .All the local users are’ chroot()’ /jailed to their /home/user direcory. Go to last line adding in the line
vim /etc/vsftpd/vsftpd.conf

chroot_local_user=YES

Option B: chroot only selected users

If you want only selected ftp user restricted to their home directory, uncomment/delete the # sign at line 94 and 96. If chroot_local_user=YES was previously added , make sure that chroot_local_user=YES is removed from your vsftpd.conf file.
vim /etc/vsftpd/vsftpd.conf

91 # You may specify an explicit list of local users to chroot() to their home
92 # directory. If chroot_local_user is YES, then this list becomes a list of
93 # users to NOT chroot().
94 chroot_list_enable=YES
95 # (default follows)
96 chroot_list_file=/etc/vsftpd/chroot_list

Step2 (if selected option B above): create a file named chroot_list under /etc/vsftpd/

The following example, we are creating chroot_list and insert the user james in the list
cd /etc/vsftpd/

vim chroot_list

james

Step3: Restart vsFTPD services
service vsftpd restart
Shutting down vsftpd: [ OK ]
Starting vsftpd for vsftpd: [ OK ]

10 Apache Security and Hardening Tips

10 Apache Security and Hardening Tips

Tip No. 1: Disable Apache Signature and/or Apache Banner

# ServerSignature Off
# ServerTokens ProductOnl

Tip No. 2: The Trace HTTP Request

Add the following to the web-server’s configuration file. For example alter the following file in Ubuntu: /etc/apache2/apache2.conf .

* TraceEnable off

Tip 3: Remove PHP scripts that print debug info using phpinfo()

The built-in PHP function phpinfo() prints a lot of interesting internal information about the PHP environment.
It can include list of which PHP modules are enabled, and the location of various files on the web-server and other sensitive information.
Our web security scanner finds a lot of such files. It is recommended to remove these test files from a production website.

Here is a tip hpw to find such files. Look for the files with the following name: test.php, info.php, i.php and phpinfo.php in your website directory and remove them.

Tip 4: Disable directory indexing

Directory indexing is a features found in every web-server by default. When directory indexing is enabled, the web-site prints a list of files found in the website directories
when the default page does not exists (for example index.php). Directories reported can be viewed by any visitor.
It is vulnerable in the sense that these directories can contain configuration, private and backup files which can be used by the attackers
to take your server under control.

You can fix this problem by disabling the Apache autoindex module.
In some Apache installations it is called mod_autoindex.so. In Ubuntu, you just need to remove the following files:

* /etc/apache2/mods-enabled/autoindex.load
* /etc/apache2/mods-enabled/autoindex.conf

So you can do it running the following commands:

* rm -f /etc/apache2/mods-enabled/autoindex.load
* rm -f /etc/apache2/mods-enabled/autoindex.conf

Tip 5: Disable WebDAV

ake sure that WebDAV is disabled in production websites. When WebDAV is enabled, the following commands are supported by Apache: OPTIONS, PROPFIND, etc.
These commands are sensitive from computer security point of view.

* /etc/apache2/mods-enabled/dav.load
* /etc/apache2/mods-enabled/dav_fs.conf
* /etc/apache2/mods-enabled/dav_fs.load
* /etc/apache2/mods-enabled/dav_lock.load

Tip 6: Create a chroot’ed Apache environment

Tip 7: Enable PHP basedir

Tip 8: Web Stats

Tip 9: Use Google

Most of the webmasters use common web scripts and CMS or blog software. We recommend you to frequently search for security updates using Google and register for security news at your blog/CMS website.

Tip 10: Additional Steps

If your webserver runs together with MySQL server it brings additional potential security problem. MySQL can read any files located on you server including the one located in different chrooted environments. It happens because of the FILE permission. By default only MySQL root has it.
For more info about MySQL security take a look at this article ( link to GreenSQL) .

Fedora

Building High Performance webserver On Centos in Dell Servers

Performance tuning a CentOS LAMP web server for high traffic volumes

This document is prepared and Posted on August 17, 2010 by William Jamieson – Thank you very much William 🙂

Performance tune a LAMP server to handle approximately 70 full page loads per second which equated to 4,250 concurrent virtual users. We ended up doubling this expectation to 140 full page loads per second without striking issue. If this speed was maintained for 24 hours it would equate to over 12 million hits per day. This article will let you know how we achieved it.

The load tests were conducted using the HP performance center; a technology that HP obtained as part of its acquisition of Mercury for approximately USD$4.5 billion in 2006.

To find out more about the load testing software visit http://en.wikipedia.org/wiki/HP_LoadRunner

Goal:
Handle 4,250 concurrent users generating approximately 70 full page loads per second.

1 full page load consisted of:
– 1 dynamically generated PHP file using MySQL
– 4 JavaScript files
– 7 CSS files
– 8 image files

Original starting environment:
– ServerModel: Dell R300
– RAM: 2GB (2 x 1GB chips)
– Operating System: CentOS release 5.5 (Final)
– Apache: v2.2.3 (running in prefork mode)
– MySQL: v5.0.77
– PHP: v5.1.6 (as an apache module)
– eAccelerator: v0.9.5.3
– 120Mbits of bandwidth

Round 1: Initial Test
Round 1: Configuration

At the start of the process we were pretty much using the default configurations for the entire lamp stack. Linux was running iptables and ip6tables in its default configuration. eAccelerator was operating with 32MB of memory with optimization and caching enabled.

Apache (/etc/httpd/conf/httpd.conf):
For more info on variables for Apache 2.0.x go to: http://httpd.apache.org/docs/2.0/mod/mpm_common.html

StartServers 8
MinSpareServers 5
MaxSpareServers 20
ServerLimit 256
MaxClients 256
MaxRequestsPerChild 4000

MySQL (/etc/my.cnf):
For more info on variables for MySQL 5.0.x go to: http://dev.mysql.com/doc/refman/5.0/en/server-system-variables.html
[mysqld]
max_connections = 100
max_user_connections = 0
max_connect_errors = 10
max_allowed_packet = 1M
table_cache = 64
sort_buffer_size = 2M
read_buffer_size = 131072
read_rnd_buffer_size = 262144
myisam_sort_buffer_size = 8M
thread_cache_size = 0
query_cache_size= 0
thread_concurrency = 10
Round 1: Results

With these settings we got up to 30 page loads per second which was 42% of our target. Interestingly, we were only operating at about 8% CPU and about 50% of our memory capacity when we hit this limit.
Round 1: Review

Looking at the apache error logs we were getting a large number of MySQL errors:
mysql_connect() [function.mysql-connect]: Too many connections in xxx.php on line 15

So the MySQL configuration seemed to be our bottleneck:

Round 2
Round 2: Configuration

We did our first major review of the Apache and MySQL performance settings and adjusted them accordingly. We doubled the Apache settings and used the ‘huge’ configuration as supplied with mysql (/usr/share/doc/mysql-server-5.0.77/my-huge.cnf).

Apache (/etc/httpd/conf/httpd.conf):
For more info on variables for Apache 2.0.x go to: http://httpd.apache.org/docs/2.0/mod/mpm_common.html

StartServers 16
MinSpareServers 10
MaxSpareServers 40
ServerLimit 512
MaxClients 512
MaxRequestsPerChild 8000

MySQL (/etc/my.cnf):
For more info on variables for MySQL 5.0.x go to: http://dev.mysql.com/doc/refman/5.0/en/server-system-variables.html
[mysqld]
# Memory usage
skip-locking
max_connections = 500
max_user_connections = 500
max_connect_errors = 999999
key_buffer = 384M
max_allowed_packet = 1M
table_cache = 512
sort_buffer_size = 2M
read_buffer_size = 2M
read_rnd_buffer_size = 8M
myisam_sort_buffer_size = 64M
thread_cache_size = 8
query_cache_size = 32M
# Try number of CPU’s*2 for thread_concurrency (eHound has 4 CPU’s)
thread_concurrency = 8

# Disable Federated by default
skip-federated

[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid

[mysqldump]
quick
max_allowed_packet = 16M

[mysql]
no-auto-rehash

[isamchk]
key_buffer = 256M
sort_buffer_size = 256M
read_buffer = 2M
write_buffer = 2M

[myisamchk]
key_buffer = 256M
sort_buffer_size = 256M
read_buffer = 2M
write_buffer = 2M

[mysqlhotcopy]
interactive-timeout

As an extra precaution we locked the network card in the server to use 1Gbit:
#ethtool -s eth0 speed 1000 duplex full

Edit the configuration for the network card:
#vim /etc/sysconfig/network-scripts/ifcfg-eth0

Add the following line:
ETHTOOL_OPTS=’autoneg on speed 1000 duplex full’

Restart the network:
#service network restart
Round 2: Results

With these settings we got up to 58 full page loads per second which was 59% of our target. Interestingly, we were still only operating at about 10% CPU capacity when we hit this limit but we were using approximately 70-80% of our memory.

Our MySQL errors had disappeared and there were no more errors in the Apache logs.
Round 2: Review

We were concerned that the system was starting to use swap memory which was slowing the server to a halt.

Round 3
Round 3: Configuration

We added an additional 2GB of RAM to the server so it now contained 4 x 1GB chips.
Round 3: Results

With the new RAM we still only got up to 58 full page loads per second which was 59% of our target. We were still only operating at about 10% CPU capacity but now we were only using about 40% of our memory.
Round 3: Review

Still no errors in the Apache logs and the load test farm was not receiving Apache errors. In fact it was reporting that it could not even connect to the server. This led us to believe that it was either a lack of bandwidth or a NIC/network/firewall configuration issue. After checking with our datacenter, we found that there were no inhibiting factors that would cause the problem described.

We increased the Apache & MySQL Limits and ran a different style of test.

Round 4
Round 4: Configuration

In this test we only loaded the dynamic components of the page as generated by PHP and MySQL and served by Apache. This meant that we told the load test farm not to download static content such as images, CSS or JavaScript files.

Also we increased the MySQL and Apache limits as follows:

Apache (/etc/httpd/conf/httpd.conf):
For more info on variables for Apache 2.0.x go to: http://httpd.apache.org/docs/2.0/mod/mpm_common.html

StartServers 280
MinSpareServers 100
MaxSpareServers 300
ServerLimit 1536
MaxClients 1536
MaxRequestsPerChild 32000

MySQL (/etc/my.cnf):
For more info on variables for MySQL 5.0.x go to: http://dev.mysql.com/doc/refman/5.0/en/server-system-variables.html
[mysqld]
# Memory usage
skip-locking
max_connections = 764
max_user_connections = 764
max_connect_errors = 999999
key_buffer = 256M
max_allowed_packet = 1M
table_cache = 256
sort_buffer_size = 1M
read_buffer_size = 1M
read_rnd_buffer_size = 4M
myisam_sort_buffer_size = 64M
thread_cache_size = 8
query_cache_size= 16M
# Try number of CPU’s*2 for thread_concurrency (eHound has 4 CPU’s)
thread_concurrency = 8

# Disable Federated by default
skip-federated

[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid

[mysqldump]
quick
max_allowed_packet = 16M

[mysql]
no-auto-rehash

[isamchk]
key_buffer = 128M
sort_buffer_size = 128M
read_buffer = 2M
write_buffer = 2M

[myisamchk]
key_buffer = 128M
sort_buffer_size = 128M
read_buffer = 2M
write_buffer = 2M

[mysqlhotcopy]
interactive-timeout
Round 4: Results

The results of this test were very interesting. We got up to 263 page loads without any issue. This consumed a lot more bandwidth than test 3 so we knew that bandwidth was not the issue. However the number of connections that both tests started to fail at were very similar.
Round 4: Review

So we knew we had a connection limit issue.

We also knew that the eAccelerator optcode cache was not dying at these high volumes, nor was MySQL, PHP or Apache.

We reviewing the kernel messages and found thousands of the following messages that were logged at the time of testing:
#cat /var/log/messages* | grep ‘Aug 15’

Aug 15 01:04:27 localhost kernel: printk: 1395 messages suppressed.
Aug 15 01:04:27 localhost kernel: ip_conntrack: table full, dropping packet.
Aug 15 01:04:32 localhost kernel: printk: 1561 messages suppressed.
Aug 15 01:04:32 localhost kernel: ip_conntrack: table full, dropping packet.
Aug 15 01:04:37 localhost kernel: printk: 1274 messages suppressed.
Aug 15 01:04:37 localhost kernel: ip_conntrack: table full, dropping packet.
Aug 15 01:04:42 localhost kernel: printk: 1412 messages suppressed.

Further investigation revealed that the iptables/ip6tables was activated and limiting the number of connections to the box because its table was full. Ordinarily when I set up a linux server I turn iptables off because I place hardware firewalls in front of the servers. However I didn’t have the opportunity to setup this box initially, so they were still activated. I however didn’t need them, so I deactivated them.

If you still need to keep iptables running you can simply adjust the following settings:
Check the current connections limit (only works if iptables is running):
#sysctl net.ipv4.netfilter.ip_conntrack_max
65536

Change the connections limit:
#vim /etc/sysctl.conf

Add the following lines:
# conntrack limits
#inet.ipv4.netfilter.ip_conntrack_max = 65536
net.ipv4.netfilter.ip_conntrack_max = 196608

Reload the config file:
#sysctl -p

Check the new connections limit:
#sysctl net.ipv4.netfilter.ip_conntrack_max
196608

Check the current buckets limit (only works if iptables is running):
#cat /proc/sys/net/ipv4/netfilter/ip_conntrack_buckets
8192

To change the buckets limit:
#vim /etc/modprobe.conf

Add the following lines:
options ip_conntrack hashsize=32768

Reboot the server:
#shutdown -r now

Check the new buckets limit:
#cat /proc/sys/net/ipv4/netfilter/ip_conntrack_buckets
24576


Alternatively if you don’t need iptables like me, you can just disable them:
#service iptables stop
#service ip6tables stop
#chkconfig iptables off
#chkconfig ip6tables off

Round 5
Round 5: Configuration

This test used exactly the same configuration with iptables disabled.
Round 5: Results

Success!!! We got to 4,250 concurrent users which is about 70 pages per second (loading all additional image, CSS and JavaScript files also) with zero errors and a 0.7 second average response time. This used about 120Mbits worth of bandwidth pipe. The datacenter ended up running out of pipe before the server had any issues.

At this rate we were running at about:
– 15% CPU utilisation
– 30% Memory usage (with 4GB RAM installed)
– 400 apache threads
– 100% Bandwidth
Round 5: Review

Key findings:
– Increase your Apache and MySQL limits
– Turn off iptables
– Ensure that you have enough RAM
– Ensure that you are checking logs from MySQL, Apache, and the kernel to pick up any errors and give you clues as to how to best solve them

Round 6
Round 6: Configuration

This test used exactly the same configuration as round 5 with 250Mbit pipe instead of a 120Mbit pipe.
Round 6: Results

Success!!! We got to 140 full page loads per second (including additional images, CSS and JavaScript files also) with zero errors and still a stable 0.7 second average response time. This used the full 250Mbits worth of bandwidth pipe. The datacenter ended up running out of pipe again before the server had any issues.

At this rate we were running at about:
– 30% CPU utilisation
– 40% Memory usage (with 4GB RAM installed)
– 800 apache threads
– 100% Bandwidth
Round 6: Review

Key findings:
– Even with 250Mbits of pipe, bandwidth is still the bottleneck in this configuration.

Round 7
Round 7: Configuration

Even though our server was performing fine, we were given another server to experiment on with much higher specs.

It was a Dell R710 with 48GB of RAM and 8 2.53MHz Xeon processors running in hyper-threading mode (essentially making 16 processors).

We also had this box connected to a dedicated 4Gbit optical internet feed to give it as much bandwidth as it needed.

Everything on the box was configured the same except for Apache and MySQL (which we took the last settings and multipled them by 4) and sysctl.

Apache (/etc/httpd/conf/httpd.conf):
For more info on variables for Apache 2.0.x go to: http://httpd.apache.org/docs/2.0/mod/mpm_common.html

StartServers 1120
MinSpareServers 400
MaxSpareServers 1200
ServerLimit 6144
MaxClients 6144
MaxRequestsPerChild 128000

MySQL (/etc/my.cnf):
For more info on variables for MySQL 5.0.x go to: http://dev.mysql.com/doc/refman/5.0/en/server-system-variables.html
[mysqld]
# Memory usage
skip-locking
max_connections = 3056
max_user_connections = 3056
max_connect_errors = 999999
key_buffer = 1024M
max_allowed_packet = 4M
table_cache = 1024
sort_buffer_size = 4M
read_buffer_size = 4M
read_rnd_buffer_size = 16M
myisam_sort_buffer_size = 256M
thread_cache_size = 32
query_cache_size= 64M
# Try number of CPU’s*2 for thread_concurrency (eHound has 4 CPU’s)
thread_concurrency = 32

# Disable Federated by default
skip-federated

[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid

[mysqldump]
quick
max_allowed_packet = 64M

[mysql]
no-auto-rehash

[isamchk]
key_buffer = 512M
sort_buffer_size = 512M
read_buffer = 8M
write_buffer = 8M

[myisamchk]
key_buffer = 512M
sort_buffer_size = 512M
read_buffer = 8M
write_buffer = 8M

[mysqlhotcopy]
interactive-timeout

We also added the following lines to sysctl:
ip_conntrack_max = 196608
net.ipv4.ip_local_port_range = 1025 65535
net.ipv4.tcp_max_tw_buckets = 1000000
net.core.somaxconn = 10000
net.ipv4.tcp_max_syn_backlog = 2000
net.ipv4.tcp_fin_timeout = 30
Round 7: Results

We got to 200 full page loads per second (including additional images, CSS and JavaScript files also) with zero errors and still a stable 0.8 second average response time. This test used 330Mbits or about 8% worth of the bandwidth available. We stopped the test simply because we didn’t need to go any higher, but potentially could have gone much higher.

At this rate we were running at about:
– 16% CPU utilisation
– 6% Memory usage (with 48GB RAM installed)
– 1227 apache threads
– 8% Bandwidth
Round 7: Review

Key findings:
– Bandwidth seem to be a much bigger bottleneck than server capability.

swap issues on Linux and clear the swap usage

swap issues on Linux

clear the swap usage

free -to (Total memory usage)

free -m (Memory usage of swap)

swapoff -a && swapon -a ( swap off and on)

free

cat /proc/swaps
sync; echo 3 > /proc/sys/vm/drop_caches

To free pagecache:
# echo 1 > /proc/sys/vm/drop_caches

To free dentries and inodes:
# echo 2 > /proc/sys/vm/drop_caches

To free pagecache, dentries and inodes:
echo 3 > /proc/sys/vm/drop_caches

Note: Works well on Production Servers

PCI Compliance Disable ETags Apache

PCI Compliance Disable ETags

PCI Compliance
Disable ETags

To alleviate security risks arising from disclosure of information about files and their properties by Apache Web server, disable FileETag directive. For PCI Compliance it is required to disable ETags

Create a file at /etc/httpd/conf.d/no-etags.conf with the following:

Header unset ETag
FileETag None

Then of course restart Apache.

http://httpd.apache.org/docs/2.2/mod/core.html#FileETag