August 2025
M T W T F S S
 123
45678910
11121314151617
18192021222324
25262728293031

Categories

August 2025
M T W T F S S
 123
45678910
11121314151617
18192021222324
25262728293031

Apache Error: “semget: No space left on device”

If Apache fails, and will not successfully start again, check the error log. If you see an error similar to the following, it could indicate that your server has run out of semaphores.

semget: No space left on device

To see how many semaphores are being used, SSH to your server as root and run the following:

ipcs -s

In order to get Apache started again, we must clear the semaphores. Run this for-loop to flush them:

for whatever in `ipcs -s | awk ‘{print $2}’`; do ipcrm -s $whatever; done

On older servers that command may not work. In these cases, you may need to do the following:

/sbin/service httpd stop
ipcs -s | grep nobody | gawk ‘{ print $2 }’ | xargs -n 1 ipcrm sem
/sbin/service httpd start

If this is a common problem for you, you may want to increase the semaphore limits on your server. You can do that by adding the following to the /etc/sysctl.conf file:

# Increases the semaphore limits & extend Apache’s uptime.
kernel.msgmni = 512
kernel.sem = 250 128000 32 512

Then load the new settings into the kernel:

sysctl -p

Note: This post assumes you are running Apache on a Linux server, are familiar with the command line, and have root access to the server.

How to install LAMP (Linux Apache, MariaDB & PHP) on CentOS 7

yum install mariadb mariadb-server mysql

INSTALL MARIA DB

MariaDB is the default database server in CentOS 7, so go ahead and install it with yum using the following command:

## yum install mariadb mariadb-server mysql

Once installed, add bind-address = 127.0.0.1 to /etc/my.cnf.d/server.cnf to bind MariaDB to localhost only:

## vim /etc/my.cnf.d/server.cnf

[mysqld]
#log-bin=mysql-bin
#binlog_format=mixed
bind-address = 127.0.0.1

[root@testserver7 my.cnf.d]# systemctl restart mariadb
[root@testserver7 my.cnf.d]# systemctl status mariadb
mariadb.service – MariaDB database server
Loaded: loaded (/usr/lib/systemd/system/mariadb.service; enabled)
Active: active (running) since Fri 2015-02-06 09:17:59 SGT; 2s ago
Process: 9179 ExecStartPost=/usr/libexec/mariadb-wait-ready $MAINPID (code=exited, status=0/SUCCESS)
Process: 9150 ExecStartPre=/usr/libexec/mariadb-prepare-db-dir %n (code=exited, status=0/SUCCESS)
Main PID: 9178 (mysqld_safe)
CGroup: /system.slice/mariadb.service
??9178 /bin/sh /usr/bin/mysqld_safe –basedir=/usr
??9348 /usr/libexec/mysqld –basedir=/usr –datadir=/var/lib/mysql –plugin-dir=/usr/lib64/mysql/plugin –log-erro…

Feb 06 09:17:57 testserver7 systemd[1]: Starting MariaDB database server…
Feb 06 09:17:57 testserver7 mysqld_safe[9178]: 150206 09:17:57 mysqld_safe Logging to ‘/var/log/mariadb/mariadb.log’.
Feb 06 09:17:57 testserver7 mysqld_safe[9178]: 150206 09:17:57 mysqld_safe Starting mysqld daemon with databases from /…/mysql
Feb 06 09:17:59 testserver7 systemd[1]: Started MariaDB database server.
Hint: Some lines were ellipsized, use -l to show in full.
Hint: Some lines were ellipsized, use -l to show in full.
[root@testserver7 my.cnf.d]# systemctl enable mariadb
ln -s ‘/usr/lib/systemd/system/mariadb.service’ ‘/etc/systemd/system/multi-user.target.wants/mariadb.service’
[root@testserver7 my.cnf.d]#
[
/usr/bin/mysqladmin -u root password ‘test123’

/usr/bin/mysqladmin -u root -h testserver7 password ‘test123′

update mysql.user set password=PASSWORD(“test123”) where User=’root’;

Reset root password
[root@testserver7 my.cnf.d]# mysql -u root
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MariaDB connection id is 1
Server version: 5.5.41-MariaDB MariaDB Server

Copyright (c) 2000, 2014, Oracle, MariaDB Corporation Ab and others.

Type ‘help;’ or ‘\h’ for help. Type ‘\c’ to clear the current input statement.

MariaDB [(none)]> update mysql.user set password=PASSWORD(“test123”) where User=’root’;
Query OK, 3 rows affected (0.00 sec)
Rows matched: 3 Changed: 3 Warnings: 0

MariaDB [(none)]> flush privileges;
Query OK, 0 rows affected (0.00 sec)

MariaDB [(none)]> quit
Bye
[root@testserver7 my.cnf.d]#

## mysql_secure_installation

Enter current password for root (enter for none): ENTER
Set root password? [Y/n] Y
Remove anonymous users? [Y/n] Y
Disallow root login remotely? [Y/n] Y
Remove test database and access to it? [Y/n] Y
Reload privilege tables now? [Y/n] Y

root@testserver7 my.cnf.d]# systemctl restart httpd
[root@testserver7 my.cnf.d]# systemctl status httpd
httpd.service – The Apache HTTP Server
Loaded: loaded (/usr/lib/systemd/system/httpd.service; enabled)
Active: active (running) since Fri 2015-02-06 09:47:33 SGT; 17s ago
Process: 10372 ExecStop=/bin/kill -WINCH ${MAINPID} (code=exited, status=0/SUCCESS)
Main PID: 10377 (httpd)
Status: “Total requests: 0; Current requests/sec: 0; Current traffic: 0 B/sec”
CGroup: /system.slice/httpd.service
??10377 /usr/sbin/httpd -DFOREGROUND
??10379 /usr/sbin/httpd -DFOREGROUND
??10380 /usr/sbin/httpd -DFOREGROUND
??10381 /usr/sbin/httpd -DFOREGROUND
??10382 /usr/sbin/httpd -DFOREGROUND
??10383 /usr/sbin/httpd -DFOREGROUND

Feb 06 09:47:28 testserver7 systemd[1]: Starting The Apache HTTP Server…
Feb 06 09:47:33 testserver7 httpd[10377]: AH00558: httpd: Could not reliably determine the server’s fully qualified dom…essage
Feb 06 09:47:33 testserver7 systemd[1]: Started The Apache HTTP Server.
Hint: Some lines were ellipsized, use -l to show in full.
[root@testserver7 my.cnf.d]# systemctl enable httpd

/etc/httpd/conf.d

vi harden.conf

TraceEnable off

## Disable Signature
ServerSignature Off

## Disable Banner
ServerTokens Prod

[root@testserver7 conf.d]# apachectl configtest
AH00558: httpd: Could not reliably determine the server’s fully qualified domain name, using 192.168.1.3. Set the ‘ServerName’ directive globally to suppress this message
Syntax OK
[root@testserver7 conf.d]#

K, let’s continue with PHP configuration. Edit /etc/php.ini and set the following:

## vim /etc/php.ini

date.timezone = America/New_York
memory_limit = 64M
expose_php = Off

restart Apache using systemctl for the changes to take effect:

## systemctl restart httpd
## systemctl status httpd

Test if PHP module is loaded in Apache using:

## httpd -M | grep php

also

## php -v

You can create a test info.php script too using the following command:

## echo -e “ /var/www/html/info.php

and access it with the following command:

## curl -I $(curl -s icanhazip.com)/info.php

SET-UP APACHE VHOSTS

Now, you would have to set-up Apache Virtual Host Directives, to be able to host multiple domains, even SSL powered ones using one IP address. So, create /etc/httpd/conf.d/vhosts.conf and add the following:

## cat /etc/httpd/conf.d/vhosts.conf

# Load my vhosts
IncludeOptional vhosts.d/*.conf

This tells Apache, to load the configuration files ending with .conf which reside in /etc/httpd/vhosts.d. As you can guess, this is the place where we put our Virtual Hosts, so let’s set-up one for domain1.com and another for domain2.net
VHOST for domain1.com

## cat /etc/httpd/vhosts.d/domain1.com.conf


ServerAdmin webmaster@domain1.com
DocumentRoot “/var/www/html/domain1.com”
ServerName domain1.com
ServerAlias www.domain1.com
ErrorLog “/var/log/httpd/domain1.com-error_log”
CustomLog “/var/log/httpd/domain1.com-access_log” combined


DirectoryIndex index.html index.php
Options FollowSymLinks
AllowOverride All
Require all granted

VHOST for domain2.net

## cat /etc/httpd/vhosts.d/domain2.net.conf


ServerAdmin webmaster@domain2.net
DocumentRoot “/var/www/html/domain2.net”
ServerName domain2.net
ServerAlias www.domain2.net
ErrorLog “/var/log/httpd/domain2.net-error_log”
CustomLog “/var/log/httpd/domain2.net-access_log” combined


DirectoryIndex index.html index.php
Options FollowSymLinks
AllowOverride All
Require all granted

Finally, restart Apache for the changes to take effect:

## apachectl configtest
## systemctl restart httpd
## systemctl status httpd

Failed to activate new LV

If you like this article, please +1 or Recommend via FB with the provided buttons above:
Article ID: 135
by: Reese K.
Posted: 15 Oct, 2013
Last updated: 15 Oct, 2013
Views: 1486

Failed to activate new LV
EDS etlprod2 ~ # lvcreate -l 14645 -n lv_dwstore vg_etlprod2
Not activating vg_etlprod2/lv_dwstore since it does not pass activation filter.
Failed to activate new LV.
Resolution:

Within my LVM configuration, I defined host tags by setting hosttags =1 within the tags section as follows in /etc/lvm/lvm.conf.

tags { hosttags = 1 }

Because of this, I was unable to create a new logical volume within a volume group. Why is this? Because the volume group was not tagged. After tagging the volume group, I was then allowed to create a logical volume.

EDS etlprod2 ~ # vgs -o vg_name,vg_tags vg_etlprod2
VG VG Tags
vg_etlprod2
EDS etlprod2 ~ # vgchange –addtag etlprod2 vg_etlprod2
Volume group “vg_etlprod2” successfully changed
EDS etlprod2 ~ # lvcreate -l 14645 -n lv_dwstore vg_etlprod2
Logical volume “lv_dwstore” created
EDS etlprod2 ~ #

Netmasks Table

Netmask Table

Decimal Binary CIDR Hex
——————————————————————-
255.255.255.255 11111111.11111111.11111111.11111111 /32 ffffffff
255.255.255.254 11111111.11111111.11111111.11111110 /31 fffffffe
255.255.255.252 11111111.11111111.11111111.11111100 /30 fffffffc
255.255.255.248 11111111.11111111.11111111.11111000 /29 fffffff8
255.255.255.240 11111111.11111111.11111111.11110000 /28 fffffff0
255.255.255.224 11111111.11111111.11111111.11100000 /27 ffffffe0
255.255.255.192 11111111.11111111.11111111.11000000 /26 ffffffc0
255.255.255.128 11111111.11111111.11111111.10000000 /25 ffffff80
255.255.255.0 11111111.11111111.11111111.00000000 /24 ffffff00

255.255.254.0 11111111.11111111.11111110.00000000 /23 fffffe00
255.255.252.0 11111111.11111111.11111100.00000000 /22 fffffc00
255.255.248.0 11111111.11111111.11111000.00000000 /21 fffff800
255.255.240.0 11111111.11111111.11110000.00000000 /20 fffff000
255.255.224.0 11111111.11111111.11100000.00000000 /19 ffffe000
255.255.192.0 11111111.11111111.11000000.00000000 /18 ffffc000
255.255.128.0 11111111.11111111.10000000.00000000 /17 ffff8000
255.255.0.0 11111111.11111111.00000000.00000000 /16 ffff0000

255.254.0.0 11111111.11111110.00000000.00000000 /15 fffe0000
255.252.0.0 11111111.11111100.00000000.00000000 /14 fffc0000
255.248.0.0 11111111.11111000.00000000.00000000 /13 fff80000
255.240.0.0 11111111.11110000.00000000.00000000 /12 fff00000
255.224.0.0 11111111.11100000.00000000.00000000 /11 ffe00000
255.192.0.0 11111111.11000000.00000000.00000000 /10 ffc00000
255.128.0.0 11111111.10000000.00000000.00000000 /9 ff800000
255.0.0.0 11111111.00000000.00000000.00000000 /8 ff000000

254.0.0.0 11111110.00000000.00000000.00000000 /7 fe000000
252.0.0.0 11111100.00000000.00000000.00000000 /6 fc000000
248.0.0.0 11111000.00000000.00000000.00000000 /5 f8000000
240.0.0.0 11110000.00000000.00000000.00000000 /4 f0000000
224.0.0.0 11100000.00000000.00000000.00000000 /3 e0000000
192.0.0.0 11000000.00000000.00000000.00000000 /2 c0000000
128.0.0.0 10000000.00000000.00000000.00000000 /1 80000000
0.0.0.0 00000000.00000000.00000000.00000000 /0 00000000

turning off auto indent and comments in vi

It’s not always convenient to have auto comments or auto indent when pasting text enabled within a vi session.
To turn them off, at least within the current vi session:

:set comments=

:set paste

Extract contents of initrd

How to extract and view contents of initrd

Depending on your version of linux, the /boot/initrd may be one of two formats:

cpio archive
ext2 filesystem data
The easiest way to determine what you’re working with is by using the file command. Here are examples showing the output for each file type listed in the above bullet points:

[root@linux ~]# file /boot/initramfs-2.6.32-220.17.1.el6.x86_64.img
/boot/initramfs-2.6.32-220.17.1.el6.x86_64.img: gzip compressed data, from Unix, last modified: Thu Jun 28 18:33:36 2012, max compression

[root@linux ~]# file /boot/initramfs64-catalyst
initramfs64-catalyst: gzip compressed data, from Unix, max compression
To be sure what you’re dealing with, supply the -z option to the file command as this will attempt to look inside the compressed files to see what the contents are:

[root@linux ~]# file -z /boot/initrd-2.6.9-023stab053.2-smp.img
/boot/initrd-2.6.9-023stab053.2-smp.img: Linux rev 1.0 ext2 filesystem data (gzip compressed data, from Unix, max compression)
[root@linux ~]# file -z /boot/initramfs64-catalyst
initramfs64-catalyst: ASCII cpio archive (SVR4 with no CRC) (gzip compressed data, from Unix, max compression)
Mount ext2 filesystem data (gzip compressed) initrd contents

I like to work with copies of the initrd, so I created a dir in /root/initrd and work with it there:

[root@linux root]# mkdir /root/initrd; cd /root/initrd
[root@linux initrd]# cp /boot/initrd-2.6.9-023stab053.2-smp.img .
Some online documents indicate the initrd should be renamed with the .gz extension so that gunzip can be used to extract the compressed file, but most linux distros ship with the handy zcat command, which saves us a step from having to rename files. This is the method I prefer:

[root@linux initrd]# zcat initrd-2.6.9-023stab053.2-smp.img > initrd-2.6.9-023stab053.2-smp
[root@linux initrd]# file -z initrd-2.6.9-023stab053.2-smp
initrd-2.6.9-023stab053.2-smp: Linux rev 1.0 ext2 filesystem data
Now it’s ready to be mounted as a loop device:

[root@linux initrd]# mount -o loop ./initrd-2.6.9-023stab053.2-smp /mnt
[root@linux initrd]# ls -l /mnt
total 9
drwxr-xr-x 2 root root 1024 Jun 25 15:40 bin
drwxr-xr-x 2 root root 1024 Jun 25 15:40 dev
drwxr-xr-x 4 root root 1024 Jun 25 15:40 etc
drwxr-xr-x 2 root root 1024 Jun 25 15:40 lib
-rwxr-xr-x 1 root root 869 Jun 25 15:40 linuxrc
drwxr-xr-x 2 root root 1024 Jun 25 15:40 loopfs
drwxr-xr-x 2 root root 1024 Jun 25 15:40 proc
lrwxrwxrwx 1 root root 3 Jun 25 15:40 sbin -> bin
drwxr-xr-x 2 root root 1024 Jun 25 15:40 sys
drwxr-xr-x 2 root root 1024 Jun 25 15:40 sysroot
Extract cpio archive initrd contents

[root@linux initrd]# cp /boot/initramfs64-catalyst .
[root@linux initrd]# zcat initramfs64-catalyst | cpio -id
[root@vpsrep tmp]# ls -l
total 387232
drwxr-xr-x 2 root root 4096 Jun 26 21:57 bin
drwxr-xr-x 2 root root 4096 Jun 26 21:57 boot
drwxr-xr-x 3 root root 4096 Jun 26 21:58 dev
drwxr-xr-x 32 root root 4096 Jun 26 21:58 etc
drwxr-x— 5 root root 4096 Jun 26 21:58 evolution
drwxr-xr-x 2 root root 4096 Jun 26 21:58 home
lrwxrwxrwx 1 root root 15 Jun 26 21:58 init -> /evolution/init
-rw-r–r– 1 root root 396057600 Jun 26 21:55 initramfs64-catalyst.cpio
lrwxrwxrwx 1 root root 5 Jun 26 21:58 lib -> lib64
drwxr-xr-x 2 root root 4096 Jun 26 21:58 lib32
drwxr-xr-x 9 root root 4096 Jun 26 21:58 lib64
drwxr-xr-x 2 root root 4096 Jun 26 21:58 media
drwxr-xr-x 2 root root 4096 Jun 26 21:58 mnt
drwxr-xr-x 2 root root 4096 Jun 26 21:58 opt
drwxr-xr-x 2 root root 4096 Jun 26 21:58 proc
drwx—— 2 root root 4096 Jun 26 21:58 root
drwxr-xr-x 2 root root 4096 Jun 26 21:58 sbin
drwxr-xr-x 2 root root 4096 Jun 26 21:58 sys
drwxrwxrwt 2 root root 4096 Jun 26 21:58 tmp
drwxr-xr-x 12 root root 4096 Jun 26 21:58 usr
drwxr-xr-x 9 root root 4096 Jun 26 21:58 var
Note: the –verbose option may be supplied to list the files to the console as they are being extracted, or adding v to the options in the example above, as in cpio -idv

cpio -itv will simply list the contents without extracting them.

The -itv options are short hand for –extract –list –verbose
The -idv options are short hand for –extract –make-directories –verbose

Repackaging

If some some reason a change needs to be made to the initrd and it needs to be buttoned back up, this is how to do it:

ext2 filesystem data

copy whatever files you want to whereever the initrd ext2 filesystem was mounted. Once finished, umount the filesystem, gzip it (which will give it the .gz extension), then rename it to the original initrd file that was copied or some other name like initrd-2.6.9-023stab053.2-smp-mpath.img. If a new name is given, be sure to update the boot loader configuration files like /boot/grub/grub.conf.

cpio archive

From within the directory the contents were extracted, first remove or move the .cpio file to another parent directory outside to ensure it’s not part of the new cpio archive then recreate the archive:

find . | cpio –quiet -c -o | gzip -9 -n > ../initramfs64-catalyst

ip_conntrack: table full, dropping packet

ip_conntrack: table full, dropping packet
At one point, there was high call volume into our support center of customers complaining about severe lag. One common denominator was that the customer base who called in happened to all reside on the same server, so investigation into the matter focused on that one particular system.

The server’s load average was really low, and had plenty of free RAM, though connectivity to customers hosted websites were lagging. After running dmesg, I noticed “ip_conntrack: table full, dropping packet”. After observing netstat -an for a bit, it was clear the server was being used to send SPAM. After blocking the connections and securing the customer SMTP passwords, the counts came down and the lag ceased.

The following command can be used to see what the max setting is for this kernel parameter:

/sbin/sysctl net.ipv4.ip_conntrack_max

or

cat /proc/sys/net/ipv4/ip_conntrack_max

To see how many you are using at present:

wc -l /proc/net/ip_conntrack

or

cat /proc/sys/net/ipv4/netfilter/ip_conntrack_count

The setting can be adjusted, and if to be made permanent, make the change in /etc/sysctl.conf. In this example, the max setting is increased to 65535.

echo “net.ipv4.ip_conntrack_max = 65535” > /etc/sysctl.conf
/sbin/sysctl -w

To increase it temporarily (non-persistent across reboots)

echo 131072 > /proc/sys/net/ipv4/ip_conntrack_max

The file /boot/grub/stage1 not read correctly

The file /boot/grub/stage1 not read correctly
After pxe booting a server and installing linux from a cpio archive, grub failed to load using the grub-install command. The error presented was as follows:

root@pxe:~# mkdir /target
root@pxe:~# mount /dev/sda1 /target
root@pxe:~# chroot /target
root@pxe:~# grub-install /dev/sda
The file /boot/grub/stage1 not read correctly.
Since command line didn’t work, I tried installing grub via the grub shell:

root@pxe:~# grub

GNU GRUB version 0.97 (640K lower / 3072K upper memory)

[ Minimal BASH-like line editing is supported. For the first word, TAB
lists possible command completions. Anywhere else TAB lists the possible
completions of a device/filename.]
grub> root (hd0,0)
root (hd0,0)
Filesystem type is ext2fs, partition type 0xfd
grub> setup (hd0)
setup (hd0)
Checking if “/boot/grub/stage1” exists… no
Checking if “/grub/stage1” exists… no

Error 2: Bad file or directory type
grub>
This too failed, with a different error message. You may have noticed the first error message given while attempting to install grub with the grub-install command yielded:

The file /boot/grub/stage1 not read correctly

While attempting to install grub via the grub shell yielded:

Error 2: Bad file or directory type

After some research, grub only works on an inode size of 128, and the pxe system formatted the file system with an inode size of 256.

root@pxe:~# tune2fs -l /dev/sda1 | grep -i ‘inode size’
Inode size: 256
The system being reimaged was an older legacy system, and the pxe system had been upgraded months prior. Having remembered this, I compared versions of mke2fs and found that the upgraded version creates 256-byte inodes by default, where the older version was 128. To correct this, I updated the pxe imaging scripts responsible for formatting the disks with the -I option as follows:

root@pxe:~# mke2fs -L / -I 128 -F -j -O dir_index /dev/sda1
Once recreating the file system with a defined inode size of 128, the grub-install command worked successfully.

root@pxe:~# tune2fs -l /dev/sda1 | grep -i ‘inode size’
Inode size: 128

How to add a new LUN using EMC powerpath

How to add a new LUN using EMC Navisphere and powerpath
Overview:

create a new LUN on an EMC Clariion CX3-40 via Navisphere
assign LUN to proper Storage Group
use powermt to bring LUN in and configure it
rescan scsi bus on Linux server for new LUN
bring new LUN under LVM control and create file system

Resizing an Online Multipath Device

Resizing an Online Multipath Device

If you need to resize an online multipath device, use the following procedure:

1. Resize your physical device

2. Use the following command to find the paths to the LUN:

# multipath -l
3. Resize your paths. FOr SCSI devices, writing a 1 to the rescan file for the device causes the SCSI driver to rescan:

# echo 1 > /sys/block/device_name/device/rescan
4. Resize your multipath device by running the multipathd resize command:

# multipath -k’reisze map mpathX’
5. Resize the file system (assuming no LVM or DOS partition are used):

# resize2fs /dev/mapper/mpathX