Automatically Filter SpamBot on DigitalOcean

Setup iptables

Install iptables-persistent, so that the iptables rules will be saved even reboot the machine:

# apt-get update
# apt-get install iptables-persistent

Script for manually add iptables

Use following scritp for manually add iptables items:

#!/bin/sh
# This script runs once per hour, Directly remove the ips which post comments
# more than 4 times per hour. And who comments less than 3 times we should sent
# its ip to old ips file. The old ips files will be used for analyse once per day
# The run frequency is controlled by  crontab.   

######################################################
# Before Start, empty the deathSentence
######################################################
>/var/log/apache2/deathSentence

######################################################
# First cat the file and try to found the bot ip list
######################################################
# Pipe 1: The one who called POST method should be monitored
# Pipe 2: Get the ip address who called POST method.
# Pipe 3: Sort the ip addresses. 
# Pipe 4: Calculate the repeated times. First column, times; Second column, ip address.
# Pipe 5: Sort via first column(times) numerically(Not textly!) .
# Pipe 6: If the Call POST time bigger than 4 in one hour, catch it!
# Pipe 7: Yes we caught this thief! Get its ipaddr.
# Write these thieves into the death sentence
cat /var/log/apache2/other_vhosts_access.log | grep "POST" | awk '{print $2}' | sort | uniq --count | sort -n | awk '$1>4' | awk {'print $2'}>/var/log/apache2/deathSentence
# Those who comments but equal or more than 4 times will be append to wishList
cat /var/log/apache2/other_vhosts_access.log | grep "POST" | awk '{print $2}' | sort | uniq --count | sort -n | awk '$1<5' | awk {'print $2'}>>/var/log/apache2/wishList

######################################################
# Second we add this bot ip list into the netfilter
######################################################
for i in `cat /var/log/apache2/deathSentence`
do
	#echo $i
	iptables -A INPUT -s $i -j DROP
done

######################################################
# Finally empty the other_vhosts_access.log
######################################################
>/var/log/apache2/other_vhosts_access.log

Oh, also add myself into the blacklist, so un-lock me:

$ iptables -A INPUT -s 1xx.x.x.x -j ACCEPT

Since those wishList should also be cared, wrote following scripts for judge, every 4 hours will be make a decision.

#!/bin/sh
# This script runs once 4 hours, used for processing the /var/log/apache2/wishList
# ip address lists. Those bad guys who were in wishList, if their total appear times
# bigger than 4 times, will be added to iptable's drop rules.
>/var/log/apache2/deathSentence_4hour

######################################################
# Read the ip list and store those bad guys into deathSentence_4hour
######################################################
cat /var/log/apache2/wishList | sort | uniq --count | sort -n | awk '$1>4' | awk {'$print $2'}>/var/log/apache2/deathSentence_4hour

######################################################
# Now you got the bad guys, add them into iptables
######################################################
for i in `cat /var/log/apache2/deathSentence_4hour`
do
        #echo $i
        iptables -A INPUT -s $i -j DROP
done

######################################################
# Finally empty the wishList
######################################################
>/var/log/apache2/wishList

Crontab It!

Run auto_add_bot_ip.sh at every minute 0 of 1 hour, then run auto_judge_wishlist.sh at every minute 10 of every 4 hours.

# m h  dom mon dow   command
0 */1 * * * /root/code/auto_add_bot_ip.sh
10 */4 * * * /root/code/auto_judge_wishlist.sh

Manually delete spam comments for WP

Login to mysql commandline via:

# mysql -uroot -p
mysql> use wordpress
.........
Database changed

Display the COLUMNS of wp_comments:

mysql> SHOW COLUMNS FROM wp_comments;
+----------------------+---------------------+------+-----+---------------------+----------------+
| Field                | Type                | Null | Key | Default             | Extra          |
+----------------------+---------------------+------+-----+---------------------+----------------+
| comment_ID           | bigint(20) unsigned | NO   | PRI | NULL                | auto_increment |
| comment_post_ID      | bigint(20) unsigned | NO   | MUL | 0                   |                |
| comment_author       | tinytext            | NO   |     | NULL                |                |
| comment_author_email | varchar(100)        | NO   | MUL |                     |                |
| comment_author_url   | varchar(200)        | NO   |     |                     |                |
| comment_author_IP    | varchar(100)        | NO   |     |                     |                |
| comment_date         | datetime            | NO   |     | 0000-00-00 00:00:00 |                |
| comment_date_gmt     | datetime            | NO   | MUL | 0000-00-00 00:00:00 |                |
| comment_content      | text                | NO   |     | NULL                |                |
| comment_karma        | int(11)             | NO   |     | 0                   |                |
| comment_approved     | varchar(20)         | NO   | MUL | 1                   |                |
| comment_agent        | varchar(255)        | NO   |     |                     |                |
| comment_type         | varchar(20)         | NO   |     |                     |                |
| comment_parent       | bigint(20) unsigned | NO   | MUL | 0                   |                |
| user_id              | bigint(20) unsigned | NO   |     | 0                   |                |
| comment_mail_notify  | tinyint(4)          | NO   |     | 0                   |                |
+----------------------+---------------------+------+-----+---------------------+----------------+
16 rows in set (0.00 sec)

If you want to display the last 30 minutes’ comments:

mysql> SELECT * FROM wp_comments WHERE comment_date  BETWEEN TIMESTAMPADD(MINUTE, -30, NOW()) AND NOW();

Delete last 30 minutes’ comments:

mysql> DELETE FROM wp_comments WHERE comment_date  BETWEEN TIMESTAMPADD(MINUTE, -30, NOW()) AND NOW();
Query OK, 536 rows affected (0.18 sec)

Select and Delete 10 day’s comments:

mysql> select * from wp_comments where datediff(now(), comment_date)<10;
mysql> delete from wp_comments where datediff(now(), comment_date)<10;
Query OK, 31029 rows affected (1.34 sec)

Disable postfix on startup:

# update-rc.d postfix disable

Build ChromiumOS

First Time Build

This build failed for I could not get the repository sync.

I setup the environment on 159’s /media/nfs:

$ git clone https://chromium.googlesource.com/chromium/tools/depot_tools.git
$ export PATH="$PATH":`pwd`/depot_tools
$ echo $PATH
/home/ubuntu/bin:/home/ubuntu/bin:/home/ubuntu/bin:/home/ubuntu/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/media/nfs/ChromiumOS/depot_tools
$ cat script.sh 
#!/bin/sh
cat >./sudo_editor<<EOF
#!/bin/sh
echo Defaults !tty_tickets > $1
echo Defaults timestamp_timeout=180 >> $1
EOF
chmod +x ./sudo_editor
sudo EDITOR=./sudo_editor visudo -f /etc/sudoers.d/relax_requirements
$ export BOARD=x86-generic
$ repo init -u https://git.chromium.org/chromiumos/manifest.git
$ repo sync

Second Time Build

Trusty@Linux59:~/Code/ChromiumOS> pwd
/home/Trusty/Code/ChromiumOS
Trusty@Linux59:~/Code/ChromiumOS>  git clone https://chromium.googlesource.com/chromium/tools/depot_tools.git
Trusty@Linux59:~/Code/ChromiumOS> export PATH=`pwd`/depot_tools:"$PATH"
Trusty@Linux59:~/Code/ChromiumOS> mkdir chromiumos
Trusty@Linux59:~/Code/ChromiumOS> cd chromiumos/
Trusty@Linux59:~/Code/ChromiumOS/chromiumos> repo init -u https://chromium.googlesource.com/chromiumos/manifest.git --repo-url https://chromium.googlesource.com/external/repo.git 
Trusty@Linux59:~/Code/ChromiumOS/chromiumos> repo sync
Trusty@Linux59:~/Code/ChromiumOS/chromiumos> cros_sdk
root's password:
(cr) ((c405e7b...)) Trusty@Linux59 ~/trunk/src/scripts $ export BOARD=x86-generic
(cr) ((c405e7b...)) Trusty@Linux59 ~/trunk/src/scripts $ ./setup_board --board=${BOARD}
# ./set_shared_user_password.sh
# ./build_packages --board=${BOARD}

Build OpenWRT For X86

Prepare

Install following packages:

$ sudo apt-get install build-essential subversion git-core libncurses5-dev zlib1g-dev gawk flex quilt libssl-dev xsltproc libxml-parser-perl

Code

Get the source code from OpenWRT.org:

$ git clone git://git.openwrt.org/openwrt.git

Then Prepare for menuconfig:

$ cd openwrt
$ ./scripts/feeds update -a
$ ./scripts/feeds install -a
$ make menuconfig

Select x86 for Target System.
[] ext4–> Target Images –> ext4
[] Build VMware image files (VMDK)

You could also select for VDI or other formats.

Luci- > collection - > select luci.

Then we could type make for making out the images.

Enable nfs server of 53

Only enabled the nfs server and use the max disk for building, the nfs server runs Redhat RHEL6.2, the same procedure could be applied to CentOS Based system.
Steps:
Query for installed packages in server:

$ rpm -qa nfs-utils
$ rpm -qa rpcbind

Edit the nfs based directory:

# cat /etc/exports
/home/Trusty/share/       *(rw,sync,no_subtree_check,no_root_squash)

Start the service and test:

# service rpcbind start
# service nfs start

In client machine, just type following command for mount the remote nfs directory:

$ sudo mount -t nfs 1xx.xxx.xxx.xx:/home/Trusty/share /mnt/

Make nfs server automatically start when system boot:

# chkconfig nfs on
# chkconfig rpcbind on

Client Machine(59), do following for automatically mount nfs:

$ vim /etc/fstab
# Using NFS
1xx.xxx.xxx.xx:/home/Trusty/share /media/nfs/     nfs     rsize=8192,wsize=8192,timeo=14,intr     0       0
$ mount -a

Then everytime this clent machine startup the remote nfs directory will be mounted to local directory.

If you are ubuntu client, then you should install nfs-client via;

sudo apt-get install nfs-common