This site is a mess…

Let’s face it: this site is a mess.

It never came off the ground after a decided to use WordPress. Fooling around with stuff like multi language support has not done it any good. Problem was/is that I spend more time on other people’s sites than my own. Time to change a few things. Time to start doing what I like: post stuff about technical problems I encounter.

I made myself a todo list:

  • Check the relevance of  “old stuff” I wrote which still seems to turn up on google. Like a piece I wrote in something like 2004 about Slackware’s init system. To my surprise I found a reference to it in the AOLS FAQ file.
  • Write more on stuff like my old skool backup system. I made something like rdiff backup. I used stuff you normally already use, like cp and rsync. (To be honest… never heard of rdiff backup until after I wrote it)
  • Don’t worry about language… just do english like I did before.
  • Don’t worry about SEO… just write.
  • In general have more fun with it.

Lets see where this brings me…. here goes nothing 😀

Help! Ik krijg een e-book.

Staande_Cover_NLDe laatste weken ben ik bezig geweest met het maken van e-books. Meer specifiek: epub bestanden. Reden hiervoor is mijn vriendin die een boek aan het schrijven is… en het is nu eenmaal leuk om dat wat je schrijft ook in een e-reader te kunnen lezen.

Cool om te doen. Technisch gezien leuk. Eigenlijk is een epub een gecomprimeerd bestand met daarin html bestanden, afbeelding, style sheets en er kan zelfs javascript in zitten heb ik me laten vertellen. Niet dat dat mijn vriendin boeit… het moet gewoon openen als je er op klikt. En dat is voor veel mensen toch wel een probleem heb ik inmiddels begrepen.

Continue reading Help! Ik krijg een e-book.

Ipset block for China

denkenPreface

Some time ago I wrote an article (Blacklist met iptables) explaining how to use ipset and iptables to make a blacklist. This became a topic again because my girlfriend (the one having the site HappyMinds) got frustrated more and more by all those people from Russia and China ruining her stats. And I have to admit: what are they doing on a site with limited content in English, most of it is in Dutch. So the idea is to extend the blacklist with an ipset block for China (as an example).

In addition to that I switched (back) to using Ubuntu as a server platform. A good opportunity to look at the differences with CenOS.

chinese hackerThe idea

In the previous article I described how you can make an ipset table from a list of IP numbers and use that list in an iptables firewall. Obvious benefit being that by using ipset you don’t have to make a firewall rule per IP number, which greatly enhances the firewall performance.
That doesn’t cut it for blocking a country/region. It would be better to block the subnets in use by that country. The people at IpDeny have been so friendly as to compile such a list and publish it. This list (or lists actually) can be downloaded, just as the IP numbers in the previous example, and put into an ipset table.

ipsetThe implementation

The script to download the list of subnets and compile it into an ipset table resembles the script from the previous article. There are some minor differences. I named the script make_zone_blacklist, if you use it and give it another name than don’t forget to change the name in you firewall script as well. The country coded being used can be looked up at the Ipdeny site.

#! /bin/bash

# make_zone_blacklist
# load the zone blacklist for an iptables based firewall
# Written by Peter Kaagman <prjv.kaagman@gmail.com>;
# version 1.0.0 20150718

# Set the path so you know which executable you run
# being paranoid?
PATH=/usr/bin:/usr/sbin:/sbin:/bin
# The base url for the list at ipdeny
url=http://www.ipdeny.com/ipblocks/data/countries
# The zones we are going to deny access
zones="cn ru"

function load_blacklist {
  #
  # Load the blacklist in a copy list
  ipset destroy zone_blacklist2 >& /dev/null # destroy is if needed
  ipset create zone_blacklist2 hash:net # (re)create it
  #
  # Iterate the zones
  for zone in $zones
  do
    # Download the new list
    # and itterate it
    for CIDR in `wget -q $url/$zone.zone -O -|grep -v ^#`; do
      # add the ip to the newly created blacklist2
      ipset add zone_blacklist2 $CIDR
    done
  done
  #
  # Swap this list with the excisting one
  # The list blacklist should be created by you iptables script
  ipset swap zone_blacklist zone_blacklist2
  #
  # Remove zone_blacklist2
  ipset destroy zone_blacklist2
}

#echo Creating blacklist
load_blacklist

As the the other example this script also assumes that the ipset table (zone_blacklist) excists. It has to be created before hand. My firewall example takes care of that.

There are some minor differences with script creating the IP blacklist:

  • The type of the ipset table is different to make it able to contain subnets instead of IP numbers.
  • The source of the download differs
  • The script is able to download several country codes

iptablesThe firewall script

The firewall script needs to be adapted to accommodate the new ipset table:

Add a variable for loading the table

iptables=/sbin/iptables
ipset=/sbin/ipset
make_blacklist=/home/pkn/scripts/bash/firewall/make_blacklist.sh
make_zone_blacklist=/home/pkn/scripts/bash/firewall/make_zone_blacklist.sh
int_if=eth0			# internal (local) interface, e.g. eth0
int_ip=192.168.178.2		# internal (local) IP, e.g. 192.168.1.94

I use these kind of variables quite often. Whenever I move the script around I only have to change this one reference in the firewall script. When using this script you will have to adapt it to your needs off course.

Creating the ipset table and the filter rules.

# default policy
$iptables -P INPUT   DROP
$iptables -P FORWARD DROP
$iptables -P OUTPUT  DROP

# drop zone_blacklist
# first create the ipset zone_blacklist
# make_zone_blacklist assumes it excists
$ipset create zone_blacklist hash:net >& /dev/null
# no network at the moment,
# so we load the list later on

# log on match
$iptables -A INPUT -m set --match-set zone_blacklist src -p TCP -j LOG --log-prefix 'Zone_BlackListed '
# and drop on match
$iptables -A INPUT -m set --match-set zone_blacklist src -p TCP -j DROP

# drop blacklist
# first create the ipset blacklist
# make_blacklist assumes it excists
$ipset create blacklist hash:ip >& /dev/null
# no network at the moment,
# so we load the list later on

# log on match
$iptables -A INPUT -m set --match-set blacklist src -p TCP -j LOG --log-prefix 'BlackListed '
# and drop on match
$iptables -A INPUT -m set --match-set blacklist src -p TCP -j DROP

# drop broadcast (do not log)

I’ve chosen to first drop the zone_blacklist entries. This because I think this list is going to be hit a lot sooner than the blacklist. Following the “drop ASAP” paradigm.

Loading the ipset table.

# log all the rest before dropping
$iptables -A INPUT   -j LOG --log-prefix "IN "
$iptables -A INPUT   -j REJECT --reject-with icmp-port-unreachable
$iptables -A OUTPUT  -j LOG --log-prefix "OU "
$iptables -A OUTPUT  -j REJECT --reject-with icmp-port-unreachable
$iptables -A FORWARD -j LOG --log-prefix "FW "
$iptables -A FORWARD -j REJECT --reject-with icmp-port-unreachable

# firewall loaded, so lets load the blacklists
echo "Loading blacklist"
$make_blacklist
echo "Loading zone_blacklist"
$make_zone_blacklist

I load the ipset tables just after the logging block. At this point all the network connection I want will be restored.

ubuntu bootingHow to survive a reboot

In the previous article I explained how you can survive a reboot on a CentOS system. In essence that meant executing some code whenever an interface comes on line. Ubuntu has a similar option but (IMHO) a bit simpler though. Ubuntu has an directory (/etc/network/ip-up.d) in which you can place a script, there are all ready some present. I’ve placed a script with the following content in that directory and made it executable:

#! /bin/bash

/home/pkn/scripts/bash/firewall/firewall.sh start

To periodically refresh the blacklist you can place the same script in /etc/cron.daily. That way the firewall is refreshed on a daily basis. You could use to only run the blacklist scripts, in that case the firewall stays loaded.

reflectieSo now are we don?

Well… I don’t know a lot…. but I do know this: you’re never completely done. With this kind of filtering you are always one step behind on the reality. So… no.. we are not done :D.
Spamhaus has 2 list which look promising: drop and edrop. Perhaps I’ll incorporate those. Analysing the log files to identify abusers seems promising to. But first I’m going to do some stats with Tobi Oetikers library. I’m kind of a stats junkie :D.

Ipset block voor China

denkenInleiding

Enige tijd geleden heb ik een artikel (Blacklist met iptables) geschreven over het gebruik van iptables om een blacklist te maken.  Dit werd weer actueel omdat mijn vriendin (je weet wel… van HappyMinds) zich meer en meer ging ergeren aan de “bezoekers” uit China en Rusland. Er is een meer dan sterk vermoeden dat die alleen maar langs komen voor rottigheid, temeer omdat haar site slechts heel beperkt te lezen is in een andere taal dan Nederlands. De bedoeling is dan ook om een uitbreiding te maken op de blacklist door een ipset block te maken voor bijvoorbeeld China.

Komt bij dat ik overgestapt ben (weer terug naar eigenlijk) naar Ubuntu als server platform. Een leuke gelegenheid om de verschillen eens te bekijken.
Continue reading Ipset block voor China

WordPress on a Raspberry Pi B using Nginx

pi-wordpressI already mentioned I host my girlfriends WordPress site – and this site – on a desktop computer. That desktop computer is an old left over Dell Latitude. For some reason this computer makes quite a bit of noise. Which wouldn’t be a problem if my “server room” wasn’t actually my spare bedroom. I could try to resolve the noise issue – it is the fan making a mess of it – but I’ve never been a hardware guy.  That, and a thread on the NedLinux forum (in dutch, sorry about that :D) about what we are doing with our Raspberry Pi, made me decide to make a web-server out of the Raspberry Pi B I’ve got lying around.

So I started on a journey with the ultimate goal to have a hosting system for a WordPress site which is quit and has an acceptable response time. And keeping in mind the journey, which should be as valuable as the goal.

Having trouble actually finding my Pi I started the journey with a very basic setup on an Alix 2d13 board. Did some pfSense lab work on that board but finished that project some time ago. I installed a Debian system on it with Apache2, PHP5 and MySQL…. it turned out to be fun to do but slooooooow. Exit Alix board, also since I found my Pi in the meantime.

nginxTalking to a co-worker about the project the idea of replacing Apache2 as the actual web server emerged. A http servers like lighthttp an nginx would suffice for my needs, perhaps even better. Lacking any experience with lighthttp or nginx made the journey more interesting. After reading up on the performance of several lightweight http servers I decided to give Nginx a try.

raspbianI decided to install Raspbian on my Pi. Giving me an environment which I’m familiar to with the ease of “burning” an image to an SD card.   This is all pretty well documented on the Raspberry Pi download page. This image is actually being maintained by de Raspberry community, not the Raspbian community.

Raspbian gives you the apt package manger. This makes it easy to install your packages. For my Pi I installed the following packages using apt:

  • nginx
  • php5, php5-cli and php5-fpm
  • mysql-server and phpmyadmin

As a side note: Not really sure where it came from – perhaps it was already there on the Raspbian image – but as it turned out I had Apache 2 installed. I stopped the process and made sure with “# update-rc.d -f apache2 remove” that it wouldn’t start again. You can’t have 2 daemons claiming port 80 and 443.

php-fpm-logoGetting Nginx to serve up PHP generated content turned out to be the biggest issue. There is no php-mod for Nginx. That’s why php5-fpm is installed. Turns out Nginx is a proxy which sends the request to a “PHP CGI script engine”.  Php-fpm is used to spawn the necessary threads.  Nginx and PHP are seperate processes. They communicate either via a unix socket or a TCP connection.

The first thing I did – after getting Nginx to server up static content – was to make sure Nginx would serve up the result of phpinfo(). This sounds trivial – and should be just that – but Nginx is not Apache which I’m familiar with. The Nginx Wiki is full of warnings not the get misinformed by all kinds of blogs giving bogus information on how to do things. So I kinda stuck to the instructions given on their own site.

The page I ended up with on using was Martin’s Nginx, PHP, Primer. I explains quite clearly on how to serve up dynamic content. Only thing which confused me a bit was that Martin includes the file “fastcgi.conf” which Nginx expected to find in “/etc/nginx”. The debian installation I have did have a file named “fastcgi_params”. Nginx did not complain… so I figured that was the one he intended.

Next thing I needed to figure out for myself was the pgp-fpm thing. I wanted to make sure I understood the way the two processes (nginx and php) were talking to each other. Unix socket or TCP.  Turned out this configuration could be found in “/etc/ph[p5/fpm”.  A file “www.conf” in the subdirectory “pool.d” controls such things.

After I corrected the syntax error in  “index.php” (I used the function php_info() instead of phpinfo() (hey… I’m a Perl guy)) it worked as expected. No worries there. On to WordPress.

Writing this blog I’m wondering why I didn’t just paste in the WordPress code. But instead I googled  for “WordPress Nginx” and found some references on the Nginx Wiki. And they have a complete page devoted on that subject. No a big surprise since they state on their homepage that wordpress.com is hosted on nginx.

But anyway… after using that config WordPress just works. An empty WordPress with one article added loads in 3-4 seconds. No plugins enabled. Not to fast I know. The troubles started when I added content and enabled plugins. The aforementioned girlfriends site loads in 10 seconds plus, testing done from within the same subnet. Not really acceptable.  By using the php-apc module I managed to reduce the load time to 8 seconds plus. Impressive…. yes… but not enough.

failedIn conclusion: I did not reach my goal. The Pi B simple does not have the power to support a website with scripted backend. Sure I could have enable a caching plugin. In effect making the site content static for visitors, but not for the writers of the site.  But the journey was fun. Getting Nginx to work was kinda like going back to the basics of things. It made me decide to order a Rasberry Pi 2 model b. A Pi with a quad core 1ghz processor. But that is something for another blog 😀

Going multilingual

 Borderless Bilbo’s-Stekkie.

multilingualFor some years now I write articles to share the things I learn. Most often they are about technical stuff which – so I’ve learned – are of interest to people outside the Netherlands.  So I’m often in doubt to write in either dutch or English. The site should probably be multilingual. But how to go about something like that?

WordPress solves this problem for me with a plugin – WPML in my case – to write multilingual sites. So from now on articles posted on this site will be in dutch and English.  Maybe even the older articles.

We worden multilingual

 Bilbos-Stekkie gaat over de grens

multilingualIk schrijf al jarenlang stukjes om de kennis die ik op doe te delen. Vaak zijn dat technische zaken die – heb ik ervaren – tot over de grens gelezen worden. Ik zit dan ook vaak in dubio of ik in het nederlands of engels moet schrijven.  Eigenlijk zou de site multilingual moeten zijn. Maar hoe pak je zoiets aan?

WorpPress lost dit voor me op met een plugin – WPML in mijn geval – om een multilingual site te kunnen hebben.  Artikelen zullen vanaf nu dan ook in het nederlands én engels verschijn. Misschien zelfs met terugwerkende kracht.

Blacklist met iptables

inleiding firewallInleiding

Mijn vriendin heeft een website: HappyMinds. Ik host die site voor haar – net als deze site – op een desktop pc waar ik CentOS 5 op heb staan. En zeg nou zelf… wat is er nu leuker als je net je eigen website hebt dan de bezoeker stats in de gaten houden? Dat doen we allemaal toch. Tussen het schrijven van haar blogs en de stats heeft ze heel veel plezier aan die site. Er was in het begin een issue met bezoekende russen, maar dat heeft zich allemaal opgelost.

Tot er op een dag een script kiddie voorbij komt die eens gaat kijken of hij in kan breken in die site. Zelf negeer ik dat soort dingen. Zorg dat de software up to date is, de wachtwoorden redelijk sterk zijn, enz. Maar je hebt wel gelijk enkele duizenden page views daar waar het er eerst tientallen waren. Dit heeft een vernietigend effect op de statistieken.
Continue reading Blacklist met iptables

Blacklist met iptables

inleiding firewallInleiding

Mijn vriendin heeft een website: HappyMinds. Ik host die site voor haar – net als deze site – op een desktop pc waar ik CentOS 5 op heb staan. En zeg nou zelf… wat is er nu leuker als je net je eigen website hebt dan de bezoeker stats in de gaten houden? Dat doen we allemaal toch. Tussen het schrijven van haar blogs en de stats heeft ze heel veel plezier aan die site. Er was in het begin een issue met bezoekende russen, maar dat heeft zich allemaal opgelost.

Tot er op een dag een script kiddie voorbij komt die eens gaat kijken of hij in kan breken in die site. Zelf negeer ik dat soort dingen. Zorg dat de software up to date is, de wachtwoorden redelijk sterk zijn, enz. Maar je hebt wel gelijk enkele duizenden page views daar waar het er eerst tientallen waren. Dit heeft een vernietigend effect op de statistieken.
Continue reading Blacklist met iptables

WhatsUp

whatsupToen WhatApp overgenomen werd door Facebook was er een hoop commotie over privacy zaken. Nu heb ik een instelling van “alsikprivacywildaninternetikwelniet” dus dat gedeelte van de discussie ging aardig langs me heen. Wat mij wel aansprak was de bespreking van WhatsApp alternatieven, en met name Telegram. Ongeveer dezelfde functionaliteit en het werkte ook nog via een webclient en er was zelfs een desktop app. Ik was om… ik kon eindelijk “WhatsAppen” met het comfort van mijn laptop/desktop. Het was niet dat WhatsAp het niet meer deed, slecht zou zijn, het was gewoon ingehaald door iets beters.

Groot nieuws deze week! WhatsApp heeft eindelijk ook een webclient! Ehhhh… zeggen ze tenminste. Ik heb mazzel… de nieuwste versie van WhatsApp op mijn telefoon… ergens in een hoekje verstopt op mijn laptop staat ook nog Chrome (gebruik dat ding eigenlijk niet, vrijheid van keuze, je weet wel). En verdomd het werkt. Maar je zal maar geen ondersteunde versie van WhatsApp hebben, Chrome niet hebben. Wat dan?

Toch maar weer Telegram?

Collega van me heeft er een leuk blog over gevonden.  De webclient van WhatsApp? Er mankeert nogal wat aan.