Tag Archives: webserver

geek howto

HOWTO sort web-server logs to find top users

March 31, 2010 – 11:07 am

The other day I came across a situation where a web-server was getting hammered, and we needed to know who the offend(ers) were. While watching a logfile tail by at high speeds is always fun, we wanted to be able to sort the web-server access log and find top users, to be able to narrow down where the traffic was coming from. While we don’t want to block users that want to access our data, sometimes we need to throttle things back so one requester doesn’t overwhelm all the available bandwidth and make the site unusable for others. So after some playing around and digging on Google, we came up with a nice, succinct one liner to do this, here it is:

cat /path/to/access.log | awk '{print $1}' | sort | uniq -c | sort -n | tail
This is a preview of HOWTO sort web-server logs to find top users. Read the full post (161 words, 1 image, estimated 39 secs reading time)
Tags , , , , , , , , , , , , , , | Permalink | Comments (5)
geek howto

HOWTO: webserver in 100 lines of Bash

September 4, 2008 – 6:48 pm

I’m a big Bash fan, I know Perl is the more popular scripting language, and I’m slowly using it more, but hey, if I need something done, I can do it quicker in Bash (keeping in mind that I’m a systems guy, not a dev guy). While at work looking up Bash related syntax I came across a page describing how to run a webserver with 100 lines of Bash. It uses the old school GNU utility Netcat (nc) for communication between the pipes, and just a ton of basic logic and functions to pass it on to the user. It’s one of those things I look at and can’t believe it works, but it does. Of course security is unknown, as is the original author, but I consider this a reference on how to do networking things in Bash; who knows what I’ll use (parts) of it for. If anyone has details on who originally wrote this I’m all ears.[sourcecode language='xml']#!/bin/bash

This is a preview of HOWTO: webserver in 100 lines of Bash. Read the full post (395 words, 1 image, estimated 1:35 mins reading time)
Tags , , , , , , , , , | Permalink | Comments (8)
geek linux

HOWTO: Configure nginx for Debian / Ubuntu

May 5, 2008 – 12:20 pm

nginxUPDATE: I’m reworking my config blending in the security ideas found on camomel.org they’re really thought things through on this, this should make for a very secure environment.

This is a preview of HOWTO: Configure nginx for Debian / Ubuntu. Read the full post (349 words, 1 image, estimated 1:24 mins reading time)
Tags , , , , , | Permalink | Comments (12)
geek howto

HOWTO: determine optimal fastcgi settings for Lighttpd

February 28, 2008 – 11:50 am

PHP Fast-cgiAnyone building a server with a LAMP stack today has tons of options, mine have evolved to using Varnish -> Lighttpd -> Xcache -> PHP5 -> MySQL. Once I had Lighttpd (aka Lighty) installed and running PHP pages I looked to optimize the configuration and push it as hard as possible for more speed. Of course lately I’ve been getting unexplained slowdowns, with many instances of php5-cgi appearing to be taking up almost all of my available CPU on `top`. Reading up on things it appears that I had max_procs, along with PHP_FCGI_CHILDREN, set far too high for the load I’m getting. When you start lighty it gives you the number of processes you’ve define, and then those in turn spawn the number of children you’ve specified. While my settings were too high, they were really overshooting things when you take into account that I’m using Xcache (which provides PHP pre-caching) and Varnish (for HTTP acceleration). So even though one of my dynamic sites that I’m working on to ‘monetize’ things is getting 700-800 hits each day, my caching strategy is taking the load away from the ever available Lighty. Because of this, lighty has much less to do, so giving it a ton of processes to just sit there and eat memory until they’re zombified is a waste. After reading the lighty FAQ and other posts specific to this, I’ve settled on the this for my fastcgi config block within my lighttpd.conf file.

This is a preview of HOWTO: determine optimal fastcgi settings for Lighttpd. Read the full post (325 words, 1 image, estimated 1:18 mins reading time)
Tags , , , , , , , , | Permalink | Comment (0)