look out honey 'cause I'm using technology

Posts Tagged ‘bash’

HOWTO monitor your servers via Twitter

Alert: your server has failed!

The other day I got inspired to write a script that would allow me to monitor my servers via Twitter. The idea of having a column in Tweetdeck set aside to inform me of my servers’ statuses’ sounded cool, plus, it’s quicker than checking email.  I know sending tweets from the command-line had been done before, but after seeing briealeida’s post titled Tweeting Cron Jobs I really got inspired. While hers was written in Perl, I didn’t want to go that route since I had a few, self imposed, restrictions I wanted to stick to. One, I only wanted to use standard shell commands, the ones you get by default in Linux, so you would have absolutely no dependencies to install for this to work. Two, I wanted to see how much info I could stuff into a 140 character tweet, and still have it make sense. While I’m still working on adding more info, the current state of the script gives me a quick snapshot of seven specifics metrics on a selected server, which I’m quite happy with. To try it yourself only takes a few minutes. (more…)


HOWTO sort web-server logs to find top users

Wario is being greedy...

The other day I came across a situation where a web-server was getting hammered, and we needed to know who the offend(ers) were. While watching a logfile tail by at high speeds is always fun, we wanted to be able to sort the web-server access log and find top users, to be able to narrow down where the traffic was coming from. While we don’t want to block users that want to access our data, sometimes we need to throttle things back so one requester doesn’t overwhelm all the available bandwidth and make the site unusable for others. So after some playing around and digging on Google, we came up with a nice, succinct one liner to do this, here it is:

cat /path/to/access.log | awk '{print $1}' | sort | uniq -c | sort -n | tail

(more…)


HOWTO: send commandline email with attachments

You've got mail!

Are you like me, do you have scripts running on servers and you need to know what they know? If there’s output in a file you can sed/grep/awk info out of them and have them emailed to you, but if you don’t know specifically what you’re looking for you may need the entire file/log/whatever. You’ll need a utility called uuencode, which is a utility that, (more…)


HOWTO: webserver in 100 lines of Bash

I’m a big Bash fan, I know Perl is the more popular scripting language, and I’m slowly using it more, but hey, if I need something done, I can do it quicker in Bash (keeping in mind that I’m a systems guy, not a dev guy). While at work looking up Bash related syntax I came across a page describing how to run a webserver with 100 lines of Bash. It uses the old school GNU utility Netcat (nc) for communication between the pipes, and just a ton of basic logic and functions to pass it on to the user. It’s one of those things I look at and can’t believe it works, but it does. Of course security is unknown, as is the original author, but I consider this a reference on how to do networking things in Bash; who knows what I’ll use (parts) of it for. If anyone has details on who originally wrote this I’m all ears.[sourcecode language='xml']#!/bin/bash

function debug {
local severity=”$1″
shift
local message=”$@”

echo -n “`date -u`” 1>&2
echo -ne ‘\t’ 1>&2
echo -n “$severity” 1>&2
echo -ne ‘\t’ 1>&2
echo “$message” 1>&2
}

function fix_path {
echo -n “$1″ | head -n 1 | sed ‘s|^[/.-]*||’ | sed ‘s|/\.*|/|g’
}

function serve_dir {
local dir=”`fix_path “$1″`”
if [ "$dir" = "" ]; then
dir=”./”
fi
echo ‘HTTP/1.1 200 OK’
echo ‘Content-type: text/html;charset=UTF-8′
echo
echo LISTING “$dir”
echo ‘

ls -p “$dir” | sed -e ‘s|^\(.*\)$|\1
|’
}

function serve_file {
echo ‘HTTP/1.1 200 OK’
echo ‘Content-type: application/x-download-this’
echo
local file=”`fix_path “$1″`”
debug INFO serving file “$file”
cat “$file”
}

function process {
local url=”`gawk ‘{print $2}’ | head -n 1`”
case “$url” in
*/)
debug INFO Processing “$url” as dir
serve_dir “$url”
break
;;
*)
debug INFO Processing “$url” as file
serve_file “$url”
;;
esac
}

function serve {
local port=”$1″
local sin=”$2″
local sout=”$3″

while debug INFO Running nc; do

nc -l -p “$port” < "$sin" > “$sout” &amp;
pid=”$!”

debug INFO Server PID: “$pid”

trap cleanup SIGINT
head -n 1 “$sout” | process > “$sin”
trap – SIGINT

debug INFO Killing nc

kill “$pid”
done

debug INFO Quiting server
}

function cleanup {
debug INFO Caught signal, quitting…
rm -Rf “$tmp_dir”
exit
}

tmp_dir=”`mktemp -d -t http_server.XXXXXXXXXX`”
sin=”$tmp_dir”/in
sout=”$tmp_dir”/out
pid=0
port=”$1″

mkfifo “$sin”
mkfifo “$sout”

debug INFO Starting server on port “$port”
serve “$port” “$sin” “$sout”
cleanup[/sourcecode]