HOWTO: webserver in 100 lines of Bash
I’m a big Bash fan, I know Perl is the more popular scripting language, and I’m slowly using it more, but hey, if I need something done, I can do it quicker in Bash (keeping in mind that I’m a systems guy, not a dev guy). While at work looking up Bash related syntax I came across a page describing how to run a webserver with 100 lines of Bash. It uses the old school GNU utility Netcat (nc) for communication between the pipes, and just a ton of basic logic and functions to pass it on to the user. It’s one of those things I look at and can’t believe it works, but it does. Of course security is unknown, as is the original author, but I consider this a reference on how to do networking things in Bash; who knows what I’ll use (parts) of it for. If anyone has details on who originally wrote this I’m all ears.[sourcecode language='xml']#!/bin/bash
function debug {
local severity=”$1″
shift
local message=”$@”
echo -n “`date -u`” 1>&2
echo -ne ‘\t’ 1>&2
echo -n “$severity” 1>&2
echo -ne ‘\t’ 1>&2
echo “$message” 1>&2
}
function fix_path {
echo -n “$1″ | head -n 1 | sed ’s|^[/.-]*||’ | sed ’s|/\.*|/|g’
}
function serve_dir {
local dir=”`fix_path “$1″`”
if [ "$dir" = "" ]; then
dir=”./”
fi
echo ‘HTTP/1.1 200 OK’
echo ‘Content-type: text/html;charset=UTF-8′
echo
echo LISTING “$dir”
echo ‘
‘
ls -p “$dir” | sed -e ’s|^\(.*\)$|\1
|’
}
function serve_file {
echo ‘HTTP/1.1 200 OK’
echo ‘Content-type: application/x-download-this’
echo
local file=”`fix_path “$1″`”
debug INFO serving file “$file”
cat “$file”
}
function process {
local url=”`gawk ‘{print $2}’ | head -n 1`”
case “$url” in
*/)
debug INFO Processing “$url” as dir
serve_dir “$url”
break
;;
*)
debug INFO Processing “$url” as file
serve_file “$url”
;;
esac
}
function serve {
local port=”$1″
local sin=”$2″
local sout=”$3″
while debug INFO Running nc; do
nc -l -p “$port” < "$sin" > “$sout” &
pid=”$!”
debug INFO Server PID: “$pid”
trap cleanup SIGINT
head -n 1 “$sout” | process > “$sin”
trap – SIGINT
debug INFO Killing nc
kill “$pid”
done
debug INFO Quiting server
}
function cleanup {
debug INFO Caught signal, quitting…
rm -Rf “$tmp_dir”
exit
}
tmp_dir=”`mktemp -d -t http_server.XXXXXXXXXX`”
sin=”$tmp_dir”/in
sout=”$tmp_dir”/out
pid=0
port=”$1″
mkfifo “$sin”
mkfifo “$sout”
debug INFO Starting server on port “$port”
serve “$port” “$sin” “$sout”
cleanup[/sourcecode]
HOWTO: Configure nginx for Debian / Ubuntu
UPDATE: I’m reworking my config blending in the security ideas found on camomel.org they’re really thought things through on this, this should make for a very secure environment.
I’m always trying new software, and with the webserver I’ve moved from Apache 1.3 to 2.0 to 2.2, and then later I moved everything over to Lighttpd, which I’ve liked, save for some memory issues that popped up. Now, enter a web server named nginx (engine x), written by a Russian hacker. It’s already proved it’s meddle by running some of the largest Russian sites for years now. It has the speed of Lighttpd, but with none of that memory weirdness, plus it uses a fraction of the CPU, so scaling should be smooth for highly visited sites. It also does cool things like load balancing, reverse proxy, IMAP and POP proxy, etc, so I can see it being used in a variety of ways on a network. It took me some time to understand how to configure it, which was a case of me just making it harder than it really is, so I wanted to post it here. Look for updates as we go along, but this is currently backing a Production site I manage.
user www-data www-data;
worker_processes 5;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr $host $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" "$http_user_agent" '
'"$request_time" "$gzip_ratio"';
access_log /var/log/nginx/access.log main;
error_log /var/log/nginx/error.log;
sendfile on;
tcp_nopush on;
tcp_nodelay off;
keepalive_timeout 65;
gzip on;
gzip_http_version 1.1;
gzip_vary on;
gzip_comp_level 6;
gzip_buffers 16 8k;
#gzip_proxied expired no-cache no-store private auth;
gzip_proxied any;
gzip_min_length 1000;
gzip_types text/plain text/html text/css application/json application/x-javascript
text/xml application/xml application/xml+rss text/javascript;
server {
listen 80;
client_max_body_size 50M;
server_name server.domain.com;
root /var/www;
index index.html index.php;
access_log /var/log/nginx/access.log main;
error_page 500 502 503 504 /500.html;
location = /500.html {
root /var/www;
}
location ~* ^.+.(jpg|jpeg|gif)$ {
root /var/www;
expires 30d;
}
location ~ \.php$ {
include /etc/nginx/fastcgi_params;
fastcgi_pass 127.0.0.1:9000;
fastcgi_index index.php;
fastcgi_param SCRIPT_FILENAME /var/www$fastcgi_script_name;
fastcgi_param QUERY_STRING $query_string;
fastcgi_param REQUEST_METHOD $request_method;
fastcgi_param CONTENT_TYPE $content_type;
fastcgi_param CONTENT_LENGTH $content_length;
}
}
}HOWTO: determine optimal fastcgi settings for Lighttpd
Anyone building a server with a LAMP stack today has tons of options, mine have evolved to using Varnish -> Lighttpd -> Xcache -> PHP5 -> MySQL. Once I had Lighttpd (aka Lighty) installed and running PHP pages I looked to optimize the configuration and push it as hard as possible for more speed. Of course lately I’ve been getting unexplained slowdowns, with many instances of php5-cgi appearing to be taking up almost all of my available CPU on `top`. Reading up on things it appears that I had max_procs, along with PHP_FCGI_CHILDREN, set far too high for the load I’m getting. When you start lighty it gives you the number of processes you’ve define, and then those in turn spawn the number of children you’ve specified. While my settings were too high, they were really overshooting things when you take into account that I’m using Xcache (which provides PHP pre-caching) and Varnish (for HTTP acceleration). So even though one of my dynamic sites that I’m working on to ‘monetize’ things is getting 700-800 hits each day, my caching strategy is taking the load away from the ever available Lighty. Because of this, lighty has much less to do, so giving it a ton of processes to just sit there and eat memory until they’re zombified is a waste. After reading the lighty FAQ and other posts specific to this, I’ve settled on the this for my fastcgi config block within my lighttpd.conf file. (more…)
Hi, I'm fak3r and I've been online for over 15 years, blogging for 10. I work with open source software, various hardware and whatever else that I can hack on. I enjoy learning by doing, painting infrequently and listening to more music than the law allows. Feel free to follow me and comment on my thoughts and ideas; we're all in it together kid.
