look out honey 'cause I'm using technology

Posts Tagged ‘webserver’

HOWTO: webserver in 100 lines of Bash

I’m a big Bash fan, I know Perl is the more popular scripting language, and I’m slowly using it more, but hey, if I need something done, I can do it quicker in Bash (keeping in mind that I’m a systems guy, not a dev guy). While at work looking up Bash related syntax I came across a page describing how to run a webserver with 100 lines of Bash. It uses the old school GNU utility Netcat (nc) for communication between the pipes, and just a ton of basic logic and functions to pass it on to the user. It’s one of those things I look at and can’t believe it works, but it does. Of course security is unknown, as is the original author, but I consider this a reference on how to do networking things in Bash; who knows what I’ll use (parts) of it for. If anyone has details on who originally wrote this I’m all ears.[sourcecode language='xml']#!/bin/bash

function debug {
local severity=”$1″
shift
local message=”$@”

echo -n “`date -u`” 1>&2
echo -ne ‘\t’ 1>&2
echo -n “$severity” 1>&2
echo -ne ‘\t’ 1>&2
echo “$message” 1>&2
}

function fix_path {
echo -n “$1″ | head -n 1 | sed ’s|^[/.-]*||’ | sed ’s|/\.*|/|g’
}

function serve_dir {
local dir=”`fix_path “$1″`”
if [ "$dir" = "" ]; then
dir=”./”
fi
echo ‘HTTP/1.1 200 OK’
echo ‘Content-type: text/html;charset=UTF-8′
echo
echo LISTING “$dir”
echo ‘

ls -p “$dir” | sed -e ’s|^\(.*\)$|\1
|’
}

function serve_file {
echo ‘HTTP/1.1 200 OK’
echo ‘Content-type: application/x-download-this’
echo
local file=”`fix_path “$1″`”
debug INFO serving file “$file”
cat “$file”
}

function process {
local url=”`gawk ‘{print $2}’ | head -n 1`”
case “$url” in
*/)
debug INFO Processing “$url” as dir
serve_dir “$url”
break
;;
*)
debug INFO Processing “$url” as file
serve_file “$url”
;;
esac
}

function serve {
local port=”$1″
local sin=”$2″
local sout=”$3″

while debug INFO Running nc; do

nc -l -p “$port” < "$sin" > “$sout” &amp;
pid=”$!”

debug INFO Server PID: “$pid”

trap cleanup SIGINT
head -n 1 “$sout” | process > “$sin”
trap – SIGINT

debug INFO Killing nc

kill “$pid”
done

debug INFO Quiting server
}

function cleanup {
debug INFO Caught signal, quitting…
rm -Rf “$tmp_dir”
exit
}

tmp_dir=”`mktemp -d -t http_server.XXXXXXXXXX`”
sin=”$tmp_dir”/in
sout=”$tmp_dir”/out
pid=0
port=”$1″

mkfifo “$sin”
mkfifo “$sout”

debug INFO Starting server on port “$port”
serve “$port” “$sin” “$sout”
cleanup[/sourcecode]


HOWTO: Configure nginx for Debian / Ubuntu

nginxUPDATE: I’m reworking my config blending in the security ideas found on camomel.org they’re really thought things through on this, this should make for a very secure environment.

I’m always trying new software, and with the webserver I’ve moved from Apache 1.3 to 2.0 to 2.2, and then later I moved everything over to Lighttpd, which I’ve liked, save for some memory issues that popped up.  Now, enter a web server named nginx (engine x), written by a Russian hacker. It’s already proved it’s meddle by running some of the largest Russian sites for years now.  It has the speed of Lighttpd, but with none of that memory weirdness, plus it uses a fraction of the CPU, so scaling should be smooth for highly visited sites.  It also does cool things like load balancing, reverse proxy, IMAP and POP proxy, etc, so I can see it being used in a variety of ways on a network.  It took me some time to understand how to configure it, which was a case of me just making it harder than it really is, so I wanted to post it here.  Look for updates as we go along, but this is currently backing a Production site I manage.

user					www-data www-data;
worker_processes  			5;
pid 					/var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
include				/etc/nginx/mime.types;
default_type			application/octet-stream;
log_format main 		'$remote_addr $host $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" "$http_user_agent" '
'"$request_time" "$gzip_ratio"';
access_log			/var/log/nginx/access.log  main;
error_log			/var/log/nginx/error.log;
sendfile 			on;
tcp_nopush        		on;
tcp_nodelay      		off;
keepalive_timeout		65;
gzip				on;
gzip_http_version		1.1;
gzip_vary			on;
gzip_comp_level 		6;
gzip_buffers			16 8k;
#gzip_proxied			expired no-cache no-store private auth;
gzip_proxied 			any;
gzip_min_length			1000;
gzip_types			text/plain text/html text/css application/json application/x-javascript
text/xml application/xml application/xml+rss text/javascript;
server {
listen			80;
client_max_body_size	50M;
server_name 		server.domain.com;
root 			/var/www;
index  			index.html index.php;
access_log  		/var/log/nginx/access.log  main;
error_page   		500 502 503 504  /500.html;
location = /500.html {
root		/var/www;
}
location ~* ^.+.(jpg|jpeg|gif)$ {
root		/var/www;
expires         30d;
}
location ~ \.php$ {
include /etc/nginx/fastcgi_params;
fastcgi_pass 127.0.0.1:9000;
fastcgi_index index.php;
fastcgi_param SCRIPT_FILENAME /var/www$fastcgi_script_name;
fastcgi_param QUERY_STRING $query_string;
fastcgi_param REQUEST_METHOD $request_method;
fastcgi_param CONTENT_TYPE $content_type;
fastcgi_param CONTENT_LENGTH $content_length;
}
}
}

HOWTO: determine optimal fastcgi settings for Lighttpd

PHP Fast-cgiAnyone building a server with a LAMP stack today has tons of options, mine have evolved to using Varnish -> Lighttpd -> Xcache -> PHP5 -> MySQL. Once I had Lighttpd (aka Lighty) installed and running PHP pages I looked to optimize the configuration and push it as hard as possible for more speed. Of course lately I’ve been getting unexplained slowdowns, with many instances of php5-cgi appearing to be taking up almost all of my available CPU on `top`. Reading up on things it appears that I had max_procs, along with PHP_FCGI_CHILDREN, set far too high for the load I’m getting. When you start lighty it gives you the number of processes you’ve define, and then those in turn spawn the number of children you’ve specified. While my settings were too high, they were really overshooting things when you take into account that I’m using Xcache (which provides PHP pre-caching) and Varnish (for HTTP acceleration). So even though one of my dynamic sites that I’m working on to ‘monetize’ things is getting 700-800 hits each day, my caching strategy is taking the load away from the ever available Lighty. Because of this, lighty has much less to do, so giving it a ton of processes to just sit there and eat memory until they’re zombified is a waste. After reading the lighty FAQ and other posts specific to this, I’ve settled on the this for my fastcgi config block within my lighttpd.conf file. (more…)


We like








We support


EFF - Electronic Frontier Foundation       TOR - The Onion Router       HRC - Human Rights Campaign