Man in the middle

You typically hear the expression man in the middle in the context of an attack where someone is actively eavesdropping on communications that are intended to be private. However, it is also an invaluable tool for debugging network programs. One of my favorite tools is netcat and this tool makes it trivial to implement a simple eavesdropping script. This can also be done with tools such as tcpdump, but I find that netcat is a bit simpler for most tasks and it is more likely to be available on the machine.

The script I typically use is shown below. Essentially it has an endless loop that listens on a port, tees the data that comes in to a request log file, sends the input to a remote server, tees the response from the remote server to a response log, and then writes the data back to a named pipe that is connected to stdin on the netcat process that was listening.

#!/bin/bash

function logFile {
   echo "$(date +%Y-%m-%d-%H-%M-%S).${1}.log"
}

function serveRequests {
   port=$1
   remoteHost=$2
   remotePort=$3
   while true; do
       rm -f backpipe
       mkfifo backpipe
       cat backpipe |
           nc -l $port |
           tee -a $(logFile request) |
           nc $remoteHost $remotePort |
           tee -a $(logFile response) >backpipe
   done
}

port=${1:-12345}
remoteHost=${2:-localhost}
remotePort=${3:-80}
serveRequests $port $remoteHost $remotePort

Parallel and Bar

I recently discovered two UNIX command line tools that are really useful. For both tools, I had simple solutions that I've been using for a long time to provide similar functionality. The first tool is GNU Parallel. As the name implies, this tool provides a way to run many commands in parallel. Whats more is that it works in a manner that is consistent with xargs. To get an idea of all the cool things you can do with this tool, I highly recommend looking through the examples in the man page.

Before finding GNU Parallel, I used a simple shell script to accomplish similar tasks. Of course, my version is much more limited in what it can do, it reads one command per line from stdin and runs it in the background. The number of parallel jobs is controlled by using the bash jobs builtin to determine how many tasks are already running. The complete script is shown below.

#!/bin/bash

# Print usage
function usage {
    local prg=`basename $0`
    cat <<USAGE

Usage: $prg [options]

    Options:
    -h                   Print this help message
    -v                   Be verbose with log messages
    -n <tasks>           The number of tasks to run at a time.

USAGE
    exit 1
}

# Check arguments
numProcs=4
verbose="false"
while getopts "n:vh" option; do
    case $option in
        (n) numProcs="$OPTARG" ;;
        (v) verbose="true" ;;
        (h) usage ;;
    esac
done
shift $(($OPTIND-1))

# Read commands from stdin
while read line; do

    # Can we run another?
    while [ "$(jobs -r | wc -l)" -ge "$numProcs" ]; do
        sleep 1
    done

    # Run task
    sh -c "$line" &
done

# Wait for jobs to finish
wait

The second command is the command line progress bar. This command writes progress information to stderr while copying data from stdin to stdout. If you ever need to copy a big file and want some basic stats so you can see that things are still working, this is the tool for you. The output and options are certainly much nicer than the crude C program I was using:

#include <stdio.h>
#include <stdlib.h>
#include <inttypes.h>
#include <unistd.h>

#define PAGESIZE 4096
#define STDIN    0
#define STDOUT   1

int
main() {
    char buffer[PAGESIZE];
    ssize_t length;
    int counter;
    uint64_t bytesWritten = 0;

    while ((length = read(STDIN, buffer, PAGESIZE)) > 0) {
        write(STDOUT, buffer, length);
        bytesWritten += length;
        ++counter;
        if (counter % 10 == 0) {
            fprintf(stderr, "%"PRIu64" bytes\r", bytesWritten);
        }
    }
    fprintf(stderr, "%"PRIu64" bytes\n", bytesWritten);

    return EXIT_SUCCESS;
}

Damn Java Socket Exception Messages

When creating an error message you should think about what information would be useful for understanding what went wrong. This should especially be true if you are creating a library that is likely to be used by many other systems. Providing good error messages up front means that even if the programmers using the library do not check and customize the messages, the user will still get a reasonable result. For some use cases, such as scripting, it can also be useful because it may be a quick one-off program where the goal is to quickly automate a repetitive task or perform some analysis. In this case, having useful default error messages can speed up the initial development so you get your answer faster.

In my case, I was checking logs for a system that crawls pages and hence attempts to resolve and connect to thousands of hosts. This system logs the exceptions, but unfortunately did not provide a customized message when doing so. Analyzing the logs showed that the two most common error messages were: 1) a failure to resolve the hostname and 2) failing to connect to an HTTP server on the host. An example error message for the first case is java.net.UnknownHostException: some-host-that-does-not-exist. This message is quite useful as the exception name explains the problem and the message tells me the name of the host that could not be resolved. An example message for the second case is java.net.SocketTimeoutException: connect timed out. This message is explains the problem, but doesn't given me the crucial information of what it was trying to connect to.

Though this can easily be fixed in the application code, it is disappointing that the default message is so bad. I have noticed that my opinion of a programming language or technology seems to go down steadily the longer I am forced to use it at work. Is the grass greener on one of the other sides? How do other languages, or rather the networking libraries they provide, fair for this use case? I looked at 13 options to see how many would give a decent error message for both use cases. The results were not very encouraging. Only one option, Go, had reasonable messages for both. For the host not found case 4 options included the hostname. Only two options provided the host and port in the failed to connect case. The results are summarized in the table below the fold with links to the source code and raw error messages.

Closing Arbitrary File Descriptors in Bash

While writing a post on file descriptors, I ended up on a tangent trying to answer a question that seemed like it should be simple and obvious. How do you close arbitrary file descriptors in a bash shell script? The chapter on I/O Redirection in the Advanced Bash-Scripting Guide shows the typical examples that you see everywhere. All of these examples use a hardcoded number to identify the descriptor, not a variable. Try to use a variable and bash gets confused thinking the value is the name of a command you wish to execute. For example:

#!/bin/bash
num=5
exec $num>&-

When you run it yields:

$ ./test.sh 
./test.sh: line 3: exec: 5: not found

I eventually found the answer on another blog that had a post about daemonizing bash. The technique is to use the built-in eval command. The variable will be substituted in a string and the string will get executed by eval. It is kind of obvious once I saw how someone else had done it, but for whatever reason it did not occur to me. Since I like examples, here is a script with a closefd function for closing file descriptors:

#!/bin/bash

function closefd {
    local fd=$1
    echo "closing $fd"
    eval "
        exec $fd>&-
        exec $fd<&-
    "
}

function listfds {
    local pid=$1
    local label=$2
    ls /proc/$pid/fd | sort -n | awk '{printf("%s ", $1)}' | sed "s/^/$label: /"
    echo ""
}

# Record the PID of this shell, need to make sure we are tracking fds of this
# shell and not some forked child
pid=$$

# Run commands
listfds $pid "before"
closefd 5
listfds $pid "middle"
closefd 13
listfds $pid "after"

You can run this script with a simple wrapper that opens up a bunch of files that will be inherited when a new process is forked. In my case I used the testclose C program from the forking file descriptors post after removing the call to closefrom. Running the script shows output like:

$ ./testclose closefd.sh
opened 20 files
before: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 255 
closing 5
middle: 0 1 2 3 4 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 255 
closing 13
after: 0 1 2 3 4 6 7 8 9 10 11 12 14 15 16 17 18 19 20 21 22 255 
child 20710 exited with status 0