Friday, August 27, 2010
Saturday, July 31, 2010
Miracle of the Herrings
Not being Catholic, I found it strange that the church would bother making such a ridiculous claim. Why not just celebrate Thomas Aquinas for what he did? What is the point of making up some pathetic story and labeling it a miracle? I was curious if this was just made up for television so I tried to find (i.e., searched for a few minutes online, not a serious scholarly effort) an official document from the church on why Thomas Aquinas was declared a saint. I didn't have much luck finding what I was looking for on the www.vatican.va website. Many results came up for Thomas Aquinas, but not what I was looking for.
The best reference I was able to find was The Sanctity and Miracles of St. Thomas Aquinas, but I have no idea about the veracity of the source. It does partially corroborate the story about the herrings:
Asked about miracles - whether he knew of any worked through the merits of Thomas either before or after death - the witness said that when Thomas died his body was buried at first before the high altar, but then the monks, fearing it might be taken from them, transferred it secretly to St. Stephen's Chapel in the same abbey-church. But about seven months later Thomas appeared in a dream to a brother James, who was prior at the time, and said:'Take me back where I was at first.' So they took him back, with due solemnity. (This dream was and still is commonly talked about in the monastery.) And when the tomb was opened a delicious fragrance came out, filling all the chapel and cloister: whereupon the community sang the Mass Os justi meditabitur sapientiam, etc., in honour of Thomas as of a saint; they thought the Mass Pro defunctis hardly suitable for such a man.In this story it explicitly states that Thomas Aquinas did not eat the fish as Stephen Fry stated in the video. However, the book Saint Thomas Aquinas: the person and his work (by Jean-Pierre Torrell, the chapter "The Last Months and Death" on page 291 in the version scanned by Google) suggests he may have eaten some of it:
All this the witness knew because he was there and saw it for himself; it happened about seven months after Thomas's death; but he could not be sure of the month or the day. Asked who were present, he said 'the whole community'.... Asked who had called him to the place where the fragrance was smelt, he said he himself smelled it; it drew him to where the tomb was.
IX. Asked if he knew of other miracles attributed to brother Thomas, the witness said that he had heard of many; and in particular that when Thomas lay sick in the castle of Maenza and was urged to eat something, he answered, 'I would eat fresh herrings, if I had some.' Now it happened that a pedlar called just then with salted fish. He was asked to open his baskets, and one was found full of fresh herrings, though it had contained only salted fish. But when the herrings were brought to Thomas, he would not eat them.
The witness spoke too of a Master Reginald, a cripple, who was cured at the tomb of brother Thomas.
Asked how he knew of these two miracles, he replied that that about the fish he had from brother William of Tocco, prior of the Friar Preachers at Benevento, who himself had it from several people at Maenza, where the event occurred. The other story he had from brother Octavian (mentioned above) who averred that he had seen it happen. And in the monastery these miracles were common knowledge.
It was there that he fell ill and totally lost his appetite; the doctor called to take care of him—John of Guido, from Piperno—asked what he would like to eat and received a disconcerting response: some fresh herring, which he once enjoyed when he was in the Ile de France. Miraculously, some were found. But according to Tocco, it was the others who ate them, since the patient no longer wanted them. An eyewitness assures us, however, that he ate some of it: de quibus etiam arengis comedit dictus frater Thomas.History is messy. But it doesn't really make a difference whether or not he ate some of the fish in terms of the miracle. A NY Times book review When the Lights Went Out in Europe gives a similar story about the herrings:
When St. Thomas Aquinas lay dying, in 1274, it was said that he asked for herrings, which were unknown thereabouts. Yet sure enough they soon obligingly turned up at the local fishmongers. Even in the early 14th century, when Thomas's candidacy for sainthood was under investigation, and at least two miracles were required for admission, this unlikely tale did not wash -- not least because it emerged that the witnesses had no way of telling whether what they had seen were herrings or not.Though I would prefer an official document from the Catholic church, from what I can tell the miracle of the herrings is a real claim made to support the canonization of Thomas Aquinas. It would be nice if the Catholic church had an easily searchable database of all the saints and the records for how they qualified for sainthood. However, if all of the "miracles" are this pathetic, then it is probably better for public relations not to make the information more accessible.
Friday, July 30, 2010
Damn boost::program_options
I was dismayed by the awful help screen on one of our internal tools. The help message looked something like:
Of course, this was an internal tool that is no longer being maintained. Poking at the source I found it was based on boost::program_options and was just writing the options_description to stdout. I found that troubling as I have used and recommended this library many times. The root of the problem seemed to be that the default value was based on a file on the system specified by an environment variable. On my system that path is quite long and there was only room for a description that was one character wide. I put together a quick test program to illustrate the problem:
However, to my surprise when I ran the test program I could not reproduce the issue. The output when I supply a large default value was still sane, it just wraps the description to the next line. For example:
A little research showed the problem was fixed in boost 1.42.0. And sure enough, if I try with boost 1.41.0, like what the internal tool was using, I can easily reproduce the problem:
And luckily for me, there isn't any work to fix the tool other than bump the version of boost it depends on.
$ ./a.out -h Allowed options: -h [ --help ] s h ... skipping ... e d -c [ --config ] arg (=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa) c o n f i g u r a t i o n f i l e
// g++ -I/opt/local/include -L/opt/local/lib -lboost_program_options-mt boostopt.cpp
#include <cstdlib>
#include <iostream>
#include <string>
#include <boost/program_options.hpp>
using namespace std;
using namespace boost::program_options;
int
main(int argc, char **argv) {
char *file = getenv("CONFIG");
string config((file == NULL) ? "-" : file);
options_description desc("Allowed options");
desc.add_options()
("help,h", "show this help message, some additional text "
"that is here for no other reason than to make "
"the message wrap when the help is printed")
("config,c", value<string>(&config)->default_value(config),
"configuration file")
;
variables_map vm;
try {
store(parse_command_line(argc, argv, desc), vm);
notify(vm);
} catch (const std::exception &e) {
cerr << "Error: " << e.what() << endl
<< endl << desc << endl;
return 1;
}
if (vm.count("help")) {
cerr << desc << endl;
return 2;
}
return 0;
}
$ env CONFIG=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa ./a.out -h
Allowed options:
-h [ --help ] show this help message, some additional
text that is here for no other reason
than to make the message wrap when the
help is printed
-c [ --config ] arg (=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa)
configuration file
$ env CONFIG=aaa ./a.out -h
Allowed options:
-h [ --help ] show this help message, some additional text that
is here for no other reason than to make the
message wrap when the help is printed
-c [ --config ] arg (=aaa) configuration file
$ env CONFIG=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa ./a.out -h
Allowed options:
-h [ --help ] sh
ow
... skipping ...
te
d
-c [ --config ] arg (=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa) co
nf
ig
ur
at
io
n
fi
le
$ env CONFIG=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa ./a.out -h
Allowed options:
-h [ --help ] s
h
... skipping ...
e
d
-c [ --config ] arg (=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa) c
o
n
f
i
g
u
r
a
t
i
o
n
f
i
l
e
Sunday, July 25, 2010
Closing Arbitrary File Descriptors in Bash
While writing a post on file descriptors, I ended up on a tangent trying to answer a question that seemed like it should be simple and obvious. How do you close arbitrary file descriptors in a bash shell script? The chapter on I/O Redirection in the Advanced Bash-Scripting Guide shows the typical examples that you see everywhere. All of these examples use a hardcoded number to identify the descriptor, not a variable. Try to use a variable and bash gets confused thinking the value is the name of a command you wish to execute. For example:
When you run it yields:
I eventually found the answer on another blog that had a post about daemonizing bash. The technique is to use the built-in eval command. The variable will be substituted in a string and the string will get executed by eval. It is kind of obvious once I saw how someone else had done it, but for whatever reason it did not occur to me. Since I like examples, here is a script with a closefd function for closing file descriptors:
You can run this script with a simple wrapper that opens up a bunch of files that will be inherited when a new process is forked. In my case I used the testclose C program from the forking file descriptors post after removing the call to closefrom. Running the script shows output like:
#!/bin/bash num=5 exec $num>&-
$ ./test.sh ./test.sh: line 3: exec: 5: not found
#!/bin/bash function closefd { local fd=$1 echo "closing $fd" eval " exec $fd>&- exec $fd<&- " } function listfds { local pid=$1 local label=$2 ls /proc/$pid/fd | sort -n | awk '{printf("%s ", $1)}' | sed "s/^/$label: /" echo "" } # Record the PID of this shell, need to make sure we are tracking fds of this # shell and not some forked child pid=$$ # Run commands listfds $pid "before" closefd 5 listfds $pid "middle" closefd 13 listfds $pid "after"
$ ./testclose closefd.sh opened 20 files before: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 255 closing 5 middle: 0 1 2 3 4 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 255 closing 13 after: 0 1 2 3 4 6 7 8 9 10 11 12 14 15 16 17 18 19 20 21 22 255 child 20710 exited with status 0
Forking File Descriptors
When create a new process with a call to fork, one thing the child process will retain is a copy of the parent's open file descriptors:
This method is simple and it basically works, but it bothers me that it is closing lots of file descriptors that aren't open. More importantly though, with this implementation we cannot determine if the close of some descriptors failed. So here is an improved version that checks the error codes and returns -1 on failure and the number of closed descriptors on success:
Notice that we are still closing file descriptors that are not open. I'm not aware of a simple and portable scheme for listing the set of open descriptors for a process so I'll live with the annoyance for now. To make sure it works as expected we can create a simple test program that opens some files, forks a child, has the child use closefrom, and then exec's a process that will list out the file descriptors that are open. Here is the example:
For the program to exec, I'll use a simple shell script that lists the contents of the procfs directory /proc/PID/fd:
Note that using procfs to get the listing of filed descriptors is not portable and may not work for you. If you need a more general scheme looking at what lsof is probably a good starting point. Back to the main topic, if the script works you should get output like:
Note file descriptor 255 shown in the listing is added by the shell that is executing the script. It was not open until the after the exec so it was not closed.
The child process shall have its own copy of the parent's file descriptors. Each of the child's file descriptors shall refer to the same open file description with the corresponding file descriptor of the parent.This can be useful, for example, it enables IPC by remapping the standard input, output, and error descriptors to pipes that were created before the fork and then using exec to replace the child image with a new command. However, this behavior can also be an annoyance if one is not careful. An example I hit recently was that the application was occasionally hanging when it shouldn't be. Poking at the system it was blocked on a read to a file descriptor that should have been closed. Debugging further the file descriptor for writing to the pipe was indeed closed in the parent process but, a fork and exec had occurred before it was closed in a different part of the code and the child process still had an open handle to write to that pipe. This failure was of course sporadic and difficult to reproduce because it only occurred if the timing was just right.
Once the problem was diagnosed though, it seemed like it should be a trivial problem to fix. We just need to make sure we close out all of the file descriptors we aren't interested in. In the past I did this using a closefrom system call. This call does not seem to be widely supported though. The naive way to do this is to simply close everything in the range of possible values for file descriptors. On POSIX systems the range can be determined via the _SC_OPEN_MAX setting:
int closefrom_v1(int fd) { int max = sysconf(_SC_OPEN_MAX); for (int i = fd; i <= max; ++i) { close(i); } return 0; }
int closefrom_v2(int fd) { int count = 0; int max = sysconf(_SC_OPEN_MAX); for (int i = fd; i <= max; ++i) { if (close(i) == -1) { if (errno != EBADF) { return -1; } } else { ++count; } } return count; }
#include <unistd.h> #include <assert.h> #include <stdio.h> #include <stdlib.h> #include <errno.h> #include <sys/wait.h> int closefrom_v1(int fd) { int max = sysconf(_SC_OPEN_MAX); for (int i = fd; i <= max; ++i) { close(i); } return 0; } int closefrom_v2(int fd) { int count = 0; int max = sysconf(_SC_OPEN_MAX); for (int i = fd; i <= max; ++i) { if (close(i) == -1) { if (errno != EBADF) { return -1; } } else { ++count; } } return count; } int main(int argc, char **argv) { if (argc != 2) { printf("Usage: %s <cmd>\n", argv[0]); exit(1); } // Open a bunch of files so the child will have something to close int numFDs = 42; for (int i = 0; i < numFDs; ++i) { FILE *f = fopen("/dev/null", "w"); if (f == NULL) { perror("open failed"); exit(1); } assert(f != NULL); } printf("opened %d files\n", numFDs); // Fork a child and exec pid_t pid = fork(); if (pid == -1) { perror("could not fork"); } else if (pid == 0) { printf("closed %d files\n", closefrom_v2(3)); if (execlp(argv[1], (char *) 0) == -1) { perror("could not exec"); } } else { int status; waitpid(pid, &status, 0); printf("child %d exited with status %d\n", pid, status); } return 0; }
#!/bin/sh ls /proc/$$/fd
$ ./testclose listfds.sh opened 42 files closed 42 files 0 1 2 255 child 31588 exited with status 0
Thursday, July 15, 2010
Thursday, June 24, 2010
InputStreamReader and invalid UTF-8
While writing some unit tests for some code to read in UTF-8 encoded data, I was surprised to find that the java InputStreamReader class did not throw an exception with bad data. Consider the following example program:
For correct UTF-8 input it does what you would expect:
However, if you feed it bad data:
The unix iconv tool gives an error as expected. The java version is printing out the unicode replacement character twice. According to the javadocs for CharsetDecoder, the default behavior is supposed to be to report the error:
Now, if the conversion is not possible it will fail loudly with an exception:
import java.io.*; public class IConv { public static void main(String[] args) throws IOException { if (args.length != 2) { System.err.println("Usage: iconv <from> <to>"); System.exit(1); } Reader input = new InputStreamReader(System.in, args[0]); Writer output = new OutputStreamWriter(System.out, args[1]); char[] buffer = new char[4096]; int length; while ((length = input.read(buffer)) != -1) { output.write(buffer, 0, length); } output.flush(); } }
$ printf "\xE2\x82\xAC\n" | iconv -f utf-8 -t utf-8 | od -t x1 0000000 e2 82 ac 0a 0000004 $ printf "\xE2\x82\xAC\n" | java IConv utf-8 utf-8 | od -t x1 0000000 e2 82 ac 0a 0000004
$ printf "\xE2\x82\xFC\n" | iconv -f utf-8 -t utf-8 | od -t x1 iconv: illegal input sequence at position 0 0000000 $ printf "\xE2\x82\xFC\n" | java IConv utf-8 utf-8 | od -t x1 0000000 ef bf bd ef bf bd 0000006
How a decoding error is handled depends upon the action requested for that type of error, which is described by an instance of the CodingErrorAction class. The possible error actions are to ignore the erroneous input, report the error to the invoker via the returned CoderResult object, or replace the erroneous input with the current value of the replacement string. The replacement has the initial value "\uFFFD"; its value may be changed via the replaceWith method.Looking at the javadocs for InputStreamReader, I didn't see any indication that it deviates from the the default behavior. But sure enough, look at the InputStreamReader source code and it creates a decoder using CodingErrorAction.REPLACE. One caveat, I used Sun/Oracle's java 6 VM for testing, but the source code I looked at was for the Apache Harmony VM because it conveniently came up in the search results. So now that we know why it is happening, we can fix it to throw an exception by explicitly specifying a CharsetDecoder configured to use CodingErrorAction.REPORT:
The default action for malformed-input and unmappable-character errors is to report them. The malformed-input error action may be changed via the onMalformedInput method; the unmappable-character action may be changed via the onUnmappableCharacter method.
import java.io.*; import java.nio.charset.*; public class IConv { private static CharsetDecoder decoder(String encoding) { return Charset.forName(encoding).newDecoder() .onMalformedInput(CodingErrorAction.REPORT) .onUnmappableCharacter(CodingErrorAction.REPORT); } private static CharsetEncoder encoder(String encoding) { return Charset.forName(encoding).newEncoder() .onMalformedInput(CodingErrorAction.REPORT) .onUnmappableCharacter(CodingErrorAction.REPORT); } public static void main(String[] args) throws IOException { if (args.length != 2) { System.err.println("Usage: iconv <from> <to>"); System.exit(1); } Reader input = new InputStreamReader(System.in, decoder(args[0])); Writer output = new OutputStreamWriter(System.out, encoder(args[1])); char[] buffer = new char[4096]; int length; while ((length = input.read(buffer)) != -1) { output.write(buffer, 0, length); } output.flush(); } }
$ printf "\xE2\x82\xFC\n" | java IConv utf-8 utf-8 | od -t x1 Exception in thread "main" java.nio.charset.MalformedInputException: Input length = 2 at java.nio.charset.CoderResult.throwException(CoderResult.java:260) at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:319) at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:158) at java.io.InputStreamReader.read(InputStreamReader.java:167) at java.io.Reader.read(Reader.java:123) at IConv.main(IConv.java:29) 0000000 $ printf "\xE2\x82\xAC\n" | java IConv utf-8 iso-8859-1 | od -t x1 Exception in thread "main" java.nio.charset.UnmappableCharacterException: Input length = 1 at java.nio.charset.CoderResult.throwException(CoderResult.java:261) at sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:266) at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:106) at java.io.OutputStreamWriter.write(OutputStreamWriter.java:190) at IConv.main(IConv.java:30) 0000000
Subscribe to:
Posts (Atom)