04:12

pgbadger

30 June 2012

For years, the standard log analysis tool for PostgreSQL has been pgfouine (For those wondering, a “fouine” in French is a beech marten; as the saying goes, I am none the wiser, if somewhat better informed.) However, pgfouine seems to have stalled as a project; there haven’t been updates in a while, it requires a patch to work with PostgreSQL 9.1, and it frequently chokes on complex or difficult-to-parse log files. And, well, it’s written in PHP.

Thus, I’m pleased to note a new-ish log analyse, pgbadger. It’s written in Perl, at least as fast as pgfouine, and can process log files that pgfouine can’t handle. It can read either CSV or standard log format, and can directly read *.gz files. It also produces a wider range of reports that pgfouine, including some very useful locking reports. I threw 25GB of logs with near 80 million lines at it without it complaining; it processed between 225 and 335 log lines per second on my laptop.

I am not sure why PostgreSQL log analyzers have adopted a small-mammal naming convention, but I’m pleased to have something else burrowing in the garden.

There are 6 comments.

Simon at 05:59, 30 June 2012:

‘fouine’ is more properly translated – at least into British English – as ‘ferret’. So it’s for ‘ferreting out’ things from the log.

Marco at 07:58, 30 June 2012:

Simon, I think ‘ferret’ is ‘furet’ in french. So, IMHO, ‘beech marten’ is the proper translation for ‘fouine’ :-)

moltonel at 02:33, 2 July 2012:

“Fouine” is also the familiar verb “fouiner” (1st person singular, present tense) which means “to search”, particularly in a messy environement.

As for the php bashing… Like many languages it has good and bad parts, which take an experienced programmer to sort out. Some languages are cleaner… But perl definitely isn’t one of those ;)

That said, I welcome the arrival of pgbadger, another tool in the toolbox.

-dg at 18:31, 7 July 2012:

The pgfouine report formats are prettier, but it is unusable for any large amount of logs. pgbadger has a few bugs, but it does work for reasonable volume.

Dan at 17:49, 25 August 2012:

Martens and Badgers are both in the Mustelidae family and even both in the Mustelinae subfamily if we believe the taxonomy. So it, err, all makes perfect sense.

Sumit at 02:20, 12 December 2012:

Hi,

I’ve been using Pgbadger1.2 on EDB9 the issue is that sometimes the log is parsed ie the time taken to parse the log is shown in the report but its an empty report.. Is that a bug with the version .
Also i was looking at using zcat for .tgz log files but its not parsing the logs instead getting the following error:
FATAL: unable to detect log file format from /pg_log/edb-2012-11-15.tgz, please use -f option.

even after using -f the issue still remains ..

Thanks