postgresql when it's not your job

16:09

WordPress to Django+PostgreSQL, Part 3: Installing Apache, Python 2.6, psycopg2, and mod_wsgi

17 October 2009

In part 2 of this series, we got PostgreSQL up and running. In this part, we’ll install the remaining components to get Django up and running under Apache: Apache itself, Python 2.6, psycopg2, and mod_wsgi. So far, we’ve just accepted the default configuration parameters that initdb provided. Those will get us up and running, and setting the various PostgreSQL config parameters is worth at least one post all to itself. (In the meantime, see Josh Berkus’ talk “The Mighty GUCS”.)

Installing Apache couldn’t be easier:

[xof@blog ~]$ sudo yum install httpd httpd-devel

(We need httpd-devel to build mod_wsgi, the module we’ll use to get at Python programs from Apache.)

 Python and Centos: A Brief Diversion

The good news about Python on Centos is that it comes pre-installed. (It has to be; yum, among other tools, is written in Python.) The bad news is that the version that it comes with is …

[xof@blog ~]$ python -V
Python 2.4.3

… not exactly current. The most recent version of Python on the 2.x line as of this writing is 2.6.3. The very bad news is that you cannot just upgrade the base system Python install to 2.6; it breaks some of the system tools.

Fortunately, a solution exists: Doing a parallel installation of Python. I’m deeply indebted to this posting from Perplexed Labs for guidance on the process.

On with it: Installing Python 2.6

Since the RPMs for Centos are all Python 2.4, we need to install Python 2.6 from source. So, we grab the tarball, download it, and install it.

[xof@blog ~]$ cd builds
[xof@blog builds]$ wget http://www.python.org/ftp/python/2.6.3/Python-2.6.3.tgz
[xof@blog builds]$ tar xvfz Python-2.6.3.tgz
[xof@blog builds]$ cd Python-2.6.3

Note the --prefix option to put the 2.6 install in a different location.

[xof@blog Python-2.6.3]$ ./configure --prefix=/opt/python2.6 --with-threads --enable-shared
[xof@blog Python-2.6.3]$ make
[xof@blog Python-2.6.3]$ sudo make install

Add a symbolic link to put Python 2.6 in our path:

[xof@blog Python-2.6.3]$ sudo ln -s /opt/python2.6/bin/python /usr/bin/python2.6

Add the shared libraries to ldconfig and to /usr/lib64:

[xof@blog Python-2.6.3]$ sudo sh
sh-3.2# cat >> /etc/ld.so.conf.d/opt-python2.6.conf
/opt/python2.6/lib
sh-3.2# /sbin/ldconfig
sh-3.2# exit
[xof@blog Python-2.6.3]$ cd /usr/lib64
[xof@blog Python-2.6.3]$ sudo ln -s /opt/python2.6/lib/libpython2.6.so.1.0 libpython2.6.so.1.0 
[xof@blog Python-2.6.3]$ sudo ln -s libpython2.6.so.1.0  libpython2.6.so

How does it look?

[xof@blog Python-2.6.3]$ python2.6
Python 2.6.3 (r263:75183, Oct 17 2009, 18:29:55) 
[GCC 4.1.2 20080704 (Red Hat 4.1.2-44)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> 

Great. Onwards.

Installing setuptools

It’s very handy to have the Python package setuptools, so let’s download that and install it from the appropriate Python egg:

[xof@blog Python-2.6.3]$ cd ..
[xof@blog builds]$ wget http://pypi.python.org/packages/2.6/s/setuptools/setuptools-0.6c9-py2.6.egg
[xof@blog builds]$ sudo sh setuptools-0.6c9-py2.6.egg --prefix=/opt/python2.6

Installing psycopg2

And now we can install psycopg2. First, I have to say that the documentation on pretty much any aspect of psycopg2 is not exactly everything one can ask for; reading the INSTALL file in the tarball is your friend…

[xof@blog builds]$ wget http://initd.org/pub/software/psycopg/psycopg2-2.0.13.tar.gz
[xof@blog builds]$ tar xvfz psycopg2-2.0.13.tar.gz
[xof@blog builds]$ cd psycopg2-2.0.13

One of the files in the tarball is setup.cfg; this contains a setting we need to fix. We point pg_config to the right location for the binary.

# "pg_config" is the preferred method to locate PostgreSQL headers and
# libraries needed to build psycopg2. If pg_config is not in the path or
# is installed under a different name uncomment the following option and
# set it to the pg_config full path.
pg_config=/usr/local/pgsql/bin/pg_config

And away we go!

[xof@blog psycopg2-2.0.13]$ python2.6 setup.py build
[xof@blog psycopg2-2.0.13]$ sudo python2.6 setup.py install

So, did that work?

[xof@blog psycopg2-2.0.13]$ python2.6
Python 2.6.3 (r263:75183, Oct 17 2009, 18:29:55) 
[GCC 4.1.2 20080704 (Red Hat 4.1.2-44)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import psycopg2
>>> conn = psycopg2.connect("dbname='template1' user='postgres' host='localhost'")
>>> cur = conn.cursor()
>>> cur.execute("""SELECT datname from pg_database""")
>>> rows = cur.fetchall()
>>> print rows
[('template1',), ('template0',), ('postgres',)]

Installing mod_wsgi

mod_wsgi is an Apache module serving up Python programs. It’s generally considered a superior candidate to mod_python.

[xof@blog builds]$ wget http://modwsgi.googlecode.com/files/mod_wsgi-2.6.tar.gz
[xof@blog builds]$ tar xvfz mod_wsgi-2.6.tar.gz
[xof@blog builds]$ cd mod_wsgi-2.6
[xof@blog mod_wsgi-2.6]$ ./configure --with-python=/opt/python2.6/bin/python
[xof@blog mod_wsgi-2.6]$ make
[xof@blog mod_wsgi-2.6]$ sudo make install

To be paranoid, we should check two things. One: Did it link against the correct Python shared library?

xof@blog mod_wsgi-2.6]$ ldd /usr/lib64/httpd/modules/mod_wsgi.so
    linux-vdso.so.1 =>  (0x00007fffcbbfe000)
    libpython2.6.so.1.0 => /opt/python2.6/lib/libpython2.6.so.1.0 (0x00007f9ac3553000)
    libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f9ac332c000)
    libdl.so.2 => /lib64/libdl.so.2 (0x00007f9ac3128000)
    libutil.so.1 => /lib64/libutil.so.1 (0x00007f9ac2f25000)
    libm.so.6 => /lib64/libm.so.6 (0x00007f9ac2ca1000)
    libc.so.6 => /lib64/libc.so.6 (0x00007f9ac294b000)
    /lib64/ld-linux-x86-64.so.2 (0x000000323e400000)

Looks good. Second, how big is the shared library it created?

xof@blog mod_wsgi-2.6]$ ls -l /usr/lib64/httpd/modules/mod_wsgi.so
-rwxr-xr-x 1 root root 339625 Oct 17 23:02 /usr/lib64/httpd/modules/mod_wsgi.so

We’re looking for a size in the 332 kilobyte or smaller range, to confirm that it linked against the Python library as a shared, not static, library; for a discussion of this issue, see the “Lack of Shared Library” heading in the installation instructions.

In the next part, we’ll configure Apache, install Django, and bring up a test application.

WordPress to Django+PostgreSQL: Part 2, Configuring PostgreSQL at 16:17, 17 October 2009:

[…] Next, in part 3, we’ll install Python 2.6, Apache, mod_wsgi and psycopg2. […]

WordPress to Django+PostgreSQL, Part 4: Configuring Apache, Django, PostgreSQL at 08:36, 19 October 2009:

[…] October 2009In the previous installment of this series (in which we’re migrating this blog from WordPress to Django and PostgreSQL), […]

usenetshack at 17:19, 26 October 2009:

Thanks for the guide, worked flawlessly with the latest CentOS 5.4 and Python 2.6.4. Cheers.

pulegium at 02:32, 16 January 2010:

Great guide! Just a note for those who want to make multiple installations (in case you have large number of production servers where you want to have Django running as described in this post) you might want to create a python RPM that you can just include during a kickstart or yum update. Here’s a quick guide how to create python 2.6 RPM for centos:

http://www.grenadepod.com/2009/12/26/building-python-2-6-4-rpm-for-centos-5-4/

carter at 20:27, 28 February 2010:

On the “Installing psycopg2” step:

I set pg_config=/usr/local/pgsql/bin/pg_config

then,installed with:

$ python2.6 setup.py build
$ python2.6 setup.py install

but when I start $ python2.6:

$ python2.6

on the next step, at the python2.6 prompt

>>>

When I enter:

>>> import psycopg2

it errors with:

Traceback (most recent call last):
File “”, line 1, in
File “/opt/python2.6/lib/python2.6/site-packages/psycopg2/__init__.py”, line 60, in
from _psycopg import BINARY, NUMBER, STRING, DATETIME, ROWID
ImportError: libpq.so.5: cannot open shared object file: No such file or directory

any ideas?

velvet at 17:57, 1 May 2010:

thank you for this post, it’s a life saver!

Marcob at 00:55, 24 August 2010:

Great article series!

As an update, I must say that, now, Psycopg2 doc is simply great, thanks to Daniele “piro” Varrazzo. Have a look at http://initd.org/psycopg/

Farshid at 14:41, 22 November 2010:

Thanks for the post.
It is nice when things do work.
I had the problem that carter mentioned: ImportError: libpq.so.5: cannot open shared object file: No such file or directory

I started from begining and followed it step by step including setting up the path and restarting the machine.
Keep up the good work

Cassie at 11:06, 2 March 2012:

Hi
My system has default python 2.4.3 installed and I would like to update to python2.6. I have installed a python2.6.5 before but the system could not find it. In order to avoid any version conflicting, I tried to install another python2.6.6 again following all your steps posted here. The only difference is I created the folder as /opt/python2.6.6 instead of /opt/python2.6. I edited the PATH as “export PATH=/usr/bin/python2.6.6:$PATH”. However, the system still can not run bash “python2.6.6” or “python2.6”. I still can only run python 2.4.3.

I am wondering if you can give me some advises where to check. Thank you very much,

Xof at 17:30, 2 March 2012:

The Python executable is “python” regardless of the version, so you might see if that’s the problem.

Cassie at 14:03, 8 March 2012:

Do you mean put a simlink as “ln -s /opt/python2.6/bin/python /usr/bin/python” ?
Doesn’t it mess up with existing system, python 2.4.3 ? Thanks

Cassie at 15:07, 8 March 2012:

Hi
I change the simlink to “ln -s /opt/python2.6/bin/python2.6 /usr/bin/python” and now it works ! :) Thank you very much,