Building a Scientific Toolbox on MacOS X 10.6 Snow Leopard (64-bit build)

Table of Contents
  1. General Data Exploration Tools
  2. Building a Python 2.7 Configuration for Scientific Programming
  3. Items to install at the Application level
  4. Library installations
  5. Python Components
  6. Utilities
  7. Revision History
General Data Exploration Tools
There are a number of freely available tools for examining scientific datasets.  It is far from an exhaustive list.  It merely represents programs which I have actually used.

Astronomical datasets, particularly images, are usually stored in a file format called FITS (Flexible Image Transport System).  FITS files can contain multiple segments, called extensions, which can contain different types of data.  The PRIMARY extension is usually a two- or three-dimensional array which can contain an image, a sequence of images, or a 3-dimensional data cube.

There are two free tools useful for examining FITS data files and a third useful for exploring 3-D datasets.

SAOImage DS9
This program is best suited for viewing FITS images, from sources such as Space Telescope, SOHO, SDO, and STEREO.

This program allows examination and editing of a wider range of FITS files than SAOImage DS9.

VisIt is a tool which can read and view a wide variety of 3-dimensional datasets.

Building a Python 2.7 Configuration for Scientific Programming

Why Python?
  • Free and Open source.  There is no need to pay for each license or installation.  This is a particularly good feature if you are running on a small cluster and can distribute the computing load across multiple processors.
  • Python is an object-oriented language (Wikipedia).
  • Python is an interpreted language(Wikipedia).  While tending to be slower than compiled languages, in sciences, the development time is often the largest time commitment.  This speeds up development.  Once the program is relatively stable, many options exist for optimizing code components for faster execution.
Python 2.7
The newest available Python as of this writing is version 3.2.  However, many of the libraries I am interested in for this project are still evolving in their support for the newest version.  Python 2.7 lets us have support for these libraries while providing a migration path to version 3 and above using the from __future__ import mechanism.

Why 64-bit?
In science, we often need to explore very large datasets.  64-bit processors have a much larger memory addressing space.  This space is needed for data structures requiring more than 2GB of memory.  You'd be surprised how quickly a 3-dimensional double-precision floating-point array can get that large.

Why build from source?
While many would argue this is a more difficult route, unless you can obtain a prepackaged installation kit that contains everything you want, configured exactly the way you want, you may encounter problems of compatibility between different installations.

It also has the advantage that you learn more about your programming environment, when can make it easier to solve other problems you may have to deal with in scientific software development.

General Instructions for SciPy on Mac
As with many scientific packages being built from source, the Apple Development Package XCode and all its compilers are required.  X11 is also needed for a number of applications.

Items to install at the Application Level

Install Cmake 2.8.3 from disk image
- this installs command line tools in /usr/local/bin
- Facilitates installation of VTK

MacTeX image (dated 2010 September)
 (this also installed ImageMagick 6.2.6, freetype2, ghostscript 8.51, and others)
May not be necessary but is sometimes helpful
- might want to install imageMagick as 64-bit

Python 2.7
Built-in python 2.5 & 2.6 are in /System/Library/Frameworks/ and linked from /usr/bin/
Mac Installer disk image(2.7) for OSX 10.5 and Later (x86-64)
Install with Shell profile updater to make it the default python
Adds /Library/Frameworks/Python.framework/Versions/2.7/bin to PATH

Library Installations

GNU Fortran
gfortran 4.2.3
Run the installer.  The components are installed in /usr/local/bin
Make sure this is in your PATH
Data compression library
- needed for libpng

% cd zlib-1.2.5
% export LDFLAGS="-arch x86_64"
% export FFLAGS="-arch x86_64"
% ./configure --prefix=/usr/local --64
% make
% make check
% sudo make install
Image format library
- needed for PIL, wxPython, matplotlib

% export LDFLAGS="-arch x86_64"
% export FFLAGS="-arch x86_64"
% cd libpng 1.4.4
% ./configure
% make
% make check
% sudo make install

Installs in /usr/local
* Careful with PATH, this can conflict with Apple's libPng.dylib, causing other applications to fail.
- needed for PIL

% export LDFLAGS="-arch x86_64"
% export FFLAGS="-arch x86_64"

% cd jpeg-8c
% cp /usr/share/libtool/config/config.sub .
% cp /usr/share/libtool/config/config.guess .
% ./configure --enable-shared --enable-static
% make
% sudo mkdir -p /usr/local/man/man1
% sudo make install

Other notes
- needed for PIL

% export LDFLAGS="-arch x86_64"
% export FFLAGS="-arch x86_64"

% cd tiff-3.9.4/
% ./configure --mandir=/usr/local/share/man --with-apple-opengl-framework CFLAGS="-O2 -D_FILE_OFFSET_BITS=64" CXXFLAGS="-O2 -D_FILE_OFFSET_BITS=64"
% make
% sudo make install
Freetype 2.4.4
Open Source font management

% ./configure
% make
% sudo make install

Fast-Fourier Transform library

% ./configure --enable-threads \
    CC="gcc -arch i386 -arch x86_64" CXX="g++ -arch i386 -arch x86_64" \
    CPP="gcc -E" CXXCPP="g++ -E" F77=gfortran
% make
% sudo make install
% sudo ln -s /usr/local/fftw-3.2.2 /usr/local/fftw

Other Notes:
Visualization Toolkit (VTK)
Requires X11 for visual output

VTK 5.6.1
Configure with CMake on command line
% ccmake .

PYTHON_LIBRARY /Library/Frameworks/Python.framework/Versions/2.7/lib/libpython2.7.dylib
VTK_DATA_ROOT   /Users/Shared/VTKData

# Let it build its own library for PNG, TIFF, ZLIB, JPEG
# Make sure it finds X11 libraries (but you don't have to set it for output to X11.  That is apparently handled elsewhere.
# let ccmake generate configure script

% make -j 2  # takes about an hour
% sudo make install

# install python wrapper
% cd Wrapping/Python
% python build
% sudo python install

# update paths
% export DYLD_LIBRARY_PATH=/usr/local/lib/vtk-5.6/
% export LD_LIBRARY_PATH=/usr/local/lib
% export PYTHONPATH=/usr/local/lib:$PYTHONPATH
% export VTK_DATA_ROOT=/Users/Shared/VTKData

Python Components

setuptools v0.6c11
% cd setuptools-0.6c11
% python build
% sudo python install
GUI for other graphics-related modules such as matplotlib

nose 0.11.3
Testing module for numpy

% python build
% sudo python install
% python -c 'import numpy; numpy.test()'
Ran 2966 tests in 10.728s

Python Imaging Library, PIL 1.1.7

JPEG_ROOT = "/usr/local/lib","/usr/local/include"
ZLIB_ROOT = "/usr/lib","/usr/include"
TIFF_ROOT = "/usr/local/lib","/usr/local/include"
FREETYPE_ROOT = "/usr/local/lib","/usr/local/include"

% python build_ext -i
( gave warnings on freetype about architecture)
% python  (57 tests passed)
% sudo python install


% gnutar xzf numpy-1.5.1.tar.gz
% cd numpy-1.5.1
% python build --fcompiler=gnu95
% sudo python install
% python -c 'import numpy; numpy.test()'
Ran 3006 tests in 9.662s

Other Notes
  • It is not clear if numpy still needs the LAPACK, ATLAS, FFTW libraries.  Does it link to Apple's transparently or build local versions?

% cd scipy-0.9.0
% python build
% sudo python install

 >>> import scipy
 >>> scipy.test()
Ran 4724 tests in 71.324s


matplotlib v1.01

Plotting and graphing package for Python

% export CFLAGS="-O -g -arch x86_64"
% export LDFLAGS="-arch x86_64"
% gnutar xzf matplotlib-1.0.0.tar.tar.gz
% cd matplotlib-1.0.0.tar

changes in
    '_darwin' : ['/usr/local'],
in the routine,
def add_base_flags(module):
Modify add the designated lines
+    module.include_dirs.append('/usr/local/include')
+    module.include_dirs.append('/usr/local/include/freetype2')

% python build
% sudo python install

Other Notes
  • Compile Matplotlib for Python on Snow Leopard 
  • Building Matplotlib for MacOS X
  • Caveat: If you have used matplotlib with an earlier version of python, you might receive errors such as "Cannot_Open_Resource" when trying to access fonts.  This can be due to the paths to your older installation cached in your local '.matplotlib/' configuration directory (in your home directory).  Deleting this directory so matplotlib will rebuild it from scratch seems to fix the problem.
Module for reading and writing FITS files

% gnutar xzf pyfits-2.4.0.tar.gz
% cd pyfits-2.4.0
% python build
% sudo python install

Sphinx v1.0.4
Module for documentation generation

% cd Sphinx-1.0.4
% python build
% sudo python install

After installing::
% cd doc
% sphinx-build . _build/html

Simple Open Source rendering tool.

I wanted to build this module under 64-bit, however, the configuration scripts seem to have a problem with this option.  This may be a compatibility problem with the libPng installed by Apple or version 1.4.4 installed in this process.  Therefore, I chose the option of forcing everything to build under 32-bit.

% cd povray-3.6.1/
% cd libraries/png
% ./configure
% make
% cd ../..
% ./configure COMPILED_BY="your name " CFLAGS="-m32" CXXFLAGS="-m32"
% make
% make check
% sudo make install

Other Notes
Some POVray user group posts provided some of the information I needed to get this to build.


% file [filename]

Tells you what type of library or file you have: 32-bit, 64-bit, PowerPC, Intel, etc.

Hopefully I've credited all the sources I collected for assistance in making this installation work (at least for me).  If you discover any errors or other issues, please e-mail me at

Revision History
2011-04-17: Initial posting

No comments: