about graph-tool and mac os x lion

Hello,

I see recently many people have had one problem or another to run graph-tool on mac os x.
I've been able to build graph-tool on Lion, with apple gcc 4.2, boost
1.47 and python 2.7.2 (these last two from homebrew).
The problem is that graph_tool doesn't import:

ImportError Traceback (most recent call last)
/Users/hammer/TIS/java-hacking/giraph/<ipython-input-1-100bbe2bc9d9>
in <module>()
----> 1 from graph_tool.all import *

/usr/local/Cellar/python/2.7.2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/graph_tool/__init__.py
in <module>()
    89 import scipy.stats
    90
---> 91 from dl_import import *
    92 dl_import("import libgraph_tool_core as libcore")
    93 import libgraph_tool_core as libcore # for pylint

/usr/local/Cellar/python/2.7.2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/graph_tool/dl_import.py
in <module>()
    25 # handle strange python installations, by importing from
the deprecated dl

    26 # module

---> 27 from dl import RTLD_LAZY, RTLD_GLOBAL
    28
    29 __all__ = ["dl_import"]

ImportError: No module named dl

I noticed that DLFCN module is missing on my python install. I tried
to substitute the RTLD_LAZY and RTLD_GLOBAL values (0x1 and 0x8) taken
from the include file dlfcn.h into the setdlopenflags call in that
same dl_import.py, but i get a trap back:

Fatal Python error: Interpreter not initialized (version mismatch?)
Abort trap: 6

Do you have any idea? How did you guys get graph_tool working on mac os?

thanks!

[...]

I've committed a fix for this in the git version. Could you test it and
see if you get the same problem?

Cheers,
Tiago

hi tiago,

thanks for answering.

actually i'm having problems running ./configure on the git version.

$ ./autogen.sh && ./configure

i get:

[..]
checking python2.7 module: numpy... yes
checking for /usr/local/Cellar/python/2.7.2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/core/include/numpy/arrayobject.h... yes
checking python2.7 module: scipy... yes
checking /usr/local/Cellar/python/2.7.2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/weave/scxx/object.h usability... yes
checking /usr/local/Cellar/python/2.7.2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/weave/scxx/object.h presence... yes
checking for /usr/local/Cellar/python/2.7.2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/weave/scxx/object.h... yes
checking expat.h usability... yes
checking expat.h presence... yes
checking for expat.h... yes
checking CGAL/version.h usability... yes
checking CGAL/version.h presence... yes
checking for CGAL/version.h... yes
./configure: line 18961: syntax error near unexpected token `CAIROMM,'
./configure: line 18961: `PKG_CHECK_MODULES(CAIROMM, cairomm-1.0)'

i have cairo and carom installed from homebrew.

Hi Claudio,

Hi Tiago,

thanks for the quick fix. I've been able to configure and build it successfully. the import now works but I get the same interpreter mismatch error as before. I'm using homebrew for building both python, cgal, graphviz, boost, numpy and scipy and they are all built with apple gcc 4.2 and linked to the homebrewed python in /usr/local/.

Here's the error by the interpreter:
Fatal Python error: Interpreter not initialized (version mismatch?)
Abort trap: 6

I've checked for the configure output of graph_tool, and there's something which it might be creating problems:

[..]

checking for python... /usr/local/bin/python
checking for a version of Python >= '2.1.0'... yes
checking for a version of Python 2.5... yes
checking for the distutils Python package... yes
checking for Python include path... -I/usr/local/Cellar/python/2.7.2/Frameworks/Python.framework/Versions/2.7/include/python2.7
checking for Python library path... -L/usr/local/Cellar/python/2.7.2/Frameworks/Python.framework/Versions/2.7/lib/python2.7 -lpython2.7
checking for Python site-packages path... /usr/local/Cellar/python/2.7.2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages
checking python extra libraries... -ldl -framework CoreFoundation
checking python extra linking flags... -u _PyMac_Error Python.framework/Versions/2.7/Python
checking consistency of all components of python development environment... yes
checking for python build information...
checking for python2.7... python2.7
checking for main in -lpython2.7... yes
checking python2.7/Python.h usability... yes
checking python2.7/Python.h presence... yes
checking for python2.7/Python.h... yes
  results of the Python check:
    Binary: python2.7
    Library: python2.7
    Include Dir: /Developer/SDKs/MacOSX10.7.sdk/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7
checking whether python version >= 2.5... yes
checking for python version... 2.7
checking for python platform... darwin
checking for python script directory... ${prefix}/lib/python2.7/site-packages
checking for python extension module directory... ${exec_prefix}/lib/python2.7/site-packages
checking for boostlib >= 1.38.0... yes
checking whether the Boost::Python library is available... yes
checking for exit in -lboost_python... no
checking for exit in -lboost_python... (cached) no
checking for exit in -lboost_python-mt... yes
checking whether the Boost::Iostreams library is available... yes
checking for exit in -lboost_iostreams... no
checking for exit in -lboost_iostreams... (cached) no
checking for exit in -lboost_iostreams-mt... yes
checking whether the Boost::Regex library is available... yes
checking for exit in -lboost_regex... no
checking for exit in -lboost_regex... (cached) no
checking for exit in -lboost_regex-mt... yes
checking whether the Boost::Graph library is available... yes
checking for exit in -lboost_graph... no
checking for exit in -lboost_graph... (cached) no
checking for exit in -lboost_graph-mt... yes
checking for main in -lCGAL... yes
checking python2.7 module: numpy... yes
checking for /usr/local/Cellar/python/2.7.2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/core/include/numpy/arrayobject.h... yes
checking python2.7 module: scipy... yes
checking /usr/local/Cellar/python/2.7.2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/weave/scxx/object.h usability... yes
checking /usr/local/Cellar/python/2.7.2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/weave/scxx/object.h presence... yes
checking for /usr/local/Cellar/python/2.7.2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/weave/scxx/object.h… yes

For what I can see the python libs are correctly found in /usr/local/ except for these lines:

checking for python2.7/Python.h... yes
  results of the Python check:
    Binary: python2.7
    Library: python2.7
    Include Dir: /Developer/SDKs/MacOSX10.7.sdk/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7

where apparently the original Xcode python is found. why's that? do you have any suggestions on configure options for it to stick it to the correct versions? i've tried some options based on python-config but unsuccessfully.
By looking at the way graph_tool is built, i don't really get what's going on, because both -I and -L options are /usr/local/ paths.
I attach here the config.log, hoping it can help.

config.log (79.4 KB)

ATT00001.txt (1.84 KB)

Hi Tiago,

thanks for the quick fix. I've been able to configure and build it
successfully. the import now works but I get the same interpreter
mismatch error as before. I'm using homebrew for building both python,
cgal, graphviz, boost, numpy and scipy and they are all built with
apple gcc 4.2 and linked to the homebrewed python in /usr/local/.

Here's the error by the interpreter:
Fatal Python error: Interpreter not initialized (version mismatch?)
Abort trap: 6

I've checked for the configure output of graph_tool, and there's
something which it might be creating problems:

[...]

For what I can see the python libs are correctly found in /usr/local/ except for these lines:

checking for python2.7/Python.h... yes
  results of the Python check:
    Binary: python2.7
    Library: python2.7
    Include Dir: /Developer/SDKs/MacOSX10.7.sdk/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7

where apparently the original Xcode python is found. why's that? do
you have any suggestions on configure options for it to stick it to
the correct versions? i've tried some options based on python-config
but unsuccessfully.
By looking at the way graph_tool is built, i don't really get what's
going on, because both -I and -L options are /usr/local/ paths. I
attach here the config.log, hoping it can help.

By looking at your config.h, it seems that the paths are in order:

#define PYTHON_DIR "/usr/local/Cellar/python/2.7.2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages"
#define CXXFLAGS " -Wall -ftemplate-depth-150 -Wno-deprecated -Wno-unknown-pragmas -O99 -fvisibility=default -fvisibility-inlines-hidden -Wno-unknown-pragmas"
#define CPPFLAGS " -I/usr/local/Cellar/python/2.7.2/Frameworks/Python.framework/Versions/2.7/include/python2.7 -I/usr/local/include -I/usr/local/Cellar/python/2.7.2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/core/include/numpy -I/usr/local/Cellar/python/2.7.2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy"
#define LDFLAGS " -L/usr/local/Cellar/python/2.7.2/Frameworks/Python.framework/Versions/2.7/lib/python2.7 -lpython2.7"

You should check if the compiled binary is indeed linked with the
correct python library. You should do the same check with your
boost-python library.

Cheers,
Tiago

Hi!

I am also trying to get things working at moment under Lion without success so far. At the moment I suspect that threading is an issue. How can I explicitly prevent graph-tool from being built against threaded libs? Is boost maybe responsible for that?

Cheers,

Sebastian

Hi Tiago,

to be honest, I can't see how lib boost can be linked to python. I ldd it and couldn't find any linkage to python.
But the problem is indeed with libgraphtool being linked to Xcode python:

$ otool -L libgraph_tool_core.so
libgraph_tool_core.so:
  /System/Library/Frameworks/Python.framework/Versions/2.7/Python (compatibility version 2.7.0, current version 2.7.1)
  /usr/local/lib/libboost_iostreams-mt.dylib (compatibility version 0.0.0, current version 0.0.0)
  /usr/local/lib/libboost_python-mt.dylib (compatibility version 0.0.0, current version 0.0.0)
  /usr/local/lib/libboost_regex-mt.dylib (compatibility version 0.0.0, current version 0.0.0)
  /usr/lib/libexpat.1.dylib (compatibility version 7.0.0, current version 7.2.0)
  /usr/local/lib/libCGAL.8.0.0.dylib (compatibility version 8.0.0, current version 8.0.0)
  /usr/lib/libstdc++.6.dylib (compatibility version 7.0.0, current version 52.0.0)
  /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 159.1.0)

I don't get why/how it happened. Any idea?

Hi Claudio,

Hi Tiago,

to be honest, I can't see how lib boost can be linked to python. I ldd
it and couldn't find any linkage to python.

boost-python must be linked with python...

But the problem is indeed with libgraphtool being linked to Xcode python:

$ otool -L libgraph_tool_core.so
libgraph_tool_core.so:
  /System/Library/Frameworks/Python.framework/Versions/2.7/Python (compatibility version 2.7.0, current version 2.7.1)
  /usr/local/lib/libboost_iostreams-mt.dylib (compatibility version 0.0.0, current version 0.0.0)
  /usr/local/lib/libboost_python-mt.dylib (compatibility version 0.0.0, current version 0.0.0)
  /usr/local/lib/libboost_regex-mt.dylib (compatibility version 0.0.0, current version 0.0.0)
  /usr/lib/libexpat.1.dylib (compatibility version 7.0.0, current version 7.2.0)
  /usr/local/lib/libCGAL.8.0.0.dylib (compatibility version 8.0.0, current version 8.0.0)
  /usr/lib/libstdc++.6.dylib (compatibility version 7.0.0, current version 52.0.0)
  /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 159.1.0)

I don't get why/how it happened. Any idea?

It may be that your DYLD_LIBRARY_PATH environment variable does not have
the correct value. Try adding the homebrew python path to it.

Cheers,
Tiago

Hi Claudio,

Hi Tiago,

to be honest, I can't see how lib boost can be linked to python. I ldd
it and couldn't find any linkage to python.

boost-python must be linked with python…

true! don't know why it passed my otool -L libboost* | grep python.

$ otool -L libboost_python-mt.dylib
libboost_python-mt.dylib:
  /usr/local/lib/libboost_python-mt.dylib (compatibility version 0.0.0, current version 0.0.0)
  /System/Library/Frameworks/Python.framework/Versions/2.7/Python (compatibility version 2.7.0, current version 2.7.1)
  /usr/lib/libstdc++.6.dylib (compatibility version 7.0.0, current version 52.0.0)
  /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 159.0.0)

Should I also build again libboost by myself (instead of using the homebrew version)?

But the problem is indeed with libgraphtool being linked to Xcode python:

$ otool -L libgraph_tool_core.so
libgraph_tool_core.so:
  /System/Library/Frameworks/Python.framework/Versions/2.7/Python (compatibility version 2.7.0, current version 2.7.1)
  /usr/local/lib/libboost_iostreams-mt.dylib (compatibility version 0.0.0, current version 0.0.0)
  /usr/local/lib/libboost_python-mt.dylib (compatibility version 0.0.0, current version 0.0.0)
  /usr/local/lib/libboost_regex-mt.dylib (compatibility version 0.0.0, current version 0.0.0)
  /usr/lib/libexpat.1.dylib (compatibility version 7.0.0, current version 7.2.0)
  /usr/local/lib/libCGAL.8.0.0.dylib (compatibility version 8.0.0, current version 8.0.0)
  /usr/lib/libstdc++.6.dylib (compatibility version 7.0.0, current version 52.0.0)
  /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 159.1.0)

I don't get why/how it happened. Any idea?

I must admit i'm pretty unfamiliar with DYLD_LIBRARY_PATH. Should I set it at compile time or runtime?

true! don't know why it passed my otool -L libboost* | grep python.

$ otool -L libboost_python-mt.dylib
libboost_python-mt.dylib:
  /usr/local/lib/libboost_python-mt.dylib (compatibility version 0.0.0, current version 0.0.0)
  /System/Library/Frameworks/Python.framework/Versions/2.7/Python (compatibility version 2.7.0, current version 2.7.1)
  /usr/lib/libstdc++.6.dylib (compatibility version 7.0.0, current version 52.0.0)
  /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 159.0.0)

Should I also build again libboost by myself (instead of using the homebrew version)?

No, I think the problem is happening at runtime...

I must admit i'm pretty unfamiliar with DYLD_LIBRARY_PATH. Should I
set it at compile time or runtime?

Runtime. The variable tells the runtime linker where the shared
libraries are located. Try the following:

export DYLD_LIBRARY_PATH="$DYLD_LIBRARY_PATH:/usr/local/Cellar/python/2.7.2/Frameworks/Python.framework/Versions/2.7/lib/python2.7"

And then run the same otool commands as before.

Cheers,
Tiago

If you disable openmp, graph-tool will not pull in thread libs
explicitly. But it may be that the boost libs are compiled with
threads...

Cheers,
Tiago

Hi Tiago,

I re-homebrew-ed boost 1.47 and now otool shows linkage to homebrew-ed python. Still graph-tool is linked to Xcode's python (even exporting DYLD_LIBRARY_PATH your way).
Anyway, I tried importing again graph_tool and this time I've got a new problem :slight_smile:

In [1]: from graph_tool.all import *

Hi Tiago,

I re-homebrew-ed boost 1.47 and now otool shows linkage to homebrew-ed
python. Still graph-tool is linked to Xcode's python (even exporting
DYLD_LIBRARY_PATH your way). Anyway, I tried importing again
graph_tool and this time I've got a new problem :slight_smile:

[...]

ImportError: dlopen(/usr/local/Cellar/python/2.7.2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/graph_tool/libgraph_tool_core.so, 8): Library not loaded: /usr/local/lib/libboost_thread-mt.dylib
  Referenced from: /usr/local/lib/libCGAL.8.0.0.dylib
  Reason: image not found

which is correct as:

$ ls -l /usr/local/lib/libboost_thread-mt.dylib
ls: /usr/local/lib/libboost_thread-mt.dylib: No such file or directory

Apparently the only dylib missing is that for thread_mt.

Any idea what's going on?

It looks like CGAL is linked to the wrong boost... You should check with
otool, and maybe recompile it.

Cheers,
Tiago

Hi Tiago,

I re-installed boost which is now linked to the correct python and re-installed CGAL.

Now, at import I get a segfault:

In [1]: from graph_tool.all import *
Segmentation fault: 11

The only weird thing remaining, i guess, is the libgraph_tool_core.so linked to the Xcode python, I guess it could be the reason of this.

Any idea of what's happening?

Yes, this is most likely due to the fact you are linking with two
different python libraries... You have to link graph_tool with the
correct one as well.

Cheers,
Tiago