parallel processing

graph-tool
#1

Hi!

I am intending to use the Parallel construct from the joblib library which basically runs an embarassingly parallel for loop via the multiprocessing facility. The error I get is

Traceback (most recent call last):
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/threading.py", line 522, in __bootstrap_inner
    self.run()
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/threading.py", line 477, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/multiprocessing/pool.py", line 225, in _handle_tasks
    put(task)
RuntimeError: Pickling of "graph_tool.libgraph_tool_core.Vertex" instances is not enabled (http://www.boost.org/libs/python/doc/v2/pickle.html)

I thought graph-tool graphs are compatible with Python pickle-stuff - or do I have to watch out somewhere?

Thanks for the help,

#2

Hi Sebastian,

RuntimeError: Pickling of "graph_tool.libgraph_tool_core.Vertex" instances is not enabled (http://www.boost.org/libs/python/doc/v2/pickle.html)

I thought graph-tool graphs are compatible with Python pickle-stuff -
or do I have to watch out somewhere?

You can only pickle Graph objects, not Vertex or Edge objects.
PropertyMaps can only be pickled if they are made "internal" to the
graph, by storing them in the appropriate "*_properties" dictionary of a
Graph object.

If you want to pickle vertex or edges, you have to convert them to ints
or tuple of ints, respectively, and then recover them later with the
Graph.vertex() and Graph.edge() function.

Cheers,
Tiago