shortest_distance / concurrency / graph view

Hello,

I have a rather fuzzy question about the topology.shortest_distance
function. I'll try to explain my problem as clearly as I can. If I fail to
be clear enough, don't waste your time, just let me know.

I have a pool of workers using one single graph view instance, each work
calls a function f described below. The graph view has an edge filter.

def f(g, origin, targets_ids, weights):
    d, p = topology.shortest_distance(g, source=origin, target=targets_ids,
weights=weights, pred_map=True)
    i = score(d)
    path = topology.shortest_path(g, source=origin, target=targets_ids[i],
pred_map=p)

    return path

Once in a while, the path from function f is not consistant with the graph
view (it goes through disabled edges).

I tried to reproduce the error in a single thread environment. The exact
same inputs do not fail as observed previously (ie. with multiple workers).

It seems highly unlikely, but would it be possible for the predecessor map
"p" to be overwritten by the actions of an another worker in the pool ? I'm
asking because I ran out of ideas to explain this problem, although it
sounds dumb.

Bests,
François.

attachment.html (1.91 KB)

I don't see how that would happen, but without a complete (and minimal)
working example, it is not possible to say much.

Best,
Tiago

Hello,

I suspect the copy_property function not to be thread safe. Remember that
the issue I observe occurs when I re-use the predecessor map returned by
shortest_distance. The copy_property is then called from
topology/__init__.py:1441 and the parameter full is None.

Here how I think it might fail with two concurrent shortest_distance calls.

Timeline
  >
  *time t=0 : Worker #1 executes the following statements from
(graph_tool/__init__.py:2216)*
  > g.set_directed(True)
  > g.clear_filters()
* time t=1 : Worker #2 executes the following statements from
(graph_tool/__init__.py:2207)*
  > is_directed = g.is_directed() *# True (as a result of worker #1)*
  > efilt = g.get_edge_filter() *# Empty **(as a result of worker #1)*
  > vfilt = g.get_vertex_filter() *# Empty **(as** a result of worker #1)*
  *time t=2 : **Worker #1 continues until **(graph_tool/__init__.py:2248)
and set back the graph g in its original (ie. t<0) state.*
* | *g.set_directed(is_directed)
  > g.set_edge_filter(efilt[0], efilt[1])
  > g.set_vertex_filter(vfilt[0], vfilt[1])
  *time t=3 : Worker #2 **continues until (graph_tool/__init__.py:2248) and
set back in its incorrect **(ie. t=1) **state.*

Is there something I miss ?

Bests,
François.

attachment.html (11 KB)

I didn't realize all the workers shared a reference to the same graph.

Indeed this can be improved. To be honest, thread safety from the python
side is something that I often do not pay attention to, since it is almost
never worth it. But please open an issue in the website to make
copy_property() thread safe, and I will get around to it.

Best,
Tiago

Hello thanks for the blazing fast answer! From what I understand, if I
provide a graphView per thread the problem should not occur, is that
correct?

Regards,
F.

attachment.html (4.19 KB)

Yes, I believe so.

Best,
Tiago

Ok, I'll give it a try and keep you updated.
Bests,
François.

attachment.html (4.39 KB)