Hi, I have the same question. Upon running code attempting to use GraphViews, I get an error during pickling:
error: 'i' format requires -2147483648 <= number <= 2147483647
More specifically, it looks like a line inside joblib is unhappy: CustomizablePickler(buffer, self._reducers).dump(obj)
And this takes us to a struct packing line: header = struct.pack("!i", n)
So, if I had to guess, I'd suspect joblib is trying to pickle the whole graph rather than the GraphView reference, or something like this. Was either of you able to get code to successfully parallelize using GraphViews to avoid copying?
This is on python 3.6.5 with graph_tool version '2.26 (commit , )' , joblib version '0.11'
Below is a minimal breaking example, if it helps. I am also happy to provide other information such as tracebacks.
def toy_func(g): return g.vertex_properties['skim'][0][1]
vmr = [0, 1] g = load_graph(path) # 22,000 vertex directed graph (a road network) skim_table = shortest_distance(g, weights=g.edge_properties["weight"]) g.properties['skim'] = skim_table p(joblib.delayed(toy_func)(GraphView(g)) for i in range(10))
-- Sent from: http://main-discussion-list-for-the-graph-tool-project.982480.n3.nabble.com/