Memory address changing when accessing an edge of the graph
Hi all, When manipulating a graph, I create a list of edges with a certain relation to a vertex (they are the edges common to the vertex‘s neighbour). I create this list at some moment in the code, and don’t update it. Later down the execution, I try to remove one of those edges from the list prior to its deletion. That use to work without problem, until today (but I have ran the code since about a month ago). Now when I try to |remove()| the edge, I get a ValueError because python tels me the edge is not in the list. When I look in the list, I see a reference to an edge with the correct source and target, but a different memory address, so I suspect that might be why it can be found… Actually, when accessing an edge form a graph, the edge's address changes: | graph.edge(4747,2693, all_edges=True) [<Edge objectwith source'4747' and target'2693' at0x7fd2d437c048>] graph.edge(4747,2693, all_edges=False) <Edge objectwith source'4747' and target'2693' at0x7fd2d437c0d8> graph.edge(4747,2693, all_edges=False) <Edge objectwith source'4747' and target'2693' at0x7fd2d437c168>| Note the latest digits of the address aren't the same. I see the same kind of things for the vertices, and it produces other bugs down the road (like this one when trying to add an edge between two vertices): |TypeError: No registered converter was able to extract a C++ reference to type graph_tool::PythonVertexfrom this Python object of type NoneType| What is strange is that I can't reproduce the bug in more simple settings, where edge list is created at the same execution level as the call to remove one of those. I seem to recall inspecting the addresses when debugging, to check that my object were consistent, that‘s why it’s strange that they change now. So I'm a bit confused here… Best, Guillaume
Forgot to mention, I was using a fresh python3-graph-tool package from ubuntu, with python3.4 and gt._version '2.2.35 (commit c32ffd6c, Thu Sep 11 16:39:47 2014 +0200)' Best, Guillaume Le 11/10/2014 17:11, Guillaume Gay a écrit :
Hi all,
When manipulating a graph, I create a list of edges with a certain relation to a vertex (they are the edges common to the vertex‘s neighbour). I create this list at some moment in the code, and don’t update it.
Later down the execution, I try to remove one of those edges from the list prior to its deletion. That use to work without problem, until today (but I have ran the code since about a month ago).
Now when I try to |remove()| the edge, I get a ValueError because python tels me the edge is not in the list. When I look in the list, I see a reference to an edge with the correct source and target, but a different memory address, so I suspect that might be why it can be found…
Actually, when accessing an edge form a graph, the edge's address changes:
| graph.edge(4747,2693, all_edges=True) [<Edge objectwith source'4747' and target'2693' at0x7fd2d437c048>]
graph.edge(4747,2693, all_edges=False) <Edge objectwith source'4747' and target'2693' at0x7fd2d437c0d8>
graph.edge(4747,2693, all_edges=False) <Edge objectwith source'4747' and target'2693' at0x7fd2d437c168>|
Note the latest digits of the address aren't the same. I see the same kind of things for the vertices, and it produces other bugs down the road (like this one when trying to add an edge between two vertices):
|TypeError: No registered converter was able to extract a C++ reference to type graph_tool::PythonVertexfrom this Python object of type NoneType|
What is strange is that I can't reproduce the bug in more simple settings, where edge list is created at the same execution level as the call to remove one of those.
I seem to recall inspecting the addresses when debugging, to check that my object were consistent, that‘s why it’s strange that they change now.
So I'm a bit confused here…
Best,
Guillaume
On 11.10.2014 17:11, Guillaume Gay wrote:
Hi all,
When manipulating a graph, I create a list of edges with a certain relation to a vertex (they are the edges common to the vertex‘s neighbour). I create this list at some moment in the code, and don’t update it.
Later down the execution, I try to remove one of those edges from the list prior to its deletion. That use to work without problem, until today (but I have ran the code since about a month ago).
Now when I try to |remove()| the edge, I get a ValueError because python tels me the edge is not in the list. When I look in the list, I see a reference to an edge with the correct source and target, but a different memory address, so I suspect that might be why it can be found…
Actually, when accessing an edge form a graph, the edge's address changes:
| graph.edge(4747, 2693, all_edges=True) [<Edge object with source '4747' and target '2693' at 0x7fd2d437c048>]
graph.edge(4747, 2693, all_edges=False) <Edge object with source '4747' and target '2693' at 0x7fd2d437c0d8>
graph.edge(4747, 2693, all_edges=False) <Edge object with source '4747' and target '2693' at 0x7fd2d437c168>|
This is entirely normal. Whenever Graph.edge() or Graph.vertex() is called, a new edge or vertex descriptor instance is created. Many different vertex or edge descriptors can point to the same edge or vertex, i.e. they are not unique. The same thing happens for many other python objects: >> a = 2.2 >> id(a) 140516113676112 >> a = 2.2 >> id(a) 140516113676184 This should not be a problem, since vertex or edge descriptors which point to the same vertex or edge should be considered equal, regardless if they are distinct instances: >> g.edge(76, 62) == g.edge(76, 62) True
Note the latest digits of the address aren't the same. I see the same kind of things for the vertices, and it produces other bugs down the road (like this one when trying to add an edge between two vertices):
|TypeError: No registered converter was able to extract a C++ reference to type graph_tool::PythonVertex from this Python object of type NoneType|
I don't think this is related to the issue above. Best, Tiago -- Tiago de Paula Peixoto <tiago@skewed.de>
Hi, thanks, that was not clear to me. But then, I don't see where this issue comes from, it was working nicely before, and I didn't modify my code. Is it bad strategy to store lists of edges references then, and it was working just by chance before? Would the code be more robust if I stored (source, target) tuples instead? On a side note, can I downgrade the deb package, if you keep the older ones around? Thanks for your time and patience with silly questions, Best, Guillaume Le 12/10/2014 01:57, Tiago de Paula Peixoto a écrit :
On 11.10.2014 17:11, Guillaume Gay wrote:
Hi all,
When manipulating a graph, I create a list of edges with a certain relation to a vertex (they are the edges common to the vertex‘s neighbour). I create this list at some moment in the code, and don’t update it.
Later down the execution, I try to remove one of those edges from the list prior to its deletion. That use to work without problem, until today (but I have ran the code since about a month ago).
Now when I try to |remove()| the edge, I get a ValueError because python tels me the edge is not in the list. When I look in the list, I see a reference to an edge with the correct source and target, but a different memory address, so I suspect that might be why it can be found…
Actually, when accessing an edge form a graph, the edge's address changes:
| graph.edge(4747, 2693, all_edges=True) [<Edge object with source '4747' and target '2693' at 0x7fd2d437c048>]
graph.edge(4747, 2693, all_edges=False) <Edge object with source '4747' and target '2693' at 0x7fd2d437c0d8>
graph.edge(4747, 2693, all_edges=False) <Edge object with source '4747' and target '2693' at 0x7fd2d437c168>| This is entirely normal. Whenever Graph.edge() or Graph.vertex() is called, a new edge or vertex descriptor instance is created. Many different vertex or edge descriptors can point to the same edge or vertex, i.e. they are not unique. The same thing happens for many other python objects:
>> a = 2.2 >> id(a) 140516113676112 >> a = 2.2 >> id(a) 140516113676184
This should not be a problem, since vertex or edge descriptors which point to the same vertex or edge should be considered equal, regardless if they are distinct instances:
>> g.edge(76, 62) == g.edge(76, 62) True
Note the latest digits of the address aren't the same. I see the same kind of things for the vertices, and it produces other bugs down the road (like this one when trying to add an edge between two vertices):
|TypeError: No registered converter was able to extract a C++ reference to type graph_tool::PythonVertex from this Python object of type NoneType| I don't think this is related to the issue above.
Best, Tiago
_______________________________________________ graph-tool mailing list graph-tool@skewed.de http://lists.skewed.de/mailman/listinfo/graph-tool
Here is maybe a code snippet demonstrating what I can't understand: |>>> j_e = eptm.graph.edge(4747, 2693)
print('looking for: {}'.format(j_e))
looking for: (4747, 2693)
## List of edges we're looking in:''') for e in eptm.cells.junctions[mother_cell]: print(e)
(4747, 2693) (4747, 2692) (980, 2693) (4989, 980) (4989, 4990) (4990, 2692)
print('\n Is {} in the list? {}'.format( j_e, j_e in eptm.cells.junctions[mother_cell]))
Is (4747, 2693) in the list? False| Here, |eptm| and |eptm.cells| are just container classes, and |eptm.cells.junctions| is a dictionary with the graphs vertices as keys. Best, Guillaume Le 12/10/2014 01:57, Tiago de Paula Peixoto a écrit :
On 11.10.2014 17:11, Guillaume Gay wrote:
Hi all,
When manipulating a graph, I create a list of edges with a certain relation to a vertex (they are the edges common to the vertex‘s neighbour). I create this list at some moment in the code, and don’t update it.
Later down the execution, I try to remove one of those edges from the list prior to its deletion. That use to work without problem, until today (but I have ran the code since about a month ago).
Now when I try to |remove()| the edge, I get a ValueError because python tels me the edge is not in the list. When I look in the list, I see a reference to an edge with the correct source and target, but a different memory address, so I suspect that might be why it can be found…
Actually, when accessing an edge form a graph, the edge's address changes:
| graph.edge(4747, 2693, all_edges=True) [<Edge object with source '4747' and target '2693' at 0x7fd2d437c048>]
graph.edge(4747, 2693, all_edges=False) <Edge object with source '4747' and target '2693' at 0x7fd2d437c0d8>
graph.edge(4747, 2693, all_edges=False) <Edge object with source '4747' and target '2693' at 0x7fd2d437c168>| This is entirely normal. Whenever Graph.edge() or Graph.vertex() is called, a new edge or vertex descriptor instance is created. Many different vertex or edge descriptors can point to the same edge or vertex, i.e. they are not unique. The same thing happens for many other python objects:
>> a = 2.2 >> id(a) 140516113676112 >> a = 2.2 >> id(a) 140516113676184
This should not be a problem, since vertex or edge descriptors which point to the same vertex or edge should be considered equal, regardless if they are distinct instances:
>> g.edge(76, 62) == g.edge(76, 62) True
Note the latest digits of the address aren't the same. I see the same kind of things for the vertices, and it produces other bugs down the road (like this one when trying to add an edge between two vertices):
|TypeError: No registered converter was able to extract a C++ reference to type graph_tool::PythonVertex from this Python object of type NoneType| I don't think this is related to the issue above.
Best, Tiago
_______________________________________________ graph-tool mailing list graph-tool@skewed.de http://lists.skewed.de/mailman/listinfo/graph-tool
On 12.10.2014 16:19, Guillaume Gay wrote:
Here is maybe a code snippet demonstrating what I can't understand:
|>>> j_e = eptm.graph.edge(4747, 2693)
print('looking for: {}'.format(j_e))
looking for: (4747, 2693)
## List of edges we're looking in:''') for e in eptm.cells.junctions[mother_cell]: print(e)
(4747, 2693) (4747, 2692) (980, 2693) (4989, 980) (4989, 4990) (4990, 2692)
print('\n Is {} in the list? {}'.format( j_e, j_e in eptm.cells.junctions[mother_cell]))
Is (4747, 2693) in the list? False|
Here, |eptm| and |eptm.cells| are just container classes, and |eptm.cells.junctions| is a dictionary with the graphs vertices as keys.
Well, I can't reproduce this: >>> l = [g.edge(76, 62)] >>> g.edge(76, 62) in l True Are you sure your list contains Edge instances, and not tuples? Unfortunately, without a self-contained example, I cannot investigate further. Best, Tiago -- Tiago de Paula Peixoto <tiago@skewed.de>
Hi, The list do contain edges, and `eptm.cells.junctions` is a vertex PropertyMap with 'object' dtype, not a dictionary. What makes debugging difficult, is that if I update the edge list just before I look for the edge, it is indeed found, it looks like it's kind of 'lost' between the time I create the edge list (at the initialization of the container class) and the time I try to remove the edge from the list. Very strange... The same code base works fine with graph-tool version 2.2.28 (I had a machine not yet updated on which I can run the code for now). I'll try to work out a minimal example demonstrating the issue. Thanks, Guillaume Le 14/10/2014 08:59, Tiago de Paula Peixoto a écrit :
On 12.10.2014 16:19, Guillaume Gay wrote:
Here is maybe a code snippet demonstrating what I can't understand:
|>>> j_e = eptm.graph.edge(4747, 2693)
print('looking for: {}'.format(j_e)) looking for: (4747, 2693) ## List of edges we're looking in:''') for e in eptm.cells.junctions[mother_cell]: print(e) (4747, 2693) (4747, 2692) (980, 2693) (4989, 980) (4989, 4990) (4990, 2692) print('\n Is {} in the list? {}'.format( j_e, j_e in eptm.cells.junctions[mother_cell])) Is (4747, 2693) in the list? False|
Here, |eptm| and |eptm.cells| are just container classes, and |eptm.cells.junctions| is a dictionary with the graphs vertices as keys. Well, I can't reproduce this:
>>> l = [g.edge(76, 62)] >>> g.edge(76, 62) in l True
Are you sure your list contains Edge instances, and not tuples?
Unfortunately, without a self-contained example, I cannot investigate further.
Best, Tiago
_______________________________________________ graph-tool mailing list graph-tool@skewed.de http://lists.skewed.de/mailman/listinfo/graph-tool
Hi Tiago, I managed to get a self contained example. The strange behaviour appears when filtering/unfiltering between the moment I update the edge list propertymap and the moment I query this list. Here's the code: https://gist.github.com/glyg/2005df62820d35e7efe4 And here's the output: ********************************* gt version: 2.2.35 (commit c32ffd6c, Thu Sep 11 16:39:47 2014 +0200) sytem info: 3.4.1 |Anaconda 2.1.0 (64-bit)| (default, Sep 10 2014, 17:10:18) [GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] ********************************* A simple graph: <Graph object, undirected, with 400 vertices and 1121 edges at 0x7fe8ad23fa58> Filtered graph has 200 vertices vertex 0 neighbours: ['(0, 21)', '(0, 20)'] Looking for (0, 21) in the neighbours: False Looking for (0, 21) in the neighbours: True Le 14/10/2014 08:59, Tiago de Paula Peixoto a écrit :
On 12.10.2014 16:19, Guillaume Gay wrote:
Here is maybe a code snippet demonstrating what I can't understand:
|>>> j_e = eptm.graph.edge(4747, 2693)
print('looking for: {}'.format(j_e)) looking for: (4747, 2693) ## List of edges we're looking in:''') for e in eptm.cells.junctions[mother_cell]: print(e) (4747, 2693) (4747, 2692) (980, 2693) (4989, 980) (4989, 4990) (4990, 2692) print('\n Is {} in the list? {}'.format( j_e, j_e in eptm.cells.junctions[mother_cell])) Is (4747, 2693) in the list? False|
Here, |eptm| and |eptm.cells| are just container classes, and |eptm.cells.junctions| is a dictionary with the graphs vertices as keys. Well, I can't reproduce this:
>>> l = [g.edge(76, 62)] >>> g.edge(76, 62) in l True
Are you sure your list contains Edge instances, and not tuples?
Unfortunately, without a self-contained example, I cannot investigate further.
Best, Tiago
_______________________________________________ graph-tool mailing list graph-tool@skewed.de http://lists.skewed.de/mailman/listinfo/graph-tool
On 16.10.2014 10:31, Guillaume Gay wrote:
Hi Tiago,
I managed to get a self contained example. The strange behaviour appears when filtering/unfiltering between the moment I update the edge list propertymap and the moment I query this list.
This is indeed a bug. It has been fixed now in git: https://git.skewed.de/count0/graph-tool/commit/26f7d07b3359098fcc551b0a11597... Thanks for the self-contained minimal example! Best, Tiago -- Tiago de Paula Peixoto <tiago@skewed.de>
Thanks to you for the quick fix... I'll try to compile it now.... Guillaume Le 16/10/2014 12:53, Tiago de Paula Peixoto a écrit :
On 16.10.2014 10:31, Guillaume Gay wrote:
Hi Tiago,
I managed to get a self contained example. The strange behaviour appears when filtering/unfiltering between the moment I update the edge list propertymap and the moment I query this list. This is indeed a bug. It has been fixed now in git:
https://git.skewed.de/count0/graph-tool/commit/26f7d07b3359098fcc551b0a11597...
Thanks for the self-contained minimal example!
Best, Tiago
_______________________________________________ graph-tool mailing list graph-tool@skewed.de http://lists.skewed.de/mailman/listinfo/graph-tool
participants (2)
-
Guillaume Gay -
Tiago de Paula Peixoto