Takes enormous memory and then eventually gets terminated.
Hi, I have a graph with around 9500 nodes and 37000 edges. I want to determine the SBM corresponding to this network, but when I run minimize_blockmodel_dl function on it, I get a 'killed:9' error. I checked the memory consumption and it exceeded 50 GB at one point. Is this expected? Or is there something I am doing wrong? -- View this message in context: http://main-discussion-list-for-the-graph-tool-project.982480.n3.nabble.com/... Sent from the Main discussion list for the graph-tool project mailing list archive at Nabble.com.
Hi Isuki, No, that is not expected. We'll need more information to help you, can you provide at least: 1. graph-tool version you're using 3. the graph, exported by graph-tool 2. a minimal script that reproduces the issue Cheers, ale .~´ Le mardi 30 mai 2017 à 02:13 -0700, isukritgupta a écrit :
Hi, I have a graph with around 9500 nodes and 37000 edges. I want to determine the SBM corresponding to this network, but when I run minimize_blockmodel_dl function on it, I get a 'killed:9' error. I checked the memory consumption and it exceeded 50 GB at one point.
Is this expected? Or is there something I am doing wrong?
-- View this message in context: http://main-discussion-list-for-the-gra ph-tool-project.982480.n3.nabble.com/Takes-enormous-memory-and-then- eventually-gets-terminated-tp4027236.html Sent from the Main discussion list for the graph-tool project mailing list archive at Nabble.com. _______________________________________________ graph-tool mailing list graph-tool@skewed.de https://lists.skewed.de/mailman/listinfo/graph-tool
1. graph_tool version: 2.22 (commit 44bf2b92, Thu Mar 2 23:08:39 2017 +0000) 2. How do I put the graph over here? 3. Here: import numpy as np from graph_tool.all import * f_network = np.genfromtxt("Name of Edge List File", delimiter=',') f_network = f_network.astype(int) for edge in f_network: g.add_edge(edge[0], edge[1], add_missing = True) state_dc = minimize_blockmodel_dl(g, deg_corr = True, verbose = True); state_ndc = minimize_blockmodel_dl(g, deg_corr = False, verbose = False); Thanks, Sukrit -- View this message in context: http://main-discussion-list-for-the-graph-tool-project.982480.n3.nabble.com/... Sent from the Main discussion list for the graph-tool project mailing list archive at Nabble.com.
binary_edgelist_hprd.csv <http://main-discussion-list-for-the-graph-tool-project.982480.n3.nabble.com/file/n4027240/binary_edgelist_hprd.csv> sbm_ppi.py <http://main-discussion-list-for-the-graph-tool-project.982480.n3.nabble.com/file/n4027240/sbm_ppi.py> Uploaded both the CSV and python code. Thanks, Sukrit -- View this message in context: http://main-discussion-list-for-the-graph-tool-project.982480.n3.nabble.com/... Sent from the Main discussion list for the graph-tool project mailing list archive at Nabble.com.
Also, I created the gt file for the network, but it is around 800 MB! So, cannot upload it. -- View this message in context: http://main-discussion-list-for-the-graph-tool-project.982480.n3.nabble.com/... Sent from the Main discussion list for the graph-tool project mailing list archive at Nabble.com.
On 01.06.2017 09:35, isukritgupta wrote:
Also, I created the gt file for the network, but it is around 800 MB! So, cannot upload it.
It's pretty obvious what is happening. If you look in your file, there are node indexes that are huge, eg. 100049587. So after you add your edges, the graph will have more than 100 million nodes, the vast majority of which have degree zero. So what you need to do is to map these non-contiguous vertices to a contiguous range. You can do this by hand, or you can add the add_edge_list() function that does it for you if you pass the hashed=True parameter: https://graph-tool.skewed.de/static/doc/graph_tool.html#graph_tool.Graph.add... Best, Tiago -- Tiago de Paula Peixoto <tiago@skewed.de>
You're right. This solved it! Thanks, Sukrit ________________ Sukrit Gupta <http://sites.google.com/view/sukrit>, B.Eng. (CS, PEC, In). C: 0091 856.680.0668 | E: *isukritgupta@gmail.com <isukritgupta@gmail.com>* On Thu, Jun 1, 2017 at 4:48 PM, Tiago de Paula Peixoto <tiago@skewed.de> wrote:
On 01.06.2017 09:35, isukritgupta wrote:
Also, I created the gt file for the network, but it is around 800 MB! So, cannot upload it.
It's pretty obvious what is happening.
If you look in your file, there are node indexes that are huge, eg. 100049587.
So after you add your edges, the graph will have more than 100 million nodes, the vast majority of which have degree zero.
So what you need to do is to map these non-contiguous vertices to a contiguous range. You can do this by hand, or you can add the add_edge_list() function that does it for you if you pass the hashed=True parameter:
https://graph-tool.skewed.de/static/doc/graph_tool.html# graph_tool.Graph.add_edge_list
Best, Tiago
-- Tiago de Paula Peixoto <tiago@skewed.de> _______________________________________________ graph-tool mailing list graph-tool@skewed.de https://lists.skewed.de/mailman/listinfo/graph-tool
participants (4)
-
Alexandre Hannud Abdo -
isukritgupta -
Sukrit Gupta
-
Tiago de Paula Peixoto