Hi, I have a graph with around 9500 nodes and 37000 edges. I want to
determine the SBM corresponding to this network, but when I run
minimize_blockmodel_dl function on it, I get a 'killed:9' error. I checked
the memory consumption and it exceeded 50 GB at one point.
Is this expected? Or is there something I am doing wrong?
Hi Isuki,
No, that is not expected.
We'll need more information to help you, can you provide at least:
1. graph-tool version you're using
3. the graph, exported by graph-tool
2. a minimal script that reproduces the issue
Cheers,
ale
.~´
1. graph_tool version: 2.22 (commit 44bf2b92, Thu Mar 2 23:08:39 2017 +0000)
2. How do I put the graph over here?
3. Here:
import numpy as np
from graph_tool.all import *
f_network = np.genfromtxt("Name of Edge List File", delimiter=',')
f_network = f_network.astype(int)
for edge in f_network:
g.add_edge(edge[0], edge[1], add_missing = True)
state_dc = minimize_blockmodel_dl(g, deg_corr = True, verbose = True);
state_ndc = minimize_blockmodel_dl(g, deg_corr = False, verbose = False);
Thanks,
Sukrit
binary_edgelist_hprd.csv
<http://main-discussion-list-for-the-graph-tool-project.982480.n3.nabble.com/file/n4027240/binary_edgelist_hprd.csv>
sbm_ppi.py
<http://main-discussion-list-for-the-graph-tool-project.982480.n3.nabble.com/file/n4027240/sbm_ppi.py>
Uploaded both the CSV and python code.
Thanks,
Sukrit
Also, I created the gt file for the network, but it is around 800 MB!
So, cannot upload it.
It's pretty obvious what is happening.
If you look in your file, there are node indexes that are huge,
eg. 100049587.
So after you add your edges, the graph will have more than 100 million
nodes, the vast majority of which have degree zero.
So what you need to do is to map these non-contiguous vertices to a
contiguous range. You can do this by hand, or you can add the
add_edge_list() function that does it for you if you pass the
hashed=True parameter:
https://graph-tool.skewed.de/static/doc/graph_tool.html#graph_tool.Graph.add_edge_list
Best,
Tiago