Hi everyone,
I was wondering if it would be possible to provide some more examples of how to run a nested mixed membership SBM with edge weights. The new version seems to have removed the "overlap=True" option for state_args in the minimize_* functions.
Is this the correct way to do it now?
import graph_tool as gta
import numpy as np g = .... # build graph e_score = .... #Set edge weights state_args = dict( deg_corr=deg_corr, base_type=gta.inference.overlap_blockmodel.OverlapBlockState, B=2*g.num_edges(), #B_max deg_corr=True, recs=[e_score], rec_types=["real-normal"]) state = gta.inference.minimize_nested_blockmodel_dl( g, state_args=state_args, multilevel_mcmc_args=dict(verbose=True)) # improve solution with merge-split state = state.copy(bs=state.get_bs() + [np.zeros(1)] * 4, sampling=True)
for i in range(100):
if i%10==0: print(".", end="") ret = state.multiflip_mcmc_sweep(niter=10, beta=np.inf, verbose=True)
I am currently running this for a fully connected bipartite graph with 3454 nodes and 55008 edges. I understand it would take longer than the non-overlapping version, but do you have any suggestions on how to speed it up? The non-overlapping version takes about 15 minutes, while the overlapping version is still running after 1 day.
Thanks for your help, Eli
Am 11.11.21 um 03:03 schrieb Eli Draizen:
Hi everyone,
I was wondering if it would be possible to provide some more examples of how to run a nested mixed membership SBM with edge weights. The new version seems to have removed the "overlap=True" option for state_args in the minimize_* functions.
Indeed, I will add more examples about this. Could you please open an issue in the website so I don't forget?
Is this the correct way to do it now?
import graph_tool as gta import numpy as np g = .... # build graph e_score = .... #Set edge weights state_args = dict( deg_corr=deg_corr, base_type=gta.inference.overlap_blockmodel.OverlapBlockState, B=2*g.num_edges(), #B_max deg_corr=True, recs=[e_score], rec_types=["real-normal"]) state = gta.inference.minimize_nested_blockmodel_dl( g, state_args=state_args, multilevel_mcmc_args=dict(verbose=True)) # improve solution with merge-split state = state.copy(bs=state.get_bs() + [np.zeros(1)] * 4, sampling=True) for i in range(100): if i%10==0: print(".", end="") ret = state.multiflip_mcmc_sweep(niter=10, beta=np.inf, verbose=True)
This is correct. But note that the "sampling=True" option is no longer needed.
I am currently running this for a fully connected bipartite graph with 3454 nodes and 55008 edges. I understand it would take longer than the non-overlapping version, but do you have any suggestions on how to speed it up? The non-overlapping version takes about 15 minutes, while the overlapping version is still running after 1 day.
The new version will contain a much faster code for the overlapping case!
But in the mean-time, what you can do is to fit the non-overlapping model first, and use that as a starting point to the MCMC with overlap. You do that simply by doing:
state = state.copy(state_args=dict(overlap=True))
Best, Tiago
Hi Tiago,
Just checking to see the status on this. I've tried the newer version (2.44) but am still having the same issues. Your proposed solution of copying the state and adding the overlap work for the non-nested block models, but I was having trouble with the nested version. I am also now running into an issue where I can only run non degree corrected sbms, otherwise it hangs (logs attached). If you have any suggestions, I would really appreciate it.
Thanks, Eli
-> = copied state adding the following options, then reran mcmc + = defined in minimization function followed by mcmc
NDC = yes NDC+Overlap = slow NDC->Overlap = yes NDC+Nested = slow NDC+Nested+Overlap = slow NDC+Nested->Overlap (not implemented)
NDC+Nested+Overlap -> DC = slow
DC+Overlap+Nested = hangs (This is the goal) DC+Overlap = hangs DC+Nested->Overlap (not implemented)
On Thu, Nov 11, 2021 at 3:16 AM Tiago de Paula Peixoto tiago@skewed.de wrote:
Am 11.11.21 um 03:03 schrieb Eli Draizen:
Hi everyone,
I was wondering if it would be possible to provide some more examples of how to run a nested mixed membership SBM with edge weights. The new version seems to have removed the "overlap=True" option for state_args in the minimize_* functions.
Indeed, I will add more examples about this. Could you please open an issue in the website so I don't forget?
Is this the correct way to do it now?
import graph_tool as gta import numpy as np g = .... # build graph e_score = .... #Set edge weights state_args = dict( deg_corr=deg_corr, base_type=gta.inference.overlap_blockmodel.OverlapBlockState, B=2*g.num_edges(), #B_max deg_corr=True, recs=[e_score], rec_types=["real-normal"]) state = gta.inference.minimize_nested_blockmodel_dl( g, state_args=state_args, multilevel_mcmc_args=dict(verbose=True)) # improve solution with merge-split state = state.copy(bs=state.get_bs() + [np.zeros(1)] * 4,sampling=True)
for i in range(100): if i%10==0: print(".", end="") ret = state.multiflip_mcmc_sweep(niter=10, beta=np.inf, verbose=True)This is correct. But note that the "sampling=True" option is no longer needed.
I am currently running this for a fully connected bipartite graph with 3454 nodes and 55008 edges. I understand it would take longer than the non-overlapping version, but do you have any suggestions on how to speed it up? The non-overlapping version takes about 15 minutes, while the overlapping version is still running after 1 day.
The new version will contain a much faster code for the overlapping case!
But in the mean-time, what you can do is to fit the non-overlapping model first, and use that as a starting point to the MCMC with overlap. You do that simply by doing:
state = state.copy(state_args=dict(overlap=True))Best, Tiago
-- Tiago de Paula Peixoto tiago@skewed.de _______________________________________________ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-leave@skewed.de
Hello,
I think I was able to figure it out using the code from the "Characterizing the posterior distribution" section. I start by equilibrating a degree-corrected NestedBlockState with edge weights and then inferring partition modes with ModeClusterState. Is this the same as running minimize_nested_blockmodel_dl? Are the minimize* functions not used anymore?
Also, is it normal to have inferred only one mode?
Thanks for your help, Eli
On Wed, Mar 2, 2022 at 4:48 PM Eli Draizen ed4bu@virginia.edu wrote:
Hi Tiago,
Just checking to see the status on this. I've tried the newer version (2.44) but am still having the same issues. Your proposed solution of copying the state and adding the overlap work for the non-nested block models, but I was having trouble with the nested version. I am also now running into an issue where I can only run non degree corrected sbms, otherwise it hangs (logs attached). If you have any suggestions, I would really appreciate it.
Thanks, Eli
-> = copied state adding the following options, then reran mcmc
- = defined in minimization function followed by mcmc
NDC = yes NDC+Overlap = slow NDC->Overlap = yes NDC+Nested = slow NDC+Nested+Overlap = slow NDC+Nested->Overlap (not implemented)
NDC+Nested+Overlap -> DC = slow
DC+Overlap+Nested = hangs (This is the goal) DC+Overlap = hangs DC+Nested->Overlap (not implemented)
On Thu, Nov 11, 2021 at 3:16 AM Tiago de Paula Peixoto tiago@skewed.de wrote:
Am 11.11.21 um 03:03 schrieb Eli Draizen:
Hi everyone,
I was wondering if it would be possible to provide some more examples
of
how to run a nested mixed membership SBM with edge weights. The new version seems to have removed the "overlap=True" option for state_args in the minimize_* functions.
Indeed, I will add more examples about this. Could you please open an issue in the website so I don't forget?
Is this the correct way to do it now?
import graph_tool as gta import numpy as np g = .... # build graph e_score = .... #Set edge weights state_args = dict( deg_corr=deg_corr, base_type=gta.inference.overlap_blockmodel.OverlapBlockState, B=2*g.num_edges(), #B_max deg_corr=True, recs=[e_score], rec_types=["real-normal"]) state = gta.inference.minimize_nested_blockmodel_dl( g, state_args=state_args, multilevel_mcmc_args=dict(verbose=True)) # improve solution with merge-split state = state.copy(bs=state.get_bs() + [np.zeros(1)] * 4,sampling=True)
for i in range(100): if i%10==0: print(".", end="") ret = state.multiflip_mcmc_sweep(niter=10, beta=np.inf, verbose=True)This is correct. But note that the "sampling=True" option is no longer needed.
I am currently running this for a fully connected bipartite graph with 3454 nodes and 55008 edges. I understand it would take longer than the non-overlapping version, but do you have any suggestions on how to
speed
it up? The non-overlapping version takes about 15 minutes, while the overlapping version is still running after 1 day.
The new version will contain a much faster code for the overlapping case!
But in the mean-time, what you can do is to fit the non-overlapping model first, and use that as a starting point to the MCMC with overlap. You do that simply by doing:
state = state.copy(state_args=dict(overlap=True))Best, Tiago
-- Tiago de Paula Peixoto tiago@skewed.de _______________________________________________ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-leave@skewed.de
-- PhD Candidate, Phil Bourne's lab University of Virginia
Am 04.03.22 um 06:37 schrieb Eli Draizen:
I think I was able to figure it out using the code from the "Characterizing the posterior distribution" section. I start by equilibrating a degree-corrected NestedBlockState with edge weights and then inferring partition modes with ModeClusterState. Is this the same as running minimize_nested_blockmodel_dl?
No, this is not the same. ModeClusterStates characterizes the whole distribution of answers, while minimize_blockmodel_dl() finds the single most likely/compressive partition.
Are the minimize* functions not used anymore?
Yes they are. They have different goals.
Also, is it normal to have inferred only one mode?
Yes, it can happen. This means the solutions are clustered around a single typical solution, rather than several different ones.
This is discussed in this paper:
https://dx.doi.org/10.1103/PhysRevX.11.021003
Best, Tiago
-
Eli Draizen -
Tiago de Paula Peixoto