Posterior entropy approximations = 0?
Hello, I am sampling the posterior distribution in order to carry out some model selection (much like outlined in the cookbook). Interestingly both the Bethe as well as the mean field approximations of the posterior entropy appear to be returning an answer of 0.0. I was thus after some opinions: Are there cases where I might reasonably expect this to be the correct answer/how would people check that it is not simply being caused by the approximations not being applicable to my data set? Thank you for any help in advance! -- Sent from: http://main-discussion-list-for-the-graph-tool-project.982480.n3.nabble.com/
Am 22.05.2018 um 12:56 schrieb P-M:
I am sampling the posterior distribution in order to carry out some model selection (much like outlined in the cookbook). Interestingly both the Bethe as well as the mean field approximations of the posterior entropy appear to be returning an answer of 0.0. I was thus after some opinions: Are there cases where I might reasonably expect this to be the correct answer/how would people check that it is not simply being caused by the approximations not being applicable to my data set?
Differently from Bethe, the MF approximation can only return zero if the true entropy is also zero. This means that the posterior distribution is concentrated around a single partition with probability one, and others with probability zero. This should only occur if the SBM is a perfect fit, e.g. for very dense networks sampled from the SBM ensemble. For real data this should almost never happen. Best, Tiago -- Tiago de Paula Peixoto <tiago@skewed.de>
Thank you, that confirms my suspicion that something strange is going on here. The average description length across 200,000 sweeps appears to be roughly three times that returned by "minimize_nested_blockmodel_dl" so it seems to me like the posterior distribution is not centered around a single partition. Do you have any more thoughts on what might cause this? (I am trying to compile the current git version too to see whether the problem persists there.) Best, Philipp -- Sent from: http://main-discussion-list-for-the-graph-tool-project.982480.n3.nabble.com/
Am 23.05.2018 um 16:13 schrieb P-M:
Thank you, that confirms my suspicion that something strange is going on here. The average description length across 200,000 sweeps appears to be roughly three times that returned by "minimize_nested_blockmodel_dl" so it seems to me like the posterior distribution is not centered around a single partition.
I don't see how you can conclude this from the reason given.
Do you have any more thoughts on what might cause this? (I am trying to compile the current git version too to see whether the problem persists there.)
It is difficult to blindly guess, without being given more information. Best, Tiago -- Tiago de Paula Peixoto <tiago@skewed.de>
Tiago Peixoto wrote
Thank you, that confirms my suspicion that something strange is going on here. The average description length across 200,000 sweeps appears to be roughly three times that returned by "minimize_nested_blockmodel_dl" so it seems to me like the posterior distribution is not centered around a single partition.
I don't see how you can conclude this from the reason given.
If I have a single peak in the posterior with a likelihood of 1 should I not expect that sampling from the posterior and maximising the posterior likelihood return the same value? Following that, if the average description length obtained by sampling from the posterior is different I must have had other partitions with non-zero probabilities contributing which contradicts the entropy of zero. Am I making a logical mistake/missing something here? Tiago Peixoto wrote
It is difficult to blindly guess, without being given more information.
Once i have verified whether the problem persists with the current git version I will try and provide an MWE. The data is empirical however so I would be surprised by a perfect fit. Best, Philipp -- Sent from: http://main-discussion-list-for-the-graph-tool-project.982480.n3.nabble.com/
Am 23.05.2018 um 18:26 schrieb P-M:
If I have a single peak in the posterior with a likelihood of 1 should I not expect that sampling from the posterior and maximising the posterior likelihood return the same value? Following that, if the average description length obtained by sampling from the posterior is different I must have had other partitions with non-zero probabilities contributing which contradicts the entropy of zero. Am I making a logical mistake/missing something here?
There could be several of such isolated peaks of the posterior, where the MCMC gets trapped. -- Tiago de Paula Peixoto <tiago@skewed.de>
participants (2)
-
P-M -
Tiago de Paula Peixoto