Save state to disk for long running nested block state analysis
G'day! Just wondering whether it would be possible to implement, e.g. as a parameter of the minimize_nested_block_model function, a way to save intermediate results to disk at certain intervals and to resume the analysis, e.g. after a forced shut-down, or any other interruption. If I'd have some hints I might try to have a go at this feature myself. Or is this already possible with some lower level functions? Cheers! Felix --
On 06.10.2017 02:54, Felix Victor Münch wrote:
G'day! Just wondering whether it would be possible to implement, e.g. as a parameter of the minimize_nested_block_model function, a way to save intermediate results to disk at certain intervals and to resume the analysis, e.g. after a forced shut-down, or any other interruption. If I'd have some hints I might try to have a go at this feature myself. Or is this already possible with some lower level functions?
This would be possible to implement, but it complicates the algorithm quite a bit. Instead, I find this is better achieved with an external checkpoint infrastructure. I use DMTCP and it works quite well: http://dmtcp.sourceforge.net/ It does not require kernel modifications or privileged access, so it is suitable for clusters. Best, Tiago -- Tiago de Paula Peixoto <tiago@skewed.de>
participants (2)
-
Felix Victor Münch -
Tiago de Paula Peixoto