Hi everyone,
I am conducting tests where I perform several parallel tests of the SIR model (they are not using n iterations because I need to find out when the model stabilizes, where si (t) = 0). Parallel execution has been carried out with the multiprocessing library. In larger networks (2000 nodes), when performing the procedure of removing parallel edges (the graphs are converted to undirected), the execution freeze. Can you imagine the reason for this behavior? If I don't execute the remove_parallel_edges function, everything happens normally ... Execution hangs when iterate_sync will be performed. A serial run works normally (without multiprocessing).
Would anyone have a tip?
Best regards,
Ronaldo Alves
Dear Ronaldo,
When asking questions like this, it is imperative to include a minimal working example that reproduces the problem. Otherwise there is not much we can do to help.
Best, Tiago
Am 30.06.20 um 16:07 schrieb Ronaldo Alves:
Hi everyone,
I am conducting tests where I perform several parallel tests of the SIR model (they are not using n iterations because I need to find out when the model stabilizes, where si (t) = 0). Parallel execution has been carried out with the multiprocessing library. In larger networks (2000 nodes), when performing the procedure of removing parallel edges (the graphs are converted to undirected), the execution freeze. Can you imagine the reason for this behavior? If I don't execute the remove_parallel_edges function, everything happens normally ... Execution hangs when iterate_sync will be performed. A serial run works normally (without multiprocessing).
Would anyone have a tip?
Best regards,
Ronaldo Alves _______________________________________________ graph-tool mailing list graph-tool@skewed.de https://lists.skewed.de/mailman/listinfo/graph-tool
Sorry about that. I tried to reduce it to the maximum so that it could be possible to simulate this behavior. https://pastebin.com/cyCwAX0z
If I remove line 18, everything goes normally. This behavior does not occur on smaller networks (if reduce a number of nodes on graph, everything is ok).
Note that cpu is not used at all.
Regards,
Ronaldo Alves
Em ter., 30 de jun. de 2020 às 11:20, Tiago de Paula Peixoto tiago@skewed.de escreveu:
Dear Ronaldo,
When asking questions like this, it is imperative to include a minimal working example that reproduces the problem. Otherwise there is not much we can do to help.
Best, Tiago
Am 30.06.20 um 16:07 schrieb Ronaldo Alves:
Hi everyone,
I am conducting tests where I perform several parallel tests of the SIR model (they are not using n iterations because I need to find out when the model stabilizes, where si (t) = 0). Parallel execution has been carried out with the multiprocessing library. In larger networks (2000 nodes), when performing the procedure of removing parallel edges (the graphs are converted to undirected), the execution freeze. Can you imagine the reason for this behavior? If I don't execute the remove_parallel_edges function, everything happens normally ... Execution hangs when iterate_sync will be performed. A serial run works normally (without multiprocessing).
Would anyone have a tip?
Best regards,
Ronaldo Alves _______________________________________________ graph-tool mailing list graph-tool@skewed.de https://lists.skewed.de/mailman/listinfo/graph-tool
-- Tiago de Paula Peixoto tiago@skewed.de
graph-tool mailing list graph-tool@skewed.de https://lists.skewed.de/mailman/listinfo/graph-tool
Am 30.06.20 um 17:13 schrieb Ronaldo Alves:
Sorry about that. I tried to reduce it to the maximum so that it could be possible to simulate this behavior. https://pastebin.com/cyCwAX0z
If I remove line 18, everything goes normally. This behavior does not occur on smaller networks (if reduce a number of nodes on graph, everything is ok).
Note that cpu is not used at all.
Thanks for the working example.
graph-tool uses OpenMP internally to perform computations in parallel, and this does not mix well with the processing library, at least not if it's using fork().
You should disable OpenMP in the beginning of your code with:
import graph_tool as gt gt.openmp_set_num_threads(1)
Best, Tiago
Hi Thiago,
I suspected there was something between Openmp and multiprocessing, but I didn't know how to disable it. Thank you for this trick. It really worked.
I was very curious about whether or not removing the parallel edges directly impact the execution.
Thank you.
Em ter., 30 de jun. de 2020 às 12:36, Tiago de Paula Peixoto tiago@skewed.de escreveu:
Am 30.06.20 um 17:13 schrieb Ronaldo Alves:
Sorry about that. I tried to reduce it to the maximum so that it could be possible to simulate this behavior. https://pastebin.com/cyCwAX0z
If I remove line 18, everything goes normally. This behavior does not occur on smaller networks (if reduce a number of nodes on graph, everything is ok).
Note that cpu is not used at all.
Thanks for the working example.
graph-tool uses OpenMP internally to perform computations in parallel, and this does not mix well with the processing library, at least not if it's using fork().
You should disable OpenMP in the beginning of your code with:
import graph_tool as gt gt.openmp_set_num_threads(1)
Best, Tiago
-- Tiago de Paula Peixoto tiago@skewed.de
graph-tool mailing list graph-tool@skewed.de https://lists.skewed.de/mailman/listinfo/graph-tool