Hi all, I'm a current R-igraph user trying to understand the feasibility of switching to graph-tool. I have some memory constraints on the server I'm using and I'm trying to understand how graph-tool operates. The performance comparison documentation says the following: "Graph-tool's performance comes at the cost of increased time and memory required during compilation. This is mostly due to the in-place graph filtering functionality..". Since filtering seems to me to be a chiefly visualization-related task, does graph-tool typically require large amounts of RAM due to filtering only for visualization, but not for other tasks? Or is it the case that filtering is continually occurring under the hood and hence graph-tool uses large amounts of memory in general? Thanks! -- Sent from: https://nabble.skewed.de/
Ni! Hi Nikhrao, Please note that the phrase you've extracted contains the words «during compilation» while you seem to be talking about runtime memory usage. Runtime memory usage will depend on the algorithm you'll be running and choices of the implementation, as there's often a tradeoff between memory usage and speed. You can count that graph-tool strikes a good balance that will serve most scenarios, and in some cases there are parameters to adjust for extreme situations. Having never used igraph, I'd assume it strives for a similar goal. I strongly recommend you to try it yourself on your actual use case. Otherwise, you'd have to provide much more detail about it for us to give you a theoretical answer that won't be worth as much as simply running the algorithm you want on the kind of data you expect to treat. Now, if you just want a "general idea" about memory consumption, I personally see no technical reason to expect big differences when comparing algorithms supported by both libraries. Best, ale .~´ On Mon, Feb 8, 2021 at 11:14 PM nikhrao <nikhrao@umich.edu> wrote:
Hi all,
I'm a current R-igraph user trying to understand the feasibility of switching to graph-tool. I have some memory constraints on the server I'm using and I'm trying to understand how graph-tool operates.
The performance comparison documentation says the following: "Graph-tool's performance comes at the cost of increased time and memory required during compilation. This is mostly due to the in-place graph filtering functionality..". Since filtering seems to me to be a chiefly visualization-related task, does graph-tool typically require large amounts of RAM due to filtering only for visualization, but not for other tasks? Or is it the case that filtering is continually occurring under the hood and hence graph-tool uses large amounts of memory in general?
Thanks!
-- Sent from: https://nabble.skewed.de/ _______________________________________________ graph-tool mailing list graph-tool@skewed.de https://lists.skewed.de/mailman/listinfo/graph-tool
Hi Alexandre, Thanks for the response! Yes, sorry for the confusion. I think I have a key misunderstanding (I haven't used python before). When you refer to compilation, does python compile the code I write as I write it? Or rather, is the code compiled once I decide to run it, and it's at this step that the memory usage is high due to filtering? For context, the networks that I'm working with have around 400,000 vertices and 300,000 edges. If I tried fairly rudimentary tasks such as computing the degree for each vertex or computing eigenvector centrality, would these require large amounts of RAM (say, 90-100 GB) due the filtering capabilities that graph-tool has? I'm just trying to get an idea of the upper bound of memory requirements that I may need on my server. Thanks, N -- Sent from: https://nabble.skewed.de/
Ni! The algorithms in graph-tool are written in C++, just like the algorithms in igraph are writen in C. In both cases, the C++/C code gets compiled before installation. Graph-tool's python library, like the higher level libraries for igraph, are only interfaces to the underlying compiled code. Therefore, there is no relationship between "filtering capabilities" increasing compile time, and the runtime speed or memory consumption of graph-tool. For the kind of things you describe, if igraph does it within your memory limits, graph-tool should do it as well. And for the numbers you give, it should be orders of magnitude lower than what you're expecting. Cheers, .~´ On Tue, Feb 9, 2021 at 12:12 AM nikhrao <nikhrao@umich.edu> wrote:
Hi Alexandre,
Thanks for the response! Yes, sorry for the confusion. I think I have a key misunderstanding (I haven't used python before). When you refer to compilation, does python compile the code I write as I write it? Or rather, is the code compiled once I decide to run it, and it's at this step that the memory usage is high due to filtering?
For context, the networks that I'm working with have around 400,000 vertices and 300,000 edges. If I tried fairly rudimentary tasks such as computing the degree for each vertex or computing eigenvector centrality, would these require large amounts of RAM (say, 90-100 GB) due the filtering capabilities that graph-tool has? I'm just trying to get an idea of the upper bound of memory requirements that I may need on my server.
Thanks, N
-- Sent from: https://nabble.skewed.de/ _______________________________________________ graph-tool mailing list graph-tool@skewed.de https://lists.skewed.de/mailman/listinfo/graph-tool
Hi Alexandre, Thanks for the response! So if I'm understanding correctly, when the documentation says "Graph-tool's performance comes at the cost of increased time and memory required during compilation" that is referring to a one-time event at the time of installation? Thanks, N -- Sent from: https://nabble.skewed.de/
Yes. And you can entirely avoid compilation if you install from binary packages such as those available for Debian, Ubuntu, Conda, and Homebrew, or if you use a Docker container. From what I can tell of your current level of understanding, I strongly advise you to do that. Please see < https://git.skewed.de/count0/graph-tool/-/wikis/installation-instructions>. Let me just advance that questions about how to operate your operating system's package manager, or docker, or compilers, are off-topic for this list. You can try stackoverflow if you need to learn those kinds of things. You may post here about installation issues if you have precisely followed the installation instructions and something doesn't work that you believe is related to graph-tool. Best, .~´ On Tue, Feb 9, 2021 at 12:55 AM nikhrao <nikhrao@umich.edu> wrote:
Hi Alexandre,
Thanks for the response! So if I'm understanding correctly, when the documentation says "Graph-tool's performance comes at the cost of increased time and memory required during compilation" that is referring to a one-time event at the time of installation?
Thanks, N
-- Sent from: https://nabble.skewed.de/ _______________________________________________ graph-tool mailing list graph-tool@skewed.de https://lists.skewed.de/mailman/listinfo/graph-tool
participants (2)
-
Alexandre Hannud Abdo -
nikhrao