I'm a current R-igraph user trying to understand the feasibility of
switching to graph-tool. I have some memory constraints on the server I'm
using and I'm trying to understand how graph-tool operates.
The performance comparison documentation says the following: "Graph-tool's
performance comes at the cost of increased time and memory required during
compilation. This is mostly due to the in-place graph filtering
functionality..". Since filtering seems to me to be a chiefly
visualization-related task, does graph-tool typically require large amounts
of RAM due to filtering only for visualization, but not for other tasks? Or
is it the case that filtering is continually occurring under the hood and
hence graph-tool uses large amounts of memory in general?
Please note that the phrase you've extracted contains the words «during
compilation» while you seem to be talking about runtime memory usage.
Runtime memory usage will depend on the algorithm you'll be running and
choices of the implementation, as there's often a tradeoff between memory
usage and speed.
You can count that graph-tool strikes a good balance that will serve most
scenarios, and in some cases there are parameters to adjust for extreme
situations. Having never used igraph, I'd assume it strives for a similar
goal.
I strongly recommend you to try it yourself on your actual use case.
Otherwise, you'd have to provide much more detail about it for us to give
you a theoretical answer that won't be worth as much as simply running the
algorithm you want on the kind of data you expect to treat.
Now, if you just want a "general idea" about memory consumption, I
personally see no technical reason to expect big differences when comparing
algorithms supported by both libraries.
Thanks for the response! Yes, sorry for the confusion. I think I have a key
misunderstanding (I haven't used python before). When you refer to
compilation, does python compile the code I write as I write it? Or rather,
is the code compiled once I decide to run it, and it's at this step that the
memory usage is high due to filtering?
For context, the networks that I'm working with have around 400,000 vertices
and 300,000 edges. If I tried fairly rudimentary tasks such as computing the
degree for each vertex or computing eigenvector centrality, would these
require large amounts of RAM (say, 90-100 GB) due the filtering capabilities
that graph-tool has? I'm just trying to get an idea of the upper bound of
memory requirements that I may need on my server.
The algorithms in graph-tool are written in C++, just like the algorithms
in igraph are writen in C. In both cases, the C++/C code gets compiled
before installation.
Graph-tool's python library, like the higher level libraries for igraph,
are only interfaces to the underlying compiled code.
Therefore, there is no relationship between "filtering capabilities"
increasing compile time, and the runtime speed or memory consumption of
graph-tool.
For the kind of things you describe, if igraph does it within your memory
limits, graph-tool should do it as well. And for the numbers you give, it
should be orders of magnitude lower than what you're expecting.
Thanks for the response! So if I'm understanding correctly, when the
documentation says "Graph-tool's
performance comes at the cost of increased time and memory required during
compilation" that is referring to a one-time event at the time of
installation?
And you can entirely avoid compilation if you install from binary packages
such as those available for Debian, Ubuntu, Conda, and Homebrew, or if you
use a Docker container. From what I can tell of your current level of
understanding, I strongly advise you to do that. Please see < https://git.skewed.de/count0/graph-tool/-/wikis/installation-instructions>\.
Let me just advance that questions about how to operate your operating
system's package manager, or docker, or compilers, are off-topic for this
list. You can try stackoverflow if you need to learn those kinds of things.
You may post here about installation issues if you have precisely followed
the installation instructions and something doesn't work that you believe
is related to graph-tool.