Most Efficient Way to Save Lots of Small Graphs

graph-tool
1

Hi all,

I just wanted to check what the most performance efficient way is to save
lots of small graphs. I have around 120k small graphs (<10 vertices) that
are currently being saved via pickle. However, I suspect that saving them
as a single large graph or as multiple separate .gt files might be more
performance efficient.

Thanks in advance.
M

attachment.html (430 Bytes)

2

Hi Michael,

Are the nodes always the same or do they overlap somehow from graph to graph?

Haiko

attachment.html (1.48 KB)

3

Hi Haiko,

Every node/vertex is distinct, although they are often encoded as the same
values in the data. For example, one graph might have three vertices (1, 2,
3) and two edges (1-2, 2-3). Another graph which is independent from the
previous one could have four vertices (1, 2, 3, 4) and three edges (1-3,
1-2, 3-4).

Thanks,
M

attachment.html (2.51 KB)

4

Hi Michael,

It seems that you're only interested in the relational structure and your nodes could be numbered in any way. If that is the case, you could create a graph object with as many nodes as your largest network and then use edge property maps to use a different (integer increasing) edge property for each network.

This technically works, but I don't know how efficient it is related to your old way of storing information.

Haiko

attachment.html (4.12 KB)

5

Yes. That's what I was thinking. I'm just not sure if the work would
produce a significant enough gain. However, I can use a large random graph
with lots of random property maps to get a baseline. I'll try it out and
report back.

attachment.html (5.07 KB)