Compiled binaries are kinda HUGE

Hello,

I just compiled graph-tool from source, it took a while but it worked (besides an conflict between Gtk 3 and 4, but that’s for another story).

But I found the binary library are really huge… Some examples:

  • inference/libgraph_tool_inference.so: 3,6 GB
  • topology/libgraph_tool_topology.so: 809,4 MB
  • libgraph_tool_core.so: 327,3 MB
  • generation/libgraph_tool_generation.so: 326,9 MB
  • and so on…

What did I do wrong with compilation options? I just used ./configure and make install commands.

Thank you!

Unfortunately, it’s not possible to provide a solution to your problem or an answer to your question with the information provided.

To enable us to understand the situation, you need to provide all the items below:

  1. Your exact graph-tool version.
  2. Your operating system.
  3. A minimal working example that shows the problem.

Item 3 above is very important! If you provide us only the part of the code that you believe causes the problem, then it is not possible to understand the context that may have contributed to it.

You need to provide us a complete example that runs and reproduces the problem you are observing.

To post code, please use triple back-ticks for proper formatting:

for x in range(100):
    print("hello world")

or upload your files as attachments.

In your case you need to provide the full contents of the config.log file and the entire output of the ./configure script.

2.59

GNU/Linux Fedora Rawhide

I have no example because it’s about compilation result size, I already provided compiled binary size as an example. But I can provide my ./configure output if it helps.

marc@marcbook ~/Git/graph-tool dev master
$ ./configure --prefix=$HOME/.local --with-python-module-path=$HOME/.pyenv/versions/dev/lib/python3.12/site-packages/

checking build system type... x86_64-pc-linux-gnu
checking host system type... x86_64-pc-linux-gnu
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a race-free mkdir -p... /usr/bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking how to create a pax tar archive... gnutar
checking whether make supports nested variables... (cached) yes
checking whether make supports the include directive... yes (GNU style)
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether the compiler supports GNU C... yes
checking whether gcc accepts -g... yes
checking for gcc option to enable C11 features... none needed
checking whether gcc understands -c and -o together... yes
checking dependency style of gcc... gcc3
checking for stdio.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for strings.h... yes
checking for sys/stat.h... yes
checking for sys/types.h... yes
checking for unistd.h... yes
checking for wchar.h... yes
checking for minix/config.h... no
checking whether it is safe to define __EXTENSIONS__... yes
checking whether _XOPEN_SOURCE should be defined... no
checking for library containing strerror... none required
checking for g++... g++
checking whether the compiler supports GNU C++... yes
checking whether g++ accepts -g... yes
checking for g++ option to enable C++11 features... none needed
checking dependency style of g++... gcc3
checking how to print strings... printf
checking for a sed that does not truncate output... /usr/bin/sed
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for fgrep... /usr/bin/grep -F
checking for ld used by gcc... /usr/bin/ld
checking if the linker (/usr/bin/ld) is GNU ld... yes
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
checking the name lister (/usr/bin/nm -B) interface... BSD nm
checking whether ln -s works... yes
checking the maximum length of command line arguments... 1572864
checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop
checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop
checking for /usr/bin/ld option to reload object files... -r
checking for file... file
checking for objdump... objdump
checking how to recognize dependent libraries... pass_all
checking for dlltool... no
checking how to associate runtime and link libraries... printf %s\n
checking for ar... ar
checking for archiver @FILE support... @
checking for strip... strip
checking for ranlib... ranlib
checking command to parse /usr/bin/nm -B output from gcc object... ok
checking for sysroot... no
checking for a working dd... /usr/bin/dd
checking how to truncate binary pipes... /usr/bin/dd bs=4096 count=1
checking for mt... no
checking if : is a manifest tool... no
checking for dlfcn.h... yes
checking for objdir... .libs
checking if gcc supports -fno-rtti -fno-exceptions... no
checking for gcc option to produce PIC... -fPIC -DPIC
checking if gcc PIC flag -fPIC -DPIC works... yes
checking if gcc static flag -static works... no
checking if gcc supports -c -o file.o... yes
checking if gcc supports -c -o file.o... (cached) yes
checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking whether -lc should be explicitly linked in... no
checking dynamic linker characteristics... GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... yes
checking whether to build static libraries... no
checking how to run the C++ preprocessor... g++ -E
checking for ld used by g++... /usr/bin/ld -m elf_x86_64
checking if the linker (/usr/bin/ld -m elf_x86_64) is GNU ld... yes
checking whether the g++ linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking for g++ option to produce PIC... -fPIC -DPIC
checking if g++ PIC flag -fPIC -DPIC works... yes
checking if g++ static flag -static works... no
checking if g++ supports -c -o file.o... yes
checking if g++ supports -c -o file.o... (cached) yes
checking whether the g++ linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking dynamic linker characteristics... (cached) GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking for pkg-config... /usr/bin/pkg-config
checking pkg-config is at least version 0.9.0... yes
checking whether g++ supports C++17 features by default... yes
configure: WARNING: C++17 is not yet standardized, so the checks may change in incompatible ways anytime
checking whether to enable debug info... no
checking whether to enable parallel algorithms with openmp... checking for OpenMP flag of C++ compiler... -fopenmp
checking whether to enable cairo drawing... yes
checking for expat... yes
checking for a Python interpreter with version >= 3.6... python
checking for python... /home/marc/.pyenv/shims/python
checking for python version... 3.12
checking for python platform... linux
checking for GNU default python prefix... ${prefix}
checking for GNU default python exec_prefix... ${exec_prefix}
checking for python script directory (pythondir)... ${PYTHON_PREFIX}/lib/python3.12/site-packages
checking for python extension module directory (pyexecdir)... ${PYTHON_EXEC_PREFIX}/lib64/python3.12/site-packages
checking for python3.12... (cached) /home/marc/.pyenv/shims/python
checking for a version of Python >= '2.1.0'... yes
checking for a version of Python == '3.12.1'... yes
checking for the distutils Python package... yes
checking for Python include path... -I/usr/include/python3.12
checking for Python library path... -L/usr/lib64 -lpython3.12
checking for Python site-packages path... /home/marc/.pyenv/versions/dev/lib/python3.12/site-packages
checking python extra libraries... -ldl -lm
checking python extra linking flags... -Xlinker -export-dynamic
checking consistency of all components of python development environment... yes
checking for boostlib >= 1.58.0 (105800)... yes
checking whether the Boost::Python library is available... yes
checking whether boost_python312 is the correct library... yes
checking whether the Boost::IOStreams library is available... yes
checking for exit in -lboost_iostreams... yes
checking whether the Boost::Regex library is available... yes
checking for exit in -lboost_regex... yes
checking whether the Boost::Context library is available... yes
checking for exit in -lboost_context... yes
checking whether the Boost::Coroutine library is available... yes
checking for exit in -lboost_coroutine... yes
checking whether the Boost::Graph library is available... yes
checking for exit in -lboost_graph... yes
checking whether the Boost::Thread library is available... yes
checking for exit in -lboost_thread... yes
checking for __gmpz_init in -lgmp... yes
checking for __gmpz_init in -lgmp... (cached) yes
checking whether CGAL is available in /usr... yes
checking whether to enable debbuging with valgrind... no
checking python module: numpy... yes
checking for numpy/arrayobject.h... yes
checking for cairomm-1.16... no
checking for cairomm-1.0... yes
checking python module: cairo... yes
checking for py3cairo.h... yes
checking whether to enable sparsehash... yes
checking for libsparsehash... yes
checking for sparsehash/dense_hash_set... yes
configure: pkgconfig directory is ${libdir}/pkgconfig
checking our pkgconfig libname... graph-tool-py3.12
checking our pkgconfig version... 2.59
checking our pkgconfig_libdir... ${libdir}/pkgconfig
expanded our pkgconfig_libdir... /home/marc/.local/lib/pkgconfig
checking our pkgconfig_libfile... graph-tool-py3.12.pc
checking our package / suffix... graph-tool / 
checking our pkgconfig description... graph-tool Python library
checking our pkgconfig requires... 
checking our pkgconfig ext libs... -L/usr/lib64 -lpython3.12 -lboost_python312
checking our pkgconfig cppflags... -I/home/marc/.pyenv/versions/dev/lib/python3.12/site-packages//graph_tool/include -I/home/marc/.pyenv/versions/dev/lib/python3.12/site-packages//graph_tool/include/boost-workaround -I/home/marc/.pyenv/versions/dev/lib/python3.12/site-packages//graph_tool/include/pcg-cpp -I/home/marc/.pyenv/versions/3.12-dev/envs/dev/lib/python3.12/site-packages/cairo/include -I/usr/include/python3.12 -pthread -I/usr/include -I/home/marc/.pyenv/versions/dev/lib64/python3.12/site-packages/numpy/core/include
checking our pkgconfig ldflags... 
noninstalled pkgconfig -L /home/marc/Git/graph-tool/src/.libs
noninstalled pkgconfig -I /home/marc/Git/graph-tool/.
checking that generated files are newer than configure... done
configure: creating ./config.status
config.status: creating Makefile
config.status: creating config.h
config.status: config.h is unchanged
config.status: executing depfiles commands
config.status: executing libtool commands
config.status: executing graph-tool-py3.12.pc commands
config.status: creating graph-tool-py3.12.pc.in
config.status: creating graph-tool-py3.12.pc
config.status: creating graph-tool-py3.12-uninstalled.pc
config.status: creating graph-tool-py3.12-uninstalled.sh

================================================================================
                             CONFIGURATION SUMMARY                              
================================================================================
Using python version:   3.12.1
Python interpreter:     /home/marc/.pyenv/shims/python
Installation path:      /home/marc/.pyenv/versions/dev/lib/python3.12/site-packages//graph_tool

C++ compiler (CXX):     g++
C++ compiler version:   13

Prefix:                 /home/marc/.local
Pkgconfigdir:           ${libdir}/pkgconfig

Python CPP flags:       -I/home/marc/.pyenv/versions/3.12-dev/envs/dev/lib/python3.12/site-packages/cairo/include -I/usr/include/python3.12
Python LD flags:        -L/usr/lib64 -lpython3.12
Boost CPP flags:        -pthread -I/usr/include
Boost LD flags:         -L/usr/lib64 -lboost_iostreams -lboost_python312 -lboost_regex -lboost_context -lboost_coroutine
Numpy CPP flags:        -I/home/marc/.pyenv/versions/dev/lib64/python3.12/site-packages/numpy/core/include
Sparsehash CPP flags:   
CGAL CPP flags:         -I/usr/include
CGAL LD flags:          
Expat CPP flags:        
Expat LD flags:          -lexpat
Cairomm CPP flags:      -I/usr/include/cairomm-1.0 -I/usr/lib64/cairomm-1.0/include -I/usr/include/sigc++-2.0 -I/usr/lib64/sigc++-2.0/include -I/usr/include/cairo -I/usr/include/freetype2 -I/usr/include/pixman-1 -I/usr/include/sysprof-6 -pthread -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -I/usr/include/libxml2 -I/usr/include/harfbuzz -I/usr/include/libpng16
Cairomm LD flags:       -lcairomm-1.0 -lsigc-2.0 -lcairo
OpenMP compiler flags:  -fopenmp
OpenMP LD flags:        
Extra CPPFLAGS:         -DBOOST_ALLOW_DEPRECATED_HEADERS -DNDEBUG 
Extra CXXFLAGS:         -fopenmp -O3 -fvisibility=default -fvisibility-inlines-hidden -Wno-deprecated -Wall -Wextra -ftemplate-backtrace-limit=0 -g -O2
Extra LDFLAGS:          

Using OpenMP:           yes
Using sparsehash:       yes
Using cairo:            yes
================================================================================

Please read the previous message very carefully, and note that I also asked for the contents of the config.log file.

In any case, the issue is already clear: You are compiling with debugging enabled (see the -g option in your CXXFLAGS), which is inflating the binary sizes with unnecessary debugging symbols. You should remove that option.

(Also, you are overriding the default optimization level with a lower value, i.e. -O2 instead if -O3. There’s no point in that.)

I did nothing for that, is it specific to the git version? How to disable them?

This comes from your setup. You probably have some value set for the environment variable CXXFLAGS that you need to unset.

I echoed $CFLAGS $CXXFLAGS and $CPPFLAGS, there are all empty. It seems they are added by the ./configure script itself.

Just run ./configure CXXFLAGS=

It worked. But weird issue… I still didn’t understand.

Also, can we use jobs? It seems using “-jX” doesn’t work.

configure: error: *** A compiler with support for C++17 language features is required.

Please, as I already explained at great length, don’t post error messages without giving the exact command you have attempted.

Sorry, I thought it was obvious, I just typed ./configure CXXFLAGS="-j4".

Also, I’m curious why the configure script is automatically adding “-g -O3” flags to the compiler.

It’s not obvious because it makes no sense at all. The option -j4 is not a valid compiler option. This option should be passed to make.

You seem unfamiliar with manual compilation. Why not use the fedora package? python3-graph-tool - Fedora Packages

Oh you’re right indeed.

Yeah, it was a long time I had to deal with C++, make and such. I’m a bit confused because this is the first time I have to dive into these subtilties.

I’m using an embedded virtualenv, I can’t rely on system distribution.

You didn’t provide an answer to that though.

You should try using conda and its environments. No compilation is necessary.

I can’t debug your system remotely, sorry. Maybe it’s just some default that gets added when it’s empty.

I mean… I already used ./configure and make [install] commands before, this is the first time I had that kind of issue. Maybe it is related to the ./autogen.sh script? The configure script was created through this one.