Hi, I don't know if it is right to pollute this list with my problems in trying to compile graph-tool for Win64. I'll try to make it brief. Using MSYS2/MinGW64, I managed to compile the whole src/graph modules and install them as *.pyd modules in the site-packages/graph_tool directory, along with the Python files. Unfortunately, I get a segfault when importing graph_tool.libgraph_tool_core . The segfault happens deep in libboost_python3. Before posting a long backtrace, maybe someone may offer some clue on what is happening. I have posted some details about what I did below. If someone is interested and/or has ideas on how o go on, I'll be glad to share and work on this problem. (And I must say that I would love to get this tool to work under Windows, because it seems to be one of the best of its kind!) Best regards, Fabrice The background is: python 3.4, boost 1.57, clang 3.5.1 (g++ 4.9.2 as provided with mingw64 couldn't compile a few objects without segfaulting). Is it possible that there is a conflict among those libs/tools (of thread model for example)? Currently I have configured like this: ./configure --enable-debug --disable-optimize CFLAGS="-I/mingw64/include" LDFLAGS="-L/mingw64/lib" CC=clang CXX='clang -std=c++11 -m64 -DMS_WIN64=1' CXXCPP='clang -E -std=c++11 -m64 -DMS_WIN64=1' PYTHON=python3 Then I run 'make -k -j 8' to compile all the objects. Linking the various dlls fails, because under Windows, you can't link graph_tool_centrality.dll without referencing grap_tool_core.dll.a. However the build process is not prepared to do that and in the right order. So I manually replicated what libtool does, ie : clang -std=c++11 -shared -nostdlib C:/Local/MSys64/mingw64/x86_64-w64- mingw32/lib/dllcrt2.o C:/Local/MSys64/mingw64/x86_64-w64-mingw32/lib/crtbegin.o .libs/*.o C:/Local/MSys64/mingw64/x86_64-w64-mingw32/lib/crtend.o - L/mingw64/lib -O3 - Wl,--export-all-symbols -Wl,--as-needed -o .libs/libgraph_tool_core.dll -Wl,-- enable-auto-image-base -Xlinker --out-implib -Xlinker .libs/libgraph_tool_core.dll.a -lpython3.4m -lboost_iostreams -lboost_python3 - lboost_regex -lboost_system -lCGAL -lgmp -lcairomm-1.0 -lcairo -lexpat -lbz2 - lstdc++ -lpthread -lgcc_s -lgcc -lmoldname -lmingwex -lmingw32 -lmsvcrt - ladvapi32 -lshell32 -luser32 -lkernel32 for s in ${subdirs}; do (cd $s ; clang -std=c++11 -shared -nostdlib C:/Local/MSys64/mingw64/x86_64-w64- mingw32/lib/dllcrt2.o C:/Local/MSys64/mingw64/x86_64-w64-mingw32/lib/crtbegin.o .libs/*.o C:/Local/MSys64/mingw64/x86_64-w64-mingw32/lib/crtend.o ../.libs/libgraph_tool_core.dll.a -L/mingw64/lib -O3 -Wl,--export-all- symbols -Wl,--as-needed -o .libs/libgraph_tool_$s.dll -Wl,--enable-auto-image- base -Xlinker --out-implib -Xlinker .libs/libgraph_tool_$s.dll.a -lpython3.4m - lboost_iostreams -lboost_python3 -lboost_regex -lboost_system -lCGAL -lgmp - lcairomm-1.0 -lcairo -lexpat -lbz2 -lstdc++ -lpthread -lgcc_s -lgcc -lmoldname -lmingwex -lmingw32 -lmsvcrt -ladvapi32 -lshell32 -luser32 -lkernel32) done for i in `find . -name \*.dll` ; do cp $i /mingw64/lib/python3.4/site- packages/graph_tool/`basename $i .dll`.pyd; done At this point, the modules are installed. The only thing to change on the Python side is to remove the calls to getdlopenflags() and setdlopenflags() which are not available to Windows.
On 03.05.2015 14:25, Fabrice Popineau wrote:
Hi,
I don't know if it is right to pollute this list with my problems in trying to compile graph-tool for Win64. I'll try to make it brief. Using MSYS2/MinGW64, I managed to compile the whole src/graph modules and install them as *.pyd modules in the site-packages/graph_tool directory, along with the Python files. Unfortunately, I get a segfault when importing graph_tool.libgraph_tool_core . The segfault happens deep in libboost_python3.
Before posting a long backtrace, maybe someone may offer some clue on what is happening. I have posted some details about what I did below.
The backtrace would actually be helpful.
The background is: python 3.4, boost 1.57, clang 3.5.1 (g++ 4.9.2 as provided with mingw64 couldn't compile a few objects without segfaulting). Is it possible that there is a conflict among those libs/tools (of thread model for example)?
Did you guarantee that Boost.Python was compiled with exactly the same Python version that you used to compile graph-tool? It could also help if you compile everything with the same compiler.
Currently I have configured like this: ./configure --enable-debug --disable-optimize CFLAGS="-I/mingw64/include" LDFLAGS="-L/mingw64/lib" CC=clang CXX='clang -std=c++11 -m64 -DMS_WIN64=1' CXXCPP='clang -E -std=c++11 -m64 -DMS_WIN64=1' PYTHON=python3 Then I run 'make -k -j 8' to compile all the objects. Linking the various dlls fails, because under Windows, you can't link graph_tool_centrality.dll without referencing grap_tool_core.dll.a. However the build process is not prepared to do that and in the right order. So I manually replicated what libtool does, ie :
clang -std=c++11 -shared -nostdlib C:/Local/MSys64/mingw64/x86_64-w64- mingw32/lib/dllcrt2.o C:/Local/MSys64/mingw64/x86_64-w64-mingw32/lib/crtbegin.o .libs/*.o C:/Local/MSys64/mingw64/x86_64-w64-mingw32/lib/crtend.o - L/mingw64/lib -O3 - Wl,--export-all-symbols -Wl,--as-needed -o .libs/libgraph_tool_core.dll -Wl,-- enable-auto-image-base -Xlinker --out-implib -Xlinker .libs/libgraph_tool_core.dll.a -lpython3.4m -lboost_iostreams -lboost_python3 - lboost_regex -lboost_system -lCGAL -lgmp -lcairomm-1.0 -lcairo -lexpat -lbz2 - lstdc++ -lpthread -lgcc_s -lgcc -lmoldname -lmingwex -lmingw32 -lmsvcrt - ladvapi32 -lshell32 -luser32 -lkernel32
for s in ${subdirs}; do (cd $s ; clang -std=c++11 -shared -nostdlib C:/Local/MSys64/mingw64/x86_64-w64- mingw32/lib/dllcrt2.o C:/Local/MSys64/mingw64/x86_64-w64-mingw32/lib/crtbegin.o .libs/*.o C:/Local/MSys64/mingw64/x86_64-w64-mingw32/lib/crtend.o ../.libs/libgraph_tool_core.dll.a -L/mingw64/lib -O3 -Wl,--export-all- symbols -Wl,--as-needed -o .libs/libgraph_tool_$s.dll -Wl,--enable-auto-image- base -Xlinker --out-implib -Xlinker .libs/libgraph_tool_$s.dll.a -lpython3.4m - lboost_iostreams -lboost_python3 -lboost_regex -lboost_system -lCGAL -lgmp - lcairomm-1.0 -lcairo -lexpat -lbz2 -lstdc++ -lpthread -lgcc_s -lgcc -lmoldname -lmingwex -lmingw32 -lmsvcrt -ladvapi32 -lshell32 -luser32 -lkernel32) done
for i in `find . -name \*.dll` ; do cp $i /mingw64/lib/python3.4/site- packages/graph_tool/`basename $i .dll`.pyd; done
At this point, the modules are installed.
I can't vouch for this way of compiling things, since I'm very unfamiliar with the windows build environment. But doing things by hand like this seems very invasive. A better approach would be to investigate why autotools is not building things right, and fix that. A quick google search reveals things like: http://gnuwin32.sourceforge.net/packages/autoconf.htm
The only thing to change on the Python side is to remove the calls to getdlopenflags() and setdlopenflags() which are not available to Windows.
This may be fatal. In GNU/Linux, for instance, this is necessary to guarantee lazy evaluation of symbols when importing. I have no idea if this is necessary in windows, and if so, what are the alternative approaches to dlopen. -- Tiago de Paula Peixoto <tiago@skewed.de>
Tiago de Paula Peixoto <tiago <at> skewed.de> writes:
I can't vouch for this way of compiling things, since I'm very unfamiliar with the windows build environment. But doing things by hand like this seems very invasive. A better approach would be to investigate why autotools is not building things right, and fix that. A quick google search reveals things like: http://gnuwin32.sourceforge.net/packages/autoconf.htm
Actually, I know quite well this gnuwin32. I'm building with MSYS2/MinGW64 for x86_64. DLLs do not work quite the same under Windows and under Linux. Libtool is of not much help here. The way you build your set of modules can't work like this under Windows. You need to link every other dll with libgraph_tool_core.dll.a There is no automatic import of symbols. (Actually, it is possible to write an init function for the dll, but nothing like this is provided automatically) The segfault I got was because g++ (actually cc1plus.exe) ran out of resources. It used about 4Gb and crashed. However, I managed to overcome the problem and I have the whole set of dlls (renamed *.pyd) modules that I can import in Python. Given the g++ crash, I tried to compile them with clang, but the layout of bjects must be different. The dlls compiled fine, but I couldn't import them into python. Anyway ...
The only thing to change on the Python side is to remove the calls to getdlopenflags() and setdlopenflags() which are not available to Windows.
This may be fatal. In GNU/Linux, for instance, this is necessary to guarantee lazy evaluation of symbols when importing. I have no idea if this is necessary in windows, and if so, what are the alternative approaches to dlopen.
... I may have a problem here. I don't know yet. I have exported all symbols by default in all *.pyd modules. Currently, I can't yet import all from graph_tool because of a bug in pycairo. Slowly progressing. Fabrice
Fabrice Popineau <fabrice.popineau <at> gmail.com> writes:
... I may have a problem here. I don't know yet. I have exported all symbols by default in all *.pyd modules. Currently, I can't yet import all from graph_tool because of a bug in pycairo. Slowly progressing.
So after fixing pycairo, I tried the small example "Building a price network" from the quickstart doc. Everything runs well till the point the price.xml.gz file is saved. It segfaults afterwards in the stats module at this line # # Let's plot its in-degree distribution in_hist = vertex_hist(g, "in") I must add that I have used the git version of graph_tool. I hope it is stable enough. Fabrice
On 06.05.2015 11:15, Fabrice Popineau wrote:
Everything runs well till the point the price.xml.gz file is saved. It segfaults afterwards in the stats module at this line
It is hard to say what it could be without a backtrace.
I must add that I have used the git version of graph_tool. I hope it is stable enough.
I get no segfaults with the git version. Best, Tiago -- Tiago de Paula Peixoto <tiago@skewed.de>
participants (2)
-
Fabrice Popineau -
Tiago de Paula Peixoto