在离线环境中安装GraphScope

部分用户可能面临在无法访问互联网的环境中安装GraphScope的问题。

我们提供了一些示例解决方案来解决这些问题。

从wheel包安装

用户可以下载graphscope的所有wheel文件及其依赖项,以便离线使用。

  1. 创建一个requirements.txt文件,列出所有需要的包。可以通过运行以下命令实现:

    echo "graphscope" > requirements.txt

  2. 下载requirements.txt文件中列出的所有软件包的wheel文件。您可以通过运行以下命令来完成此操作:

    pip download -r requirements.txt

  3. 将wheel文件复制到您计算机上可以离线访问的文件夹中。

  4. 当您需要离线使用这些软件包时,可以通过运行以下命令进行安装:

    pip install --no-index --find-links=./wheels .

这将从您下载的wheel文件中安装requirements.txt文件里列出的所有软件包。

从源码安装

我们提供一个示例,从基础操作系统开始,下载所有依赖项以供离线使用,以及编译选项来编译安装依赖项和graphscope分析引擎。

操作系统:CentOS 7

依赖项:

  • GCC 8.3.1

  • make 3.82

  • which (boost依赖项)

  • perl (boost依赖项)

  • Python 3.8 (vineyard依赖项)

  • cmake v3.24.3

  • openmpi 4.0.5

  • gflags 2.2.2

  • glog 0.6.0

  • apache-arrow 10.0.1

  • boost 1.74.0

  • openssl 1.1.1

  • zlib 1.2.11

  • protobuf 21.9

  • grpc 1.49.1

  • libgrape-lite 主分支

  • vineyard 0.13.5

  • libclang (Python包)

下载依赖项的位置:

你可以在github上获取GraphScope的源代码

下载依赖项和GraphScope

curl -LO https://graphscope.oss-cn-beijing.aliyuncs.com/dependencies/dependencies.tar.gz
curl -LO https://graphscope.oss-cn-beijing.aliyuncs.com/dependencies/libgrape-lite.tar.gz
git clone https://github.com/alibaba/graphscope
tar czf graphscope.tar.gz graphscope

将所有文件复制到目标环境并解压到指定位置

mkdir -p workdir
tar xzf dependencies.tar.gz -C workdir/
tar xzf libgrape-lite.tar.gz -C workdir/dependencies/
tar xzf graphscope.tar.gz -C workdir/dependencies/

安装基础软件包、gcc-g++和python

yum install centos-release-scl-rh -y
yum install which perl make devtoolset-8-gcc-c++ rh-python38-python-pip -y
source /opt/rh/devtoolset-8/enable
source /opt/rh/rh-python38/enable

安装第三方依赖项和GraphScope分析引擎

#!/bin/bash
set -euxo pipefail
# This directory is the path to the `dependencies` directory in the previous step
export DIR=/data/dependencies
# This would be the install prefix of dependencies and GraphScope
export PREFIX=/data/install

mkdir -p ${PREFIX}
export PATH=${PREFIX}/bin/:${PATH}/
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH:-""}:${PREFIX}/lib:${PREFIX}/lib64

# Install libclang
python3 -m pip install ${DIR}/libclang-16.0.0-py2.py3-none-manylinux2010_x86_64.whl

# Install cmake
bash ${DIR}/cmake-3.24.3-linux-x86_64.sh --skip-license --prefix=${PREFIX}

# Install gflags
cd ${DIR}/gflags-2.2.2
cmake . -DBUILD_SHARED_LIBS=ON \
    -DCMAKE_PREFIX_PATH="${PREFIX}" \
    -DCMAKE_INSTALL_PREFIX="${PREFIX}"
make -j && make install

# Install glog
cd ${DIR}/glog-0.6.0
cmake . -DBUILD_SHARED_LIBS=ON \
    -DCMAKE_PREFIX_PATH=${PREFIX} \
    -DCMAKE_INSTALL_PREFIX=${PREFIX}
make -j && make install

# Install boost
cd ${DIR}/boost_1_74_0
./bootstrap.sh --prefix=${PREFIX} --with-libraries=system,filesystem,context,program_options,regex,thread,random,chrono,atomic,date_time
./b2 install link=shared runtime-link=shared variant=release threading=multi

# Install openssl
cd ${DIR}/openssl-OpenSSL_1_1_1h
./config --prefix="${PREFIX}" -fPIC -shared
make -j
make install

# Install apache-arrow
cd ${DIR}/arrow-apache-arrow-10.0.1
mkdir -p build && cd build
cmake ../cpp \
    -DCMAKE_PREFIX_PATH=${PREFIX} \
    -DCMAKE_INSTALL_PREFIX=${PREFIX} \
    -DARROW_COMPUTE=ON \
    -DARROW_WITH_UTF8PROC=OFF \
    -DARROW_CSV=ON \
    -DARROW_FILESYSTEM=ON \
    -DARROW_IPC=ON \
    -DARROW_BUILD_BENCHMARKS=OFF \
    -DARROW_BUILD_EXAMPLES=OFF \
    -DARROW_BUILD_INTEGRATION=OFF \
    -DARROW_BUILD_UTILITIES=OFF \
    -DARROW_BUILD_TESTS=OFF \
    -DARROW_ENABLE_TIMING_TESTS=OFF \
    -DARROW_FUZZING=OFF \
    -DARROW_JEMALLOC=OFF \
    -DARROW_BUILD_SHARED=ON \
    -DARROW_BUILD_STATIC=OFF
make -j && make install

# Install openmpi
cd ${DIR}/openmpi-4.0.5
./configure --enable-mpi-cxx --disable-dlopen --prefix=${PREFIX}
make -j$(nproc)
make install

# Install protobuf
cd ${DIR}/protobuf-21.9
./configure --prefix="${PREFIX}" --enable-shared --disable-static
make -j$(nproc)
make install

# Install zlib
cd ${DIR}/zlib-1.2.11
cmake . -DCMAKE_INSTALL_PREFIX="${PREFIX}" \
    -DCMAKE_PREFIX_PATH="${PREFIX}" \
    -DBUILD_SHARED_LIBS=ON
make -j$(nproc)
make install

# Install grpc
cd ${DIR}/grpc
cmake . -DCMAKE_INSTALL_PREFIX="${PREFIX}" \
    -DCMAKE_PREFIX_PATH="${PREFIX}" \
    -DBUILD_SHARED_LIBS=ON \
    -DgRPC_INSTALL=ON \
    -DgRPC_BUILD_TESTS=OFF \
    -DgRPC_BUILD_CSHARP_EXT=OFF \
    -DgRPC_BUILD_GRPC_CSHARP_PLUGIN=OFF \
    -DgRPC_BUILD_GRPC_NODE_PLUGIN=OFF \
    -DgRPC_BUILD_GRPC_OBJECTIVE_C_PLUGIN=OFF \
    -DgRPC_BUILD_GRPC_PHP_PLUGIN=OFF \
    -DgRPC_BUILD_GRPC_PYTHON_PLUGIN=OFF \
    -DgRPC_BUILD_GRPC_RUBY_PLUGIN=OFF \
    -DgRPC_BACKWARDS_COMPATIBILITY_MODE=ON \
    -DgRPC_PROTOBUF_PROVIDER=package \
    -DgRPC_ZLIB_PROVIDER=package \
    -DgRPC_SSL_PROVIDER=package \
    -DOPENSSL_ROOT_DIR="${PREFIX}" \
    -DCMAKE_CXX_FLAGS="-fpermissive"
make -j
make install

# Install libgrape-lite
cd ${DIR}/libgrape-lite
cmake . -DCMAKE_PREFIX_PATH="${PREFIX}" \
    -DCMAKE_INSTALL_PREFIX="${PREFIX}"
make -j
make install

# Install vineyard
cd ${DIR}/v6d-0.13.4
cmake . -DCMAKE_PREFIX_PATH="${PREFIX}" \
    -DCMAKE_INSTALL_PREFIX="${PREFIX}" \
    -DBUILD_VINEYARD_TESTS=OFF \
    -DBUILD_SHARED_LIBS=ON \
    -DBUILD_VINEYARD_PYTHON_BINDINGS=OFF
make -j4
make install

# Install graphscope
cd ${DIR}/graphscope/analytical_engine
mkdir -p build && cd build
cmake ..  -DCMAKE_PREFIX_PATH=${PREFIX} \
    -DCMAKE_INSTALL_PREFIX=${PREFIX} \
    -DNETWORKX=OFF \
    -DENABLE_JAVA_SDK=OFF \
    -DBUILD_TESTS=OFF
make -j
make install

echo "Install complete, you could find the binary file in the ${PREFIX}/bin"