如何调试Turbomind#

Turbomind 是用 C++ 实现的，不像 Python 那样容易调试。本文档提供了调试 Turbomind 的基本方法。

先决条件#

首先，根据从源代码安装中的命令完成本地编译。

配置Python调试环境#

由于许多大公司目前使用Centos 7作为在线生产环境，我们将以Centos 7为例来说明这个过程。

获取 `glibc` 和 `python3` 版本#

rpm -qa | grep glibc
rpm -qa | grep python3

结果应该类似于这样：

[username@hostname workdir]# rpm -qa | grep glibc
glibc-2.17-325.el7_9.x86_64
glibc-common-2.17-325.el7_9.x86_64
glibc-headers-2.17-325.el7_9.x86_64
glibc-devel-2.17-325.el7_9.x86_64

[username@hostname workdir]# rpm -qa | grep python3
python3-pip-9.0.3-8.el7.noarch
python3-rpm-macros-3-34.el7.noarch
python3-rpm-generators-6-2.el7.noarch
python3-setuptools-39.2.0-10.el7.noarch
python3-3.6.8-21.el7_9.x86_64
python3-devel-3.6.8-21.el7_9.x86_64
python3.6.4-sre-1.el6.x86_64

根据上述信息，我们可以看到glibc的版本是2.17-325.el7_9.x86_64，而python3的版本是3.6.8-21.el7_9.x86_64。

下载并安装 `debuginfo` 库#

从 http://debuginfo.centos.org/7/x86_64 下载 glibc-debuginfo-common-2.17-325.el7.x86_64.rpm、glibc-debuginfo-2.17-325.el7.x86_64.rpm 和 python3-debuginfo-3.6.8-21.el7.x86_64.rpm。

rpm -ivh glibc-debuginfo-common-2.17-325.el7.x86_64.rpm
rpm -ivh glibc-debuginfo-2.17-325.el7.x86_64.rpm
rpm -ivh python3-debuginfo-3.6.8-21.el7.x86_64.rpm

升级GDB#

sudo yum install devtoolset-10 -y
echo "source scl_source enable devtoolset-10" >> ~/.bashrc
source ~/.bashrc

验证#

gdb python3

输出应该类似于这样：

[username@hostname workdir]# gdb python3
GNU gdb (GDB) Red Hat Enterprise Linux 9.2-10.el7
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
   <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from python3...
(gdb)

如果显示Reading symbols from python3，则配置已成功。

对于其他操作系统，请参考DebuggingWithGdb。

设置符号链接#

设置符号链接后，无需每次使用pip在本地安装。

# Change directory to lmdeploy, e.g.
cd /workdir/lmdeploy

# Since it has been built in the build directory
# Link the lib directory
cd lmdeploy && ln -s ../build/lib . && cd ..
# (Optional) Link compile_commands.json for clangd index
ln -s build/compile_commands.json .

开始调试#

# Use gdb to start the API server with Llama-2-13b-chat-hf, e.g.
gdb --args python3 -m lmdeploy serve api_server /workdir/Llama-2-13b-chat-hf

# Set directories in gdb
Reading symbols from python3...
(gdb) set directories /workdir/lmdeploy

# Set a breakpoint using the relative path, e.g.
(gdb) b src/turbomind/models/llama/BlockManager.cc:104

# When it shows
# ```
# No source file named src/turbomind/models/llama/BlockManager.cc.
# Make breakpoint pending on future shared library load? (y or [n])
# ```
# Just type `y` and press enter

# Run
(gdb) r

# (Optional) Use https://github.com/InternLM/lmdeploy/blob/main/benchmark/profile_restful_api.py to send a request

python3 profile_restful_api.py --backend lmdeploy --dataset-path /workdir/ShareGPT_V3_unfiltered_cleaned_split.json --num_prompts 1

使用GDB#

请参考GDB执行命令并祝调试愉快。

如何调试Turbomind

目录