Envoy Proxy使用介绍教程(二): envoy源代码阅读、集成开发环境(IDE)

Tags: envoy 

目录

说明

这篇笔记比较老,不再更新维护,请移步最新的手册:envoy相关笔记

@2019-01-17 15:16:31 IDE建议使用vscode,直接跳转到章节:在Mac上准备Envoy的开发环境

之前用Clion折腾了很久,发现Clion不能正确解读用bazel-cmakelists生成的CMakeLists.txt。后面关于Clion的内容先保留着,还是有一定价值的。

使用C/C++开发使用的Clion阅读envoy代码,Clion使用cmake管理构建项目的,envoy使用的是bazel。 Clion中可以安装一个名为bazel的插件的,不知为何安装之后没有效果,软件界面也和网上的教程不匹配。

用Clion阅读、编写Envoy代码

这种方式不能索引envoy依赖的外部代码! 建议直接参照第三节中的方法,用vscode + bazel-cmakelists。

如果只是用Clion阅读、编写Envoy代码,将Envoy代码导入后,需要将envoy项目目录中的自动生成的CMakeLists.txt文件内容修改为如下:

cmake_minimum_required(VERSION 3.13)
project(envoy)

set(CMAKE_CXX_STANDARD 14)

include_directories(include)
include_directories(source)

# add_executable保持原状
add_executable(envoy
     api/test/build/build_test.cc
     api/test/validate/pgv_test.cc
     ..省略...)

include目录只需要包含includesource目录就可以了,因为envoy的代码中引用头文件的时候,使用的是相对于include和source的路径,例如:

#include "exe/main_common.h"             // 引用的是:source/exe/main_common.h
#include "absl/debugging/symbolize.h"    // 部分找不到头文件是envoy依赖的外部库

用bazel-cmakelists生成CMakeLists.txt

这一步比较折腾,如果不在意bazel-cmakelists的用法,可以直接使用下一节中的操作,相关文件已经设置好了。

如果要在Clion中编译envoy,并且希望能够看到envoy依赖的代码,可以尝试一下Github上一个名为bazel-cmakelists的工具, 这个工具可以为使用Bazel的C++项目生成CMakeLists.txt。

本地需要安装有bazel,mac上可用brew安装bazel

brew tap bazelbuild/tap
brew tap-pin bazelbuild/tap
brew install bazelbuild/tap/bazel

下载bazel-cmakelists,将其放在envoy项目根目录中:

cd envoy
git clone https://github.com/lizan/bazel-cmakelists.git

bazel-cmakelists命令执行过程中会调用bazel命令构建指定的target,如果bazel build //source/exe:envoy-static能够运行,可以直接在envoy项目目录中执行下面的命令:

./bazel-cmakelists/bazel-cmakelists --targets //source/exe:envoy-static

如果是在容器中构建envoy,转换过程比较麻烦。

需要在ci/do_ci.sh中增加一个名为cmake的任务:

elif [[ "$1" == "cmake" ]]; then
  setup_gcc_toolchain
  echo "bazel cmake..."
  cd "${ENVOY_CI_DIR}"
  ../bazel-cmakelists/bazel-cmakelists --targets //source/exe:envoy-static --output=../CMakeLists.txt
  exit 0

修改./bazel-cmakelists/bazel-cmakelists

# 将下面这行
bazel_args=['build']
# 修改为
bazel_args=['build', '--strategy=Genrule=standalone', '--spawn_strategy=standalone', '--verbose_failures', '--package_path', '%workspace%:/source', '--action_env=HOME', '--action_env=PYTHONUSERBASE', '--jobs=4', '--show_task_finish', '-c', 'opt']

还需要将./bazel-cmakelists/bazel-cmakelists中所有用到bazel命令的地方加上参数'--package_path', '%workspace%:/source',例如:

for fn in subprocess.check_output(['bazel', 'query', '--package_path', '%workspace%:/source', '--noimplicit_deps', query]).splitlines():
xml = subprocess.check_output(['bazel', 'query','--package_path', '%workspace%:/source',  '--noimplicit_deps', query, '--output', 'xml'])
  ...

然后执行下面的命令:

$ ./ci/run_envoy_docker.sh './ci/do_ci.sh cmake'
building using 4 CPUs
gcc/g++ toolchain configured
bazel cmake...
$TEST_TMPDIR defined: output root default is '/build/tmp' and max_idle_secs default is '15'.
Starting local Bazel server and connecting to it...
$TEST_TMPDIR defined: output root default is '/build/tmp' and max_idle_secs default is '15'.
Loading:
Loading: 0 packages loaded
Loading: 0 packages loaded
    currently loading: source/exe
Analyzing: target //source/exe:envoy-static (3 packages loaded)
Analyzing: target //source/exe:envoy-static (9 packages loaded)
Analyzing: target //source/exe:envoy-static (84 packages loaded)
Analyzing: target //source/exe:envoy-static (176 packages loaded)
Analyzing: target //source/exe:envoy-static (187 packages loaded)
Analyzing: target //source/exe:envoy-static (259 packages loaded)
Analyzing: target //source/exe:envoy-static (269 packages loaded)
Analyzing: target //source/exe:envoy-static (285 packages loaded)
INFO: Analysed target //source/exe:envoy-static (290 packages loaded).
INFO: Found 1 target...
[1 / 7] [-----] BazelWorkspaceStatusAction stable-status.txt
...

得到CMakeLists.txt以后,还需要修正其中的路径,因为这个文件是在容器生成的,文件中的路径是容器中的路径,如果与容器外路径不一致,需要修改为容器外的路径,例如:

set(INCLUDE_DIRECTORIES
    /build/tmp/_bazel_bazel/436badd4919a15958fa3800a4e21074a/execroot/ci/external/com_github_google_jwt_verify
    /build/tmp/_bazel_bazel/436badd4919a15958fa3800a4e21074a/external/com_github_grpc_grpc/third_party/address_sorting
    /build/tmp/_bazel_bazel/436badd4919a15958fa3800a4e21074a/execroot/ci/bazel-out/k8-opt/genfiles/external/com_github_gabime_spdlog/include
    /build/tmp/_bazel_bazel/436badd4919a15958fa3800a4e21074a/execroot/ci/bazel-out/k8-opt/genfiles/external/com_github_eile_tclap/include
    /build/tmp/_bazel_bazel/436badd4919a15958fa3800a4e21074a/execroot/ci/bazel-out/k8-opt/genfiles/external/com_lyft_protoc_gen_validate
    /build/tmp/_bazel_bazel/436badd4919a15958fa3800a4e21074a/execroot/ci/external/bazel_tools
    /build/tmp/_bazel_bazel/436badd4919a15958fa3800a4e21074a/external/com_github_gabime_spdlog/include
    /build/tmp/_bazel_bazel/436badd4919a15958fa3800a4e21074a/external/boringssl/src/include
    /build/tmp/_bazel_bazel/436badd4919a15958fa3800a4e21074a/execroot/ci/bazel-out/k8-opt/genfiles/external/boringssl/src/includ

这里的/build是容器中的路径,容器外的路径是用变量ENVOY_DOCKER_BUILD_DIR指定的,默认是tmp

在Mac上准备Envoy的开发环境

建议使用vscode,用cilon的时候,即使用baze-cmakelists生成了CMakeLists.txt,cilon还是找不到头文件。 vscode就没有这个问题,按照后面的操作生成CMakeLists.txt之后,直接将eonvy目录导入vscode就可以了,非常好用,envoy自身的头文件和依赖的外部头文件都能找到。

安装vscode,直接到vscode的网站上下载

安装bazel:

brew tap bazelbuild/tap
brew tap-pin bazelbuild/tap
brew install bazelbuild/tap/bazel

下载envoy代码,这里切换到v1.8.0版本中:

git clone https://github.com/envoyproxy/envoy.git
cd envoy
git checkout v1.8.0

envoy目录中下载经过修改的bazel-cmakelists

cd envoy
git clone https://github.com/lijiaocn/envoy-bazel-cmakelists.git

ci/do_ci.sh中加入一个新的任务:

elif [[ "$1" == "cmake" ]]; then
  setup_gcc_toolchain
  echo "bazel cmake..."
  cd "${ENVOY_CI_DIR}"
  ../envoy-bazel-cmakelists/bazel-cmakelists --targets //source/exe:envoy-static --output=../CMakeLists.txt
  exit 0

在本地准备bazel的缓存目录,这里指定为/build,与docker中路径相同,这样就不需要调整生成的CMakeLists.txt文件了:

export ENVOY_DOCKER_BUILD_DIR=/build
sudo mkdir $ENVOY_DOCKER_BUILD_DIR
sudo chown -R  lijiao  $ENVOY_DOCKER_BUILD_DIR

如果不想每次都设置环境变量,可以在脚本ci/run_envoy_docker.sh开始的地方直接定义变量:

ENVOY_DOCKER_BUILD_DIR=/build

在Mac上还需要设置一下docker,在docker的Perferences->File Sharing面板中添加刚创建的/build目录,这一步一定要有,否则会报下面的错误:

➜  envoy git:(5d25f466c) ✗ ./ci/run_envoy_docker.sh './ci/do_ci.sh cmake'
docker: Error response from daemon: Mounts denied:
The path /build
is not shared from OS X and is not known to Docker.
You can configure shared paths from Docker -> Preferences... -> File Sharing.
See https://docs.docker.com/docker-for-mac/osxfs/#namespaces for more info.

在docker的配置面板中添加/build目录之后,需要点击配置面板中的Apply&Start才能生效。

用下面的命令生成CMakeLists.txt文件:

./ci/run_envoy_docker.sh './ci/do_ci.sh cmake'

生成CMakeLists.txt之前,会调用bazel编译envoy,envoy第一次编译的时间非常非常长,4核CPU的情况下都要几个小时。

编译过程中可能遇到下面的问题:

$TEST_TMPDIR defined: output root default is '/build/tmp' and max_idle_secs default is '15'.
Starting local Bazel server and connecting to it...
ERROR: /source/bazel/repositories.bzl:315:5: no such package '@boringssl//': BUILD file not found on package path and referenced by '//external:ssl'
ERROR: Analysis of target '//source/exe:envoy-static' failed; build aborted: Analysis failed
INFO: Elapsed time: 8.258s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (179 packages loaded)
    currently loading: @com_google_protobuf//
rm: cannot remove '/source/bazel-cmakelists': Is a directory

出现这个问题的原因不明确,有时候这一次运行和下一次运行的错误还不相同,怀疑是bazel在运行时要用git拉取很多代码,但是网速速度比较慢,有些代码拉取失败,导致后续一些列错误。 将/build目录中的文件清除后,在网络状况好的时候重试就可以了。

正常情况应当出现下面的日志:

Analyzing: target //source/exe:envoy-static (266 packages loaded)
INFO: SHA256 (https://github.com/prometheus/client_model/archive/99fa1f4be8e564e8a6b613da7fa6f46c9edafc6c.tar.gz) = 783bdaf8ee0464b35ec0c8704871e1e72afa0005c3f3587f65d9d6694bf3911b
INFO: Analysed target //source/exe:envoy-static (290 packages loaded).
INFO: Found 1 target...
[62 / 280] 4 actions running
    GoStdlib external/io_bazel_rules_go/linux_amd64_stripped/stdlib~/pkg [for host]; 72s local
    Compiling external/com_google_protobuf/src/google/protobuf/message.cc [for host]; 4s local
    Compiling external/com_google_protobuf/src/google/protobuf/compiler/zip_writer.cc [for host]; 1s local
    Compiling external/com_google_protobuf/src/google/protobuf/util/field_mask_util.cc [for host]; 0s local

还经常遇到下面的问题Server crashed during startup,原因同样不明,重新执行就好了,似乎某部外部条件导致bazel crash:

No remote cache bucket is set, skipping setup remote cache.
ENVOY_SRCDIR=/source
find: File system loop detected; '/source/ci/bazel-ci/ci' is part of the same file system loop as '/source/ci'.
find: File system loop detected; '/source/ci/bazel-ci/external/envoy/ci' is part of the same file system loop as '/source/ci'.
HEAD is now at 3e5b733... cleanup: match NOT_IMPLEMENTED_GCOVR_EXCL_LINE change (#53)
building using 4 CPUs
gcc/g++ toolchain configured
bazel release build...
Building...
$TEST_TMPDIR defined: output root default is '/build/tmp' and max_idle_secs default is '15'.
Starting local Bazel server and connecting to it...

Server crashed during startup. Now printing /build/tmp/_bazel_bazel/436badd4919a15958fa3800a4e21074a/server/jvm.out
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by io.netty.util.internal.ReflectionUtil (file:/build/tmp/_bazel_bazel/install/ca1b8b7c3e5200be14b7f27896826862/_embedded_binaries/A-server.jar) to field sun.nio.ch.SelectorImpl.selectedKeys
WARNING: Please consider reporting this to the maintainers of io.netty.util.internal.ReflectionUtil
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release

如果使用CentOS镜像编译,有可能遇到下面的问题:

In file included from source/exe/main_common.cc:6:0:
bazel-out/k8-opt/bin/source/common/common/_virtual_includes/compiler_requirements_lib/common/common/compiler_requirements.h:14:2: error: #error "Your toolchain has set _GLIBCXX_USE_CXX11_ABI to a value that uses a std::string " "implementation that is not thread-safe. This may cause rare and difficult-to-debug errors " "if std::string is passed between threads in any way. If you accept this risk, you may define " "ENVOY_IGNORE_GLIBCXX_USE_CXX11_ABI_ERROR=1 in your build."
 #error "Your toolchain has set _GLIBCXX_USE_CXX11_ABI to a value that uses a std::string "

研究了bazel半天,没找到添加预定义宏的地方,后来同事找到了一个临时方法,直接把代码修改了:

source/common/common/compiler_requirements.h中第13行前面的!去掉:

#if defined(_GLIBCXX_USE_CXX11_ABI) && _GLIBCXX_USE_CXX11_ABI != 1 &&                              \
    !defined(ENVOY_IGNORE_GLIBCXX_USE_CXX11_ABI_ERROR)

修改成:

#if defined(_GLIBCXX_USE_CXX11_ABI) && _GLIBCXX_USE_CXX11_ABI != 1 &&                              \
    defined(ENVOY_IGNORE_GLIBCXX_USE_CXX11_ABI_ERROR)

编译完成,envoy目录中生成了CMakeLists.txt文件之后,直接将envoy目录导入到vscode中,就可以愉快的读代码、写代码了。

如果还是提示头文件找不到,编辑.vscode目录的中的json文件,在includePath中添加上/build/**

"includePath": [
    "${workspaceFolder}/**",
    "/build/**"
],

然后还可以安装Bazel Tools插件。

VScode中查找C++函数的引用

方法1:cquery插件(推荐方法2)

Find all references中讨论了很久,没耐心看他们在讨论些什么…

尝试了一下讨论中提到的cquery插件,安装后发现可用。

安装cquery,在mac上可以直接用brew安装:

brew install cquery

cquery和vscode官方的c++插件冲突,后来发现C++ Intellisense更好一些。

方法2:C++ Intellisense

在vscode中安装了C++ Intellisense插件之后,还需要在本地安装gnu global tool,在mac上可以直接用brew安装:

brew install global

然后到项目顶层目录中执行命令gtags生成tag文件。


推荐阅读

Copyright @2011-2019 All rights reserved. 转载请添加原文连接,合作请加微信lijiaocn或者发送邮件: [email protected],备注网站合作

友情链接:  系统软件  程序语言  运营经验  水库文集  网络课程  微信网文  发现知识星球