Feature/anakin embed by Superjomn · Pull Request #11135 · PaddlePaddle/Paddle

Superjomn · 2018-06-03T14:53:31Z

Code of Anakin's paddle interface is a little mess, I changed some and will give a PR to its repo latter.

TODO add an offline valid test.
TODO wget the Anakin library from somewhere in CI phrase or just run it manually offline?

…/anakin_embed

luotao1 · 2018-06-04T02:25:07Z

paddle/contrib/inference/CMakeLists.txt

 endif(APPLE)

+set(ANAKIN_INCLUDE "" CACHE STRING "root of Anakin header files")
+set(ANAKIN_LIBRARY "" CACHE STRING "path of Anakin library")


可以仿照TENSORRT和CUDNN一样，合并成一个ANAKIN_ROOT么，然后自动��include和lib。

Paddle/cmake/tensorrt.cmake

Lines 5 to 16 in 71f51ff

set(TENSORRT_ROOT "/usr" CACHE PATH "TENSORRT ROOT")

find_path(TENSORRT_INCLUDE_DIR NvInfer.h

PATHS ${TENSORRT_ROOT} ${TENSORRT_ROOT}/include

$ENV{TENSORRT_ROOT} $ENV{TENSORRT_ROOT}/include

NO_DEFAULT_PATH

)

find_library(TENSORRT_LIBRARY NAMES libnvinfer.so libnvinfer.a

PATHS ${TENSORRT_ROOT} ${TENSORRT_ROOT}/lib

$ENV{TENSORRT_ROOT} $ENV{TENSORRT_ROOT}/lib

NO_DEFAULT_PATH

DOC "Path to TensorRT library.")

暂时不行。。阿拉金的库比较乱

接口没有封装，要依赖整个 repo 的头文件

library 需要手动拷，和完整 repo 的头文件没有放在一起

他们的 paddle_api.h 是个 test，没有包含在发布的头文件里

所以，现在必须显式下载 library，和完整的 github repo，分别设定路径，才能编译

后面会让他们改，这个可能是临时方案。

luotao1 · 2018-06-04T02:26:10Z

paddle/contrib/inference/CMakeLists.txt

@@ -17,6 +17,36 @@ if(APPLE)
    set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wno-error=pessimizing-move")
 endif(APPLE)


可以单独一个anakin.cmake，paddle中所有第三方库加入的时候，都单独做了一个cmake，这样会更加清晰。
20-42行可以放进anakin.cmake中。

后面可以看看，暂时他们的代码也有挺多问题的，现在给的接口也非常初步。。

单独的anakin.cmake里面也可以显示设置include和lib

可以等稍微成熟些再改地正规些，我感觉需要再跟他们沟通下。

目前的状态太初步了，他们的头文件我自己也改了很多地方才能用，后面应该他们要有大的修正才正式用起来。

luotao1 · 2018-06-04T02:28:03Z

paddle/contrib/inference/CMakeLists.txt

+set(inference_deps paddle_inference_api paddle_fluid_api)
+
+# if anakin is set enable anakin api implementation
+if (NOT ANAKIN_INCLUDE STREQUAL "")


if(ANAKIN_INCLUDE_DIR AND ANAKIN_LIBRARY) set(ANAKIN_FOUND ON) else() set(ANAKIN_FOUND OFF) endif()

需要一个ANAKIN_FOUND的开关

这个可以有

luotao1 · 2018-06-04T02:29:28Z

paddle/contrib/inference/CMakeLists.txt

+    set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wno-error=switch")
+    set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wno-error=return-type")
+    set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wno-error=non-virtual-dtor")
+    set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wno-error=cpp")


28-35行可以合并成：

if(ANAKIN_FOUND) set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wno-error=comment -Wno-error=reorder -Wno-error=format ...")

luotao1 · 2018-06-04T02:30:14Z

paddle/contrib/inference/CMakeLists.txt

 inference_api_test(test_paddle_inference_api_impl
                    ARGS test_word2vec test_image_classification)

+if (NOT ANAKIN_INCLUDE STREQUAL "")


if(ANAKIN_FOUND)

panyx0718 · 2018-06-04T02:24:49Z

paddle/contrib/inference/paddle_inference_api_anakin_engine.cc

+    const std::vector<PaddleTensor> &inputs,
+    std::vector<PaddleTensor> *output_data) {
+  for (const auto &input : inputs) {
+    CHECK(input.dtype == PaddleDType::FLOAT32);


should return false?

panyx0718 · 2018-06-04T02:25:02Z

paddle/contrib/inference/paddle_inference_api_anakin_engine.cc

+
+PaddleInferenceAnakinPredictor::PaddleInferenceAnakinPredictor(
+    const AnakinConfig &config) {
+  CHECK(Init(config));


Do this in CreateXXX and return false?

Currently, Anakin engine's creation is just a void function.

panyx0718 · 2018-06-04T02:25:48Z

paddle/contrib/inference/paddle_inference_api_anakin_engine.cc

+  // TODO(Superjomn) Tell anakin to support return code.
+  engine_.Execute();
+
+  CHECK(!output_data->empty())


maybe not need to worry about it?

I found some issue about the usage of the output_data. The current implementation for the native engine is that the output_data's memory is maintained by the engine, but the original intention is to force the users to maintain the memory outside and the engine just copies the data.

So that, it can support

fetch any target, just set the PaddleTensor's names of output_data -- given a big model, fetch any variable

engines like Anakin's output variables are located at GPU, need a memory to copied, and users can do some memory optimization such as pre-allocate a big buffer.

panyx0718 · 2018-06-04T02:27:41Z

paddle/contrib/inference/paddle_inference_api_anakin_engine.cc

+    auto *tensor = engine_.GetOutputInGPU(output.name);
+    output.shape = tensor->shape();
+    // Copy data from GPU -> CPU
+    CHECK_EQ(cudaMemcpy(output.data.data,


return false?

panyx0718 · 2018-06-04T02:31:41Z

paddle/contrib/inference/CMakeLists.txt

+# if anakin is set enable anakin api implementation
+if (NOT ANAKIN_INCLUDE STREQUAL "")
+    # Anakin's code style doesn't follow google c style.
+    set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wno-error=comment")


Does this affect other codes outside anakin?

yes, if Anakin is turn on. @luotao1 any solution?

请问是我们使用anakin头文件的时候，会出现各种错误么？

这几个 warning 都会编译失败，代码问题比较多，现在是会嵌套 include 他们所有的头文件，会有这类 warning.

有一个办法，是可以不改cmake_cxx_flags。

参考 https://eigen.tuxfamily.org/dox/DisableStupidWarnings_8h_source.html
新增一个DisableStupidWarnings.h的头文件，将所有的warning以 #pragma GCC diagnostic ignored "-Wignored-attributes" 形式给去掉，加载anakin的头文件时，同步加载这个头文件。或者直接写进anakin的头文件中。

我在使用ICC编译器时，会出现Intel编译器一堆warning的问题，亲测有效。

luotao1

ci里面没有检查使用anakin的代码正确性，是不是需要在dockerfile里面先把anakin给装上？

luotao1 · 2018-06-04T02:44:37Z

paddle/contrib/inference/CMakeLists.txt

@@ -17,6 +17,36 @@ if(APPLE)
    set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wno-error=pessimizing-move")
 endif(APPLE)


单独的anakin.cmake里面也可以显示设置include和lib

Superjomn added 7 commits June 2, 2018 08:59

init

b29e36a

update

eba331f

Merge branch 'develop' of github.com:PaddlePaddle/Paddle into feature…

9382b05

…/anakin_embed

finish

3597d70

update code style

4ac35fe

fix code style

0462466

fix compile error

08ac23f

Superjomn requested review from luotao1 and panyx0718 June 4, 2018 00:47

luotao1 reviewed Jun 4, 2018

View reviewed changes

panyx0718 reviewed Jun 4, 2018

View reviewed changes

luotao1 reviewed Jun 4, 2018

View reviewed changes

update follow reviews

546006b

panyx0718 approved these changes Jun 4, 2018

View reviewed changes

Superjomn merged commit 418c41d into PaddlePaddle:develop Jun 4, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feature/anakin embed#11135

Feature/anakin embed#11135
Superjomn merged 8 commits intoPaddlePaddle:developfrom
Superjomn:feature/anakin_embed

Superjomn commented Jun 3, 2018 •

edited

Loading

luotao1 Jun 4, 2018

Superjomn Jun 4, 2018

luotao1 Jun 4, 2018

Superjomn Jun 4, 2018

luotao1 Jun 4, 2018

Superjomn Jun 4, 2018

luotao1 Jun 4, 2018

Superjomn Jun 4, 2018

luotao1 Jun 4, 2018

Superjomn Jun 4, 2018

luotao1 Jun 4, 2018

panyx0718 Jun 4, 2018

Superjomn Jun 4, 2018

panyx0718 Jun 4, 2018

Superjomn Jun 4, 2018

panyx0718 Jun 4, 2018

Superjomn Jun 4, 2018

panyx0718 Jun 4, 2018

Superjomn Jun 4, 2018

panyx0718 Jun 4, 2018

Superjomn Jun 4, 2018

luotao1 Jun 4, 2018

Superjomn Jun 4, 2018

luotao1 Jun 4, 2018 •

edited

Loading

luotao1 left a comment •

edited

Loading

luotao1 Jun 4, 2018

Labels

3 participants

	set(TENSORRT_ROOT "/usr" CACHE PATH "TENSORRT ROOT")
	find_path(TENSORRT_INCLUDE_DIR NvInfer.h
	PATHS ${TENSORRT_ROOT} ${TENSORRT_ROOT}/include
	$ENV{TENSORRT_ROOT} $ENV{TENSORRT_ROOT}/include
	NO_DEFAULT_PATH
	)

	find_library(TENSORRT_LIBRARY NAMES libnvinfer.so libnvinfer.a
	PATHS ${TENSORRT_ROOT} ${TENSORRT_ROOT}/lib
	$ENV{TENSORRT_ROOT} $ENV{TENSORRT_ROOT}/lib
	NO_DEFAULT_PATH
	DOC "Path to TensorRT library.")

		@@ -17,6 +17,36 @@ if(APPLE)
		set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wno-error=pessimizing-move")
		endif(APPLE)

Conversation

Superjomn commented Jun 3, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

luotao1 Jun 4, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

luotao1 left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Labels

3 participants

Superjomn commented Jun 3, 2018 •

edited

Loading

luotao1 Jun 4, 2018 •

edited

Loading

luotao1 left a comment •

edited

Loading