Memory optimization for fit a line demo

[QiJune]

This PR includes:

• Control flow graph construction #7316 Simple Control Flow graph construction without if/while operator
• Variable liveness analysis based on control flow graph #7317 Variable liveness analysis
• Memory reuse policy based on liveness analysis #7318 Simple memory reuse policy without inplace attribute support
• training benchmark on some examples #7320 A simple demo on fit_a_line

The cache memory pool can be hit:

hit cache !!!! pool index is 1, var name is fc_0.tmp_1@GRAD, cache var name is square_error_cost_0.tmp_1@GRAD, var shape is [-1L, 1L]
elementwise_add_grad
hit cache !!!! pool index is 0, var name is fc_0.b_0@GRAD, cache var name is mean_0.tmp_0@GRAD, var shape is [1L]
hit cache !!!! pool index is 0, var name is fc_0.tmp_0@GRAD, cache var name is square_error_cost_0.tmp_1, var shape is [-1L, 1L]

Memory can be saved from 102400 bytes to 90112 bytes.

Memory use before optimization in a batch

I0108 20:34:28.472046 2497127360 executor.cc:114] Memory used 36864
I0108 20:34:28.472112 2497127360 executor.cc:114] Memory used 49152
I0108 20:34:28.472159 2497127360 executor.cc:114] Memory used 53248
I0108 20:34:28.472302 2497127360 executor.cc:114] Memory used 57344
I0108 20:34:28.472363 2497127360 executor.cc:114] Memory used 61440
I0108 20:34:28.472487 2497127360 executor.cc:114] Memory used 65536
I0108 20:34:28.472534 2497127360 executor.cc:114] Memory used 69632
I0108 20:34:28.472662 2497127360 executor.cc:114] Memory used 73728
I0108 20:34:28.472720 2497127360 executor.cc:114] Memory used 77824
I0108 20:34:28.472798 2497127360 executor.cc:114] Memory used 81920
I0108 20:34:28.472895 2497127360 executor.cc:114] Memory used 86016
I0108 20:34:28.472966 2497127360 executor.cc:114] Memory used 90112
I0108 20:34:28.473053 2497127360 executor.cc:114] Memory used 98304
I0108 20:34:28.473188 2497127360 executor.cc:114] Memory used 102400
I0108 20:34:28.473237 2497127360 executor.cc:114] Memory used 102400
I0108 20:34:28.473333 2497127360 executor.cc:114] Memory used 102400
I0108 20:34:28.473363 2497127360 executor.cc:126] Memory used 102400
I0108 20:34:28.473496 2497127360 executor.cc:130] Memory used after deleting local scope 36864

Memory use after optimization in a batch

I0108 20:35:11.611282 2497127360 executor.cc:114] Memory used 36864
I0108 20:35:11.611460 2497127360 executor.cc:114] Memory used 49152
I0108 20:35:11.611562 2497127360 executor.cc:114] Memory used 53248
I0108 20:35:11.611773 2497127360 executor.cc:114] Memory used 57344
I0108 20:35:11.612010 2497127360 executor.cc:114] Memory used 61440
I0108 20:35:11.612197 2497127360 executor.cc:114] Memory used 65536
I0108 20:35:11.612269 2497127360 executor.cc:114] Memory used 69632
I0108 20:35:11.612385 2497127360 executor.cc:114] Memory used 73728
I0108 20:35:11.612460 2497127360 executor.cc:114] Memory used 77824
I0108 20:35:11.612624 2497127360 executor.cc:114] Memory used 81920
I0108 20:35:11.612782 2497127360 executor.cc:114] Memory used 86016
I0108 20:35:11.612910 2497127360 executor.cc:114] Memory used 86016
I0108 20:35:11.613070 2497127360 executor.cc:114] Memory used 86016
I0108 20:35:11.613468 2497127360 executor.cc:114] Memory used 90112
I0108 20:35:11.613684 2497127360 executor.cc:114] Memory used 90112
I0108 20:35:11.613921 2497127360 executor.cc:114] Memory used 90112
I0108 20:35:11.614055 2497127360 executor.cc:126] Memory used 90112
I0108 20:35:11.614233 2497127360 executor.cc:130] Memory used after deleting local scope 36864

[reyoung]

We can just handle memory optimization for each block and do not optimize the memory between block. It may be enough.

[wangkuiyi]

Do we need a little more details in this log message, e.g., “before ....” or “after ...”?

[QiJune]

This PR mainly provides a demo to show the result and these logs are actually for debugging. And another clean PR focus on memory optimization transpiler #7356 has been merged.
So I will close this PR.