Skip to content

Commit

Permalink
[Dy2Stat] Update the doc of error handling for 2.0 (PaddlePaddle#3149)
Browse files Browse the repository at this point in the history
  • Loading branch information
liym27 authored Jan 20, 2021
1 parent af6c5b1 commit fa258b3
Show file tree
Hide file tree
Showing 2 changed files with 57 additions and 88 deletions.
81 changes: 33 additions & 48 deletions doc/paddle/guides/04_dygraph_to_static/error_handling_cn.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ def func():
func()
```

ProgramTranslator打印的警告信息如下
ProgramTranslator 打印的警告信息如下

```bash
2020-01-01 00:00:00,104-WARNING: <function inner_func at 0x125b3a550> doesn't have to be transformed to static function because it has been transformed before, it will be run as-is.
Expand All @@ -33,8 +33,8 @@ ProgramTranslator打印的警告信息如下:
重新抛出的异常具有以下特点:
- 隐藏了部分对用户无用的动转静过程调用栈;
- 转换前的代码会给出提示:"In User Code:";
- 报错信息中包含了转换前的原始动态图代码;
- 转换后的代码的异常信息,给出提示"In transformed code:";
- 报错信息中包含了转换前的原始动态图代码,并给出提示"(* user code *)"
例如,运行以下代码,在静态图构建时,即编译期会抛出异常:
Expand All @@ -55,13 +55,15 @@ func(np.ones([3, 2]))
```bash
Traceback (most recent call last):
<ipython-input-13-f9c3ea702e3a> in <module>()
func(np.ones([3, 2]))
File "paddle/fluid/dygraph/dygraph_to_static/program_translator.py", line 332, in __call__
func(np.ones([3, 2]))
File "paddle/fluid/dygraph/dygraph_to_static/program_translator.py", line 352, in __call__
error_data.raise_new_exception()
File "paddle/fluid/dygraph/dygraph_to_static/error.py", line 188, in raise_new_exception
raise new_exception
AssertionError: In user code:
AssertionError: In transformed code:
File "<ipython-input-13-f9c3ea702e3a>", line 7, in func
x = fluid.layers.reshape(x, shape=[-1, -1])
File "<ipython-input-13-f9c3ea702e3a>", line 7, in func (* user code *)
x = paddle.reshape(x, shape=[-1, -1])
File "paddle/fluid/layers/nn.py", line 6193, in reshape
attrs["shape"] = get_attr_shape(shape)
File "paddle/fluid/layers/nn.py", line 6169, in get_attr_shape
Expand All @@ -73,18 +75,19 @@ AssertionError: In user code:
1. 报错栈中,涉及代码转换过程的信息栈默认会被隐藏,不进行展示,以减少干扰信息。
2. ProgramTranslator 处理后的报错信息中,会包含提示 "In user code:",表示之后的报错栈中,包含动转静前的动态图代码,即用户写的代码:
```bash
AssertionError: In user code:
File "<ipython-input-13-f9c3ea702e3a>", line 7, in func
x = fluid.layers.reshape(x, shape=[-1, -1])
File "paddle/fluid/layers/nn.py", line 6193, in reshape
attrs["shape"] = get_attr_shape(shape)
File "paddle/fluid/layers/nn.py", line 6169, in get_attr_shape
"be -1. But received shape[%d] is also -1." % dim_idx)
2. ProgramTranslator 处理后的报错信息中,会包含提示 "In transformed code:",表示之后的报错信息栈,是在运行转换后的代码时的报错信息:
```bash
AssertionError: In transformed code:
File "<ipython-input-13-f9c3ea702e3a>", line 7, in func (* user code *)
x = paddle.reshape(x, shape=[-1, -1])
File "paddle/fluid/layers/nn.py", line 6193, in reshape
attrs["shape"] = get_attr_shape(shape)
File "paddle/fluid/layers/nn.py", line 6169, in get_attr_shape
"be -1. But received shape[%d] is also -1." % dim_idx)
```
其中,`File "<ipython-input-13-f9c3ea702e3a>", line 7, in func` 是转换前的代码位置信息,`x = fluid.layers.reshape(x, shape=[-1, -1])` 是转换前的代码
其中,`File "<ipython-input-13-f9c3ea702e3a>", line 7, in func` 是转换前的代码位置信息,`x = paddle.reshape(x, shape=[-1, -1])` 是转换前用户的动态图代码
3. 新的异常中,包含原始报错中的的报错信息,如下:
```bash
Expand All @@ -101,7 +104,7 @@ AssertionError: In user code:
@paddle.jit.to_static
def func(x):
x = paddle.to_tensor(x)
two = paddle.fill_constant(shape=[1], value=2, dtype="int32")
two = paddle.full(shape=[1], fill_value=2, dtype="int32")
x = paddle.reshape(x, shape=[1, two])
return x
Expand All @@ -113,11 +116,12 @@ func(np.ones([3]).astype("int32"))
```bash
Traceback (most recent call last):
File "<ipython-input-57-c63d6a351262>", line 10, in <module>()
func(np.ones([3]).astype("int32"))
File "paddle/fluid/dygraph/dygraph_to_static/program_translator.py", line 332, in __call__
func(np.ones([3]).astype("int32"))
File "paddle/fluid/dygraph/dygraph_to_static/program_translator.py", line 352, in __call__
error_data.raise_new_exception()
File "paddle/fluid/dygraph/dygraph_to_static/error.py", line 188, in raise_new_exception
raise new_exception
EnforceNotMet: In user code:
EnforceNotMet: In transformed code:
File "<ipython-input-57-c63d6a351262>", line 7, in func
x = paddle.reshape(x, shape=[1, two])
Expand All @@ -132,32 +136,13 @@ EnforceNotMet: In user code:
File "paddle/fluid/framework.py", line 1977, in __init__
for frame in traceback.extract_stack():
--------------------------------------
C++ Traceback (most recent call last):
--------------------------------------
0 paddle::imperative::Tracer::TraceOp(std::string const&, paddle::imperative::NameVarBaseMap const&, paddle::imperative::NameVarBaseMap const&, paddle::framework::AttributeMap, paddle::platform::Place const&, bool)
1 paddle::imperative::OpBase::Run(paddle::framework::OperatorBase const&, paddle::imperative::NameVarBaseMap const&, paddle::imperative::NameVarBaseMap const&, paddle::framework::AttributeMap const&, paddle::platform::Place const&)
2 paddle::imperative::PreparedOp::Run(paddle::imperative::NameVarBaseMap const&, paddle::imperative::NameVarBaseMap const&, paddle::framework::AttributeMap const&)
3 std::_Function_handler<void (paddle::framework::ExecutionContext const&), paddle::framework::OpKernelRegistrarFunctor<paddle::platform::CPUPlace, false, 0ul, paddle::operators::RunProgramOpKernel<paddle::platform::CPUDeviceContext, float> >::operator()(char const*, char const*, int) const::{lambda(paddle::framework::ExecutionContext const&)#1}>::_M_invoke(std::_Any_data const&, paddle::framework::ExecutionContext const&)
4 paddle::operators::RunProgramOpKernel<paddle::platform::CPUDeviceContext, float>::Compute(paddle::framework::ExecutionContext const&) const
5 paddle::framework::Executor::RunPartialPreparedContext(paddle::framework::ExecutorPrepareContext*, paddle::framework::Scope*, long, long, bool, bool, bool)
6 paddle::framework::OperatorBase::Run(paddle::framework::Scope const&, paddle::platform::Place const&)
7 paddle::framework::OperatorWithKernel::RunImpl(paddle::framework::Scope const&, paddle::platform::Place const&) const
8 paddle::framework::OperatorWithKernel::RunImpl(paddle::framework::Scope const&, paddle::platform::Place const&, paddle::framework::RuntimeContext*) const
9 paddle::operators::ReshapeKernel::operator()(paddle::framework::ExecutionContext const&) const
10 paddle::operators::ReshapeOp::ValidateShape(std::vector<int, std::allocator<int> >, paddle::framework::DDim const&)
11 paddle::platform::EnforceNotMet::EnforceNotMet(std::string const&, char const*, int)
12 paddle::platform::GetCurrentTraceBackString()
----------------------
Error Message Summary:
----------------------
InvalidArgumentError: The 'shape' in ReshapeOp is invalid. The input tensor X'size must be equal to the capacity of 'shape'. But received X's shape = [3], X's size = 3, 'shape' is [1, 2], the capacity of 'shape' is 2.
[Hint: Expected capacity == in_size, but received capacity:2 != in_size:3.] (at /paddle/paddle/fluid/operators/reshape_op.cc:206)
[operator < reshape2 > error] [operator < run_program > error]
InvalidArgumentError: The 'shape' in ReshapeOp is invalid. The input tensor X'size must be equal to the capacity of 'shape'. But received X's shape = [3], X's size = 3, 'shape' is [1, 2], the capacity of 'shape' is 2.
[Hint: Expected capacity == in_size, but received capacity:2 != in_size:3.] (at /home/teamcity/work/ef54dc8a5b211854/paddle/fluid/operators/reshape_op.cc:222)
[Hint: If you need C++ stacktraces for debugging, please set `FLAGS_call_stack_level=2`.]
[operator < reshape2 > error] [operator < run_program > error]
```

上述异常中,除了隐藏部分报错栈、报错定位到转换前的动态图代码外,报错信息中包含了C++报错栈 `C++ Traceback``Error Message Summary`,这是 Paddle 的 C++ 端异常信息,经处理后在 Python 的异常信息中显示
上述异常中,除了隐藏部分报错栈、报错定位到转换前的动态图代码外,报错信息中隐藏了C++报错栈,您可设置环境变量 `FLAGS_call_stack_level=2` 来展示 C++ 栈信息

> **注解:**
>
Expand Down
64 changes: 24 additions & 40 deletions doc/paddle/guides/04_dygraph_to_static/error_handling_en.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ When an exception occurs in the transformed code by ProgramTranslator, the excep
Among the features of the re-raised exception:
- Some useless call stacks of Dynamic-to-Static are hidden;
- A prompt will be given before the un-transformed code: "In User Code:";
- For the abnormal information of the transformed code, the prompt "in transformed code:" is given;
- The error message includes references to the original dynamic graph code before transformation;
For example, if executing the following code, an exception is raised when the static graph is built, that is, at compile time:
Expand All @@ -57,13 +57,15 @@ func(np.ones([3, 2]))
```bash
Traceback (most recent call last):
<ipython-input-13-f9c3ea702e3a> in <module>()
func(np.ones([3, 2]))
File "paddle/fluid/dygraph/dygraph_to_static/program_translator.py", line 332, in __call__
func(np.ones([3, 2]))
File "paddle/fluid/dygraph/dygraph_to_static/program_translator.py", line 352, in __call__
error_data.raise_new_exception()
File "paddle/fluid/dygraph/dygraph_to_static/error.py", line 188, in raise_new_exception
raise new_exception
AssertionError: In user code:
AssertionError: In transformed code:
File "<ipython-input-13-f9c3ea702e3a>", line 7, in func
x = fluid.layers.reshape(x, shape=[-1, -1])
File "<ipython-input-13-f9c3ea702e3a>", line 7, in func (* user code *)
x = paddle.reshape(x, shape=[-1, -1])
File "paddle/fluid/layers/nn.py", line 6193, in reshape
attrs["shape"] = get_attr_shape(shape)
File "paddle/fluid/layers/nn.py", line 6169, in get_attr_shape
Expand All @@ -75,19 +77,19 @@ The above error information can be divided into three points:
1. In the error stack, the call stacks related to the code transformation process are hidden by default and not displayed, so as to avoid confusion.
2. In the error message processed by ProgramTranslator, a prompt "In user code:" will be included, which means that the following error stacks contains the original dynamic graph code, that is, the code written by the user:
2. In the error message processed by ProgramTranslator, a prompt "In transformed code:" will be included, which means that the following error information stack is raised when running the converted code:
```bash
AssertionError: In user code:
AssertionError: In transformed code:
File "<ipython-input-13-f9c3ea702e3a>", line 7, in func
x = fluid.layers.reshape(x, shape=[-1, -1])
File "<ipython-input-13-f9c3ea702e3a>", line 7, in func (* user code *)
x = paddle.reshape(x, shape=[-1, -1])
File "paddle/fluid/layers/nn.py", line 6193, in reshape
attrs["shape"] = get_attr_shape(shape)
File "paddle/fluid/layers/nn.py", line 6169, in get_attr_shape
"be -1. But received shape[%d] is also -1." % dim_idx)
```
`File "<ipython-input-13-f9c3ea702e3a>", line 7, in func` is the location information of un-transformed code, `x = fluid.layers.reshape(x, shape=[-1, -1])` is the un-transformed code.
`File "<ipython-input-13-f9c3ea702e3a>", line 7, in func` is the location information of un-transformed code, `x = paddle.reshape(x, shape=[-1, -1])` is the un-transformed code.
3. The new exception contains the message that the exception originally reported, as follows:
```bash
Expand All @@ -104,7 +106,7 @@ If execute the following code, an exception is raised when the static graph is e
@paddle.jit.to_static
def func(x):
x = paddle.to_tensor(x)
two = paddle.fill_constant(shape=[1], value=2, dtype="int32")
two = paddle.full(shape=[1], fill_value=2, dtype="int32")
x = paddle.reshape(x, shape=[1, two])
return x
Expand All @@ -114,11 +116,12 @@ func(np.ones([3]).astype("int32"))
```bash
Traceback (most recent call last):
File "<ipython-input-57-c63d6a351262>", line 10, in <module>()
func(np.ones([3]).astype("int32"))
File "paddle/fluid/dygraph/dygraph_to_static/program_translator.py", line 332, in __call__
func(np.ones([3]).astype("int32"))
File "paddle/fluid/dygraph/dygraph_to_static/program_translator.py", line 352, in __call__
error_data.raise_new_exception()
File "paddle/fluid/dygraph/dygraph_to_static/error.py", line 188, in raise_new_exception
raise new_exception
EnforceNotMet: In user code:
EnforceNotMet: In transformed code:
File "<ipython-input-57-c63d6a351262>", line 7, in func
x = paddle.reshape(x, shape=[1, two])
Expand All @@ -133,32 +136,13 @@ EnforceNotMet: In user code:
File "paddle/fluid/framework.py", line 1977, in __init__
for frame in traceback.extract_stack():
--------------------------------------
C++ Traceback (most recent call last):
--------------------------------------
0 paddle::imperative::Tracer::TraceOp(std::string const&, paddle::imperative::NameVarBaseMap const&, paddle::imperative::NameVarBaseMap const&, paddle::framework::AttributeMap, paddle::platform::Place const&, bool)
1 paddle::imperative::OpBase::Run(paddle::framework::OperatorBase const&, paddle::imperative::NameVarBaseMap const&, paddle::imperative::NameVarBaseMap const&, paddle::framework::AttributeMap const&, paddle::platform::Place const&)
2 paddle::imperative::PreparedOp::Run(paddle::imperative::NameVarBaseMap const&, paddle::imperative::NameVarBaseMap const&, paddle::framework::AttributeMap const&)
3 std::_Function_handler<void (paddle::framework::ExecutionContext const&), paddle::framework::OpKernelRegistrarFunctor<paddle::platform::CPUPlace, false, 0ul, paddle::operators::RunProgramOpKernel<paddle::platform::CPUDeviceContext, float> >::operator()(char const*, char const*, int) const::{lambda(paddle::framework::ExecutionContext const&)#1}>::_M_invoke(std::_Any_data const&, paddle::framework::ExecutionContext const&)
4 paddle::operators::RunProgramOpKernel<paddle::platform::CPUDeviceContext, float>::Compute(paddle::framework::ExecutionContext const&) const
5 paddle::framework::Executor::RunPartialPreparedContext(paddle::framework::ExecutorPrepareContext*, paddle::framework::Scope*, long, long, bool, bool, bool)
6 paddle::framework::OperatorBase::Run(paddle::framework::Scope const&, paddle::platform::Place const&)
7 paddle::framework::OperatorWithKernel::RunImpl(paddle::framework::Scope const&, paddle::platform::Place const&) const
8 paddle::framework::OperatorWithKernel::RunImpl(paddle::framework::Scope const&, paddle::platform::Place const&, paddle::framework::RuntimeContext*) const
9 paddle::operators::ReshapeKernel::operator()(paddle::framework::ExecutionContext const&) const
10 paddle::operators::ReshapeOp::ValidateShape(std::vector<int, std::allocator<int> >, paddle::framework::DDim const&)
11 paddle::platform::EnforceNotMet::EnforceNotMet(std::string const&, char const*, int)
12 paddle::platform::GetCurrentTraceBackString()
----------------------
Error Message Summary:
----------------------
InvalidArgumentError: The 'shape' in ReshapeOp is invalid. The input tensor X'size must be equal to the capacity of 'shape'. But received X's shape = [3], X's size = 3, 'shape' is [1, 2], the capacity of 'shape' is 2.
[Hint: Expected capacity == in_size, but received capacity:2 != in_size:3.] (at /paddle/paddle/fluid/operators/reshape_op.cc:206)
[operator < reshape2 > error] [operator < run_program > error]
InvalidArgumentError: The 'shape' in ReshapeOp is invalid. The input tensor X'size must be equal to the capacity of 'shape'. But received X's shape = [3], X's size = 3, 'shape' is [1, 2], the capacity of 'shape' is 2.
[Hint: Expected capacity == in_size, but received capacity:2 != in_size:3.] (at /home/teamcity/work/ef54dc8a5b211854/paddle/fluid/operators/reshape_op.cc:222)
[Hint: If you need C++ stacktraces for debugging, please set `FLAGS_call_stack_level=2`.]
[operator < reshape2 > error] [operator < run_program > error]
```

In the above exception, in addition to hiding part of the error stack and locating the error to the un-transformed dynamic graph code, the error information includes the c++ error stack `C++ Traceback` and `Error Message Summary`, which are the exception from C++ and are displayed in Python exception after processing.
In the above exception, in addition to hiding part of the error stack and locating the error to the un-transformed dynamic graph code, the C++ error stack is hidden. You can set the environment variable `FLAGS_call_stack_level=2` to show C++ stack information.

> **NOTE:**
>
Expand Down

0 comments on commit fa258b3

Please sign in to comment.