Add nms op and batched_nms api #40962

RichardWooSJTU · 2022-03-25T12:19:18Z

PR types

New features

PR changes

OPs

Describe

add nms op in paddle/fluid/operators/detection and corresponding api batched_nms In python/paddle/vision/ops.py

English document preview:

中文文档预览：

heavengate · 2022-03-28T02:13:05Z

paddle/fluid/operators/detection/nms_op.cc

+    Tensor* output = context.Output<Tensor>("KeepBoxesIdxs");
+    int64_t* output_data = output->mutable_data<int64_t>(context.GetPlace());
+    auto threshold = context.template Attr<float>("iou_threshold");
+    VLOG(3) << "threshold= " << threshold;


这行删掉吧

heavengate · 2022-03-28T02:13:49Z

paddle/fluid/operators/detection/nms_op.cu

+
+static const int64_t threadsPerBlock = sizeof(int64_t) * 8;
+
+__host__ __device__ static inline int64_t CeilDivide(int64_t n, int64_t m) {


"host device" -> HOSTDEVICE

… add_nms_op_and_batched_nms 1. Make current branch updated. 2. Try to solve CI "build Kunlun KP build error"

qingqing01 · 2022-03-29T12:35:50Z

paddle/fluid/operators/detection/nms_op.cc

+             "Boxes is a Tensor with shape [N, 4] "
+             "N is the number of boxes "
+             "in last dimension in format [x1, x2, y1, y2] "
+             "the relation shold be ``0 <= x1 < x2 && 0 <= y1 < y2``.");


shold -> should

qingqing01 · 2022-03-29T12:49:04Z

paddle/fluid/operators/detection/nms_op.cu

+
+HOSTDEVICE static inline int64_t CeilDivide(int64_t n, int64_t m) {
+  return (n + m - 1) / m;
+}


This common function can be put in nms_op.h, then remove this function in .cc

qingqing01 · 2022-03-29T12:52:18Z

paddle/fluid/operators/detection/nms_op.cu

+template <typename T>
+static __device__ inline bool CalculateIoU(const T* const box_1,
+                                           const T* const box_2,
+                                           const float threshold) {


Same as "CeilDivide"

qingqing01 · 2022-03-29T12:59:04Z

python/paddle/tests/test_ops_batched_nms.py

+            else:
+                continue
+
+    return selected_indices


Remove duplicated code, import from other file

qingqing01 · 2022-03-29T13:02:57Z

python/paddle/vision/ops.py

+    return out
+
+
+def batched_nms(boxes, scores, category_idxs, categories, iou_threshold, top_k):


please also add unit testing for Dynamic to Static

added test_batched_nms_dynamic_to_static function in python/paddle/fluid/tests/unittest/test_batched_nms.py

XiaoguangHu01 · 2022-03-30T01:31:34Z

python/paddle/vision/ops.py

+    return out
+
+
+def batched_nms(boxes, scores, category_idxs, categories, iou_threshold, top_k):


nms和batched_nms是否是业界普遍的叫法？
batched_nms和nms这两个api是否能在对外接口层合并？比如，默认category_idxs=None，表示nms的情况

一般api不区分batched和非batched的形式，非batched认为是batched一种特殊形式，比如不会有batched_conv

batched一般理解是输入数据第0维度表示样本的batch，但这里的含义并不相同

当时使用该名称时仅参考了pytorch的设计，经调研batched_nms仅在pytorch中使用。与指导人讨论后认为batched一次确实无法较好概括该API所实现的功能。目前已经将这两个API合并为同一个nms，使用参数进行区分。

… add_nms_op_and_batched_nms merge to modify unittest CMakefile.txt to skip test_ops_nms

TCChenlong

LGTM
TODO：Fix api docs

XiaoguangHu01

LG API

Xreki · 2022-04-04T11:04:10Z

paddle/fluid/operators/detection/nms_op.cu

+    std::vector<uint64_t> mask_host(num_boxes * blocks_per_line);
+    memory::Copy(platform::CPUPlace(), mask_host.data(), context.GetPlace(),
+                 mask_dev, num_boxes * blocks_per_line * sizeof(uint64_t),
+                 context.cuda_device_context().stream());


GPU内容拷回CPU后，需要同步，不然后面用到的mask_host极有可能是脏数据。

已线下沟通下个PR修改

RichardWooSJTU added 3 commits March 25, 2022 12:12

add nms op and batched_nms api

0f7c4bf

modify description of nms op

bd9918f

fix error msg of PADDLE_ENFORCE

7ea82ed

heavengate requested review from wangxinxin08, qingqing01 and heavengate March 25, 2022 13:47

heavengate reviewed Mar 28, 2022

View reviewed changes

RichardWooSJTU added 5 commits March 28, 2022 03:44

delete debug info

b3b48de

modify HOSTDEVICE keyword

8eeef56

accelerate test

8d9a9ba

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

042eb6b

… add_nms_op_and_batched_nms 1. Make current branch updated. 2. Try to solve CI "build Kunlun KP build error"

fix rocm compile error

a17bfb1

heavengate previously approved these changes Mar 29, 2022

View reviewed changes

modify api doc and fix shape bug

918cd3f

RichardWooSJTU dismissed heavengate’s stale review via 918cd3f March 29, 2022 04:41

RichardWooSJTU added 3 commits March 29, 2022 04:55

fix topk error when compile time

264ab25

add api to __all__

a4537c7

fix doc string

085ef8a

RichardWooSJTU requested a review from heavengate March 29, 2022 07:09

RichardWooSJTU added 5 commits March 29, 2022 07:13

fix doc string

407b831

fix doc example and math error

b2e7ed5

fix doc example and math error

7b52f99

fix doc math error

905acf8

fix doc math error

e421ca1

qingqing01 reviewed Mar 29, 2022

View reviewed changes

delete duplicated code

611a872

XiaoguangHu01 reviewed Mar 30, 2022

View reviewed changes

RichardWooSJTU added 2 commits March 30, 2022 10:24

try to fix CI-Windows-Inference memory error

6186d8c

merge nms and batched_nms

9f92dcd

heavengate previously approved these changes Mar 31, 2022

View reviewed changes

fix coverage

3052238

RichardWooSJTU dismissed heavengate’s stale review via 3052238 March 31, 2022 15:53

RichardWooSJTU added 2 commits April 2, 2022 03:26

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

dfb5f20

… add_nms_op_and_batched_nms merge to modify unittest CMakefile.txt to skip test_ops_nms

skip test_ops_nms

615aff7

TCChenlong approved these changes Apr 2, 2022

View reviewed changes

XiaoguangHu01 approved these changes Apr 4, 2022

View reviewed changes

Xreki reviewed Apr 4, 2022

View reviewed changes

heavengate merged commit 7554f42 into PaddlePaddle:develop Apr 5, 2022

RichardWooSJTU deleted the add_nms_op_and_batched_nms branch April 6, 2022 02:55

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add nms op and batched_nms api #40962

Add nms op and batched_nms api #40962

RichardWooSJTU commented Mar 25, 2022 •

edited

Loading

heavengate Mar 28, 2022

RichardWooSJTU Mar 28, 2022

heavengate Mar 28, 2022 •

edited

Loading

RichardWooSJTU Mar 28, 2022

qingqing01 Mar 29, 2022

RichardWooSJTU Mar 29, 2022

qingqing01 Mar 29, 2022

RichardWooSJTU Mar 29, 2022

qingqing01 Mar 29, 2022

RichardWooSJTU Mar 29, 2022

qingqing01 Mar 29, 2022

RichardWooSJTU Mar 29, 2022

qingqing01 Mar 29, 2022

RichardWooSJTU Mar 29, 2022

XiaoguangHu01 Mar 30, 2022 •

edited

Loading

RichardWooSJTU Mar 31, 2022

TCChenlong left a comment

XiaoguangHu01 left a comment

Xreki Apr 4, 2022

RichardWooSJTU Apr 5, 2022


		static const int64_t threadsPerBlock = sizeof(int64_t) * 8;

		__host__ __device__ static inline int64_t CeilDivide(int64_t n, int64_t m) {

		return out


		def batched_nms(boxes, scores, category_idxs, categories, iou_threshold, top_k):

Add nms op and batched_nms api #40962

Add nms op and batched_nms api #40962

Conversation

RichardWooSJTU commented Mar 25, 2022 • edited Loading

PR types

PR changes

Describe

Choose a reason for hiding this comment

Choose a reason for hiding this comment

heavengate Mar 28, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

XiaoguangHu01 Mar 30, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TCChenlong left a comment

Choose a reason for hiding this comment

XiaoguangHu01 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

RichardWooSJTU commented Mar 25, 2022 •

edited

Loading

heavengate Mar 28, 2022 •

edited

Loading

XiaoguangHu01 Mar 30, 2022 •

edited

Loading