Skip to content

Pnp evaluator#5108

Merged
zhouxiao-coder merged 6 commits intoPaddlePaddle:developfrom
zhouxiao-coder:pnp-evaluator
Nov 6, 2017
Merged

Pnp evaluator#5108
zhouxiao-coder merged 6 commits intoPaddlePaddle:developfrom
zhouxiao-coder:pnp-evaluator

Conversation

@zhouxiao-coder
Copy link
Contributor

@zhouxiao-coder zhouxiao-coder commented Oct 26, 2017

This evaluator counts positive pairs, negative pairs in the ranking result.
Tested on the 1-dimension score case.
resolve #5086

@zhouxiao-coder zhouxiao-coder changed the title [WIP]Pnp evaluator Oct 31, 2017
Copy link
Contributor

@wanghaoshuang wanghaoshuang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

没啥大问题,提了一点小建议。

std::make_pair(query[i], std::vector<PredictionResult>()));
}
predictions[query[i]].push_back(PredictionResult(
score[i * width + column], label[i], has_weight ? weight[i] : 1.0));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这里是不是也可以用emplace?

predictions[query[i]].emplace_back(score[i * width + column], label[i], has_weight ? weight[i] : 1.0);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

auto query = query_t->data<int32_t>();
const T* weight = nullptr;
auto has_weight = weight_t != nullptr;
if (has_weight) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not if (weight_t != nullptr) and remove line 52 ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.


auto score = score_t->data<T>();
auto label = label_t->data<T>();
auto query = query_t->data<int32_t>();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

我们要不要考虑query id有可能不是数字的情况?
即使我们只支持数字的话,这里用size_t是不是更合适?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. 我似乎没看到我们有支持string类型tensor的design doc,所以我假定这里是原来的实现一样,只支持整数的;而且由于这里事实上是ID,所以用数字也是比较符合直觉的。之后如果有其它需要我们可以再扩展。
  2. 我参考了下其它要用到int类型input的op(lookup_table_op, top_k_op), 这里用int32_t确实和它们用int64_t不一致,所以我把这里改为int64_t了。size_t是unsigned,和这里语义有区别。
# group by query id
predictions = {}
batch_size = label.shape[0]
print "batch_size=", batch_size
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove debug code?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

self.check_output()


class TestPositiveNegativePairOpAccumulateWeight(OpTest):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TestPositiveNegativePairOpAccumulate TestPositiveNegativePairOpAccumulateWeight 有比较多的重复code, 可以考虑合并下。

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Copy link
Contributor

@wanghaoshuang wanghaoshuang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@zhouxiao-coder zhouxiao-coder merged commit c8f1389 into PaddlePaddle:develop Nov 6, 2017
@zhouxiao-coder zhouxiao-coder deleted the pnp-evaluator branch November 6, 2017 13:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

3 participants