Another implementation of CRF #4646
Conversation
|
would you please add unit test and an example? New users would benefit with a good use case to know this layer exists. It would also help to know how to properly apply it. |
|
@EderSantana yup, will do it soon. I made this just because there is another PR (#4621) for CRF, and I think it may be valuable to compare. |
linxihui
force-pushed
the
implement-crf
branch
5 times, most recently
from
80c37b4 to
e640e12
Compare
|
@EderSantana @fchollet Sorry for the delay. Unit tests and an example is added. The check fail because of some other tests |
|
the failure is only in theano under python 2.7 and unrelated to this current PR |
linxihui
force-pushed
the
implement-crf
branch
from
e640e12 to
c460e3b
Compare
linxihui
force-pushed
the
implement-crf
branch
from
c460e3b to
19b13d2
Compare
|
@EderSantana it's weird. Now it passes tests. Though there is a seed setting in |
|
|
||
| # use learn_mode = 'join', test_mode = 'viterbi' | ||
| crf = CRF(10) | ||
| model.add(crf(Embed)) |
There was a problem hiding this comment.
hi @linxihui, 'Embed' seems to be undefined.
I think this line should be model.add(crf) (followed by test_crf.py#L34)
There was a problem hiding this comment.
@himkt You're right. Thanks for pointing out. Now fixed.
linxihui
force-pushed
the
implement-crf
branch
from
ad94185 to
052c146
Compare
|
Closing outdated PR. If you still care about the content of the PR, please submit a new PR to |
|
@linxihui Is there any plan about this PR in future ? I think this PR is great work (I wanna use CRF!). I want to help if there are something I can support. |
This implementation has two modes for optimization:
1. (
join mode) optimized by maximizing join likelihood, which is optimal in theory of statistics.Note that in this case, CRF must be the output/last layer.
2. (
marginal mode) return marginal probabilities on each time step and optimized via compositelikelihood (product of marginal likelihood), i.e., using
categorical_crossentropyloss.Note that in this case, CRF can be either the last/output layer or an intermediate layer (not clear now whether there is a valuable application).
For prediction (test phrase), one can choose either Viterbi best path (class indices) or marginal
probabilities if probabilities are needed.
This implementation also supports masking, specifying activation functions (or more accurate, restrict range of chain energy) and accepts either onehot or sparse target.
NOTE: this PR requires PR #4899