SparsePerceptron?

[kordjamshidi]

It seems this is not working? Could someone try this? I get this error even when I have only true/false binary label:
Error: : An LTU must be given a single binary label classifier.

[kordjamshidi]

You can test it in here or make your own test.

[kordjamshidi]

@danyaljj : could you also have a look at this? the changes in the history are not clear due to the name changes, I can not see from which point this error has been introduced. Might be related to setLabeler and set extractor.

[danyaljj]

The error is coming from here:
https://github.com/IllinoisCogComp/lbjava/blob/434cf0a40e4f2ae08c96d3ae1b96f319eb531d67/lbjava/src/main/java/edu/illinois/cs/cogcomp/lbjava/learn/LinearThresholdUnit.java#L320-L322

Did you print the value of allowableValues for your labeler? i.e.

println(bClassifier.classifier.labeler.allowableValues())

(or sth like that)

[kordjamshidi]

It seems they change in the second iteration. But maybe you know what has changed in here because our ER example was using SparseAveragedPerceptron and there was not a problem with it.

[bhargav]

I looked at the ER examples and it seems that previously we were using SparsePerceptron or SparseNetworkLearner with a SparseAveragePerceptron as the base unit.

@danyaljj The default value of allowableValues is a new String[0] and it is not properly overridden in current DiscreteProperty class implementation.

[kordjamshidi]

I see, thanks for tracking @bhargav. Just to mention that it gives the same error with SparsePerceptron too.

[kordjamshidi]

@danyaljj @bhargav I need help for this.

[danyaljj]

On a slightly different note, why are we using SparsePerceptron directly? (instead of using it inside a SparseNetworkLearner)

[kordjamshidi]

Training a binary classifier is pure and more straightforward. Multiclass classification itself includes deciding about the policies for how to use the outcome of each classifier and it can be done in different ways. If we want to have control over that we better to use the binary units directly. I need help to solve this issue before answering any other questions :-P!

[danyaljj]

I'm not sure if I can buy this argument. In fact it's Dan's opinion as well, to always use the SparseNetworkLearner. If we have a feature for multi-class case, we will have it for binary case for free.

[kordjamshidi]

Ok, I am working on this, in addition to my point mentioned above, for me at least, I see some complexity in using SparseNetworks. Can you sketch the algorithm that it uses for updating the weights of each LTU when seeing each example? Can you test a SparseNetwork in Lbjava before training it, does it throw NPE there as well? I see the initialization has issues, while this is not the case for SparsePerceptron. I am testing the joinTraining, while it works so clean and nice with SparsePerceptron, I am not sure what is happening when I use the SparseNetwork. So, if we want to use that as the base, the Lbjava's implementation needs to be investigated and probably be improved.

[kordjamshidi]

Due to the caching problems I could not use my favorite RandomDataApp, so I found it easier to use the Badge example as a test case, so I added it to my branch. If you run this, it works perfectly with JoinTrain without any pre-training but not with JoinTrainSparseNetwork.
https://github.com/kordjamshidi/saul/tree/SL-Integration/saul-examples/src/main/scala/edu/illinois/cs/cogcomp/saulexamples/Badge

[danr-ccg]

I see that there is some discussion of this issue. We had some problems with this last year, since people were using it directly, which isn’t right.
I think that this needs to be documented better. Where is this documented? I couldn’t find it.
Thanks,
Dan

From: Daniel Khashabi [mailto:[email protected]]
Sent: Saturday, October 29, 2016 3:57 AM
To: IllinoisCogComp/saul [email protected]
Subject: Re: [IllinoisCogComp/saul] SparsePerceptron? (#441)

On a slightly different note, why are we using SparsePerceptron directly? (instead of using it inside a SparseNetworkLearner)

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_IllinoisCogComp_saul_issues_441-23issuecomment-2D257080184&d=DQMCaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=EoEIPOcXriNKNTRQxPa5uv7hPypxUE6tE_RlGOluh-c&m=hOPOJEgrvXOUoi9-vSabY86ZTJZM55eMXbrWS9CHexA&s=PprSVGjsr0fqeUy5PjLHrQsNf5_QZ_MZ_bC-eAjd-y0&e=, or mute the threadhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_APLC6rEEYQw-2DfQ94zJJHAglReuELgdGEks5q4wpHgaJpZM4KdxFA&d=DQMCaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=EoEIPOcXriNKNTRQxPa5uv7hPypxUE6tE_RlGOluh-c&m=hOPOJEgrvXOUoi9-vSabY86ZTJZM55eMXbrWS9CHexA&s=Jvnqqo40SBkzEv3xWQiRw0CbM_yvOmI-crcyhLOmZHY&e=.

[kordjamshidi]

@danr-ccg : Do you mean you have had problems with using SparsePerceptron? and are you asking for documentation for the LBJava implementation of SparsePerceptron? To my experience, it is the simplest implemented learning unit and we can build complex models based on that including various paradigms fo multi-class. However, I am not against using the SparseNetworks for the cases that we don't care about the way the multi-class decisions are made.

[danr-ccg]

The problem was in the way it was used. My recollection is that it must be used via SparseNetworks, even in the binary case, but the documentation and examples did not reflect that.

          Dan

From: Parisa Kordjamshidi [mailto:[email protected]]
Sent: Sunday, November 6, 2016 1:27 PM
To: IllinoisCogComp/saul [email protected]
Cc: Roth, Dan [email protected]; Mention [email protected]
Subject: Re: [IllinoisCogComp/saul] SparsePerceptron? (#441)

@danr-ccghttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_danr-2Dccg&d=DQMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=EoEIPOcXriNKNTRQxPa5uv7hPypxUE6tE_RlGOluh-c&m=OPZ0JYFNrCZkayhu-di_9nAP3IJGU0TjJ1Ofwz8DSjI&s=yLCv-Dv65lUxO64qHmQzc878cwvORhscHFiSeNX67tw&e= : Do you mean you have had problems with using SparsePerceptron? and are you asking for documentation for the LBJava implementation of SparsePerceptron? To my experience, it is the simplest implemented learning unit and we can build complex models based on that including various paradigms fo multi-class. However, I am not against using the SparseNetworks for the cases that we don't care about the way the multi-class decisions are made.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_IllinoisCogComp_saul_issues_441-23issuecomment-2D258703901&d=DQMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=EoEIPOcXriNKNTRQxPa5uv7hPypxUE6tE_RlGOluh-c&m=OPZ0JYFNrCZkayhu-di_9nAP3IJGU0TjJ1Ofwz8DSjI&s=Wx8u4Y6QqEbUInzX9rmu_mfuVbeeacLH0fa_yQYHvrk&e=, or mute the threadhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_APLC6pHswSZ59cV-5FiBrpYntlLeXbwb3fks5q7ioegaJpZM4KdxFA&d=DQMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=EoEIPOcXriNKNTRQxPa5uv7hPypxUE6tE_RlGOluh-c&m=OPZ0JYFNrCZkayhu-di_9nAP3IJGU0TjJ1Ofwz8DSjI&s=BCwqzSeMPMzz8H3CPA_E316FHkSgqWKaPbTqc2QXbys&e=.

[danyaljj]

Dan,
To your question on documentation of this issue: it is included in the lbjava documentations, merged about a year ago:
https://github.com/IllinoisCogComp/lbjava/blob/master/lbjava/doc/ALGORITHMS.md#note-on-binary--multiclass-classification

Daniel