getConfidenceBySymbol crashes for some images

[kevincon]

Using getConfidenceBySymbol to print out the confidence values for each character recognized works for the image_sample.jpg file included in the Template Framework Project, but try swapping it with the following image and it crashes every time:

(I know this isn't an ideal image for Tesseract, but it still shouldn't cause it to crash)

I really need to be able to use the symbol confidence values, so I really appreciate any ideas this repo's maintainers have about how to fix this issue.

Here's the code I am using to print out the confidence values:

// needed for confidence by symbol to work correctly
// see http://stackoverflow.com/questions/17393555/character-confidence-for-tesseract-3-02-using-config-file
[tesseract setVariableValue:@"T" forKey:@"save_blob_choices"];

... // setting image, calling recognize, etc.

NSArray *confidences = [tesseract getConfidenceBySymbol];
for (NSDictionary *confidence in confidences) {
    NSLog(@"%@: %@", confidence[@"text"], confidence[@"confidence"]);
}

Here's the commit I added to my fork on top of HEAD to produce this crash, if you guys want to check it out for yourselves: https://github.com/kevincon/Tesseract-OCR-iOS/commit/4bcf1ac4d2c49872381aadb867e2ce6878edd8d6

Also, +@ko since he added getConfidenceBySymbol in a pull request and might have some ideas about why it crashes for some images.

[ko]

I was able to reproduce it after adding:

[tesseract setVariableValue:@"T" forKey:@"save_blob_choices"];

The "save_blob_choices" appears to be (I haven't yet looked beyond a cursory glance) related to using the ChoiceIterator. The implemented solution for the getConfidenceBySymbol, however, only relies on the ResultIterator.

I understand that setting the "save_blob_choices" to "T" shouldn't cause the ResultIterator--or anything else, for that matter--to crash; as a workaround, if what you're looking for is strictly the confidence values of the recognized characters, have you tried without this parameter setting?

For what it's worth, a c++ implementation was able to run without error on this image using tesseract 3.03. I haven't been able to test the results of the same program against version 3.02.02 (which is what Tesseract-OCR-iOS tries to build). That would at least help in determining where the issue lies.

[kevincon]

Thanks for your response, @ko.

If I don't include the parameter "save_blob_choices" set to True, then the confidences returned by getConfidenceBySymbol for each character are the same for characters that are part of the same word recognized by Tesseract.

For example, here's the confidences returned by running the Template Framework Project (HEAD commit) on the included image_sample.jpg WITHOUT "save_blob_choices" set to True:

2014-10-07 22:06:22.495 Template Framework Project[22687:1592487] 1: 85.8671
2014-10-07 22:06:22.495 Template Framework Project[22687:1592487] 2: 85.8671
2014-10-07 22:06:22.495 Template Framework Project[22687:1592487] 3: 85.8671
2014-10-07 22:06:22.496 Template Framework Project[22687:1592487] 4: 85.8671
2014-10-07 22:06:22.496 Template Framework Project[22687:1592487] 5: 85.8671
2014-10-07 22:06:22.496 Template Framework Project[22687:1592487] 6: 85.8671
2014-10-07 22:06:22.496 Template Framework Project[22687:1592487] 7: 85.8671
2014-10-07 22:06:22.496 Template Framework Project[22687:1592487] 8: 85.8671
2014-10-07 22:06:22.496 Template Framework Project[22687:1592487] 9: 85.8671
2014-10-07 22:06:22.496 Template Framework Project[22687:1592487] 0: 85.8671

Now here are the confidences returned for image_sample.jpg WITH "save_blob_choices" set to True:

2014-10-07 22:09:38.120 Template Framework Project[22999:1595961] 1: 86.15938
2014-10-07 22:09:38.120 Template Framework Project[22999:1595961] 2: 93.0193
2014-10-07 22:09:38.120 Template Framework Project[22999:1595961] 3: 90.94276
2014-10-07 22:09:38.120 Template Framework Project[22999:1595961] 4: 85.8671
2014-10-07 22:09:38.120 Template Framework Project[22999:1595961] 5: 90.58944
2014-10-07 22:09:38.121 Template Framework Project[22999:1595961] 6: 91.65635
2014-10-07 22:09:38.121 Template Framework Project[22999:1595961] 7: 93.1501
2014-10-07 22:09:38.121 Template Framework Project[22999:1595961] 8: 91.92635
2014-10-07 22:09:38.121 Template Framework Project[22999:1595961] 9: 88.0007
2014-10-07 22:09:38.121 Template Framework Project[22999:1595961] 0: 89.97475

For my particular application, there isn't really a concept of "words", as I am just trying to OCR random character strings, so the word confidence values aren't useful to me. :(

Could you try running the C++ program you mentioned under the version of Tesseract that this iOS library uses (3.02.02) to try to find a better error message than Xcode's BAD ACCESS? Thanks so much for your help!

[ko]

Ah that is interesting; didn't expect "save_blob_choices" to have that effect.

So, I tested tesseract 3.02.02 with leptonica 1.69 with installations done via homebrew. It appears to have worked without error.

That has fingers pointing to something on the iOS side. Started taking things out and found that removing the blackAndWhite call appears to have stopped the crashing with the sample image provided.

    //[tesseract setImage:[img blackAndWhite]]; //image to check
    [tesseract setImage:img];

My results:

2014-10-09 22:03:30.317 Template Framework Project[38020:1338153] T: 82.04062
2014-10-09 22:03:30.317 Template Framework Project[38020:1338153] R: 81.41316
2014-10-09 22:03:30.317 Template Framework Project[38020:1338153] L: 87.02643
2014-10-09 22:03:30.317 Template Framework Project[38020:1338153] K: 73.63049
2014-10-09 22:03:30.317 Template Framework Project[38020:1338153] P: 68.1906
2014-10-09 22:03:30.317 Template Framework Project[38020:1338153] S: 86.40933
2014-10-09 22:03:30.317 Template Framework Project[38020:1338153] 9: 86.24488
2014-10-09 22:03:30.318 Template Framework Project[38020:1338153] 5: 83.34719
2014-10-09 22:03:30.318 Template Framework Project[38020:1338153] 0: 75.72008
2014-10-09 22:03:30.318 Template Framework Project[38020:1338153] 9: 86.83403
2014-10-09 22:03:30.318 Template Framework Project[38020:1338153] TRLKPS 9
5 0 9

Yeah... it's yet another workaround.

Haven't been able to dig into the "blackAndWhite" or "grayScale" function yet, though, defined in "UIImage+Filters.m".

[kevincon]

That's interesting that blackAndWhite might have contributed to the crashing issue. But recently I've switched to using the GPUImage library to perform OCR on real-time video frames, and I was still experiencing the crashing there. :(

The good news is that I was able to resolve my issue by updating the Tesseract and Leptonica libs for this project to 3.03RC and 1.70, respectively. Now getConfidenceBySymbol works for all images, and it doesn't need "save_blob_choices" set to T to return the correct confidence values for each symbol.

I've submitted this pull request so hopefully it will be useful for others. Thanks again for your help, @ko!

[ko]

Nice!

Glad that moving the libraries up to 3.03RC/1.70 worked out. Although I do find it odd that 3.02.02/1.69 would fail to handle the resulting UIImage from blackAndWhite (and grayScale); I suppose with over a year between 3.02.02 and 3.03RC, there are bound to be some bug and stability fixes.