feature: add hallucination detection to server

[densumesh]

Please indicate what issue this PR is related to and @ any maintainers who are relevant

[cdxker]

Dockerfile doesn't seem to work, I think we need to include ort into the cargo.toml or download libonnxruntime.so some other way

[skeptrunedev]

• fix image build
• show null instead of 0 where hallucination detection has not run

[skeptrunedev]

This ran fine on the big office computer, but was really slow on my laptop. I'm somewhat concerned about putting this into prod because I think it will necessitate putting the server on a GPU node.

I feel relatively strongly that we need to run the models separately outside of the server.

[skeptrunedev]

I think the change which got made to old messages w/out hallucination detection didn't appear on the table is also messing up actual hallucinations. Either that or the NER change zeroed out everything?

Are we just not displaying 0's or do we have a smarter system for not displaying nulls?

[skeptrunedev]

display detected hallucinations as non-zero on the table

[skeptrunedev]

I also wonder if we should add something like "low confidence queries" and "likely hallucinations" in this PR.

Feels necessary to be able to sort in descending order by both top score and hallucination score.

[skeptrunedev]

Should be high prio on the ordered queries, but this is good!

stale