Fix the sampler truncation strategy

[chenmoneygithub]

resolve #712

[mattdangerw]

Left a few comments. Overall the thing that is still striking me about this design is how special cased the ragged code for doing the pre and post processing is getting (even in token id space). Seems like we have a few phases for generation...

1. tokenize inputs
2. mask and pack inputs
3. sample
4. truncate and manipulate outputs
5. detokenize

It seems unlikely that we will ever do 2) and 4) in a way that is perfectly general, so it's unfortunate that 2), 3), and 4) all come together. It does not feel very modular.

Not something we need to solve on this PR, but let's keep thinking about that.

[chenmoneygithub]

@mattdangerw Yea this sampler implementation now contains a lot of things, including preprocessing, sampling algo and postprocessing. It's a bit hard to figure out the best user flow with the class itself, we could keep thinking about it while we are rolling out the support for GPT2 text generation, including cache, long sentence generation and so on.

[mattdangerw]

Thanks!