add PPO and stack_llama support

[sywangyi]

What does this PR do?

Fixes # (issue)

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you make sure to update the documentation with your changes?
• Did you write any new necessary tests?

[sywangyi]

depend on #612 and #507

[sywangyi]

@libinta

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[libinta]

can you also consider to add test case in test folder for the stack_llama/stack_llama_2 for better regression detection? You can have a short run for perf/acc check.

[sywangyi]

@mandy-li , finetune reward model (step 2) and then apply RLHF to the actor model and reward model (step 3) is all here

[regisss]

@sywangyi Could you remove the stack_llama folder and move everything in examples/trl similarly to #635 please?

[sywangyi]

yes, I will refactor the PR

[sywangyi]

https://habana.atlassian.net/servicedesk/customer/portal/1/HS-1425 is filed for PPO training graph. @libinta

[sywangyi]

Thanks for the detailed review @regisss , update PR based on the review comments.

[regisss]

@sywangyi Can you add evaluate and scikit-learn to requirements.txt? It is needed to run reward_modeling.py.

[regisss]

LGTM!