add support for TinyLlama model

[tjs-intel]

⚠️ Do not merge this PR before Habana DeepSpeed 1.15 is released ⚠️

What does this PR do?

The TinyLlama model only has checkpoints in the form of model.safetensors. This checkpoint needs to be included in the list of checkpoints that is passed to DeepSpeed in order for the model to function properly when initialized with DeepSpeed.

This PR adds the safetensor checkpoints to the list of checkpoints passed to DeepSpeed.

Note: This change requires an upstream commit in microsoft/DeepSpeed to be merged downstream to HabanaAI/DeepSpeed in order for DeepSpeed to support the provided safetensor format.

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you make sure to update the documentation with your changes?
• Did you write any new necessary tests?

[regisss]

@mandy-li Do you know if this commit will be part of Habana DeepSpeed 1.15?

[mandy-li]

@mandy-li Do you know if this commit will be part of Habana DeepSpeed 1.15?

@regisss , @tjs-intel , DS changes should be submitted to internal habana deepspeed-fork repo

[regisss]

@mandy-li Do you know if this commit will be part of Habana DeepSpeed 1.15?

@regisss , @tjs-intel , DS changes should be submitted to internal habana deepspeed-fork repo

deepspeedai/DeepSpeed@c8c57b8 is part of the v0.13 release of DeepSpeed. I guess it should be part of Habana DeepSpeed 1.15 or 1.16 no?

[mandy-li]

@mandy-li Do you know if this commit will be part of Habana DeepSpeed 1.15?

@regisss , @tjs-intel , DS changes should be submitted to internal habana deepspeed-fork repo

microsoft/DeepSpeed@c8c57b8 is part of the v0.13 release of DeepSpeed. I guess it should be part of Habana DeepSpeed 1.15 or 1.16 no?

Yes, will be in 1.15, but we modified it in the internal Habana DS repo.
@tjs-intel , this PR should go to habana oh-fork to test internally.

[tjs-intel]

OH-fork PR here HabanaAI#25

[tjs-intel]

HabanaAI#25 has been merged

[regisss]

HabanaAI#25 has been merged

So we need to wait for 1.15 to be merged to merge this PR right?

[tjs-intel]

@regisss I will leave that discussion to the maintainers.

The consequence of merging this before the HabanaAI/DeepSpeed 1.15 release is that the latest released version of HabanaAI/DeepSpeed (1.14) does not support safetensors. If there are any models with both .bin and .safetensors checkpoints, this change will prefer the .safetensors checkpoint and send that to DeepSpeed instead of the .bin checkpoint. This will cause DeepSpeed 1.14 to raise an exception since the .safetensors checkpoint cannot be unpickled.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[regisss]

Thanks for the clear explanation @tjs-intel! In that case, we'll wait for the release of SynapseAI v1.15 to merge this PR.
Changes look good to me, could you also run the following to make the code style check pass please?

pip install -U ruff
make style

[regisss]

Merging in the synapse_1.15 branch