Adding orpo training - #1210

Closed

Goekdeniz-Guelmez wants to merge 47 commits into

ml-explore:mainfrom

Goekdeniz-Guelmez:adding-orpo-training

Commits on Jan 18, 2025

initial commit
Goekdeniz-Guelmez
committed
1ff7888
fixing reference model loading and freezing
Goekdeniz-Guelmez
committed
582f979
update LORA.md
Goekdeniz-Guelmez
committed
1b4e196
update lora_config.yaml
Goekdeniz-Guelmez
committed
06a9f5d

Commits on Jan 19, 2025

update ACKNOWLEDGMENTS.md
Goekdeniz-Guelmez
committed
040f7c3
nits
Goekdeniz-Guelmez
committed
51fd621
initial commit
Goekdeniz-Guelmez
committed
a9b7609
remerge with dpo
Goekdeniz-Guelmez
committed
7d279b5
finish
Goekdeniz-Guelmez
committed
fa80d08
nits
Goekdeniz-Guelmez
committed
9ede9db
nits
Goekdeniz-Guelmez
committed
424cb85
update ACKNOWLEDGMENTS.md
Goekdeniz-Guelmez
committed
2a5b315
update
Goekdeniz-Guelmez
committed
ea0d11c
fixes
Goekdeniz-Guelmez
committed
363bde6
nits
Goekdeniz-Guelmez
committed
61cd253

Commits on Jan 22, 2025

Merge branch 'ml-explore:main' into adding-orpo-training
Goekdeniz-Guelmez
authored
4098c3b
niits
Goekdeniz-Guelmez
committed
0bb0011

Commits on Jan 24, 2025

removing dpo and fixing some stuff for orpo
Goekdeniz-Guelmez
committed
e368829
updates
Goekdeniz-Guelmez
committed
09ed837

Commits on Jan 25, 2025

Removed rejected_rewards handling, Updated batch unpacking to match iterator, Updated batch unpacking to match iterator, Added preference score scaling, Simplified reward calculation, Removed redun…
Goekdeniz-Guelmez
committed
d8e7834

Commits on Jan 26, 2025

clean up
Goekdeniz-Guelmez
committed
2f2ddd4
Merge branch 'main' into adding-orpo-training
Goekdeniz-Guelmez
authored
294d189
fix ACKNOWLEDGMENTS
Goekdeniz-Guelmez
committed
649d3f8

Commits on Jan 29, 2025

Merge branch 'ml-explore:main' into adding-orpo-training
Goekdeniz-Guelmez
authored
ceccb4c

Commits on Jan 31, 2025

cleaning up
Goekdeniz-Guelmez
committed
541677a

Commits on Feb 3, 2025

Merge branch 'ml-explore:main' into adding-orpo-training
Goekdeniz-Guelmez
authored
2c96da5

Commits on Feb 4, 2025

Commits on Feb 6, 2025

Commits on Feb 10, 2025

Merge branch 'main' into adding-orpo-training
Goekdeniz-Guelmez
authored
575ece6

Commits on Feb 12, 2025

Merge branch 'ml-explore:main' into adding-orpo-training
Goekdeniz-Guelmez
authored
80c64da

Commits on Feb 18, 2025

Merge branch 'ml-explore:main' into adding-orpo-training
Goekdeniz-Guelmez
authored
348f728

Commits on Feb 19, 2025

Merge branch 'ml-explore:main' into adding-orpo-training
Goekdeniz-Guelmez
authored
65d777b

Commits on Feb 21, 2025

Merge branch 'ml-explore:main' into adding-orpo-training
Goekdeniz-Guelmez
authored
de14718
better dataset handling
Goekdeniz-Guelmez
committed
5704136

Commits on Feb 22, 2025

Merge branch 'ml-explore:main' into adding-orpo-training
Goekdeniz-Guelmez
authored
2d4895d

Commits on Feb 25, 2025

Merge branch 'ml-explore:main' into adding-orpo-training
Goekdeniz-Guelmez
authored
a1c2ac2

Commits on Feb 28, 2025

Merge branch 'main' into adding-orpo-training
Goekdeniz-Guelmez
authored
6a3912b

Commits on Mar 1, 2025

updates
Goekdeniz-Guelmez
committed
bb261aa

Commits on Mar 3, 2025

Merge branch 'ml-explore:main' into adding-orpo-training
Goekdeniz-Guelmez
authored
3023ae0

Commits on Mar 8, 2025

Merge branch 'main' into adding-orpo-training
Goekdeniz-Guelmez
authored
700c3ef
fix optimizer
Goekdeniz-Guelmez
committed
73cc094

Commits on Mar 12, 2025

Merge branch 'ml-explore:main' into adding-orpo-training
Goekdeniz-Guelmez
authored
64ba244

Commits on Mar 14, 2025

removings
Goekdeniz-Guelmez
committed
1de4c86