[WINDOWS] Fix windows build (#1602)

tqchen · web-flow · commit feef76eea16c · 2024-01-14T11:08:45.000-08:00
This PR fixes the windows build.
diff --git a/cpp/tokenizers.cc b/cpp/tokenizers.cc
@@ -35,7 +35,7 @@ std::string TokenizerObj::Decode(const std::vector<int32_t>& token_ids) const {
 }
 
 Tokenizer Tokenizer::FromPath(const String& _path) {
-  std::filesystem::path path(_path);
+  std::filesystem::path path(_path.operator std::string());
   std::filesystem::path sentencepiece;
   std::filesystem::path huggingface;
   std::filesystem::path rwkvworld;

Original file line number	Diff line number	Diff line change
`@@ -35,7 +35,7 @@ std::string TokenizerObj::Decode(const std::vector<int32_t>& token_ids) const {`
`35`	`35`	`}`
`36`	`36`
`37`	`37`	`Tokenizer Tokenizer::FromPath(const String& _path) {`
`38`		`- std::filesystem::path path(_path);`
	`38`	`+ std::filesystem::path path(_path.operator std::string());`
`39`	`39`	`std::filesystem::path sentencepiece;`
`40`	`40`	`std::filesystem::path huggingface;`
`41`	`41`	`std::filesystem::path rwkvworld;`