Skip to content

Support auto translate to english #821

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 16 commits into from
Dec 31, 2024
Merged
Show file tree
Hide file tree
Changes from 8 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 35 additions & 1 deletion .github/workflows/docs-translate.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ on:
- 'docs/**'

jobs:
translate:
translate-zh:
runs-on: ubuntu-latest
steps:
- name: Checkout
Expand Down Expand Up @@ -48,3 +48,37 @@ jobs:
uses: stefanzweifel/git-auto-commit-action@v4
with:
commit_message: Update docs and translate

translate-en:
needs: translate-zh
runs-on: ubuntu-latest
env:
DEEPSEEK_API_KEY: ${{ secrets.DEEPSEEK_API_KEY }}
steps:
- name: Checkout
uses: actions/checkout@v4

- uses: pnpm/action-setup@v4
name: Install pnpm
with:
version: 9
run_install: false

- name: Install Node.js
uses: actions/setup-node@v4
with:
node-version: 20
cache: 'pnpm'

- name: Install dependencies
run: pnpm install

- name: Start Translate
run: |
cp docs/index.md docs/en/index.md
pnpm run docs:translate

- name: Commit Updated
uses: stefanzweifel/git-auto-commit-action@v4
with:
commit_message: Update docs and translate
52 changes: 52 additions & 0 deletions bin/doc-translate.github-model.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
import OpenAI from "openai";
import { promises as fs } from 'fs';
import path from 'path';

const token = process.env["GITHUB_TOKEN"];
const endpoint = "https://models.inference.ai.azure.com";
const modelName = "o1-mini";

async function translateFiles() {
const client = new OpenAI({
baseURL: endpoint,
apiKey: token,
dangerouslyAllowBrowser: true
});

const docsPath = path.join(process.cwd(), 'docs/zh-cn');

async function translateFile(filePath) {
const content = await fs.readFile(filePath, 'utf8');
const response = await client.chat.completions.create({
messages: [
{ role: "user", content: "You are a professional translator. Translate the following Markdown content from Chinese to English. Preserve all Markdown formatting." },
{ role: "user", content: content }
],
model: modelName
});

const translatedContent = response.choices[0].message.content;
const englishPath = filePath.replace('/zh-cn/', '/en/');
await fs.mkdir(path.dirname(englishPath), { recursive: true });
await fs.writeFile(englishPath, translatedContent);
console.log(`Translated: ${filePath} -> ${englishPath}`);
}

async function processDirectory(dirPath) {
const files = await fs.readdir(dirPath, { withFileTypes: true });

for (const file of files) {
const fullPath = path.join(dirPath, file.name);

if (file.isDirectory()) {
await processDirectory(fullPath);
} else if (file.name.endsWith('.md')) {
await translateFile(fullPath);
}
}
}

await processDirectory(docsPath);
}

translateFiles().catch(console.error);
73 changes: 35 additions & 38 deletions bin/doc-translate.js
Original file line number Diff line number Diff line change
@@ -1,52 +1,49 @@
import OpenAI from "openai";
import { promises as fs } from 'fs';
import { readdir, readFile, writeFile, mkdir } from 'fs/promises';
import path from 'path';

const token = process.env["GITHUB_TOKEN"];
const endpoint = "https://models.inference.ai.azure.com";
const modelName = "o1-mini";
const endpoint = "https://api.deepseek.com";
const token = process.env["DEEPSEEK_API_KEY"];

async function translateFiles() {
const client = new OpenAI({
const openai = new OpenAI({
baseURL: endpoint,
apiKey: token,
dangerouslyAllowBrowser: true
});

const docsPath = path.join(process.cwd(), 'docs/zh-cn');

async function translateFile(filePath) {
const content = await fs.readFile(filePath, 'utf8');
const response = await client.chat.completions.create({
messages: [
{ role: "user", content: "You are a professional translator. Translate the following Markdown content from Chinese to English. Preserve all Markdown formatting." },
{ role: "user", content: content }
],
model: modelName
});

async function translateContent(content) {
const completion = await openai.chat.completions.create({
messages: [
{ role: "system", content: "You are a professional translator. Translate the following Chinese markdown content to English. Keep all markdown formatting intact." },
{ role: "user", content: content }
],
model: "deepseek-chat",
});
return completion.choices[0].message.content;
}

const translatedContent = response.choices[0].message.content;
const englishPath = filePath.replace('/zh-cn/', '/en/');
await fs.mkdir(path.dirname(englishPath), { recursive: true });
await fs.writeFile(englishPath, translatedContent);
console.log(`Translated: ${filePath} -> ${englishPath}`);
}
async function translateFiles(srcDir, destDir) {
try {
const files = await readdir(srcDir, { recursive: true });

async function processDirectory(dirPath) {
const files = await fs.readdir(dirPath, { withFileTypes: true });
for (const file of files) {
if (!file.endsWith('.md')) continue;

for (const file of files) {
const fullPath = path.join(dirPath, file.name);
const srcPath = path.join(srcDir, file);
const destPath = path.join(destDir, file);
const destFolder = path.dirname(destPath);

if (file.isDirectory()) {
await processDirectory(fullPath);
} else if (file.name.endsWith('.md')) {
await translateFile(fullPath);
}
}
}
await mkdir(destFolder, { recursive: true });

await processDirectory(docsPath);
const content = await readFile(srcPath, 'utf8');
const translatedContent = await translateContent(content);
const finalContent = translatedContent.replace(/\/zh-cn\//g, '/en/');
await writeFile(destPath, finalContent);

console.log(`Translated: ${file}`);
}
} catch (error) {
console.error('Translation error:', error);
}
}

translateFiles().catch(console.error);
translateFiles('docs/zh-cn', 'docs/en');
2 changes: 1 addition & 1 deletion docs/zh-cn/components/cache.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,4 +62,4 @@ di(CacheManager::class)->store('co')->remember($key, $ttl=60, function() {

## 参考

Likes [Laravel-Cache](https://laravel.com/docs/8.x/cache)
参阅 [Laravel-Cache](https://laravel.com/docs/8.x/cache)
2 changes: 1 addition & 1 deletion docs/zh-cn/components/encryption.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ php bin/hyperf.php vendor:publish friendsofhyperf/encryption

## 使用

```shell
```php
$encryptString = encrypt($string);
$decryptString = decrypt($encryptString);
```
2 changes: 1 addition & 1 deletion docs/zh-cn/components/fast-paginate.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ select * from contacts -- The full data that you want to show your
)
```

> 运行上述查询时,您可能会遇到错误!例如 `This version of MySQL doesn't yet support 'LIMIT & IN/ALL/ANY/SOME subquery.`
> 运行上述查询时,您可能会遇到错误!例如 `This version of MySQL doesn't yet support 'LIMIT & IN/ALL/ANY/SOME subquery.`
> 在这个包中,我们将它们作为[两个独立的查询](https://github.com/hammerstonedev/fast-paginate/blob/154da286f8160a9e75e64e8025b0da682aa2ba23/src/BuilderMixin.php#L62-L79)来运行以解决这个问题!

根据您的数据集,性能提升可能会有所不同,但这种方法允许数据库检查尽可能少的数据以满足用户的需求。
Expand Down
2 changes: 1 addition & 1 deletion docs/zh-hk/components/cache.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,4 +62,4 @@ di(CacheManager::class)->store('co')->remember($key, $ttl=60, function() {

## 參考

Likes [Laravel-Cache](https://laravel.com/docs/8.x/cache)
參閲 [Laravel-Cache](https://laravel.com/docs/8.x/cache)
2 changes: 1 addition & 1 deletion docs/zh-hk/components/encryption.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ php bin/hyperf.php vendor:publish friendsofhyperf/encryption

## 使用

```shell
```php
$encryptString = encrypt($string);
$decryptString = decrypt($encryptString);
```
2 changes: 1 addition & 1 deletion docs/zh-hk/components/fast-paginate.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ select * from contacts -- The full data that you want to show your
)
```

> 運行上述查詢時,您可能會遇到錯誤!例如 `This version of MySQL doesn't yet support 'LIMIT & IN/ALL/ANY/SOME subquery.`
> 運行上述查詢時,您可能會遇到錯誤!例如 `This version of MySQL doesn't yet support 'LIMIT & IN/ALL/ANY/SOME subquery.`
> 在這個包中,我們將它們作為[兩個獨立的查詢](https://github.com/hammerstonedev/fast-paginate/blob/154da286f8160a9e75e64e8025b0da682aa2ba23/src/BuilderMixin.php#L62-L79)來運行以解決這個問題!

根據您的數據集,性能提升可能會有所不同,但這種方法允許數據庫檢查儘可能少的數據以滿足用户的需求。
Expand Down
2 changes: 1 addition & 1 deletion docs/zh-tw/components/cache.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,4 +62,4 @@ di(CacheManager::class)->store('co')->remember($key, $ttl=60, function() {

## 參考

Likes [Laravel-Cache](https://laravel.com/docs/8.x/cache)
參閱 [Laravel-Cache](https://laravel.com/docs/8.x/cache)
2 changes: 1 addition & 1 deletion docs/zh-tw/components/encryption.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ php bin/hyperf.php vendor:publish friendsofhyperf/encryption

## 使用

```shell
```php
$encryptString = encrypt($string);
$decryptString = decrypt($encryptString);
```
2 changes: 1 addition & 1 deletion docs/zh-tw/components/fast-paginate.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ select * from contacts -- The full data that you want to show your
)
```

> 執行上述查詢時,您可能會遇到錯誤!例如 `This version of MySQL doesn't yet support 'LIMIT & IN/ALL/ANY/SOME subquery.`
> 執行上述查詢時,您可能會遇到錯誤!例如 `This version of MySQL doesn't yet support 'LIMIT & IN/ALL/ANY/SOME subquery.`
> 在這個包中,我們將它們作為[兩個獨立的查詢](https://github.com/hammerstonedev/fast-paginate/blob/154da286f8160a9e75e64e8025b0da682aa2ba23/src/BuilderMixin.php#L62-L79)來執行以解決這個問題!

根據您的資料集,效能提升可能會有所不同,但這種方法允許資料庫檢查儘可能少的資料以滿足使用者的需求。
Expand Down