Skip to content

Commit 5f88d56

Browse files
tamainamei23u1-liquid
authored
perf(federation): Ed25519署名に対応する (misskey-dev#13464)
* 1. ed25519キーペアを発行・Personとして公開鍵を送受信 * validate additionalPublicKeys * getAuthUserFromApIdはmainを選ぶ * ✌️ * fix * signatureAlgorithm * set publicKeyCache lifetime * refresh * httpMessageSignatureAcceptable * ED25519_SIGNED_ALGORITHM * ED25519_PUBLIC_KEY_SIGNATURE_ALGORITHM * remove sign additionalPublicKeys signature requirements * httpMessageSignaturesSupported * httpMessageSignaturesImplementationLevel * httpMessageSignaturesImplementationLevel: '01' * perf(federation): Use hint for getAuthUserFromApId (misskey-dev#13470) * Hint for getAuthUserFromApId * とどのつまりこれでいいのか？ * use @misskey-dev/node-http-message-signatures * fix * signedPost, signedGet * ap-request.tsを復活させる * remove digest prerender * fix test? * fix test * add httpMessageSignaturesImplementationLevel to FederationInstance * ManyToOne * fetchPersonWithRenewal * exactKey * ✌️ * use const * use gen-key-pair fn. from '@misskey-dev/node-http-message-signatures' * update node-http-message-signatures * fix * @misskey-dev/[email protected] * getAuthUserFromApIdでupdatePersonの頻度を増やす * cacheRaw.date * use requiredInputs misskey-dev#13464 (comment) * update @misskey-dev/node-http-message-signatures * clean up * err msg * fix(backend): fetchInstanceMetadataのLockが永遠に解除されない問題を修正 Co-authored-by: まっちゃとーにゅ <[email protected]> * fix httpMessageSignaturesImplementationLevel validation * fix test * fix * comment * comment * improve test * fix * use Promise.all in genRSAAndEd25519KeyPair * refreshAndprepareEd25519KeyPair * refreshAndfindKey * commetn * refactor public keys add * digestプリレンダを復活させる RFC実装時にどうするか考える * fix, async * fix * !== true * use save * Deliver update person when new key generated (not tested) misskey-dev#13464 (comment) * 循環参照で落ちるのを解消？ * fix? * Revert "fix?" This reverts commit 0082f6f. * a * logger * log * change logger * 秘密鍵の変更は、フラグではなく鍵を引き回すようにする * addAllKnowingSharedInboxRecipe * nanka meccha kaeta * delivre * キャッシュ有効チェックはロック取得前に行う * @misskey-dev/[email protected] * PrivateKeyPem * getLocalUserPrivateKey * fix test * if * fix ap-request * update node-http-message-signatures * fix type error * update package * fix type * update package * retry no key * @misskey-dev/[email protected] * fix type error * log keyid * logger * db-resolver * JSON.stringify * HTTP Signatureがなかったり使えなかったりしそうな場合にLD Signatureを活用するように * inbox-delayed use actor if no signature * ユーザーとキーの同一性チェックはhostの一致にする * log signature parse err * save array * とりあえずtryで囲っておく * fetchPersonWithRenewalでエラーが起きたら古いデータを返す * use transactionalEntityManager * fix spdx * @misskey-dev/[email protected] * add comment * fix * publicKeyに配列が入ってもいいようにする misskey-dev#13950 * define additionalPublicKeys * fix * merge fix * refreshAndprepareEd25519KeyPair → refreshAndPrepareEd25519KeyPair * remove gen-key-pair.ts * defaultMaxListeners = 512 * Revert "defaultMaxListeners = 512" This reverts commit f2c412c. * genRSAAndEd25519KeyPairではキーを直列に生成する? * maxConcurrency: 8 * maxConcurrency: 16 * maxConcurrency: 8 * Revert "genRSAAndEd25519KeyPairではキーを直列に生成する?" This reverts commit d0aada5. * maxWorkers: '90%' * Revert "maxWorkers: '90%'" This reverts commit 9e0a93f. * e2e/timelines.tsで個々のテストに対するtimeoutを削除, maxConcurrency: 32 * better error handling of this.userPublickeysRepository.delete * better comment * set result to keypairEntityCache * deliverJobConcurrency: 16, deliverJobPerSec: 1024, inboxJobConcurrency: 4 * inboxJobPerSec: 64 * delete request.headers['host']; * fix * // node-fetch will generate this for us. if we keep 'Host', it won't change with redirects! * move delete host * modify comment * modify comment * fix correct → collect * refreshAndfindKey → refreshAndFindKey * modify comment * modify attachLdSignature * getApId, InboxProcessorService * TODO * [skip ci] add CHANGELOG --------- Co-authored-by: MeiMei <[email protected]> Co-authored-by: まっちゃとーにゅ <[email protected]>
1 parent 3331f39 commit 5f88d56

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

52 files changed

+1092
-690
lines changed

‎.config/docker_example.yml

+3-3
Original file line numberDiff line numberDiff line change
@@ -164,12 +164,12 @@ id: 'aidx'
164164
#clusterLimit: 1
165165

166166
# Job concurrency per worker
167-
# deliverJobConcurrency: 128
168-
# inboxJobConcurrency: 16
167+
# deliverJobConcurrency: 16
168+
# inboxJobConcurrency: 4
169169

170170
# Job rate limiter
171171
# deliverJobPerSec: 128
172-
# inboxJobPerSec: 32
172+
# inboxJobPerSec: 64
173173

174174
# Job attempts
175175
# deliverJobMaxAttempts: 12

‎.config/example.yml

+4-4
Original file line numberDiff line numberDiff line change
@@ -230,15 +230,15 @@ id: 'aidx'
230230
#clusterLimit: 1
231231

232232
# Job concurrency per worker
233-
#deliverJobConcurrency: 128
234-
#inboxJobConcurrency: 16
233+
#deliverJobConcurrency: 16
234+
#inboxJobConcurrency: 4
235235
#relationshipJobConcurrency: 16
236236
# What's relationshipJob?:
237237
# Follow, unfollow, block and unblock(ings) while following-imports, etc. or account migrations.
238238

239239
# Job rate limiter
240-
#deliverJobPerSec: 128
241-
#inboxJobPerSec: 32
240+
#deliverJobPerSec: 1024
241+
#inboxJobPerSec: 64
242242
#relationshipJobPerSec: 64
243243

244244
# Job attempts

‎.devcontainer/devcontainer.yml

+4-4
Original file line numberDiff line numberDiff line change
@@ -157,12 +157,12 @@ id: 'aidx'
157157
#clusterLimit: 1
158158

159159
# Job concurrency per worker
160-
# deliverJobConcurrency: 128
161-
# inboxJobConcurrency: 16
160+
# deliverJobConcurrency: 16
161+
# inboxJobConcurrency: 4
162162

163163
# Job rate limiter
164-
# deliverJobPerSec: 128
165-
# inboxJobPerSec: 32
164+
# deliverJobPerSec: 1024
165+
# inboxJobPerSec: 64
166166

167167
# Job attempts
168168
# deliverJobMaxAttempts: 12

‎CHANGELOG.md

+2
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,8 @@
77
- Feat: 通報を受けた際、または解決した際に、予め登録した宛先に通知を飛ばせるように(mail or webhook) #13705
88
- Feat: ユーザーのアイコン/バナーの変更可否をロールで設定可能に
99
- 変更不可となっていても、設定済みのものを解除してデフォルト画像に戻すことは出来ます
10+
- Feat: 連合に使うHTTP SignaturesがEd25519鍵に対応するように #13464
11+
- Ed25519署名に対応するサーバーが増えると、deliverで要求されるサーバーリソースが削減されます
1012
- Fix: 配信停止したインスタンス一覧が見れなくなる問題を修正
1113
- Fix: Dockerコンテナの立ち上げ時に`pnpm`のインストールで固まることがある問題
1214
- Fix: デフォルトテーマに無効なテーマコードを入力するとUIが使用できなくなる問題を修正

‎CONTRIBUTING.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -185,7 +185,7 @@ TODO
185185
## Environment Variable
186186

187187
- `MISSKEY_CONFIG_YML`: Specify the file path of config.yml instead of default.yml (e.g. `2nd.yml`).
188-
- `MISSKEY_WEBFINGER_USE_HTTP`: If it's set true, WebFinger requests will be http instead of https, useful for testing federation between servers in localhost. NEVER USE IN PRODUCTION.
188+
- `MISSKEY_USE_HTTP`: If it's set true, federation requests (like nodeinfo and webfinger) will be http instead of https, useful for testing federation between servers in localhost. NEVER USE IN PRODUCTION. (was `MISSKEY_WEBFINGER_USE_HTTP`)
189189

190190
## Continuous integration
191191
Misskey uses GitHub Actions for executing automated tests.

‎chart/files/default.yml

+4-4
Original file line numberDiff line numberDiff line change
@@ -178,12 +178,12 @@ id: "aidx"
178178
#clusterLimit: 1
179179

180180
# Job concurrency per worker
181-
# deliverJobConcurrency: 128
182-
# inboxJobConcurrency: 16
181+
# deliverJobConcurrency: 16
182+
# inboxJobConcurrency: 4
183183

184184
# Job rate limiter
185-
# deliverJobPerSec: 128
186-
# inboxJobPerSec: 32
185+
# deliverJobPerSec: 1024
186+
# inboxJobPerSec: 64
187187

188188
# Job attempts
189189
# deliverJobMaxAttempts: 12

‎packages/backend/migration/1708980134301-APMultipleKeys.js

+39
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
/*
2+
* SPDX-FileCopyrightText: syuilo and misskey-project
3+
* SPDX-License-Identifier: AGPL-3.0-only
4+
*/
5+
6+
export class APMultipleKeys1708980134301 {
7+
name = 'APMultipleKeys1708980134301'
8+
9+
async up(queryRunner) {
10+
await queryRunner.query(`DROP INDEX "public"."IDX_171e64971c780ebd23fae140bb"`);
11+
await queryRunner.query(`ALTER TABLE "user_keypair" ADD "ed25519PublicKey" character varying(128)`);
12+
await queryRunner.query(`ALTER TABLE "user_keypair" ADD "ed25519PrivateKey" character varying(128)`);
13+
await queryRunner.query(`ALTER TABLE "user_publickey" DROP CONSTRAINT "FK_10c146e4b39b443ede016f6736d"`);
14+
await queryRunner.query(`ALTER TABLE "user_publickey" DROP CONSTRAINT "PK_10c146e4b39b443ede016f6736d"`);
15+
await queryRunner.query(`ALTER TABLE "user_publickey" ADD CONSTRAINT "PK_0db6a5fdb992323449edc8ee421" PRIMARY KEY ("userId", "keyId")`);
16+
await queryRunner.query(`ALTER TABLE "user_publickey" DROP CONSTRAINT "PK_0db6a5fdb992323449edc8ee421"`);
17+
await queryRunner.query(`ALTER TABLE "user_publickey" ADD CONSTRAINT "PK_171e64971c780ebd23fae140bba" PRIMARY KEY ("keyId")`);
18+
await queryRunner.query(`ALTER TABLE "user_publickey" ADD CONSTRAINT "UQ_10c146e4b39b443ede016f6736d" UNIQUE ("userId")`);
19+
await queryRunner.query(`CREATE INDEX "IDX_10c146e4b39b443ede016f6736" ON "user_publickey" ("userId") `);
20+
await queryRunner.query(`ALTER TABLE "user_publickey" ADD CONSTRAINT "FK_10c146e4b39b443ede016f6736d" FOREIGN KEY ("userId") REFERENCES "user"("id") ON DELETE CASCADE ON UPDATE NO ACTION`);
21+
}
22+
23+
async down(queryRunner) {
24+
await queryRunner.query(`ALTER TABLE "user_publickey" DROP CONSTRAINT "FK_10c146e4b39b443ede016f6736d"`);
25+
await queryRunner.query(`DROP INDEX "public"."IDX_10c146e4b39b443ede016f6736"`);
26+
await queryRunner.query(`ALTER TABLE "user_publickey" DROP CONSTRAINT "UQ_10c146e4b39b443ede016f6736d"`);
27+
await queryRunner.query(`ALTER TABLE "user_publickey" DROP CONSTRAINT "PK_171e64971c780ebd23fae140bba"`);
28+
await queryRunner.query(`ALTER TABLE "user_publickey" ADD CONSTRAINT "PK_0db6a5fdb992323449edc8ee421" PRIMARY KEY ("userId", "keyId")`);
29+
await queryRunner.query(`ALTER TABLE "user_publickey" DROP CONSTRAINT "PK_0db6a5fdb992323449edc8ee421"`);
30+
await queryRunner.query(`ALTER TABLE "user_publickey" ADD CONSTRAINT "PK_10c146e4b39b443ede016f6736d" PRIMARY KEY ("userId")`);
31+
await queryRunner.query(`ALTER TABLE "user_publickey" ADD CONSTRAINT "FK_10c146e4b39b443ede016f6736d" FOREIGN KEY ("userId") REFERENCES "user"("id") ON DELETE CASCADE ON UPDATE NO ACTION`);
32+
await queryRunner.query(`ALTER TABLE "user_profile" ALTER COLUMN "followersVisibility" DROP DEFAULT`);
33+
await queryRunner.query(`ALTER TABLE "user_profile" ALTER COLUMN "followersVisibility" TYPE "public"."user_profile_followersVisibility_enum_old" USING "followersVisibility"::"text"::"public"."user_profile_followersVisibility_enum_old"`);
34+
await queryRunner.query(`ALTER TABLE "user_profile" ALTER COLUMN "followersVisibility" SET DEFAULT 'public'`);
35+
await queryRunner.query(`ALTER TABLE "user_keypair" DROP COLUMN "ed25519PrivateKey"`);
36+
await queryRunner.query(`ALTER TABLE "user_keypair" DROP COLUMN "ed25519PublicKey"`);
37+
await queryRunner.query(`CREATE UNIQUE INDEX "IDX_171e64971c780ebd23fae140bb" ON "user_publickey" ("keyId") `);
38+
}
39+
}

‎packages/backend/migration/1709242519122-HttpSignImplLv.js

+16
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
/*
2+
* SPDX-FileCopyrightText: syuilo and misskey-project
3+
* SPDX-License-Identifier: AGPL-3.0-only
4+
*/
5+
6+
export class HttpSignImplLv1709242519122 {
7+
name = 'HttpSignImplLv1709242519122'
8+
9+
async up(queryRunner) {
10+
await queryRunner.query(`ALTER TABLE "instance" ADD "httpMessageSignaturesImplementationLevel" character varying(16) NOT NULL DEFAULT '00'`);
11+
}
12+
13+
async down(queryRunner) {
14+
await queryRunner.query(`ALTER TABLE "instance" DROP COLUMN "httpMessageSignaturesImplementationLevel"`);
15+
}
16+
}

‎packages/backend/migration/1709269211718-APMultipleKeysFix1.js

+16
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
/*
2+
* SPDX-FileCopyrightText: syuilo and misskey-project
3+
* SPDX-License-Identifier: AGPL-3.0-only
4+
*/
5+
6+
export class APMultipleKeys1709269211718 {
7+
name = 'APMultipleKeys1709269211718'
8+
9+
async up(queryRunner) {
10+
await queryRunner.query(`ALTER TABLE "user_publickey" DROP CONSTRAINT "UQ_10c146e4b39b443ede016f6736d"`);
11+
}
12+
13+
async down(queryRunner) {
14+
await queryRunner.query(`ALTER TABLE "user_publickey" ADD CONSTRAINT "UQ_10c146e4b39b443ede016f6736d" UNIQUE ("userId")`);
15+
}
16+
}

‎packages/backend/package.json

+1-1
Original file line numberDiff line numberDiff line change
@@ -79,13 +79,13 @@
7979
"@fastify/multipart": "8.3.0",
8080
"@fastify/static": "7.0.4",
8181
"@fastify/view": "9.1.0",
82+
"@misskey-dev/node-http-message-signatures": "0.0.10",
8283
"@misskey-dev/sharp-read-bmp": "1.2.0",
8384
"@misskey-dev/summaly": "5.1.0",
8485
"@napi-rs/canvas": "^0.1.53",
8586
"@nestjs/common": "10.3.10",
8687
"@nestjs/core": "10.3.10",
8788
"@nestjs/testing": "10.3.10",
88-
"@peertube/http-signature": "1.7.0",
8989
"@sentry/node": "8.13.0",
9090
"@sentry/profiling-node": "8.13.0",
9191
"@simplewebauthn/server": "10.0.0",

‎packages/backend/src/@types/http-signature.d.ts

-82
This file was deleted.

‎packages/backend/src/const.ts

+5
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,11 @@ export const MAX_NOTE_TEXT_LENGTH = 3000;
99
export const USER_ONLINE_THRESHOLD = 1000 * 60 * 10; // 10min
1010
export const USER_ACTIVE_THRESHOLD = 1000 * 60 * 60 * 24 * 3; // 3days
1111

12+
export const REMOTE_USER_CACHE_TTL = 1000 * 60 * 60 * 3; // 3hours
13+
export const REMOTE_USER_MOVE_COOLDOWN = 1000 * 60 * 60 * 24 * 14; // 14days
14+
15+
export const REMOTE_SERVER_CACHE_TTL = 1000 * 60 * 60 * 3; // 3hours
16+
1217
//#region hard limits
1318
// If you change DB_* values, you must also change the DB schema.
1419

‎packages/backend/src/core/AccountUpdateService.ts

+21-6
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,8 @@
33
* SPDX-License-Identifier: AGPL-3.0-only
44
*/
55

6-
import { Inject, Injectable } from '@nestjs/common';
6+
import { Inject, Injectable, OnModuleInit } from '@nestjs/common';
7+
import { ModuleRef } from '@nestjs/core';
78
import { DI } from '@/di-symbols.js';
89
import type { UsersRepository } from '@/models/_.js';
910
import type { MiUser } from '@/models/User.js';
@@ -12,30 +13,44 @@ import { RelayService } from '@/core/RelayService.js';
1213
import { ApDeliverManagerService } from '@/core/activitypub/ApDeliverManagerService.js';
1314
import { UserEntityService } from '@/core/entities/UserEntityService.js';
1415
import { bindThis } from '@/decorators.js';
16+
import type { PrivateKeyWithPem } from '@misskey-dev/node-http-message-signatures';
1517

1618
@Injectable()
17-
export class AccountUpdateService {
19+
export class AccountUpdateService implements OnModuleInit {
20+
private apDeliverManagerService: ApDeliverManagerService;
1821
constructor(
22+
private moduleRef: ModuleRef,
23+
1924
@Inject(DI.usersRepository)
2025
private usersRepository: UsersRepository,
2126

2227
private userEntityService: UserEntityService,
2328
private apRendererService: ApRendererService,
24-
private apDeliverManagerService: ApDeliverManagerService,
2529
private relayService: RelayService,
2630
) {
2731
}
2832

33+
async onModuleInit() {
34+
this.apDeliverManagerService = this.moduleRef.get(ApDeliverManagerService.name);
35+
}
36+
2937
@bindThis
30-
public async publishToFollowers(userId: MiUser['id']) {
38+
/**
39+
* Deliver account update to followers
40+
* @param userId user id
41+
* @param deliverKey optional. Private key to sign the deliver.
42+
*/
43+
public async publishToFollowers(userId: MiUser['id'], deliverKey?: PrivateKeyWithPem) {
3144
const user = await this.usersRepository.findOneBy({ id: userId });
3245
if (user == null) throw new Error('user not found');
3346

3447
// フォロワーがリモートユーザーかつ投稿者がローカルユーザーならUpdateを配信
3548
if (this.userEntityService.isLocalUser(user)) {
3649
const content = this.apRendererService.addContext(this.apRendererService.renderUpdate(await this.apRendererService.renderPerson(user), user));
37-
this.apDeliverManagerService.deliverToFollowers(user, content);
38-
this.relayService.deliverToRelays(user, content);
50+
await Promise.allSettled([
51+
this.apDeliverManagerService.deliverToFollowers(user, content, deliverKey),
52+
this.relayService.deliverToRelays(user, content, deliverKey),
53+
]);
3954
}
4055
}
4156
}

‎packages/backend/src/core/CreateSystemUserService.ts

+3-4
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ import { randomUUID } from 'node:crypto';
77
import { Inject, Injectable } from '@nestjs/common';
88
import bcrypt from 'bcryptjs';
99
import { IsNull, DataSource } from 'typeorm';
10-
import { genRsaKeyPair } from '@/misc/gen-key-pair.js';
10+
import { genRSAAndEd25519KeyPair } from '@/misc/gen-key-pair.js';
1111
import { MiUser } from '@/models/User.js';
1212
import { MiUserProfile } from '@/models/UserProfile.js';
1313
import { IdService } from '@/core/IdService.js';
@@ -38,7 +38,7 @@ export class CreateSystemUserService {
3838
// Generate secret
3939
const secret = generateNativeUserToken();
4040

41-
const keyPair = await genRsaKeyPair();
41+
const keyPair = await genRSAAndEd25519KeyPair();
4242

4343
let account!: MiUser;
4444

@@ -64,9 +64,8 @@ export class CreateSystemUserService {
6464
}).then(x => transactionalEntityManager.findOneByOrFail(MiUser, x.identifiers[0]));
6565

6666
await transactionalEntityManager.insert(MiUserKeypair, {
67-
publicKey: keyPair.publicKey,
68-
privateKey: keyPair.privateKey,
6967
userId: account.id,
68+
...keyPair,
7069
});
7170

7271
await transactionalEntityManager.insert(MiUserProfile, {

0 commit comments

Comments
 (0)