Skip to content

Enable use of alternate endpoint FDQN (for when behind reverse proxies) and alternate port over HTTP for testing#1

Closed
johnandersen777 wants to merge 1 commit intojakelazaroff:mainfrom
johnandersen777:alternate_port
Closed

Enable use of alternate endpoint FDQN (for when behind reverse proxies) and alternate port over HTTP for testing#1
johnandersen777 wants to merge 1 commit intojakelazaroff:mainfrom
johnandersen777:alternate_port

Conversation

@johnandersen777
Copy link

@johnandersen777 johnandersen777 commented Jan 27, 2023

Reproduced from Alice Engineering Comms: 2023-01-26 @pdxjohnny Engineering Logs

$ PORT=8000 ADMIN_USERNAME=alice ADMIN_PASSWORD=alice npm run dev
$ curl -u alice:alice -X POST --header "Content-Type: application/json" --data @post.json -v http://localhost:8000/admin/create
$ curl -u alice:alice -X POST --header "Content-Type: application/json" --data @post.json -v http://localhost:8000/admin/follow/http://localhost:7000/bob
  • Follow failing currently, 404s, not sure why
  • If this works it will be perfect for the downstream triggers
    • Note as registry content address
      • Even metric manifest scratch works with this pattern
  • Overlay to set port for own actor
diff --git a/src/index.ts b/src/index.ts
index 676cc41..ffdabfe 100644
--- a/src/index.ts
+++ b/src/index.ts
@@ -7,7 +7,7 @@ import { admin } from "./admin.js";

 const app = express();

-app.set("actor", `https://${HOSTNAME}/${ACCOUNT}`);
+app.set("actor", `http://${HOSTNAME}:${PORT}/${ACCOUNT}`);

 app.use(
   express.text({ type: ["application/json", "application/activity+json"] })
  • Apply overlay: Need to use http for now
    • Overlay application orchestrator: shell pipeline, grep and sed
$ git grep https -- src/ | grep -v .org
src/activitypub.ts:        id: `https://${HOSTNAME}/${crypto.randomUUID()}`,
src/admin.ts:  const uri = `https://${HOSTNAME}/@${crypto.randomUUID()}`;
src/index.ts:app.set("actor", `https://${HOSTNAME}/${ACCOUNT}`);
src/request.ts:  const fragment = actor.inbox.replace("https://" + url.hostname, "");
$ sed -e 's/https:\/\/${HOSTNAME}/http:\/\/\${HOSTNAME}:\${PORT}/g' -e 's/https:\/\/" + url.hostname/http:\/\/" + url.hostname/g' $(git grep https -- src/ | grep -v .org | sed -e 's/:.*//g') | grep http | grep -v .org
$ sed -i -e 's/https:\/\/${HOSTNAME}/http:\/\/\${HOSTNAME}:\${PORT}/g' -e 's/https:\/\/" + url.hostname/http:\/\/" + url.hostname/g' $(git grep https -- src/ | grep -v .org | sed -e 's/:.*//g')
$ git diff
  • Resulting dataflows after dynamic overlay application
diff --git a/src/activitypub.ts b/src/activitypub.ts
index 11cce94..1b9dc4b 100644
--- a/src/activitypub.ts
+++ b/src/activitypub.ts
@@ -63,7 +63,7 @@ activitypub.post("/:actor/inbox", async (req, res) => {
     case "Follow": {
       await send(actor, body.actor, {
         "@context": "https://www.w3.org/ns/activitystreams",
-        id: `https://${HOSTNAME}/${crypto.randomUUID()}`,
+        id: `http://${HOSTNAME}/${crypto.randomUUID()}`,
         type: "Accept",
         actor,
         object: body,
diff --git a/src/admin.ts b/src/admin.ts
index 024ddcd..ca00c46 100644
--- a/src/admin.ts
+++ b/src/admin.ts
@@ -4,7 +4,7 @@ import { is, omit, type } from "superstruct";
 import { Router } from "express";
 import basicAuth from "express-basic-auth";

-import { ADMIN_PASSWORD, ADMIN_USERNAME, HOSTNAME } from "./env.js";
+import { ADMIN_PASSWORD, ADMIN_USERNAME, HOSTNAME, PORT } from "./env.js";
 import {
   createFollowing,
   createPost,
@@ -61,16 +61,21 @@ admin.post("/create", async (req, res) => {
   return res.sendStatus(204);
 });

-admin.post("/follow/:actor", async (req, res) => {
-  const actor: string = req.app.get("actor");

-  const object = req.params.actor;
-  const uri = `https://${HOSTNAME}/@${crypto.randomUUID()}`;
-  await send(actor, object, {
+admin.post("/follow/:actor/:hostname/:port/:proto", async (req, res) => {
+  const our_actor: string = req.app.get("actor");
+  console.log(`Follow endpoint, our actor: ${our_actor}`)
+
+  const { proto, hostname, port, actor } = req.params;
+  const object = `${proto}://${hostname}:${port}/${actor}`;
+  console.log(`Follow endpoint, object: ${object}`)
+  const uri = `http://${HOSTNAME}:${PORT}/@${crypto.randomUUID()}`;
+  console.log(`Follow endpoint, uri: ${uri}`)
+  await send(our_actor, object, {
     "@context": "https://www.w3.org/ns/activitystreams",
     id: uri,
     type: "Follow",
-    actor,
+    actor: our_actor,
     object,
   });

@@ -78,7 +83,7 @@ admin.post("/follow/:actor", async (req, res) => {
   res.sendStatus(204);
 });

-admin.delete("/follow/:actor", async (req, res) => {
+admin.delete("/follow/:actor/:hostname", async (req, res) => {
   const actor: string = req.app.get("actor");

   const object = req.params.actor;
diff --git a/src/request.ts b/src/request.ts
index 462bcbd..3665f71 100644
--- a/src/request.ts
+++ b/src/request.ts
@@ -31,7 +31,7 @@ export async function send(sender: string, recipient: string, message: object) {
   const url = new URL(recipient);

   const actor = await fetchActor(recipient);
-  const fragment = actor.inbox.replace("https://" + url.hostname, "");
+  const fragment = actor.inbox.replace("http://" + url.hostname, "");
   const body = JSON.stringify(message);
   const digest = crypto.createHash("sha256").update(body).digest("base64");
   const d = new Date();
  • YES! We got a meaningful error
$ PORT=8000 npm run dev

> dumbo@1.0.0 dev
> ts-node --esm src/index.ts

Dumbo listening on port 8000…
Follow endpoint, our actor: http://localhost:8000/alice
Follow endpoint, object: https://localhost:7000/bob
Follow endpoint, uri: http://localhost:8000/@d935a0cc-43a2-4d96-8eaf-b7dad202d836
file:///home/pdxjohnny/activitypub-starter-kit-alice/node_modules/node-fetch/src/index.js:108
                        reject(new FetchError(`request to ${request.url} failed, reason: ${error.message}`, 'system', error));
          ^
FetchError: request to https://localhost:7000/bob failed, reason: connect ECONNREFUSED 127.0.0.1:7000
    at ClientRequest.<anonymous> (file:///home/pdxjohnny/activitypub-starter-kit-alice/node_modules/node-fetch/src/index.js:108:11)
    at ClientRequest.emit (node:events:513:28)
    at ClientRequest.emit (node:domain:489:12)
    at TLSSocket.socketErrorListener (node:_http_client:496:9)
    at TLSSocket.emit (node:events:513:28)
    at TLSSocket.emit (node:domain:489:12)
    at emitErrorNT (node:internal/streams/destroy:151:8)
    at emitErrorCloseNT (node:internal/streams/destroy:116:3)
    at processTicksAndRejections (node:internal/process/task_queues:82:21) {
  type: 'system',
  errno: 'ECONNREFUSED',
  code: 'ECONNREFUSED',
  erroredSysCall: 'connect'
}
  • Try following self
$ PORT=8000 npm run --watch dev

> dumbo@1.0.0 dev
> ts-node --esm src/index.ts

Dumbo listening on port 8000…
Follow endpoint, our actor: http://localhost:8000/alice
Follow endpoint, object: http://localhost:8000/alice
Follow endpoint, uri: http://localhost:8000/@b7ec4963-659b-46bc-805a-375aa71bb96f
GET /alice 200 1412 - 2.391 ms
GET /alice 200 1412 - 0.580 ms
Error: Invalid request signature.
    at verify (file:///home/pdxjohnny/activitypub-starter-kit-alice/src/request.ts:126:24)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async file:///home/pdxjohnny/activitypub-starter-kit-alice/src/activitypub.ts:51:12
POST /alice/inbox 401 12 - 97.576 ms
file:///home/pdxjohnny/activitypub-starter-kit-alice/src/request.ts:64
    throw new Error(res.statusText + ": " + (await res.text()));
          ^
Error: Unauthorized: Unauthorized
    at send (file:///home/pdxjohnny/activitypub-starter-kit-alice/src/request.ts:64:11)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async file:///home/pdxjohnny/activitypub-starter-kit-alice/src/admin.ts:74:3
$ openssl genrsa -out keypair.pem 4096
$ openssl rsa -in keypair.pem -pubout -out publickey.crt
$ openssl pkcs8 -topk8 -inform PEM -outform PEM -nocrypt -in keypair.pem -out pkcs8.key
$ PORT=8000 ADMIN_USERNAME=alice ADMIN_PASSWORD=alice PUBLIC_KEY=publickey.crt PRIVATE_KEY=keypair.pem npm run dev

> dumbo@1.0.0 dev
> ts-node --esm src/index.ts

Dumbo listening on port 8000…
POST /admin/follow/alice/localhost/8000/http 401 0 - 1.020 ms
POST /admin/create 204 - - 16.262 ms
Follow endpoint, our actor: http://localhost:8000/alice
Follow endpoint, object: http://localhost:8000/alice
Follow endpoint, uri: http://localhost:8000/@1367d6ef-78a2-4b26-a7b2-4ca0e7a79989
GET /alice 200 611 - 1.014 ms
Error: error:1E08010C:DECODER routines::unsupported
    at Object.createPrivateKey (node:internal/crypto/keys:620:12)
    at send (file:///home/pdxjohnny/activitypub-starter-kit-alice/src/request.ts:39:22)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async file:///home/pdxjohnny/activitypub-starter-kit-alice/src/admin.ts:74:3 {
  library: 'DECODER routines',
  reason: 'unsupported',
  code: 'ERR_OSSL_UNSUPPORTED'
}
  • Create post

post.json

{
  "object": {
    "type": "Note",
    "content": "Alice is Here!"
  }
}
$ curl -u alice:alice -X POST --header "Content-Type: application/json" --data @post.json -v http://localhost:8000/admin/create
$ curl -u alice:alice -X POST -v http://localhost:8000/admin/follow/alice/localhost/8000/http
$ PORT=8000 ADMIN_USERNAME=alice ADMIN_PASSWORD=alice PUBLIC_KEY=publickey.crt PRIVATE_KEY=keypair.pem npm run dev

> dumbo@1.0.0 dev
> ts-node --esm src/index.ts

Dumbo listening on port 8000…
Follow endpoint, our actor: http://localhost:8000/alice
Follow endpoint, object: http://localhost:8000/alice
Follow endpoint, uri: http://localhost:8000/@1bba04e4-ca3d-4f9c-84c0-924f7ee5d796
GET /alice 200 611 - 3.711 ms
Error: error:0909006C:PEM routines:get_name:no start line
    at Object.createPrivateKey (node:internal/crypto/keys:620:12)
    at send (file:///home/pdxjohnny/activitypub-starter-kit-alice/src/request.ts:39:22)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async file:///home/pdxjohnny/activitypub-starter-kit-alice/src/admin.ts:74:3 {
  library: 'PEM routines',
  function: 'get_name',
  reason: 'no start line',
  code: 'ERR_OSSL_PEM_NO_START_LINE'
}
  • Perhaps a missformatted key?
    • Looks like there's a PEM start lin to me, although sometimes
      these things need to be find replaced from RSA to PEM ENCODED
$ cat keypair.pem
-----BEGIN RSA PRIVATE KEY-----
MIIJJwIBAAKCAgEAozVUsUl3mXxhSJbTGW8KaOrSzcx7FnZij6Qc5jRmuiGKUlQb
wHojhcwQUMkVYioVZR1hK80rKT9FXndDYpjoB6O1z92TRYBiwpz2T5VR/1oqtB2j
8ajGJbG43wuMvi3f5YYMzl7cySpzwRDCZSzAjryz7zDBwEu17d912ufUqT7TAbco
GbLx8yM0ONtIDi89WnXZNQk1C3issO2pb/n9YtAaXlrsrTeB99IY6I1G9qnq00Nk
SR2XW6R6+GDFWV2wcu61XKXvMT4g2U6HibrLLIVmWv+hPIvvLWweCNpg74gnq8DL
a/TMjkt0Q6UImuG3Iwdbg29KOdhS98MmrttRRq8ljsttwfwqqyLRZFNQuW2v1Zxw
C0BB7XomhkJgdHCIOWGeAULxRlQarlFstT6fGaNSlVbcHoKDX6j+XckF+13prsRz
WrZxM44v2zw8Yx2oh7LJKcvFdqow8TZBG+YnaO6w1Wel2+n92iaOC0oU+sgxtfBv
ECebzMM94YPB58Ja3hlbIz627Ut+v/TDXHmVjxueufw285GpSI7GmsZihcdB5eBM
IDE0UKnvNbqc+TncoTUXAIxXs7cvnEHusAmMONxtxXlRNOSfKaJ/PWkVwa3NvPrd
4oeIJWdLRppNd5mYA1i2CkPdd5lBAiMWwk2AzP5Hrjlf3/QyZe7mHQAfvjkCAwEA
AQKCAgAMj6o5CuJ9makTISiWKImwkYIv/LDshagITiU7QoU1hidTNs37/mqFfbMz
xIY0y/Bhm+VCrcPIpOn930f8arBRBjSUDwWqr7rqJ5J9hYyODq6CtlVL4CV/+TG1
WPo4GOfGjo6lw39SrEXEcjnD97HKSpO+sQ34cttJt40mj/k6HW+4DhS0BaXBhNAc
ICRnkxMxxyg0gMToYR1JcME9sQvjcwjUgkL9Aeyq2iy4mUcA7Qr5v5IUXoarsvkT
7L7DpgZSkjS7MLo8HAADOZwM6aeBgbLjBhGuy0ZZRJz7KXOUe/IxseD2Kh0kRoJo
QmdrJDwoIgTVFNetpAyerRJpGtAidQ5SOjwPY4h0qCQ6DymHi/L3dSuKKeBdIdY+
bq1p43K3ZwD9NSnkA7SE+cuxDyjLlTy6OYOFdP8nrpX4vLS2JaNjYZ9knA9NZWx8
jO46lQhStQbEnfSurIeeuJre8Sdmx5sTdMS3EDOoL3K0qIG4SodX7ZmBkRw/HSSD
teV7zt3SerpdNluGsTD+ezhefr1B05pRVHnZ2mvG1XRPHbaNbcLedOxyEmUg/Ld4
pq0yyM1zDYjtjMAw7Zr/rQ2Fdf5NJhKhm2+MWEdSi69Ag2bd6Myu6A8z9N0GVWjc
z+OHDiyZcpXwHCZpxu0OKNiPyNrYATBtSGk/ziHYaWIwfHENQQKCAQEA14neGh4L
FGrF0s19OdHT6EFweZ2+SNWgdUUWcias4dXGWnTdiDjmYhHTMLw6GjhCFGMBVGDa
5VfgDXDqE9qWiE61w3maqpnDe5OSul6midR2m/1nDPElvUIONSIoc2gy0x1cpA0k
3+lyd008Oz8JgyWBGDyykOTK4jpNFCfO6dQbv9AwfI+ibSdcDSw1e1LH3gh8AJf8
6xlexgSrPY+A/PSA1jGWWLgeUp9vr2A9sNlHmAMzOIoOMgGXwLGBApCDERCg75es
2sOwcMlGWACpUsGe8mvb8aRtE8ZC3Foq48CqvQkWNXUG7uNmsso+O9Yx+Ipsg3xw
8eQIO8fNjXEQRQKCAQEAwdihcDsgsil0AC+kVtOeZDZkuRozhJeX0cC86Wg9quKf
MpXxrcXqucXTs6Mj55tIiKBuIqwKIoTKOm2c/I6FzmwGWfUrq9IV5doaJfaHbOfF
s9p/TucqFqMzYuSBjnDZ/W+WAYHIc7Yv5rtsbvGDBVHrGk5septi2VF+Y1xLYN5k
h5WCDJ84W53aQCmkGEJX8g784HVuNjaGCsfLS6Hu2U9p7B0GjgUPIEWNsz0Qfw22
CUjVxMsgDJfs8+F/PgMP6dO4Bh0E5ozDjCngMcyNgujO0e5E6ENhUBvxorq2ZNs4
+reThNb8EVpNyoB0kW54TjF1O7+xOQk2WqzvFge3ZQKCAQBIc9GzGdJDpLim6ehk
XUJMWmMCnqHuPdFYWkb3TETlDUVF9u7Y8beP08oYIc2uLqcXz0gNIxmx6l4oZoJo
9huP6lVzRhdVraZmje7Osy5sOn08ZmwTj6ROADxiY/8Q/D/Jc59GjhyNIB4YOGA8
0i3SZfMFQLINvrrsXQi388G8HE7PpZ4G4QmKg6aPzwKTV/pTiqqUUIL2TGrtSXh+
kxSa812zoquVWx7mSy3x1/okzoUgdkLriIzJBnwKjCB/yjAktmBC6ctzJkDTSPVa
c653YRqbBuLCUbFQ6l5jT/QG5yb9sGZExff0qYBGLXHKD3Bwyac8c8JLrYmO/tT7
7Lu5AoIBACiUyXdNaZLiyr4fOzBSLR6dpIh7y70+XzIyP1o90Gst9lYIvge7H2C0
4ZUB2kpqX8z6iRQJIDYJxqxktjDJRYnpY4sBoJrf6GWuOzsnWUKbYvA8FdrW2iDT
GbbiT50aUwiTi7vVB7nxsiWDpzeyp9M9SxK+yEcCsLb+MI9sivtEk5cu3YL28j17
1m0ISqopeW/bY2U6MFB5KaaoHQ9AX1hvH6WmjfC9bmU7KmcTqZhvrmRTMy13uMXq
KFkGJDU/Pt2czTG6cYQyg92cBqtmP1ngkyuvzg0xzfWPZA7FN9n1awBR3jg5KZwY
Y6C5M64eimEUSY6wmtFt9EsXWRYrl2ECggEAFOi9VS+SLQKeOJ+X0WVsC/yx2yoS
TFYkI1NcHl3j/W6dFJGwanV+uAR6pJjt+obgJVlncuvRTK6BPxEmsxIb61T9W3uw
pAABeX3S6T05XA3v25l0zvCZiunkZbtyR/FfEGjMkls1vvDDqeSveqpU9y4YpAYL
UsszhZ3U1MXyvwO1Z7KWOl2BhVFI/zskbltcLPwYvI0xH8/OR7wrS5z3YdDj65Gr
/iBiuIYJTL8LZ8kprZB4mKTd8DGqNEJVyYQOG+RJLWW37/mm+SeAwABSfhanccVt
WNAXcit1N6u8ao3A0+kV6zR6pGLD8MxphtfdhKQeTOQG5QindbV6Opo5ug==
-----END RSA PRIVATE KEY-----
  • nodejs docs
    • crypto.createPrivateKey(key)#
$ PORT=8000 ADMIN_USERNAME=alice ADMIN_PASSWORD=alice PUBLIC_KEY=$(cat publickey.crt) PRIVATE_KEY=$(cat pkcs8.key) npm run dev
Dumbo listening on port 8000…
Follow endpoint, our actor: http://localhost:8000/alice
Follow endpoint, object: http://localhost:8000/alice
Follow endpoint, uri: http://localhost:8000/@3eca6f29-414e-449e-9543-19f718314593
GET /alice 200 1410 - 3.044 ms
GET /alice 200 1410 - 0.618 ms
Error: Invalid request signature.
    at verify (file:///home/pdxjohnny/activitypub-starter-kit-alice/src/request.ts:128:24)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async file:///home/pdxjohnny/activitypub-starter-kit-alice/src/activitypub.ts:51:12
POST /alice/inbox 401 12 - 111.891 ms
file:///home/pdxjohnny/activitypub-starter-kit-alice/src/request.ts:66
    throw new Error(res.statusText + ": " + (await res.text()));
          ^
Error: Unauthorized: Unauthorized
    at send (file:///home/pdxjohnny/activitypub-starter-kit-alice/src/request.ts:66:11)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async file:///home/pdxjohnny/activitypub-starter-kit-alice/src/admin.ts:74:3
  • Update fragment on activitypub.send()
diff --git a/src/request.ts b/src/request.ts
index 462bcbd..cad57a7 100644
--- a/src/request.ts
+++ b/src/request.ts
@@ -31,7 +31,7 @@ export async function send(sender: string, recipient: string, message: object) {
   const url = new URL(recipient);
 
   const actor = await fetchActor(recipient);
-  const fragment = actor.inbox.replace("https://" + url.hostname, "");
+  const fragment = url.pathname + "/inbox";
   const body = JSON.stringify(message);
   const digest = crypto.createHash("sha256").update(body).digest("base64");
   const d = new Date();
@@ -46,6 +46,7 @@ export async function send(sender: string, recipient: string, message: object) {
   const signature = crypto
     .sign("sha256", Buffer.from(data), key)
     .toString("base64");
+  console.log(`crypto.sign("sha256", data: ${data}, key: ${key}, signature: ${signature})`);
 
   const res = await fetch(actor.inbox, {
     method: "POST",
@@ -119,6 +120,7 @@ export async function verify(req: Request): Promise<string> {
       return `${header}: ${req.get(header)}`;
     })
     .join("\n");
+  console.log(`crypto.verify("sha256", data: ${comparison}, key: ${key}, signature: ${included.signature})`);
   const data = Buffer.from(comparison);
 
   // verify the signature against the headers using the actor's public key
$ dffml service dev export alice.cli:ALICE_COLLECTOR_DATAFLOW

asciicast

  • Tested with localhost.run based HTTPS
$ ssh -R 80:localhost:8000 nokey@localhost.run
$ curl -u alice:alice -X POST -v https://9e2336258d686a.lhr.life/admin/follow/alice/9e2336258d686a.lhr.life/443/https
$ curl -u alice:alice -X POST --header "Content-Type: application/json" --data @post.json -v https://9e2336258d686a.lhr.life/admin/create

asciicast

Signed-off-by: John Andersen <johnandersenpdx@gmail.com>
@johnandersen777
Copy link
Author

johnandersen777 commented Jan 27, 2023

I don't plan on maintaining this branch but wanted to ensure visibility via closed PR in case any one else finds this useful for testing / debugging.

@johnandersen777 johnandersen777 changed the title Scratch work to use alternate port Enable use of alternate endpoint FDQN (for when behind reverse proxies) and alternate port over HTTP for testing Jan 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants