Add supavisor to supabase #2712

Geczy · 2024-06-28T19:29:09Z

more necessary setup instructions here https://discord.com/channels/459365938081431553/1217542291225448611/1256342603318956042

templates/compose/supabase.yaml

RobertHH-IS · 2024-06-28T22:40:10Z

Can you post the directions here? Discord link does not lead anywhere.

Geczy · 2024-06-28T23:24:58Z

Sure. Here it is

and also need to run this in supavisor

curl  -X PUT \
  'http://172.28.0.7:4000/api/tenants/dev_tenant' \
  --header 'Accept: */*' \
  --header 'User-Agent: Thunder Client (https://www.thunderclient.com)' \
  --header 'Authorization: Bearer YOUR_SUPABASE_SERVICE_KEY_HERE' \
  --header 'Content-Type: application/json' \
  --data-raw '{
  "tenant": {
    "db_host": "supabase-db",
    "db_port": 5432,
    "db_database": "postgres",
    "ip_version": "auto",
    "enforce_ssl": false,
    "require_user": false,
    "auth_query": "SELECT rolname, rolpassword FROM pg_authid WHERE rolname=$1;",
    "users": [
      {
        "db_user": "postgres",
        "db_password": "YOUR_DB_PASSWORD_HERE",
        "pool_size": 20,
        "mode_type": "transaction",
        "is_manager": true
      }
    ]
  }
}'

there might be a way to set this up without running this, like with an init script tho

and my url looks like this

:6543/postgres?pgbouncer=true&connection_limit=1&options=reference%3Ddev_tenant

Mortalife · 2024-06-29T10:27:45Z

templates/compose/supabase.yaml

+      - PROXY_PORT_TRANSACTION=6543
+      - DATABASE_URL=ecto://postgres:${SERVICE_PASSWORD_POSTGRES}@${POSTGRES_HOST:-supabase-db}:${POSTGRES_PORT:-5432}/${POSTGRES_DB:-postgres}
+      - CLUSTER_POSTGRES=true
+      - SECRET_KEY_BASE=12345678901234567890121234567890123456789012345678903212345678901234567890123456789032123456789012345678901234567890323456789032


These probably shouldn't be hard coded

What do you recommend? They need to specifically be this length

Ideally there's a way to generate a random string either through docker-compose or coolify, I couldn't find one though.

Considering this should be unique to every user, and I don't imagine many people are changing the defaults while using these, I'd either wait for the functionality to be added into coolify, or force the user to enter the information.

You can get docker-compose to error if an environment variable is missing using ${VAR:?error} so that would mean changing it to: SECRET_KEY_BASE=${SECRET_KEY_BASE:?error}.

Supabase is a fairly popular template, so people would need to be told that they need to put in a variable that contains a string that is X character long, but I don't know where a good place for the instruction to go would be.

Failing all that, maybe setting it to SECRET_KEY_BASE=${SECRET_KEY_BASE} and finding where the default values for Dashboard User and Dashboard Password are generated and ensure a new one is generated for the 2 variables that need them.

Appears there is some functionality to do this:

coolify/bootstrap/helpers/shared.php

Lines 1969 to 2046 in e9158b7

case 'PASSWORD':

$generatedValue = Str::password(symbols: false);

break;

case 'PASSWORD_64':

$generatedValue = Str::password(length: 64, symbols: false);

break;

// This is not base64, it's just a random string

case 'BASE64_64':

$generatedValue = Str::random(64);

break;

case 'BASE64_128':

$generatedValue = Str::random(128);

break;

case 'BASE64':

case 'BASE64_32':

$generatedValue = Str::random(32);

break;

// This is base64,

case 'REALBASE64_64':

$generatedValue = base64_encode(Str::random(64));

break;

case 'REALBASE64_128':

$generatedValue = base64_encode(Str::random(128));

break;

case 'REALBASE64':

case 'REALBASE64_32':

$generatedValue = base64_encode(Str::random(32));

break;

case 'USER':

$generatedValue = Str::random(16);

break;

case 'SUPABASEANON':

$signingKey = $service->environment_variables()->where('key', 'SERVICE_PASSWORD_JWT')->first();

if (is_null($signingKey)) {

return;

} else {

$signingKey = $signingKey->value;

}

$key = InMemory::plainText($signingKey);

$algorithm = new Sha256();

$tokenBuilder = (new Builder(new JoseEncoder(), ChainedFormatter::default()));

$now = new DateTimeImmutable();

$now = $now->setTime($now->format('H'), $now->format('i'));

$token = $tokenBuilder

->issuedBy('supabase')

->issuedAt($now)

->expiresAt($now->modify('+100 year'))

->withClaim('role', 'anon')

->getToken($algorithm, $key);

$generatedValue = $token->toString();

break;

case 'SUPABASESERVICE':

$signingKey = $service->environment_variables()->where('key', 'SERVICE_PASSWORD_JWT')->first();

if (is_null($signingKey)) {

return;

} else {

$signingKey = $signingKey->value;

}

$key = InMemory::plainText($signingKey);

$algorithm = new Sha256();

$tokenBuilder = (new Builder(new JoseEncoder(), ChainedFormatter::default()));

$now = new DateTimeImmutable();

$now = $now->setTime($now->format('H'), $now->format('i'));

$token = $tokenBuilder

->issuedBy('supabase')

->issuedAt($now)

->expiresAt($now->modify('+100 year'))

->withClaim('role', 'service_role')

->getToken($algorithm, $key);

$generatedValue = $token->toString();

break;

default:

$generatedValue = Str::random(16);

break;

}

return $generatedValue;

}

And the docs for how to use them: https://coolify.io/docs/knowledge-base/add-a-service#password

Maybe this will work.

- SECRET_KEY_BASE=${SERVICE_BASE64_128_SUPAVISOR} - VAULT_ENC_KEY=${SERVICE_PASSWORD_64_SUPAVISOR}

Doesn't actually need to be 128 characters I don't think, fly deployment docs suggest some secure value: https://supabase.github.io/supavisor/deployment/fly/

Might be worth setting them both to PASSWORD_64 if only for better naming.

Mortalife · 2024-06-30T22:16:00Z

Thinking on the issue of running the dev_tenant curl.

Sure. Here it is

and also need to run this in supavisor
there might be a way to set this up without running this, like with an init script tho

and my url looks like this
:6543/postgres?pgbouncer=true&connection_limit=1&options=reference%3Ddev_tenant

Maybe this could also be a run once container like the minio create-bucket, using a curl image and overriding the entrypoint.

Geczy · 2024-07-01T01:01:33Z

the vault secret has to be 32 characters:

As a general note, if you are not using the Makefile you will have to set a
VAULT_ENC_KEY which should be at least 32 bytes long.

and this tenant business might actually be even easier than that. i think we can just add this to the sql file where i create the _supavisor schema. but we have to encrypt the password on the fly -- which we probably do need a cron image for, unless sql can do it for us?

  config :supavisor, Supavisor.Vault,
    ciphers: [
      default: {
        Cloak.Ciphers.AES.GCM,
        tag: "AES.GCM.V1", key: System.get_env("VAULT_ENC_KEY")
      }
    ]

SET statement_timeout = 0;
SET lock_timeout = 0;
SET idle_in_transaction_session_timeout = 0;
SET client_encoding = 'UTF8';
SET standard_conforming_strings = ON;

SELECT pg_catalog.Set_config('search_path', '', false);

SET check_function_bodies = false;
SET xmloption = content;
SET client_min_messages = warning;
SET row_security = OFF;

--
-- Data for Name: tenants; Type: TABLE DATA; Schema: _supavisor; Owner: postgres
--
INSERT INTO _supavisor.tenants
            (id,
             external_id,
             db_host,
             db_port,
             db_database,
             inserted_at,
             updated_at,
             default_parameter_status,
             ip_version,
             upstream_ssl,
             upstream_verify,
             upstream_tls_ca,
             enforce_ssl,
             require_user,
             auth_query,
             default_pool_size,
             sni_hostname,
             default_max_clients,
             client_idle_timeout,
             default_pool_strategy,
             client_heartbeat_interval,
             allow_list)
VALUES      ('7453afc5-d04c-45f7-b0ed-11edd51a3279',
             'dev_tenant',
             'supabase-db',
             5432,
             'postgres',
             '2024-06-28 20:14:29',
             '2024-06-28 20:14:29',
             '{"server_version": "15.1 (Ubuntu 15.1-1.pgdg20.04+1)"}',
             'auto',
             false,
             NULL,
             NULL,
             false,
             false,
             'SELECT rolname, rolpassword FROM pg_authid WHERE rolname=$1;',
             15,
             NULL,
             1000,
             0,
             'fifo',
             60,
             '{0.0.0.0/0,::/0}');

--
-- Data for Name: users; Type: TABLE DATA; Schema: _supavisor; Owner: postgres
--
INSERT INTO _supavisor.users
            (id,
             db_user_alias,
             db_user,
             db_pass_encrypted,
             pool_size,
             mode_type,
             is_manager,
             tenant_external_id,
             inserted_at,
             updated_at,
             pool_checkout_timeout,
             max_clients)
VALUES      ('524e2da5-b4cb-4917-88ed-ad6a304649e8',
             'postgres',
             'postgres',
             '-the-encrypted-password-here-',
             20,
             'transaction',
             true,
             'dev_tenant',
             '2024-06-28 20:14:29',
             '2024-06-28 20:14:29',
             60000,
             NULL);

the tricky part is to get the encrypted password there

Mortalife · 2024-07-01T11:19:00Z

  config :supavisor, Supavisor.Vault,
    ciphers: [
      default: {
        Cloak.Ciphers.AES.GCM,
        tag: "AES.GCM.V1", key: System.get_env("VAULT_ENC_KEY")
      }
    ]

Chatgpt gave me a confidently incorrect answer, but it doesn't look like pgcrypto can be used with AES-GCM, it also suggested some code using the v8 extension to run js, but honestly at this point just sending the curl is probably best option.

Shame pg_net doesn't support PUT otherwise we could've used that in sql to send the request.

Geczy · 2024-07-01T13:39:39Z

so something like this could work?

services:
  init-curl:
    image: curlimages/curl:8.8.0
    command: >
      sh -c "curl -X PUT 'http://supabase-supavisor:4000/api/tenants/dev_tenant' \
      --header 'Accept: */*' \
      --header 'User-Agent: Thunder Client (https://www.thunderclient.com)' \
      --header 'Authorization: Bearer ${SERVICE_SUPABASEANON_KEY}' \
      --header 'Content-Type: application/json' \
      --data-raw '{
        \"tenant\": {
          \"db_host\": \"${POSTGRES_HOST:-supabase-db}\",
          \"db_port\": ${POSTGRES_PORT:-5432},
          \"db_database\": \"${POSTGRES_DB:-postgres}\",
          \"ip_version\": \"auto\",
          \"enforce_ssl\": false,
          \"require_user\": false,
          \"auth_query\": \"SELECT rolname, rolpassword FROM pg_authid WHERE rolname=$1;\",
          \"users\": [
            {
              \"db_user\": \"postgres\",
              \"db_password\": \"${SERVICE_PASSWORD_POSTGRES}\",
              \"pool_size\": 20,
              \"mode_type\": \"transaction\",
              \"is_manager\": true
            }
          ]
        }
      }'"
    depends_on:
      - supabase-supavisor
    entrypoint: [ "sh", "-c" ]

  supabase-supavisor:
    ...
    depends_on:
      supabase-db:
        condition: service_healthy
      init-curl:
        condition: service_completed_successfully

Mortalife · 2024-07-01T13:45:48Z

init-curl:
    image: curlimages/curl:8.8.0

I think the only thing potentially missing is: restart: "no" inside the init-curl service, and I'd probably name the service something like supavisor-setup but yeah. Looks like it'll probably work to me.

One last thing, I wonder if those string replacements will work. If not you may need to link the environment variables to the curl container and pass them through using bash.

https://github.com/coollabsio/coolify/blob/main/templates/compose/supabase.yaml#L1053-L1071

The minio gives an example of how this is done with a docker-compose defined file.

Geczy · 2024-07-01T13:59:20Z

i updated the pr, but needs testing

Geczy · 2024-07-01T14:03:12Z

you can check with

psql -c "SELECT * from _supavisor.tenants;" "postgres://postgres:[email protected]:5432"

Update supabase.yaml

ea93d75

Geczy commented Jun 28, 2024

View reviewed changes

templates/compose/supabase.yaml Outdated Show resolved Hide resolved

Update supabase.yaml

34649ac

Geczy mentioned this pull request Jun 28, 2024

[Feature]: Add pgBouncer as a built-in application #773

Open

1 task

Mortalife reviewed Jun 29, 2024

View reviewed changes

Merge branch 'next' into patch-6

8be0075

Update supabase.yaml

5586c73

Geczy added 2 commits July 1, 2024 09:10

Update supabase.yaml

078e2b7

Update supabase.yaml

a00a76e

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add supavisor to supabase #2712

Add supavisor to supabase #2712

Geczy commented Jun 28, 2024 •

edited

Loading

RobertHH-IS commented Jun 28, 2024

Geczy commented Jun 28, 2024

Mortalife Jun 29, 2024

Geczy Jun 29, 2024

Mortalife Jun 30, 2024

Mortalife Jun 30, 2024

Mortalife Jun 30, 2024

Mortalife Jun 30, 2024

Mortalife commented Jun 30, 2024

Geczy commented Jul 1, 2024 •

edited

Loading

Mortalife commented Jul 1, 2024 •

edited

Loading

Geczy commented Jul 1, 2024

Mortalife commented Jul 1, 2024 •

edited

Loading

Geczy commented Jul 1, 2024

Geczy commented Jul 1, 2024

	case 'PASSWORD':
	$generatedValue = Str::password(symbols: false);
	break;
	case 'PASSWORD_64':
	$generatedValue = Str::password(length: 64, symbols: false);
	break;
	// This is not base64, it's just a random string
	case 'BASE64_64':
	$generatedValue = Str::random(64);
	break;
	case 'BASE64_128':
	$generatedValue = Str::random(128);
	break;
	case 'BASE64':
	case 'BASE64_32':
	$generatedValue = Str::random(32);
	break;
	// This is base64,
	case 'REALBASE64_64':
	$generatedValue = base64_encode(Str::random(64));
	break;
	case 'REALBASE64_128':
	$generatedValue = base64_encode(Str::random(128));
	break;
	case 'REALBASE64':
	case 'REALBASE64_32':
	$generatedValue = base64_encode(Str::random(32));
	break;
	case 'USER':
	$generatedValue = Str::random(16);
	break;
	case 'SUPABASEANON':
	$signingKey = $service->environment_variables()->where('key', 'SERVICE_PASSWORD_JWT')->first();
	if (is_null($signingKey)) {
	return;
	} else {
	$signingKey = $signingKey->value;
	}
	$key = InMemory::plainText($signingKey);
	$algorithm = new Sha256();
	$tokenBuilder = (new Builder(new JoseEncoder(), ChainedFormatter::default()));
	$now = new DateTimeImmutable();
	$now = $now->setTime($now->format('H'), $now->format('i'));
	$token = $tokenBuilder
	->issuedBy('supabase')
	->issuedAt($now)
	->expiresAt($now->modify('+100 year'))
	->withClaim('role', 'anon')
	->getToken($algorithm, $key);
	$generatedValue = $token->toString();
	break;
	case 'SUPABASESERVICE':
	$signingKey = $service->environment_variables()->where('key', 'SERVICE_PASSWORD_JWT')->first();
	if (is_null($signingKey)) {
	return;
	} else {
	$signingKey = $signingKey->value;
	}
	$key = InMemory::plainText($signingKey);
	$algorithm = new Sha256();
	$tokenBuilder = (new Builder(new JoseEncoder(), ChainedFormatter::default()));
	$now = new DateTimeImmutable();
	$now = $now->setTime($now->format('H'), $now->format('i'));
	$token = $tokenBuilder
	->issuedBy('supabase')
	->issuedAt($now)
	->expiresAt($now->modify('+100 year'))
	->withClaim('role', 'service_role')
	->getToken($algorithm, $key);
	$generatedValue = $token->toString();
	break;
	default:
	$generatedValue = Str::random(16);
	break;
	}

	return $generatedValue;
	}

Add supavisor to supabase #2712

Are you sure you want to change the base?

Add supavisor to supabase #2712

Conversation

Geczy commented Jun 28, 2024 • edited Loading

RobertHH-IS commented Jun 28, 2024

Geczy commented Jun 28, 2024

Mortalife Jun 29, 2024

Choose a reason for hiding this comment

Geczy Jun 29, 2024

Choose a reason for hiding this comment

Mortalife Jun 30, 2024

Choose a reason for hiding this comment

Mortalife Jun 30, 2024

Choose a reason for hiding this comment

Mortalife Jun 30, 2024

Choose a reason for hiding this comment

Mortalife Jun 30, 2024

Choose a reason for hiding this comment

Mortalife commented Jun 30, 2024

Geczy commented Jul 1, 2024 • edited Loading

Mortalife commented Jul 1, 2024 • edited Loading

Geczy commented Jul 1, 2024

Mortalife commented Jul 1, 2024 • edited Loading

Geczy commented Jul 1, 2024

Geczy commented Jul 1, 2024

Geczy commented Jun 28, 2024 •

edited

Loading

Geczy commented Jul 1, 2024 •

edited

Loading

Mortalife commented Jul 1, 2024 •

edited

Loading

Mortalife commented Jul 1, 2024 •

edited

Loading