Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 31 additions & 0 deletions server/migrations/functions.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
# PostgreSQL functions that are applied, that needs to be both migrated and recreated
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wasn't entirely sure how to play this. I wanted to do the ID generation in postgres to avoid having to retry saving customers if we got unlucky in the random ID generation. This is not really following any existing conventions. I put them here so that we can load them into the test runner as well, and it becomes fairly clear for the future how that works.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was a bit reluctant to have a Postgres function at first but... Can't deny it, that's pretty neat and efficient πŸ˜„ I like it!

# in our tests.

SEQUENCE_CREATE_CUSTOMER_SHORT_ID = "CREATE SEQUENCE customer_short_id_seq START 1;"

# ID generation algorithm based on https://instagram-engineering.com/sharding-ids-at-instagram-1cf5a71e5a5c
FUNCTION_GENERATE_CUSTOMER_SHORT_ID = """
CREATE OR REPLACE FUNCTION generate_customer_short_id(creation_timestamp TIMESTAMP WITH TIME ZONE DEFAULT clock_timestamp())
RETURNS bigint AS $$
DECLARE
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is based on the Instagram ID generation algorithm. We get fairly small, ordered but not leaking user volume, IDs that we can base26-encode to get an alphabetical identifier we can use for invoice numbers.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Instagram ID generation algorithm also has a portion of the ID as a machine shard. We could put that in from the start, as it will become a bit annoying to do a migration in the future, if we believe we will need it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What would that look like in our current setup? Just an arbitrary fixed ID?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exactly, we'd set it to 0 or 1 for now, and then a component of the id would just be fixed. It would allow us to introduce it at some point without migrating the IDs or having two realities.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, let's add it then πŸ‘

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking this over (implications on the size of the ID and size of the base26-encoded), and thought maybe it would make sense to go in the opposite direction. Switching over to seconds instead of milliesconds, and not adding the shard gives us a capacity of 1024 customers/second, and alphabetical IDs that are ~8 characters.

Keeping milliseconds and adding shard IDs we would be on ~11 characters. Not sure where we should be aiming right now, but it does feel a bit overkill.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your call :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I increased it to millisecond precision again, in the event of a spike I don't want to be guilty of it not working. I'll merge this now!

our_epoch bigint := 1672531200000; -- 2023-01-01 in milliseconds
seq_id bigint;
now_millis bigint;
result bigint;
BEGIN
-- Get sequence number modulo 1024 (10 bits)
SELECT nextval('customer_short_id_seq') % 1024 INTO seq_id;

-- Use provided timestamp (defaults to clock_timestamp())
SELECT FLOOR(EXTRACT(EPOCH FROM creation_timestamp) * 1000) INTO now_millis;

-- 42 bits timestamp (milliseconds) | 10 bits sequence = 52 bits total
-- Capacity: 1,024 IDs per millisecond (over 1 million per second)
-- Combine: (timestamp - epoch) << 10 | sequence
result := (now_millis - our_epoch) << 10;
result := result | seq_id;

RETURN result;
END;
$$ LANGUAGE plpgsql;
"""
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
"""add order_settings to organizations with invoice_numbering

Revision ID: 9b0f38cdd25d
Revises: 1ac3957fd2cf
Create Date: 2025-10-23 15:43:43.869159

"""

import sqlalchemy as sa
from alembic import op
from sqlalchemy.dialects import postgresql

# Polar Custom Imports

# revision identifiers, used by Alembic.
revision = "9b0f38cdd25d"
down_revision = "1ac3957fd2cf"
branch_labels: tuple[str] | None = None
depends_on: tuple[str] | None = None


def upgrade() -> None:
op.add_column(
"organizations",
sa.Column(
"order_settings",
postgresql.JSONB(astext_type=sa.Text()),
nullable=True,
),
)

op.execute(
"""
UPDATE organizations
SET order_settings = '{"invoice_numbering": "organization"}'::jsonb
"""
)

op.alter_column("organizations", "order_settings", nullable=False)


def downgrade() -> None:
op.drop_column("organizations", "order_settings")
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
"""add invoice_next_number to customers

Revision ID: 8c9ef8060e95
Revises: 9b0f38cdd25d
Create Date: 2025-10-24 09:15:38.708186

"""

import sqlalchemy as sa
from alembic import op

# Polar Custom Imports

# revision identifiers, used by Alembic.
revision = "8c9ef8060e95"
down_revision = "9b0f38cdd25d"
branch_labels: tuple[str] | None = None
depends_on: tuple[str] | None = None


def upgrade() -> None:
op.add_column(
"customers",
sa.Column("invoice_next_number", sa.Integer(), nullable=True),
)
op.execute(
"UPDATE customers SET invoice_next_number = 1 WHERE invoice_next_number IS NULL"
)
op.alter_column("customers", "invoice_next_number", nullable=False)


def downgrade() -> None:
op.drop_column("customers", "invoice_next_number")
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
"""add short_id to customers

Revision ID: aee75623c4bb
Revises: 8c9ef8060e95
Create Date: 2025-10-24 11:39:54.946342

"""

import sqlalchemy as sa
from alembic import op

# Polar Custom Imports
from migrations.functions import (
FUNCTION_GENERATE_CUSTOMER_SHORT_ID,
SEQUENCE_CREATE_CUSTOMER_SHORT_ID,
)

# revision identifiers, used by Alembic.
revision = "aee75623c4bb"
down_revision = "8c9ef8060e95"
branch_labels: tuple[str] | None = None
depends_on: tuple[str] | None = None


def upgrade() -> None:
op.execute(SEQUENCE_CREATE_CUSTOMER_SHORT_ID)

# Create Instagram-style ID generator
# Combines timestamp (milliseconds since epoch) + sequence number
op.execute(FUNCTION_GENERATE_CUSTOMER_SHORT_ID)

op.add_column(
"customers",
sa.Column("short_id", sa.BigInteger(), nullable=True),
)

op.execute(
"""
UPDATE customers
SET short_id = generate_customer_short_id(created_at);
"""
)

op.alter_column("customers", "short_id", nullable=False)

op.execute(
"""
ALTER TABLE customers
ALTER COLUMN short_id SET DEFAULT generate_customer_short_id();
"""
)

op.create_unique_constraint(
op.f("customers_organization_id_short_id_key"),
"customers",
["organization_id", "short_id"],
)
op.create_index(
op.f("ix_customers_short_id"), "customers", ["short_id"], unique=False
)
# ### end Alembic commands ###


def downgrade() -> None:
# ### commands auto generated by Alembic - please adjust! ###
op.drop_index(op.f("ix_customers_short_id"), table_name="customers")
op.drop_constraint(
op.f("customers_organization_id_short_id_key"), "customers", type_="unique"
)
op.drop_column("customers", "short_id")
op.execute("DROP FUNCTION IF EXISTS generate_customer_short_id();")
op.execute("DROP SEQUENCE IF EXISTS customer_short_id_seq;")
# ### end Alembic commands ###
5 changes: 5 additions & 0 deletions server/polar/enums.py
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,11 @@ def to_stripe(self) -> Literal["always_invoice", "create_prorations"]:
raise ValueError(f"Invalid proration behavior: {self}")


class InvoiceNumbering(StrEnum):
organization = "organization"
customer = "customer"


class TokenType(StrEnum):
client_secret = "polar_client_secret"
client_registration_token = "polar_client_registration_token"
Expand Down
32 changes: 32 additions & 0 deletions server/polar/models/customer.py
Original file line number Diff line number Diff line change
@@ -1,18 +1,21 @@
import dataclasses
import string
import time
from collections.abc import Sequence
from datetime import datetime
from enum import StrEnum
from typing import TYPE_CHECKING, Any
from uuid import UUID

import sqlalchemy as sa
from sqlalchemy import (
TIMESTAMP,
Boolean,
Column,
ColumnElement,
ForeignKey,
Index,
Integer,
String,
UniqueConstraint,
Uuid,
Expand All @@ -36,6 +39,21 @@
from .subscription import Subscription


def short_id_to_base26(short_id: int) -> str:
"""Convert a numeric short_id to an 8-character base-26 string (A-Z)."""
chars = string.ascii_uppercase
result = ""
num = short_id

# Convert to base-26
while num > 0:
result = chars[num % 26] + result
num = num // 26

# Pad with 'A' to ensure 8 characters
return result.rjust(8, "A")


class CustomerOAuthPlatform(StrEnum):
github = "github"
discord = "discord"
Expand Down Expand Up @@ -91,9 +109,16 @@ class Customer(MetadataMixin, RecordModel):
postgresql_nulls_not_distinct=True,
),
UniqueConstraint("organization_id", "external_id"),
UniqueConstraint("organization_id", "short_id"),
)

external_id: Mapped[str | None] = mapped_column(String, nullable=True, default=None)
short_id: Mapped[int] = mapped_column(
sa.BigInteger,
nullable=False,
index=True,
server_default=sa.text("generate_customer_short_id()"),
)
email: Mapped[str] = mapped_column(String(320), nullable=False)
email_verified: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False)
stripe_customer_id: Mapped[str | None] = mapped_column(
Expand Down Expand Up @@ -136,6 +161,8 @@ class Customer(MetadataMixin, RecordModel):
TIMESTAMP(timezone=True), nullable=True, default=None, index=True
)

invoice_next_number: Mapped[int] = mapped_column(Integer, nullable=False, default=1)

organization_id: Mapped[UUID] = mapped_column(
Uuid,
ForeignKey("organizations.id", ondelete="cascade"),
Expand Down Expand Up @@ -214,6 +241,11 @@ def remove_oauth_account(
def oauth_accounts(self) -> dict[str, Any]:
return self._oauth_accounts

@property
def short_id_str(self) -> str:
"""Get the base-26 string representation of the short_id."""
return short_id_to_base26(self.short_id)

@property
def legacy_user_id(self) -> UUID:
return self._legacy_user_id or self.id
Expand Down
19 changes: 18 additions & 1 deletion server/polar/models/organization.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@

from polar.config import settings
from polar.email.sender import DEFAULT_REPLY_TO_EMAIL_ADDRESS, EmailFromReply
from polar.enums import SubscriptionProrationBehavior
from polar.enums import InvoiceNumbering, SubscriptionProrationBehavior
from polar.kit.db.models import RateLimitGroupMixin, RecordModel
from polar.kit.extensions.sqlalchemy import StringEnum

Expand Down Expand Up @@ -74,6 +74,15 @@ class OrganizationSubscriptionSettings(TypedDict):
}


class OrganizationOrderSettings(TypedDict):
invoice_numbering: InvoiceNumbering


_default_order_settings: OrganizationOrderSettings = {
"invoice_numbering": InvoiceNumbering.organization,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Leaving this on organization for now, so we can roll this out and change the defaults after this PR.

}


class OrganizationCustomerEmailSettings(TypedDict):
order_confirmation: bool
subscription_cancellation: bool
Expand Down Expand Up @@ -179,6 +188,10 @@ def account(cls) -> Mapped[Account | None]:
JSONB, nullable=False, default=_default_subscription_settings
)

order_settings: Mapped[OrganizationOrderSettings] = mapped_column(
JSONB, nullable=False, default=_default_order_settings
)

notification_settings: Mapped[OrganizationNotificationSettings] = mapped_column(
JSONB, nullable=False, default=_default_notification_settings
)
Expand Down Expand Up @@ -259,6 +272,10 @@ def proration_behavior(self) -> SubscriptionProrationBehavior:
def benefit_revocation_grace_period(self) -> int:
return self.subscription_settings["benefit_revocation_grace_period"]

@property
def invoice_numbering(self) -> InvoiceNumbering:
return InvoiceNumbering(self.order_settings["invoice_numbering"])

@declared_attr
def all_products(cls) -> Mapped[list["Product"]]:
return relationship("Product", lazy="raise", back_populates="organization")
Expand Down
8 changes: 4 additions & 4 deletions server/polar/order/service.py
Original file line number Diff line number Diff line change
Expand Up @@ -578,7 +578,7 @@ async def _create_order_from_checkout(

organization = checkout.organization
invoice_number = await organization_service.get_next_invoice_number(
session, organization
session, organization, customer
)

repository = OrderRepository.from_session(session)
Expand Down Expand Up @@ -699,7 +699,7 @@ async def create_subscription_order(
tax_rate = tax_calculation["tax_rate"]

invoice_number = await organization_service.get_next_invoice_number(
session, subscription.organization
session, subscription.organization, customer
)

total_amount = subtotal_amount - discount_amount + tax_amount
Expand Down Expand Up @@ -791,7 +791,7 @@ async def create_trial_order(

organization = subscription.organization
invoice_number = await organization_service.get_next_invoice_number(
session, organization
session, organization, customer
)

repository = OrderRepository.from_session(session)
Expand Down Expand Up @@ -1344,7 +1344,7 @@ async def create_order_from_stripe(
)

invoice_number = await organization_service.get_next_invoice_number(
session, subscription.organization
session, subscription.organization, customer
)

repository = OrderRepository.from_session(session)
Expand Down
Loading
Loading