Skip to content

large files need more than 10 minutes to upload.#8782

Merged
frankie567 merged 1 commit intopolarsource:mainfrom
obrienciaran:s3_files_presign_ttl_increase
Jan 9, 2026
Merged

large files need more than 10 minutes to upload.#8782
frankie567 merged 1 commit intopolarsource:mainfrom
obrienciaran:s3_files_presign_ttl_increase

Conversation

@obrienciaran
Copy link
Contributor

@obrienciaran obrienciaran commented Jan 6, 2026

📋 Summary

Related Issue: Fixes #8745

Increase signed URL expiration time to 3600 seconds to allow large file uploads.

🎯 What

Increases timeout to 1 hour. This balances functionality with security.

🤔 Why

Large files fail to upload before the 10 minute signed URL expires, even with a decent connection.

🔧 How

Increases timeout to 3600 seconds.

🧪 Testing

  • I have tested these changes locally
  • All existing tests pass (uv run task test for backend, pnpm test for frontend)
  • I have added new tests for new functionality
  • I have run linting and type checking (uv run task lint && uv run task lint_types for backend)

Test Instructions

NA

🖼️ Screenshots/Recordings

📝 Additional Notes

NA

✅ Pre-submission Checklist

NA

  • My code follows the project's style guidelines
  • I have performed a self-review of my code
  • I have commented my code where necessary
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have updated the relevant tests
  • All tests pass locally
  • AI/LLM Policy: If I used AI assistance, I have tested and executed the code locally (not just "vibe-coded")

@vercel
Copy link

vercel bot commented Jan 6, 2026

@obrienciaran is attempting to deploy a commit to the polar-sh Team on Vercel.

A member of the Team first needs to authorize it.

@obrienciaran
Copy link
Contributor Author

obrienciaran commented Jan 9, 2026

Is there anybody who could take a look at this 🙏 ? I'm totally blocked by this! The final step before my product goes live is testing the checkout flow with Polar but I can't upload my files!

Maybe @Yopi @psincraian @frankie567 (apologies if I'm tagging the wrong people or missing anything from the PR, but it is a simple one).

@vercel
Copy link

vercel bot commented Jan 9, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

2 Skipped Deployments
Project Deployment Review Updated (UTC)
polar Ignored Ignored Preview Jan 9, 2026 4:12pm
polar-sandbox Ignored Ignored Preview Jan 9, 2026 4:12pm

@frankie567 frankie567 enabled auto-merge (rebase) January 9, 2026 16:12
@Yopi
Copy link
Contributor

Yopi commented Jan 9, 2026

Hi @obrienciaran, thank you for the PR, and solving the issue on your own! I think we might want to rework this or have a different way to upload large files to avoid hitting this timeout when the user has a slow internet connection, but I think for now this works -- if it's enough for you.

@frankie567 frankie567 merged commit 3081f4d into polarsource:main Jan 9, 2026
24 checks passed
@obrienciaran
Copy link
Contributor Author

Thank you very much @Yopi

It's an interesting one - I tried to upload on a few different networks, but each timed out on a 2.9gb upload with the short window. I think the solution of simply increasing the signed URL is sufficient for now. But I look forward to new developments in future 😄 Thanks for jumping on this so fast.

@obrienciaran obrienciaran deleted the s3_files_presign_ttl_increase branch January 9, 2026 16:20
@Yopi
Copy link
Contributor

Yopi commented Jan 9, 2026

No worries. The change should be out in production now @obrienciaran. I hope it's smooth sailing from now on, but otherwise let me know!

@obrienciaran
Copy link
Contributor Author

obrienciaran commented Jan 9, 2026

I just had another look @Yopi , I see that Terraform was overwriting the Pydantic class Settings(BaseSettings) so S3_FILES_PRESIGN_TTL was being reset back to 600 and not being picked up as 3600 in the upload buckets.

I've changed the Terraform render.tf files to 3600 in this PR (ticket) for both the Sandbox and Prod environments. Would you or somebody else be able to check this out if you get a moment? I wasn't able to run tests/terraform plan/apply on this as I don't have the correct permissions/keys for Terraform.

@Yopi
Copy link
Contributor

Yopi commented Jan 10, 2026

I just had another look @Yopi , I see that Terraform was overwriting the Pydantic class Settings(BaseSettings) so S3_FILES_PRESIGN_TTL was being reset back to 600 and not being picked up as 3600 in the upload buckets.

I've changed the Terraform render.tf files to 3600 in this PR (ticket) for both the Sandbox and Prod environments. Would you or somebody else be able to check this out if you get a moment? I wasn't able to run tests/terraform plan/apply on this as I don't have the correct permissions/keys for Terraform.

Should have verified that it was completed yesterday, sorry about that! I have applied the terraform from your other PR now. It looks like it's working with longer file uploads now!

@obrienciaran
Copy link
Contributor Author

Thanks for the approval. It was my fault for not double checking, I didn't realise Terraform overwrites config files like that.

I can confirm the signed URLs are now 1 hour for uploads on my end too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: 403 Forbidden error on large file uploads (Multipart S3 Timeout)

3 participants