fix: update render.yaml to being compatible with langflow 1.0

[nicoloboschi]

render.yaml is outdated and it doesn't work with the latest docker image

Fixes #2723

[github-actions]

Pull Request Validation Report

This comment is automatically generated by Conventional PR

Whitelist Report

Whitelist	Active	Result
Pull request is a draft and should be ignored	✅	❌
Pull request is made by a whitelisted user and should be ignored	❌	❌
Pull request is submitted by a bot and should be ignored	✅	❌
Pull request is submitted by administrators and should be ignored	❌	❌

Result

Pull request does not satisfy any enabled whitelist criteria. Pull request will be validated.

Validation Report

Validation	Active	Result
All commits in this pull request has valid messages	❌	✅
Pull request does not introduce too many changes	❌	✅
Pull request has a valid title	✅	✅
Pull request has mentioned issues	❌	✅
Pull request has valid branch name	❌	✅
Pull request should have a non-empty body	✅	✅

Result

Pull request satisfies all enabled pull request rules.

_{Last Modified at 30 Jul 24 12:54 UTC}

[aws-amplify-sa-east-1]

This pull request is automatically being deployed by Amplify Hosting (learn more).

Access this pull request here: https://pr-3071.dmtpw4p5recq1.amplifyapp.com

[ogabrielluiz]

LGTM

[conradchu]

@nicoloboschi This commit seems to break all existing Render installs (like ours) as it changes the sqlite DB folder. I'm trying to figure out the fix and/or restoring from backups. As it is, it looks like our Sqlite DB got wiped.

[nicoloboschi]

hey @conradchu thanks for your feedback. I'm sorry to hear that.

The previous render file was not compatible with 1.0.x docker image layout. Which version were you running?

For production deployment I'd recommend to not rely on "main" branch files but use your own repository and apply changes once you tested it carefully.
I'm not saying this change it was intentionally meant to break existing instances.

[conradchu]

Hi @nicoloboschi

We wanted to inform you that we have identified the cause of the recent data loss and are taking steps to prevent it from happening again in the future. Unfortunately, this incident resulted in the loss of some data.

Cause of the Issue

On July 1, 2024, Langflow updated their Dockerfile to use a different home path, which was deployed on our Render server. This change altered our environment variable, causing our SQLite database folder to move to a new path: /app/data. Although we had manually backed up some components and projects to JSON, we were not aware of the full extent of the changes and assumed Langflow had deployed a breaking change.

Key Events Leading to Data Loss

• Our disk backups were only configured to backup data to a specific path: /home/user, as per Langflow's render.yaml configuration.
• Langflow's committers only realized on July 29, 2024, that there was a bug in their render.yaml configuration, where the home path did not match the Dockerfile. They updated the path from /home/user to /app/data.
• By default, Render is set to re-deploy the server whenever a change is made to render.yaml.
• When the service was re-deployed, our SQLite database was lost because it was being written to a folder on an ephemeral disk on the instance.
• As a result, any attempted restores from the disk backups did not yield any previous data since our SQLite database was using an ephemeral disk.

Remediation and Future Plans

To prevent similar incidents in the future, we have:

• Forked the Langflow repository and disabled auto-deploys.
• Decided to configure our Langflow to use Postgres instead of SQLite, following the same pattern to backup the Postgres database daily to S3 like our other Render databases. This will provide better separation between the data and the app instance.

Key Takeaways

This incident highlights two important lessons:

1. Be cautious with SQLite on cloud instances: Although SQLite is a simple, disk-based database, using it with cloud instances with ephemeral and persistent disk configurations can be tricky. It is crucial to ensure that the SQLite database is being run on the proper mount path of a persisted disk to avoid data loss.
2. Use forked repositories: Using forked repositories is essential, even if we have confidence in the development team. Langflow's dev team may not be aware of the subtleties of each PaaS configuration, which can lead to unintended consequences. By using a forked repository, we can better manage and control the configuration changes and ensure the stability of our application.