Upgrade with docker-compose, execute database migration, PyTorch, TensorFlow >= 2.0, or flex not found.

[mwnu]

Self Checks

• This is only for bug report, if you would like to ask a quesion, please head to Discussions.
• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• Pleas do not modify this template :) and fill in all the required fields.

Dify version

v0.6.0-fix1

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

docker-compose down
docker-compose pull
docker-compose up -d
2.
connect to PostgreSQL and execute the following SQL:

ALTER TABLE dataset_keyword_tables ADD COLUMN data_source_type VARCHAR(255) NOT NULL DEFAULT 'database';
ALTER TABLE embeddings ADD COLUMN provider_name VARCHAR(40) NOT NULL DEFAULT '';
ALTER TABLE embeddings DROP CONSTRAINT embedding_hash_idx;
ALTER TABLE embeddings ADD CONSTRAINT embedding_hash_idx UNIQUE (model_name, hash, provider_name);

Execute the below script to complete the migrate:

docker compose exec api flask convert-to-agent-apps

✔️ Expected Behavior

complete the migrate

❌ Actual Behavior

None of PyTorch, TensorFlow >= 2.0, or Flax have been found. Models won't be available and only tokenizers, configuration and file/data utilities can be used.
Start convert to agent apps.
Congratulations! Converted 0 agent apps.

[dosubot]

Hey @mwnu, great to see you back in the world of code and containers! 🚀

Based on the information you've provided and the context I've gathered, it seems that the Dockerfile for Dify v0.6.0-fix1 does not include specific scripts or commands for installing PyTorch, TensorFlow, or Flax during the Docker image build process. This could be the reason why these libraries are not found during the database migration process.

To resolve this issue, you can modify your Dockerfile or Docker Compose setup to ensure these libraries are installed. Here are the steps you can follow:

1. Dockerfile Adjustments: Add the installation commands for PyTorch, TensorFlow, and Flax in your Dockerfile. You can use the official images or pip commands to install these libraries. For example:

   FROM python:3.8
   # Set up your environment and install other dependencies
   RUN pip install torch==1.8.1 tensorflow>=2.0 flax
   # Continue with the rest of your Dockerfile setup

2. Requirements File: If your project uses a requirements.txt file for managing Python dependencies, add PyTorch, TensorFlow, and Flax to this file. For example:

   torch==1.8.1
   tensorflow>=2.0
   flax
   

   Adjust the versions according to your compatibility needs.

3. Docker Compose: If you're using Docker Compose, ensure that the service responsible for the migration has access to an environment where these libraries are installed. This might involve building a custom image with these dependencies included.

4. Environment Variables: Ensure that any environment variable or configuration needed by these libraries for operation (e.g., CUDA version for GPU support) is correctly set in your Docker environment.

5. Volume Mounts: If your models are stored outside the Docker container, use Docker volumes to mount the model directory inside the container so that the application can access them.

After making these changes, rebuild your Docker images and run the containers to ensure that the issue is resolved. You can test the availability of these libraries by importing them in a Python shell or script running inside the container.

Please note that the exact commands and versions may vary based on your specific project requirements and the base images you are using. Always refer to the official documentation of PyTorch, TensorFlow, and Flax for the most up-to-date installation instructions compatible with your project.

For more information, you can refer to the Dockerfile and config.py in the Dify repository.

I hope this helps! If you have any further questions or run into any issues, feel free to ask.

Sources

• Error on docker-api-1
• api/config.py
• api/Dockerfile

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[crazywoola]

It depends on your application type, but this is a warning, you could ignore it.

Start convert to agent apps.
Congratulations! Converted 0 agent apps.

As this suggests, this execution has no errors.

[takatost]

This is a warning message, you can ignore it.