Skip to content

Latest commit

 

History

History
234 lines (147 loc) · 6.76 KB

installation.md

File metadata and controls

234 lines (147 loc) · 6.76 KB

Installation

The simplest way to try SecretFlow is to use offical docker image which ships with SecretFlow binary.

Or you could install SecretFlow via Python Package Index.

For advanced users, you could install SecretFlow from source.

For Windows users, you could install SecretFlow base WSL2.

After installation, don't forget to have a quick try to check if SecretFlow is good to go.

Additional: For users with available GPU devices, you could try GPU support.

Environment

Python：3.8

pip: >= 19.3

OS: CentOS 7, Ubuntu 18.04

CPU/Memory: recommended minimum requirement is 8C16G.

Option 1: from pypi

For users who want to try SecretFlow, you can install the current release.

Note that it requires python version == 3.8, you can create a virtual environment with conda if not satisfied.

conda create -n sf python=3.8
conda activate sf

After that, please use pip to install SecretFlow.

pip install -U secretflow

Option 2: from docker

You can also use SecretFlow Docker image to give SecretFlow a quick try.

The latest version can be obtained from secretflow tags.

export version={SecretFlow version}

for example

export version=0.6.13b1

then run the image.

docker run -it secretflow/secretflow-anolis8:${version}

Option 3: from source

  1. Download code and set up Python virtual environment.
git clone https://github.com/secretflow/secretflow.git
cd secretflow

conda create -n secretflow python=3.8
conda activate secretflow
  1. Install SecretFlow
python setup.py bdist_wheel

pip install dist/*.whl

Option 4: from WSL

SecretFlow does not support Windows directly now, however, a Windows user can use secretFlow by WSL(Windows Subsystem for Linux).

  1. Install WSL2 in Windows
  • You are supposed to follow the guide_zh or guide_en to install WSL(Windows Subsystem for Linux) in your Windows and make sure that the version of WSL is 2.
  • As for the distribution of GNU/Linux, Ubuntu is recommended.
  1. Install Anaconda in WSL

Just follow the installation of anaconda in GNU/Linux to install anaconda in your WSL.

  1. Install secretflow
  • create conda environment
conda create -n sf python=3.8
  • activate the environment
conda activate sf
  • use pip to install SecretFlow.
pip install -U secretflow
  1. Use WSL to develop your application

After set up of SecretFlow in WSL, you can use Pycharm Professional to Configure an interpreter using WSL or Visual Studio Code with WSL to use SecretFlow in Windows Operating System.

A quick try

Try your first SecretFlow program.

Import secretflow package.

>>> import secretflow as sf

Create a local cluster with parties alice, bob and carol.

>>> sf.init(parties=['alice', 'bob', 'carol'], address='local')

Create alice's PYU device, which can process alice's data.

>>> alice_device = sf.PYU('alice')

Let alice say hello world.

>>> message_from_alice = alice_device(lambda x:x)("Hello World!")

Print the message.

>>> message_from_alice
<secretflow.device.device.pyu.PYUObject object at 0x7fdec24a15b0>

We see that the message on alice device is a PYU Object at deriver program.

Print the text at the driver by revealing the message.

>>> print(sf.reveal(message_from_alice))
Hello World!

GPU support

Before you read

If you don't need to use GPU, please ignore this section and refer to quick try.

Introduction

NVIDIA's CUDA and cuDNN are typically used to accelerate the training and inference of machine learning models. Tensoflow and PyTorch, two widely-used machine learning frameworks, both intergrate the GPU support. In SecretFlow, PyTorch and Tensorflow are adopted as the backends for Federated Learning, of which the performance can be boosted with GPU support.

If you want to use GPU acceleration in SecretFlow, you need to complete the Preparations first to set up the environment.

In the following, there are two options to run the GPU-version SecretFlow:

  1. Use the offical GPU docker image

  2. Build the GPU docker image by yourself.

After the image is ready, you could run the container and try GPU support.

Preparations

  1. Make sure your NVIDIA driver is available and meet the version requirements:

Driver version must be >= 525.60.13 for CUDA 12 and >= 450.80.02 for CUDA 11 on Linux.

You could run NVIDIA System Management Interface (nvidia-smi) to make sure your NVIDIA driver is available and meet the version requirements.

nvidia-smi

NOTE: We currently only supply the GPU Docker image based on CUDA11. When the GPU packages of PyTorch and TensorFlow based on CUDA12 are available, we will supply the GPU Docker image based on CUDA12.

  1. Follow the NVIDIA official guide to setup NVIDIA Container Toolkit on your distributions.

After the environment is set up, you could build/run the image.

Option 1: Get the GPU docker image from the SecretFlow repository

The GPU Docker image of SecretFlow is available on the SecretFlow repository at Dockerhub and you can run the following command to get the latest GPU docker image.

docker pull secretflow/secretflow-gpu

For more information, please visit the GPU docker images at Dockerhub.

Option 2: Build the GPU docker image by yourself

You could also build the Docker image by yourself.

  1. Download code
git clone https://github.com/secretflow/secretflow.git
cd secretflow/docker
  1. Use a dockerfile file to construct the image
docker build -f  secretflow-gpu.Dockerfile -t secretflow-gpu .

Run a container and Check GPU

  1. Run a container
docker container run --runtime=nvidia  -it --gpus all secretflow-gpu bash

NOTE: The following two parameters are necessary:

  • --runtime=nvidia
  • --gpus all
  1. After the container is running, you can use the jupyter notebook GPU Check to check the access of Tensorflow and PyTorch for NVIDIA GPUs inside the container.