Skip to content

intel/intel-optimization-for-horovod

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Intel® Optimization for Horovod* is the distributed training framework for TensorFlow*. The goal is to make distributed Deep Learning workload run faster and easier to use on Intel GPU devices. It's developed based on latest release version v0.28.1 of public Horovod.

Install

Hardware Requirements

  • Intel® Data Center GPU Max Series, Driver Version: 803
Software Installation requirement
Intel® oneAPI Base Toolkit Install Intel® oneAPI Base Toolkit
TensorFlow Install tensorflow 2.15.1
Intel® Extension for TensorFlow* Install Intel® Extension for TensorFlow*
System Ubuntu 22.04, SUSE Linux Enterprise Server(SLES) 15 SP3/SP4
Python 3.9-3.11
Pip 19.0 or later (requires manylinux2014 support)

Install GPU Drivers

OS Intel GPU Install Intel GPU Driver
Ubuntu 22.04, RedHat 8.6, SLES 15 SP3/SP4 Intel® Data Center GPU Max Series Refer to the Installation Guides for latest driver installation. If install the verified Intel® Data Center GPU Max Series/Intel® Data Center GPU Flex Series 803, please append the specific version after components.

Installation Channel:

Intel® Optimization for Horovod* can be installed through the following channels:

PyPI Source
Install from pip Build from source

Install for GPU

Installing Intel® Optimization for Horovod* with different frameworks is feasible. You could choose Intel® Extension for TensorFlow* as dependency.

  1. Installing Intel® Extension for TensorFlow* and Intel® Optimization for Horovod* with command:
    pip install tensorflow==2.15.1
    pip install --upgrade intel-extension-for-tensorflow[xpu]
    pip install intel-optimization-for-horovod

Running Intel® Optimization for Horovod*

The example commands below show how to run distributed training.

  1. To run on a machine with 2 Intel GPUs, which have 4 titles totally.

    horovodrun -np 4 python train.py
  2. To run on 4 machines with 2 GPUs(4 tiles) each:

    horovodrun -np 16 -H server1:4,server2:4,server3:4,server4:4 python train.py

Running Intel® Optimization for Horovod* with tensorflow on Intel GPU

It is easy to train models with Intel® Extension for TensorFlow. You can refer to tensorflow examples for more details.