Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docker Documentation #1251

Merged
merged 31 commits into from
Jan 15, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
5155213
Updates to the Dockerfiles making them more secure. update to the ins…
Jan 14, 2024
87b5a8e
updates to the docker documentation
Jan 14, 2024
0919fb2
clean up before commit
Jan 14, 2024
98491b0
added docker instructions on how to test documentation prior to commit.
Jan 14, 2024
a93d12f
fixed the title heading in the installation.md
Jan 14, 2024
351e050
removed alternate yarn install from Dockerfile.dev
Jan 14, 2024
d6a22ea
Merge branch 'main' into dockerDocs
r3d91ll Jan 14, 2024
3acdc7e
Update sidebars.js
r3d91ll Jan 14, 2024
1c863c3
cleaned up the lint errors from the pre-commit-check.
Jan 14, 2024
2180c03
removed bad edit from sidebars.js
Jan 14, 2024
8d89e27
fixing git conflict with sidebars.js file
Jan 14, 2024
71d94be
Update Installation.md
r3d91ll Jan 14, 2024
784df48
Update Installation.md
r3d91ll Jan 14, 2024
3e1deaf
Merge branch 'main' into dockerDocs
r3d91ll Jan 14, 2024
8a7d6e6
Update Installation.md
r3d91ll Jan 14, 2024
7868951
Update Installation.md
r3d91ll Jan 14, 2024
403fbad
fixed bad link to Dockerfiles.md
Jan 14, 2024
bbd996e
Fix trailing whitespace and formatting issues
Jan 14, 2024
3ee5b9a
Update link to Dockerfiles.md
Jan 14, 2024
59248aa
somtimes its really hard to find that lost i.
Jan 14, 2024
bd80955
trailing whitespace and EOF issues resolved
Jan 14, 2024
ce33466
fixed broken link
Jan 14, 2024
ed2904e
trailing space an broken link
Jan 15, 2024
f95c5eb
removed broken links. will update one merged.
Jan 15, 2024
1888ba2
Merge branch 'main' into dockerDocs
r3d91ll Jan 15, 2024
41f9bd6
fixed glaring error about building an image from a particular branch …
Jan 15, 2024
79ec267
Merge branch 'dockerDocs' of github.com:r3d91ll/autogen into dockerDocs
Jan 15, 2024
b297ca5
Merge branch 'main' into dockerDocs
r3d91ll Jan 15, 2024
2a7159e
Merge branch 'microsoft:main' into dockerDocs
r3d91ll Jan 15, 2024
fa0c96d
removed tic mark typo from contribute.md
Jan 15, 2024
296d5a3
Merge branch 'dockerDocs' of github.com:r3d91ll/autogen into dockerDocs
Jan 15, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 18 additions & 13 deletions samples/dockers/Dockerfile.base
Original file line number Diff line number Diff line change
@@ -1,21 +1,26 @@
FROM python:3.11-slim-bookworm

RUN : \
&& apt-get update \
RUN apt-get update \
&& DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
software-properties-common \
&& DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
python3-venv \
software-properties-common sudo\
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/* \
&& :
&& rm -rf /var/lib/apt/lists/*

# Setup a non-root user 'autogen' with sudo access
RUN adduser --disabled-password --gecos '' autogen
RUN adduser autogen sudo
RUN echo '%sudo ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers
USER autogen
WORKDIR /home/autogen

RUN python3 -m venv /venv
ENV PATH=/venv/bin:$PATH
EXPOSE 8081
# Set environment variable if needed
# ENV OPENAI_API_KEY="{OpenAI-API-Key}"

RUN cd /venv; pip install pyautogen
# Pre-load popular packages as per https://learnpython.com/blog/most-popular-python-packages/
RUN pip install numpy pandas matplotlib seaborn scikit-learn requests urllib3 nltk pillow pytest beautifulsoup4
RUN pip install pyautogen numpy pandas matplotlib seaborn scikit-learn requests urllib3 nltk pillow pytest beautifulsoup4

# Expose port
# EXPOSE 8081

ENTRYPOINT []
# Start Command
CMD [ "/bin/bash" ]
41 changes: 25 additions & 16 deletions samples/dockers/Dockerfile.dev
Original file line number Diff line number Diff line change
@@ -1,33 +1,42 @@
# basic setup
# Basic setup
FROM python:3.11-slim-bookworm

# Update and install necessary packages
RUN apt-get update && apt-get -y update
RUN apt-get install -y sudo git npm
# added vim and nano for convenience
RUN apt-get install -y sudo git npm vim nano curl

# Setup user to not run as root
RUN adduser --disabled-password --gecos '' autogen-dev
RUN adduser autogen-dev sudo
# Setup a non-root user 'autogen' with sudo access
RUN adduser --disabled-password --gecos '' autogen
RUN adduser autogen sudo
RUN echo '%sudo ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers
USER autogen-dev
USER autogen
WORKDIR /home/autogen

# Set environment variable
# ENV OPENAI_API_KEY="{OpenAI-API-Key}"

# Pull repo
RUN cd /home/autogen-dev && git clone https://github.com/microsoft/autogen.git
WORKDIR /home/autogen-dev/autogen
# Clone the AutoGen repository
RUN git clone https://github.com/microsoft/autogen.git /home/autogen/autogen
WORKDIR /home/autogen/autogen

# Install autogen in editable mode (Note: extra components can be installed if needed)
# Install AutoGen in editable mode with extra components
RUN sudo pip install -e .[test,teachable,lmm,retrievechat,mathchat,blendsearch]

# Install precommit hooks
# Install pre-commit hooks
RUN pre-commit install

# For docs
# Setup Docusaurus and Yarn for the documentation website
RUN sudo npm install --global yarn
RUN sudo pip install pydoc-markdown
RUN cd website
RUN yarn install --frozen-lockfile --ignore-engines

# Pre-load popular packages as per https://learnpython.com/blog/most-popular-python-packages/
# Exposes the Yarn port for Docusaurus
EXPOSE 3000

# Pre-load popular Python packages
RUN pip install numpy pandas matplotlib seaborn scikit-learn requests urllib3 nltk pillow pytest beautifulsoup4

# override default image starting point
CMD /bin/bash
ENTRYPOINT []
# Set the default command to bash
CMD ["/bin/bash"]
34 changes: 21 additions & 13 deletions samples/dockers/Dockerfile.full
Original file line number Diff line number Diff line change
@@ -1,21 +1,29 @@
FROM python:3.11-slim-bookworm

RUN : \
&& apt-get update \
# Update and install dependencies
RUN apt-get update \
&& DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
software-properties-common \
&& DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
python3-venv \
software-properties-common sudo\
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/* \
&& :
&& rm -rf /var/lib/apt/lists/*

RUN python3 -m venv /venv
ENV PATH=/venv/bin:$PATH
EXPOSE 8081
# Setup a non-root user 'autogen' with sudo access
RUN adduser --disabled-password --gecos '' autogen
RUN adduser autogen sudo
RUN echo '%sudo ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers
USER autogen
WORKDIR /home/autogen

# Set environment variable if needed
# ENV OPENAI_API_KEY="{OpenAI-API-Key}"

RUN cd /venv; pip install pyautogen[teachable,lmm,retrievechat,mathchat,blendsearch] autogenra
# Pre-load popular packages as per https://learnpython.com/blog/most-popular-python-packages/
# Install Python packages
RUN pip install --upgrade pip
RUN pip install pyautogen[teachable,lmm,retrievechat,mathchat,blendsearch] autogenra
RUN pip install numpy pandas matplotlib seaborn scikit-learn requests urllib3 nltk pillow pytest beautifulsoup4

ENTRYPOINT []
# Expose port
EXPOSE 8081

# Start Command
CMD ["/bin/bash"]
81 changes: 81 additions & 0 deletions samples/dockers/Dockerfiles.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
# README for AutoGen Docker Samples

Welcome to the `autogen/samples/dockers` directory! Here you'll find Dockerfiles that are essential for setting up your AutoGen development environment. Each Dockerfile is tailored for different use cases and requirements. Below is a brief overview of each and how you can utilize them effectively.

## Dockerfile Descriptions

### Dockerfile.base

- **Purpose**: This Dockerfile is designed for basic setups. It includes common Python libraries and essential dependencies required for general usage of AutoGen.
- **Usage**: Ideal for those just starting with AutoGen or for general-purpose applications.
- **Building the Image**: Run `docker build -f Dockerfile.base -t autogen_base_img .` in this directory.

### Dockerfile.full

- **Purpose**: This Dockerfile is for advanced features. It includes additional dependencies and is configured for more complex or feature-rich AutoGen applications.
- **Usage**: Suited for advanced users who need the full range of AutoGen's capabilities.
- **Building the Image**: Execute `docker build -f Dockerfile.full -t autogen_full_img .`.

### Dockerfile.dev

- **Purpose**: Tailored for AutoGen project developers, this Dockerfile includes tools and configurations aiding in development and contribution.
- **Usage**: Recommended for developers who are contributing to the AutoGen project.
- **Building the Image**: Run `docker build -f Dockerfile.dev -t autogen_dev_img .`.
- **Before using**: We highly encourage all potential contributors to read the [AutoGen Contributing](https://microsoft.github.io/autogen/docs/Contribute) page prior to submitting any pull requests.

## Customizing Dockerfiles

Feel free to modify these Dockerfiles for your specific project needs. Here are some common customizations:

- **Adding New Dependencies**: If your project requires additional Python packages, you can add them using the `RUN pip install` command.
- **Changing the Base Image**: You may change the base image (e.g., from a Python image to an Ubuntu image) to suit your project's requirements.
- **Changing the Python version**: do you need a different version of python other than 3.11. Just update the first line of each of the Dockerfiles like so:
`FROM python:3.11-slim-bookworm` to `FROM python:3.10-slim-bookworm`
- **Setting Environment Variables**: Add environment variables using the `ENV` command for any application-specific configurations. We have prestaged the line needed to inject your OpenAI_key into the docker environment as a environmental variable. Others can be staged in the same way. Just uncomment the line.
`# ENV OPENAI_API_KEY="{OpenAI-API-Key}"` to `ENV OPENAI_API_KEY="{OpenAI-API-Key}"`
- **Need a less "Advanced" Autogen build**: If the Dockerfile.full is to much but you need more than advanced then update this line in the Dockerfile.full file.
`RUN pip install pyautogen[teachable,lmm,retrievechat,mathchat,blendsearch] autogenra` to install just what you need. `RUN pip install pyautogen[retrievechat,blendsearch] autogenra`
- **Can't Dev without your favorite CLI tool**: if you need particular OS tools to be installed in your Docker container you can add those packages here right after the sudo for the Dockerfile.base and Dockerfile.full files. In the example below we are installing net-tools and vim to the environment.

```code
RUN apt-get update \
&& DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
software-properties-common sudo net-tools vim\
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*
```

### Managing Your Docker Environment

After customizing your Dockerfile, build the Docker image using the `docker build` command as shown above. To run a container based on your new image, use:

```bash
docker run -it -v $(pwd)/your_app:/app your_image_name
```

Replace `your_app` with your application directory and `your_image_name` with the name of the image you built.

#### Closing for the Day

- **Exit the container**: Type `exit`.
- **Stop the container**: Use `docker stop {application_project_name}`.

#### Resuming Work

- **Restart the container**: Use `docker start {application_project_name}`.
- **Access the container**: Execute `sudo docker exec -it {application_project_name} bash`.
- **Reactivate the environment**: Run `source /usr/src/app/autogen_env/bin/activate`.

### Useful Docker Commands

- **View running containers**: `docker ps -a`.
- **View Docker images**: `docker images`.
- **Restart container setup**: Stop (`docker stop my_container`), remove the container (`docker rm my_container`), and remove the image (`docker rmi my_image:latest`).

#### Troubleshooting Common Issues

- Check Docker daemon, port conflicts, and permissions issues.

#### Additional Resources

For more information on Docker usage and best practices, refer to the [official Docker documentation](https://docs.docker.com).
Loading
Loading