Skip to content

Contributing

Nicholas Tindle edited this page Aug 8, 2024 · 27 revisions

Welcome to the AutoGPT project(s)! We appreciate your interest in contributing. This guide is intended to help (aspiring) contributors find their way, and to increase the net value of individual contributions.

Before you get into contributing to the project, it's good to be aware of our big-picture goals. These are documented in our Roadmap.

Lots of ways to contribute 🚀

This Contribution Guide is not just about writing code. The code is a means to an end, and our end goals require an all-round approach. We encourage every participant to contribute in multiple areas, because it helps to connect the different aspects of the project in a meaningful way.

  1. Writing Code 🧑🏼‍💻 – Help yourself and others with functional or technical improvements.

  2. Improving Documentation 📖 – Writing and maintaining user & developer documentation is crucial for the usability and accessibility of our projects.

  3. Managing issues and PRs 🕵🏼 – Help the project's maintainers to identify and refine valuable contributions.

  4. Community Support 🧑🏼‍🚒 – Pay it forward by helping others to get going in this new and exciting space, and create an inclusive community where no enthusiasm is wasted!

  5. Creating benchmarking challenges 📊 – Build automated tests for tasks that you want AutoGPT to be able to do.

  6. Other ways to contribute 🫱🏼‍🫲🏼

While code contributions are what drive the project forward most immediately, the above items are equally important to ensure long-term viability and user/developer adoption. We appreciate your efforts in any of these areas!

Code of Conduct

We expect any participant to adhere to our Code of Conduct. This way, we maintain a respectful and productive environment. Please familiarize yourself with the CoC if you're unsure what is considered appropriate behavior.

Related Links 🔗

Writing Code 🧑🏼‍💻

If you're ready to contribute code, do what you do best! Meanwhile, also make sure to adhere to our code guidelines and thoroughly test your changes before submitting them for review.

House rules

  1. Maintainers have limited time. Please be considerate of this.
    • Do not ask them for programming help. They are busy enough debugging their own code. ;-)
    • Please don't make more non-trivial changes than needed. This makes PRs harder to review.
    • Don't submit non-functional PRs (e.g. typo fixes, fixes for non-issues) under 10 lines in size.
  2. Pull requests must have a description stating what they change and why, or they will not be looked at.
  3. Big, complicated, or architectural changes should be checked with a Maintainer first. Just ping them with your idea in the dev channel, or make sure it is in line with a roadmapped/approved issue or proposal. See also our roadmap and project board.
  4. If you are working on a PR related to a GitHub issue, please state so! to protect yourself and others from doing duplicate work.

Pre-development checklist for enhancements

When making bigger changes, best be sure your efforts don't go to waste. If you follow this checklist, the chance that your pull request won't be accepted should be small:

  • Read the roadmap item that relates to your objective, and make sure your proposal aligns with it. If there is no related roadmap item, pitch your idea in the dev channel.
  • Check for related or similar pull requests. If one already exists, please review it. Creating another PR for the same thing is not desirable, except if:
    • the existing PR doesn't completely implement your idea, feel free to either engage on the existing PR or submit an additional one yourself.
    • the existing PR is not mergeable and the PR hasn't been updated in a while, feel free to create a new PR. Be sure to cross-link between your new PR and the existing one. This also makes it easier for maintainers to identify PRs that can be closed.
  • Create a draft PR describing the issue you want to address and the changes you want to make, and ask a Maintainer for feedback
    • If there are any GitHub issues that would be resolved by your proposed changes, link them in the draft PR

Notes

  • This checklist does not apply to simple bugfixes and small changes
  • If you have an experimental or otherwise daring idea, consider submitting it as an experiment! Generative and Agentic AI technology is a rapidly growing and evolving field. We are very interested to see experimental proposals and implementations, even if we can't always merge them into the main application. The experiments/ folder in the repo is not subject to the same practicality and integration requirements, so there you have much more freedom to develop and demonstrate techniques and technologies.

Code guidelines

Install our pre-commit config to automate code formatting and checks before each commit:

  1. Install the pre-commit package with pip install --user pre-commit.
  2. Run pre-commit install in the project's top-level folder

This section is intended to serve as a reference for code style. Don't feel obliged to read it all, but it can (and should) be referred to in feedback on pull requests.

Contents:

Formatting & Linting

We maintain a consistent code style using isort, Black, and Flake8. The settings for these linters are defined separately for each subproject. Automatic formatting (isort + Black) and linting (Flake8) is included in the pre-commit config, and checks for all three are included in our CI.

Type checking

We use Pyright for type-checking, both in pre-commit and in CI. If you use VSCode, get type error indications in your editor by setting Type Checking Mode to basic.

Type Checking Mode setting in VSCode, with 'basic' value selected

Typing guidelines:

  • Function parameters and return types should be annotated. An exception to this is Pytest fixtures that yield their output, where you should omit the return type.
  • Class attributes should always be typed, either directly by declaring them on the class, or indirectly by assigning a typed value to them in __init__ or a method.
  • TypeVar T is always an unbounded type variable. For bounded type variables, use a different letter (combination), preferably an acronym of the bounding type. Example: ST = TypeVar("ST", bound=SomeType)
  • Minimize use of # type: ignore. If you have information about a variable that the type checker can't infer, cast or assert is preferred.
  • If you put a # type: ignore somewhere because of a bug/limitation in the type checker, find the related issue and link it in a comment on the line above. (example)

Naming

  • Class (attribute) names and function names should always be descriptive. We'd rather have readable, descriptive names than short, unintelligible ones. We're less critical about names of local variables.
  • Classes, functions, variables, and constants should begin with an underscore _ if they are not supposed to be exported from their respective namespace.

Ordering of namespace contents

We prefer top-down ordering based on type, usage, and "public"/"private" status:

In a module:

  1. Imports

    • Ordering is enforced by isort
    • if TYPE_CHECKING: should always be at the end of the import section
    • Imports from within the same scoped module should be relative (from .utils import fmt_line)
  2. Constants

  3. Class definitions

  4. Functions

    1. Public (def function())
    2. Private (def _function())

    Function A uses function B -> function B is defined after function A

In a class:

  1. Constants

  2. Attribute declarations

  3. Methods

    1. __init__
    2. @property methods
    3. Magic methods (e.g. __len__, __bool__)
    4. Public (def method(..))
    5. Private (def _method(..))

    Method A uses method B -> method B is defined after method A

Test coverage

If you are adding new functionality, make sure it is covered by tests – preferably integration tests. When writing new tests, do this using pytest rather than unittest-style. For instructions on running tests, see Running tests.

Unit tests should be placed in the same directory as the thing they are testing: e.g. unit tests for some/dir/agent.py should be in some/dir/agent_test.py.

For tests involving calls to paid APIs, we use pytest-recording to cache requests and responses in cassettes. When modifying prompts, update the cassettes accordingly.

  • No prompt change: No API tokens are consumed.
  • Prompt change without a matching cached request: Add an API key to .env, and the tests will make a real API call.

In our CI pipeline, Pytest uses cassettes, and an API key is only available for CI runs on branches in the main repo.

Refactoring

Refactoring is not a problem, as long as it stays within the scope of the changes that form the core of your PR.

Other code guidelines

  • For file paths, preferably use pathlib.Path rather than str.

  • For function parameters of object types (list, dict, BaseModel, etc.), use param: Optional[list[str]] = None rather than param: list[str] = []. Otherwise, the default value (implicitly) becomes unwanted shared state:

    def fun(a = []):
      a.append("hello")
      print(a)
    
    fun()  # ['hello']
    fun()  # ['hello', 'hello']
    fun()  # ['hello', 'hello', 'hello']

Prompts, Performance & Benchmarking

There are no silver bullets or absolute optima when it comes to prompt engineering. LLMs are like black boxes, and will remain as such for the foreseeable future.

Some advice:

  • Prompt engineering is often a matter of trial and error. Don't give up too easily ;-)
  • Use agbenchmark to evaluate the performance impact of your changes

Resources

Commits & Pull Requests

To keep the commit history organized and transparent, we use conventional commit and pull request titles. See also the Conventional Commits specification. Top-level scopes:

  • agent – the AutoGPT agent, located in autogpt/
  • forge – the AutoGPT Forge SDK, located in forge/
  • frontend – the Agent Protocol front-end, located in frontend/
  • benchmark – the AGBenchmark subproject, located in benchmark/
  • autogpt_server – the AutoGPT Server subproject, located in rnd/autogpt_server/
  • autogpt_builder – the AutoGPT Builder front-end (for the autogpt_server), located in rnd/autogpt_builder/
  • docs – the user documentation website, located in docs/

The commit message as a whole should give a good overview of the changes made. The same applies to pull request descriptions.

Merge Commit Messages

By default, GitHub adds a list of all the PR's commits and their respective messages to the merge commit message. This often isn't a great representation of the changes made, e.g. when changes were reverted or changed again while iterating on the pull request.

Template workflow:

  1. Keep a compiled list of changes in the PR description. Hint: this is easy to do from a list of clear commit messages.
  2. On merge, copy this list to the merge commit message.

Pushing to master

For those with write access and bypass privileges

In some cases, it is appropriate to push changes directly to master:

  • Small, trivial changes
  • Urgent fixes for immediate breakage

It is up to you to determine whether a small set of changes is not worth others' time to review. If you are unsure, just ask in the dev channel: this is often still quicker than opening a pull request.

If you are pushing directly to master, it is very important that you do due diligence, double-checking and testing your changes before pushing them. If you don't do this and something breaks because of it, you risk having your bypass privileges revoked.

Force-pushing

  • Force-pushing to master or any other public release channel is a hard no-go. There should be branch protection rules in place to prohibit it.
  • Force-pushing over commits on a pull request that have already been reviewed unlinks the review comments, so please don't do this.

Improving documentation 📖

We have documentation in various places:

  • User docs and basic developer docs at docs.agpt.co
  • Internal & contributor docs in this wiki
  • README.md and CONTRIBUTING.md in the AutoGPT repo

These docs should all work together to guide anyone interested in using or contributing to AutoGPT. Writing good docs and maintaining them isn't easy though, and it is often forgotten when building new features, so heroes who want to contribute to our docs are very welcome! You can:

  • Edit documentation: Review and enhance existing documentation for clarity and accuracy.
  • Create new docs: If you identify gaps or missing information, consider adding new documentation to address these areas.

Keep in mind our House Rules specifically: Don't submit non-functional PRs (e.g. typo fixes, fixes for non-issues) under 10 lines in size.

Managing issues and PRs 🕵🏼

You can help us a lot to manage our workload for processing contributions by filtering, improving, and prioritizing issues and pull requests. At scale, this is the most time-consuming part of work on OSS projects.

The better contributions are organized, the greater their value.

Level 1: no permissions needed

You can already do a lot without having any extra permissions:

  • Upvote important issues and PRs

  • Downvote duplicate or low-quality issues and bad PRs

  • Review PRs, checking them for code quality and usefulness

  • Help issue/PR authors to ensure their contribution has all the necessary information

  • Find PRs that have the same goal, and advise the authors to work together or review each others work

  • Flag high-value PRs that can easily be merged or integrated to a Maintainer

  • Compile a list of important issues and/or PRs, and work with a Catalyst/Maintainer to prioritize them

  • Compile a list of duplicate or related issues, and work with a Catalyst/Maintainer to clean them up or consolidate them

  • Test PRs that are labeled as needs manual testing

If you contribute significantly at this level, we'd love to make you a Catalyst! 🚀 To be considered, DM a Maintainer a list of 20 links to contributions of yours.

Level 2: Catalyst 🦸🏼

As a Catalyst, you can label and/or close issues and pull requests. We really appreciate it if you help us out with this. It makes sense to combine this with a Community Support Lite kind of role, as you'll have a good view of existing issues.

For further info on how to Catalyze, see the Catalyst guide.

Community Support 🧑🏼‍🚒

Become an active member of our community by helping out fellow community members! This can involve answering questions, providing guidance, or sharing your expertise and insights. Your support helps foster a positive and collaborative environment. 💖

People will ask for help in various places:

If you are very active and helpful, you will be noticed by our support crew, and they can give you an official Support badge! 🚀

Creating benchmarking challenges 📊

There are two reasons to add interesting new challenges to the benchmark:

  • If AutoGPT can't do it yet, that gives us something to work towards
  • If AutoGPT can already do it, being able to measure this can help to preserve the capability

Feel free to pitch your idea in the #benchmarks channel, or if you feel confident, submit a PR to the AutoGPT Benchmark subproject directly!

Other Ways to Contribute 🫱🏼‍🫲🏼

There are many other ways to contribute to our project. Consider reporting bugs, suggesting enhancements, or just engaging with our community. Every contribution, no matter how small, helps us improve AutoGPT. :-)

Hunting & Reporting Bugs

If you encounter a bug in the project, you can contribute by reporting it. Here's how:

  1. Avoid duplicates: Search for an existing GitHub issue that describes the same problem or idea.
  2. GitHub Issue: If no such issue exists yet, create one with a clear description and title.
  3. Problem Description: Provide a step-by-step description of how to reproduce the bug.
  4. Supporting Information: Include relevant logs, screenshots, or additional details that can help in debugging.

Suggesting Enhancements

If you have ideas for new features or improvements, you can contribute by suggesting enhancements:

  1. Check the project board: Make sure your idea is not on the project board yet.
  2. Ask: Pitch your idea in the dev channel to make sure there is no immediate issue with it.
  3. GitHub Issue: Open a GitHub issue with a clear and descriptive feature request title.
  4. Detailed Description: Provide a comprehensive description of the proposed enhancement, including its benefits and potential drawbacks.
  5. Examples and Mockups: If applicable, offer examples or mockups to illustrate your idea.

AutoGPT logo

⚠️ We are working on updating this wiki ⚠️

We are working back towards a more accessible and inclusive developer experience. As long as this notice is here, beware that there may be things on this wiki that still need updating.

~ Pwuts, 2024-06-13

Start

Help

Research & Development

Operational

Miscellaneous

Clone this wiki locally