Yarp devices for Large Language Models

This repository contains the YARP plugin for Large Language Models.

Documentation

Documentation of the individual devices is provided in the official Yarp documentation page:

Prerequisites

To install GPTDevice we need:

• nlohmann-json
• cURL
• liboai

Based on liboai recommendations, it is recommended to install nlohmann-json and libcurl using vcpkg.

Install vcpkg, cURL and nlohmann-json

git clone https://github.com/Microsoft/vcpkg.git && \
    cd vcpkg && ./bootstrap-vcpkg.sh -disableMetrics && \
    ./vcpkg integrate install && ./vcpkg install curl && \
    ./vcpkg install nlohmann-json

Install liboai

git clone https://github.com/D7EAD/liboai.git && \
    cd liboai/liboai && \
    cmake -B build -S . -DCMAKE_TOOLCHAIN_FILE=<path-to-vcpkg>/scripts/buildsystems/vcpkg.cmake \
    -DCMAKE_POSITION_INDEPENDENT_CODE=true && \
    cd build && make -j8 && make install

Installation

Build with pure CMake commands

# Configure, compile and install
cmake -S. -Bbuild -DCMAKE_INSTALL_PREFIX=<install_prefix> -DCMAKE_TOOLCHAIN_FILE="<path-to-vcpkg>/scripts/buildsystems/vcpkg.cmake" -DLIBOAI_INSTALL_PATH="<path-to-liboai>
cmake --build build
cmake --build build --target install

Docker

In alternative one can build a docker image by running

cd yarp-devices-llm
docker build -t <my-img-name> .

Configuration

You need to copy .env-template into a .env file with the correct information regarding your OpenAI deployment. Note that currently GPTDevice works only with an Azure instance of GPT.

Remember to source the .env file in order to allow the device to connect to your GPT instance.

set -a
source .env
set +a

Usage

Assuming that the user has already installed YARP and the related LLM devices, one can use this basic configuration example

yarpserver

set -a
source .env
set +a
yarprobotinterface --from assets/LLMDeviceExample.xml

yarp rpc /yarpgpt/rpc
>>help
Responses:
  *** Available commands:
  setPrompt
  readPrompt
  ask
  getConversation
  deleteConversation
  help
>>setPrompt "You are a geography expert who is very concise in its answers"
Response: [ok]
>>ask "What is the capital of Italy?"
Response: [ok] Rome. #This answer may vary in the form, hopefully not in the content.

CI Status

License

This software is distributed under the terms of the BSD-3-Clause.

Maintainers

This repository is maintained by:

	@fbrand-new