Inlet and outlet not called using a pipeline as a base_model to create custom model

[sir3mat]

Hello! When I create a model by going to Workspaces -> Model -> Create a Model and selecting the model ID of a custom pipeline as the base model, the inlet and outlet functions don't seem to be triggered.

Any insights on why this might be happening? Thank you!

[explorigin]

Can you post the code of your custom pipeline?

[sir3mat]

Due to company policies, I had to remove some code related to system prompt construction.

Currently, the main issue is that when using the pipeline directly, the self.file_contents variable updates correctly. However, if I wrap the pipeline to create a custom model from the UI, the inlet (and the oultet) is not called, and the self.file_contents list doesn’t update across different chats.

Here a code example

"""
title: pipeline 
author: user
version: 1.0
license: MIT
description: A pipeline
requirements:  langfuse, openai
"""

from typing import List, Optional, Union, Generator, Iterator
import os
import uuid
from pydantic import BaseModel
import openai
import re
from langfuse import Langfuse
from langfuse.api.resources.commons.errors.unauthorized_error import UnauthorizedError
import os
from utils.pipelines.main import get_last_assistant_message

# print Setup
...

# OIFile Class to manage openwebui file data
class OIFile:
    def __init__(self, file_id: str, filename: str, content: str):
        self.id = file_id
        self.filename = filename
        self.content = content

    def __repr__(self):
        return f"File(id={self.id}, filename={self.filename}, content={len(self.content)} bytes)"


class Pipeline:
    class Valves(BaseModel):
        LLM_BASE_URL: str
        LLM_API_KEY: str
        LLM_MODEL_NAME: str
        LLM_TEMPERATURE: float
        LLM_MAX_TOKENS: int
        

    def __init__(self):
        self.name = "pipeline"
        self.valves = self._initialize_valves()
        self.file_contents = [OIFile]
        

    def _initialize_valves(self) -> Valves:
        return self.Valves(
            LLM_BASE_URL=os.getenv("LLM_BASE_URL", "url"),
            LLM_API_KEY=os.getenv("LLM_API_KEY", "empty"),
            LLM_MODEL_NAME=os.getenv(
                "LLM_MODEL_NAME", "meta-llama/Llama-3.2-3B-Instruct"
            ),
            LLM_TEMPERATURE=float(os.getenv("LLM_TEMPERATURE", 0)),
            LLM_MAX_TOKENS=int(os.getenv("LLM_MAX_TOKENS", 10000)),
        )

    async def on_startup(self):
        print.info(f"Server {self.name} is starting.")

    async def on_shutdown(self):
        print.info(f"Server {self.name} is shutting down.")
        
    async def on_valves_updated(self):
        print.info("Valves updated.")


    async def inlet(self, body: dict, user: dict) -> dict:
        print.info("Processing inlet request")

        # Extract file info for all files in the body
        self.file_contents = self._extract_file_info(body)

        # Log the extracted file information
        for file in self.file_contents:
            print.info(
                f"File info extracted: ID={file.id}, Filename={file.filename}, Content size={len(file.content)} bytes"
            )
        return body

    def _extract_file_info(self, body: dict) -> list:
        """Extracts the file info from the request body for all files."""
        files = []
        for file_data in body.get("files", []):
            file = file_data["file"]
            file_id = file["id"]
            filename = file["filename"]
            file_content = file["data"]["content"]

            # Create a OIFile object and append it to the list
            files.append(OIFile(file_id, filename, file_content))

        return files

    def pipe(
        self, body: dict, user_message: str, model_id: str, messages: List[dict]
    ) -> Union[str, Generator, Iterator]:
       
        print.info("Starting PIPE process")

        # Extract parameters from body with default fallbacks
        stream = body.get("stream", True)
        max_tokens = body.get("max_tokens", self.valves.LLM_MAX_TOKENS)
        temperature = body.get("temperature", self.valves.LLM_TEMPERATURE)

        system_prompt = self._extract_system_prompt(messages)

        # Generate and update the system prompt if required
        if self.file_contents:
            # build or retrieve a prompt
            # adding file contents



        # Call the LLM API
        return self._call_openai_api(messages, max_tokens, temperature, stream)

    def _call_openai_api(
        self,
        messages: List[dict],
        max_tokens: int,
        temperature: float,
        stream: bool,
    ) -> Union[str, Generator, Iterator]:
       
        client = openai.Client(
            api_key=self.valves.LLM_API_KEY,
            base_url=self.valves.LLM_BASE_URL,
        )

        try:
            # Call OpenAI API with the prepared parameters
            response = client.chat.completions.create(
                model=self.valves.LLM_MODEL_NAME,
                messages=messages,
                max_tokens=max_tokens,
                temperature=temperature,
                stream=stream,
                stream_options={"include_usage": True},
            )

            return response
        except Exception as e:
            print.error(f"Error during OpenAI API call: {e}")
            return f"Error: {e}"

    async def outlet(self, body: dict, user: Optional[dict] = None) -> dict:
        print(f"outlet:{__name__}")
        print(f"Received body: {body}")
        self.file_contents=[]
        return body

[explorigin]

I need to see what the "wrapping" code looks like.

[sir3mat]

Sorry, I use wrap but I should use another term.
When I create a model by going to Workspaces -> Model -> Create a Model and selecting the model ID of a custom pipeline as the base model, the inlet and outlet functions don't seem to be triggered.
The pipeline used is the one showed above (plus some function to extract the system prompt and adding the file contents and other details).
When I create the model as described at the beginning of this comment, the function inlet and outlet are not triggered

[sir3mat]

@explorigin @tjbck any update?

[sir3mat]

Moreover, i have seen this behaviour:
In this custom pipeline setup, the workflow is as follows:

Documents are loaded and fed into OpenWebUI.
OpenWebUI encodes the documents, splits them into chunks, and stores them.
The /inlet endpoint on the pipeline is called.
OpenWebUI then performs the RAG operation.
The pipeline’s /pipe endpoint is triggered, displaying the response on OpenWebUI.
Finally, the client calls the /outlet endpoint to retrieve the result.

Question on Implementation:

Why do the /inlet, /pipe, and /outlet endpoints each have different request bodies?

[sir3mat]

any updates?