feat(wren-ai-service): question recommendation pipeline

[paopa]

This PR introduces a new Question Recommendation feature to the Wren AI Service.

Key Updates:

1. Added question_recommendation to the pipes configuration in config.example.yaml
2. Updated ServiceContainer in src/globals.py to include question_recommendation
3. Added question_recommendation router to the main router in src/web/v1/routers/__init__.py
4. Implemented the Question Recommendation pipeline, service, and router

New Endpoints

The PR introduces two new endpoints for question recommendation:

1. POST /question-recommendations
2. GET /question-recommendations/{id}

How to Use the Endpoints

1. Generate Question Recommendations

To generate new question recommendations:

POST /question-recommendations

{
  // JSON string of the MDL (Model Definition Language)
  "mdl": "{ ... }",
  // Optional list of previous questions
  "previous_questions": [
    "question1",
    "question2"
  ],
  // Optional project ID
  "project_id": "project-id",
  // Optional max number of questions to generate (default: 5)
  "max_questions": 5,
  // Optional max number of categories (default: 3)
  "max_categories": 3,
  // Optional configuration settings
  "configuration": {
    // Optional language (default: "English")
    "language": "English",
    // Optional timezone settings
    "timezone": {
      // Timezone name (default: "Asia/Taipei")
      "name": "Asia/Taipei"
    }
  }
}

Response:

{
  "id": "unique-uuid"
}

2. Retrieve Question Recommendations

To check the status and retrieve the generated recommendations:

GET /question-recommendations/{id}

Response:

{
  "id": "unique-uuid",
  "status": "generating" | "finished" | "failed",
  "response": {  // Present only if status is "finished"
    "questions": [
      {
        "question": "str",
        "explanation": "str",
        "category": "str"
      },
    ]  // List of question recommendations
  },
  "error": {  // Present only if status is "failed"
    "code": "OTHERS",
    "message": "Error description"
  }
}

Usage:

1. Send a POST request to start the generation process.
2. Use the returned ID to poll the GET endpoint until the status is "finished" or "failed".

Benefits

1. Enhanced User Experience: Provides intelligent question suggestions based on the data model.
2. Improved Data Exploration: Helps users discover insights they might not have considered.
3. Scalability: Asynchronous processing allows for handling multiple requests efficiently.

Testing

1. Deploy an MDL(I suggest you can deploy an e-commerce dataset from UI)
2. And then, using the following testing request body for ensuring the endpoint

{
  "mdl": "{\"catalog\":\"wrenai\",\"schema\":\"ecommerce\",\"models\":[{\"name\":\"customers\",\"refSql\":\"select * from main.customers\",\"columns\":[{\"name\":\"City\",\"type\":\"VARCHAR\",\"isCalculated\":false,\"notNull\":false,\"properties\":{\"description\":\"The Customer City, where the customer company is located. Also called 'customer segment'.\"}},{\"name\":\"Id\",\"type\":\"VARCHAR\",\"isCalculated\":false,\"notNull\":false,\"properties\":{\"description\":\"A unique identifier for each customer in the data model.\"}},{\"name\":\"State\",\"type\":\"VARCHAR\",\"isCalculated\":false,\"notNull\":false,\"properties\":{\"description\":\"A field indicating the state where the customer is located.\"}},{\"name\":\"orders\",\"type\":\"orders\",\"relationship\":\"CustomersOrders\",\"isCalculated\":false,\"notNull\":false,\"properties\":{}},{\"name\":\"LatestRecord\",\"type\":\"DATE\",\"isCalculated\":true,\"expression\":\"max(orders.PurchaseTimestamp)\",\"notNull\":false,\"properties\":{}},{\"name\":\"NumberOfOrders\",\"type\":\"BIGINT\",\"expression\":\"count(OrderId)\"},{\"name\":\"Peak\",\"type\":\"DOUBLE\",\"expression\":\"max(TotalValue)\"}],\"timeGrain\":[{\"name\":\"PurchaseTimestamp\",\"refColumn\":\"PurchaseTimestamp\",\"dateParts\":[\"YEAR\",\"MONTH\",\"DAY\"]}]},{\"name\":\"orders\",\"refSql\":\"select * from main.orders\",\"columns\":[{\"name\":\"CustomerId\",\"type\":\"VARCHAR\",\"isCalculated\":false,\"notNull\":false,\"properties\":{\"description\":\"The unique identifier of the customer who placed the order.\"}},{\"name\":\"OrderId\",\"type\":\"VARCHAR\",\"isCalculated\":false,\"notNull\":false,\"properties\":{\"description\":\"A unique identifier for each order in the dataset.\"}},{\"name\":\"PurchaseTimestamp\",\"type\":\"TIMESTAMP\",\"isCalculated\":false,\"notNull\":false,\"properties\":{\"description\":\"The date and time when the order was placed.\"}},{\"name\":\"TotalValue\",\"type\":\"DOUBLE\",\"isCalculated\":false,\"notNull\":false,\"properties\":{\"description\":\"The total monetary value of the order.\"}},{\"name\":\"order_items\",\"type\":\"order_items\",\"relationship\":\"OrdersOrderItems\",\"isCalculated\":false,\"notNull\":false,\"properties\":{}},{\"name\":\"NumberOfOrders\",\"type\":\"BIGINT\",\"expression\":\"count(OrderId)\"},{\"name\":\"Peak\",\"type\":\"DOUBLE\",\"expression\":\"max(TotalValue)\"}],\"timeGrain\":[{\"name\":\"PurchaseTimestamp\",\"refColumn\":\"PurchaseTimestamp\",\"dateParts\":[\"YEAR\",\"MONTH\",\"DAY\"]}]}],\"cumulativeMetrics\":[],\"views\":[{\"name\":\"total_items_view\",\"statement\":\"WITH customer_order_items AS (SELECT c.State, oi.ItemNumber FROM customers c JOIN orders o ON c.Id = o.CustomerId JOIN order_items oi ON o.OrderId = oi.OrderId) SELECT State, COUNT(ItemNumber) AS Total_Items_Purchased FROM customer_order_items GROUP BY State\",\"properties\":{\"question\":\"How many items have been purchased in orders placed by customers in each state?\",\"summary\":\"The query counts the total number of items purchased by customers in each state.\",\"viewId\":\"20b69c05-e9a3-4c4d-a421-06ba7e7a385a\"}}],\"macros\":[{\"name\":\"MoM\",\"definition\":\"(calculation: Expression, timestamp: Expression) => ({{ calculation }} - COALESCE(LAG({{ calculation }}) OVER (ORDER BY {{ timestamp }}), 0)) / COALESCE(LAG({{ calculation }}) OVER (ORDER BY {{ timestamp }}), 1)\"}],\"dateSpine\":{\"unit\":\"DAY\",\"start\":\"1970-01-01\",\"end\":\"2077-12-31\",\"properties\":{}}}",
  "previous_questions": [],
  "max_questions": 5,
  "max_categories": 3,
  "configuration": {
    "language": "English",
    "timezone": {
      "name": "Asia/Taipei"
    }
  }
}

Screenshots

[cyyeh]

Thanks for your hard work, and please check out the comments I left

[cyyeh]

a nitpick