Semantic Kernels for Enterprise Application

[abhinpai]

My colleagues and I are currently working on a POC to assess the practicality of implementing semantic kernels within our enterprise applications. Our primary objectives are as follows:

1. Implementing a straightforward natural language search system to query information from our database (Postgres), effectively enabling NL2SQL capabilities.
2. Developing a natural language search system for constructing 3rd-party API request payloads and retrieving data from external sources to populate dashboards.

For the first use case, I've made significant progress following an example from the kernel-memory repository, specifically: nl2sql example.

However, I've encountered a couple of challenges:

1. The token usage is exceptionally high due to the extensive size of our database schema, which includes numerous normalized RDBMS tables. This schema is incorporated into the semantic kernel skills, contributing to the high token consumption.
   The generated SQL statements often do not align perfectly with our database schema.

2. For the second use case, I aim to prepare an API request payload based on the search query. The constructed payload will be used to make a 3rd-party API call, and the response data will be used to create dashboards. The complexity here lies in crafting a request payload that adheres to the valid parameters defined by the DTOs.

I have several questions regarding these challenges:

1. How can I minimize token usage, regardless of the size of the database schema?
2. What strategies can I employ to ensure that the system fine-tunes its outputs based on our specific database schema?
3. What is the most effective and scalable approach for constructing request payloads for any API?

[evchaki]

@abhinpai - great questions and sorry for the delay. You can try the following things:

1. Don't have the LLM create the full sql statements - instead have it call a stored procedure or view and pass over the parameters into it. This is more secure and scaleable.
2. You could use a vector DB to store part of your schema and/or the views so the LLM knows what can be called