AI Player v.1.0.3-alpha-1

Changelog :

So after a lot of research and painful hours of coding here's the new update!

What's new :

1. Fixed previous bugs.

2. Switched to a much lighter model (llama3.2) for conversations, RAG and function calling

3. A whole lot of commands

4. Reinforcement Learning (Q-learning).

5. Mod no longer requires Java 21 to run. Can run on java 17.

6. Theortically Multiplayer compatible (just install the dependencies on server side), as carpet mod can run on servers too, but I have not tested it yet. Feedback is welcome from testers on this.

7. Theoretically the mod should not require everyone to install it on multiplayer, it should be a server-sided one, haven't tested this one yet, feedback is welcome from testers.

Bot can now interact with it's envrionment based on "triggers" and then learn about it's situation and try to adapt.

The learning process is not short, don't expect the bot to learn how to deal with a situation very quickly, in fact if you want intelligent results, you may need hours of training(Something I will focus on once I fix some more bugs, add some more triggers and get this version out of the alpha stage)

To start the learning process:

/bot spawn <botName> training

Right now the bot only reacts to hostile mobs around it, will add more "triggers" so that the bot responds to more scenarios and learns how to deal with such scenarios in upcoming updates

---

# New commands :

Spawn command changed.

/bot spawn <bot> <mode: training or play>, if you type anything else in the mode parameter you will get a message in chat showing the correct usage of this command

/bot use-key <W,S, A, D, LSHIFT, SPRINT, UNSNEAK, UNSPRINT> <bot>

AI Player 1.0.2-hotfix-3

Changelog description

This is the final hotfix version of version 1.0.2
that addresses a lot of broken code, and changing of the mod's workflow at the core.

In this hotfix, once again, a lot of the code has changed.

I will keep it simple in the changelog this time.

Fixed

1. GUI issue in config manager which didn't show the currently selected language model [FIXED].
2. The bot was not responding to normal conversations [FIXED].

Even improved accuracy with the gemma2 model.

For this patch, users will need to:

1. Go to your game folder (.minecraft)/config and you will find a settings.json5 file.
Delete that

2.(If you have run the previous 1.0.2 version already then) again go back to your .minecraft. you will find a folder called "sqlite_databases". Inside that is a file called memory_agent.db

3. Delete that as well.

4. Install the gemma2 model (ollama pull gemma2) [Required]

Users can choose to keep the llama2 or mistral models and test each other's efficiency.

However the mod has a 100% chance of breaking when llama2 and mistral is used because of the responses they generate.

---

From a technical perspective

The breaking of the mod happens llama2 and mistral generate very large initial responses whose vector embeddings cannot be generated and thus the mod throws an exception.

Gemma2 however keeps it precise and to the point, thus resulting in a lot shorter response.

Gemma2 also has much more accurate information about minecraft data, like crafting recipes, biomes, mobs, etc till 1.20.5 presumably, as the test results indicate, as compared to llama2 or mistral.

AI Player 1.0.2-hotfix-2

A small update to the version 1.0.2

Thanks to Mr. Álvaro Carvalho, who suggested a few enhancements to the mod regarding the commands system and some inner code changes.

Also this version's icon is fixed and will be recognized by modmanager.

Once again the setup instructions for existing users

1. Go to your game folder (.minecraft)/config and you will find a settings.json5 file.
Delete that

2.(If you have run the previous 1.0.2 version already then) again go back to your .minecraft. you will find a folder called "sqlite_databases". Inside that is a file called memory_agent.db

3. Delete that as well.
4. Install the models, mistral, llama2 and nomic-embed-text

5.Then run the game
6. Inside the game run /configMan to set the language model to llama2.

Then spawn the bot and start talking!

patch 1.0.3 will include backwards compatibility for versions 1.20, 1.20.1 and 1.20.3 and other features as mentioned earlier!

AI Player v1.0.2

Changelog

For the general player base.

Aaaand after 2 weeks of a lot of coffee, back pain and brain ache, here's the new update.

At first it may seem that this version doesn't do much, but keep talking with the bot and you will find out :)

To-do: Install two models in ollama. [VERY IMPORTANT]

1. nomic-embed-text (ollama pull nomic-embed-text)
2. llama2 (ollama pull llama2)

In game, change your selected language model to llama2 for the best performance.

TLDR: The bot is way more intelligent now, can remember past conversations, can store current conversations and then use the data for long term storage, has gotten better at function calling like movement and block detection, no fixed responses, no need of fixed prompt checking, it all happens,

Also the connection to the language model is only made when the bot is spawned in game, thus saving a lot of memory when the bot is not in the game.

Eventually I intend to bring support for custom models on the ollama database.

dynamically, hehe.

Next patch will include more minecraft interaction, such as mining, improved movement, collision/obstacle detection, etc.

---

For the nerds.

So, for the tech savvy people, I have implemented the following features.

LONG TERM MEMORY: This mod now features concepts used in the field AI like Natural Language Processing (much better now) and something called

Retrieval Augmented Generation (RAG).

How does it work?

Well:

We convert the user input, to a set of vector embeddings which is a list of numbers.

Then physics 101!

A vector is a representation of 3 coordinates in the XYZ plane. It has two parts, a direction and a magnitude.

If you have two vectors, you can check their similarity by checking the angle between them.

The closer the vectors are to each other, the more similar they are!

Now if you have two sentences, converted to vectors, you can find out whether they are similar to each other using this process.

In this particular instance I have used a method called cosine similarity

Cosine similarity

Where you find the similarity using the formula

(x, y) = x . y / |x| . |y|

where |x| and |y| are the magnitudes of the vectors.

So we use this technique to fetch a bunch of stored conversation and event data from an SQL database, generate their vector embeddings, and then run that against the user's prompt. We get then further sort on the basis on let's say timestamps and we get the most relevant conversation for what the player said.

Pair this with function calling. Which combines Natural Language processing to understand what the player wants the bot to do, then call a pre-coded method, for example movement and block check, to get the bot to do the task.

Save this data, i.e what the bot did just now to the database and you get even more improved memory!

To top it all off, Llama 2 is the best performing model for this mod right now, so I will suggest y'all to use llama2.

In fact some of the methods won't even run without llama2 like the RAG for example so it's a must.

AI Player 1.0.1-alpha-2

Changelog

This alpha version of version 1.0.1 includes a lot of content. Well atleast in terms of code it does.

Steve (or the bot/) can now understand what users saying a lot better by using a process called "Natural Language Processing!"

This Natural language processing has been implemented for the sake of getting the in-game bot to execute actions based on what the user says, it is separate from the language model. However I intend to achieve a sort of "synchronisation" which will keep the language model informed of what is going on in-game.

Also I have added block and nearby entity detection.

The bot can now detect if there is a block in front of it, simply ask it using the sendAMessage command!

There's 99% chance that the bot will understand the intention and context of your message. Hope to achieve 100% soon.

Initially when you spawn the bot, it takes a minute or so for it to face you correctly, since it's spawn position will be fixed by the game by teleporting it somewhere near you after spawning is done (This is not the same as seen in game when you spawn the bot and instantly teleports to you).

After that the bot will accurately face the nearest entity in front of it. The range is 5 blocks in the X, Y and Z axes. To make it more realistic, I have made it so that the bot can't detect entites behind it.

Version 1.0.1

Changelog

Fixed the issue of players not being able to spawn bots when playing in offline mode.
Also, a lot of bugfixes

Version 1.0.0-alpha-1

Fixed a lot of bugs which caused instant crash during bootup of the game

Version 1.0.0-alpha-1

This is an alpha version of the mod which has some of the very basic features implemented yet.

Released for mass testing and bug-hunting.