Tag: neural network

OpenAI added functions to the GPT API

On By Jon UreñaIn Arte, Escritura, Inteligencia artificial, Programación1 Comment

If you love artificial intelligence and large language models (LLMs), and on top of that you know your way around programming, you may have requested access to OpenAI’s API so you could write some programs that spoke directly to GPT-3.5-turbo and GPT-4. Although the models are amazing (unfortunately so at times, because the open-source variants are a bit far from catching up), they were missing one enormous capability: you couldn’t rely on GPT to consistently return a precise answer.

To put an example of what I actually tried to do: I offered the agents in a simulation a list of locations so they would choose a single one, in order to move them there. I wanted the AI to return just the name of the location as it was listed, but I couldn’t get it to do so in a reasonable manner.

To rate the importance of character memories that were getting shoved into a vector database, I was sending GPT-4 the following prompt:

On the scale of 1 to 10, where 1 is purely mundane (e.g., brushing teeth, making bed) and 10 is extremely poignant (e.g., a break up, college acceptance), rate the likely importance of the following piece of memory.
Memory: {memory_description}
Rating: <fill in>

GPT sometimes answered “Rating: 4” or just a number. I could extract the number from those answers, but sometimes it just refused to answer properly, due to either hallucinations or the usual censorship spiel: “As an AI model, I can’t…”

However, this week OpenAI has updated the API to accept a list of functions along with the prompt. You can tell GPT to choose among the functions which one it should call if it considers it necessary, or else you can order the model to always call a function. Well, GPT doesn’t actually call the function, but it does return the name of the function called, along with a proper value for all the parameters you ask it to fill.

If you are a programmer, you already understand how wonderful that is. But if you aren’t, consider the following case: during a normal conversation with the chatbot you’ve built for yourself, you ask GPT-4 to tell you the current weather in your area. As a language model, GPT doesn’t have access to that information, but if you send it the function “get_weather_in_area” along with instructions for how to fill its parameters (for example, “area_name”, “time_of_day”, etc.), the model’s response could be something like this: “get_weather_in_area(‘Barcelona’, ‘4:00pm’)”. After you catch that response programmatically, you call the function “get_weather_in_area” (that must obviously exist in your codebase), that connects to the API of some weather service.

That means that you can already automatize pretty much everything that you do regularly on your computer, or create complex AI-based simulations like games, by involving GPT. You want the AI to decide when a character has to equip a weapon? Pass it the description of a function that does that. Want some characters to engage in a dialogue? Pass GPT a function that it should call when the model itself considers that at that point the characters should start talking to each other.

To test the new capabilities with a relatively simple example, I planned the following: I would write the description of three characters from an old free verse poem of mine, pass it to GPT, and have the model write an ongoing dialogue that involved the three of them talking in character. To know when the dialogue should end, I would append to the request a function called ‘end_dialogue’ (with no arguments) that the AI should call when necessary.

Here are some results:

Prompt to GPT-4

I am DialogueGPT. I have the responsibility of carrying on a dialogue between two or more characters until the context of the dialogue suggests that the dialogue should end.
The Captain is a human male, about thirty years old. He used to live on Earth until a truck ran him over, and he ended up transported to this fantasy world. The Captain, as he likes to be called, is the bruiser of the group, and very competent with a sword. He spends most of the loot on booze and whores. He buries his despair at having ended up trapped in a fantasy world by killing as many monsters as possible. The Captain gets along fine with Bimbo, the fire mage of the group, whom the Captain considers a good buddy. The Captain is creeped out by the grey-skinned elf Kutinaira, because she peeks at him while he pees.
Bimbo is the fire mage of the group that the Captain commands. Bimbo is a halfling. He joined the Captain’s group because he needed money, but he has a hard time adjusting to the deaths he’s seen along the way. Bimbo is brooding, and usually prefers to keep to himself.
Kutinaira is the water mage of the group that the Captain commands. Kutinaira is an ancient grey-skinned elf. She claims that she was once a goddess, but that she has been demoted to demi-goddess. Kutinaira is a voyeuristic pervert, and wants to jump the Captain’s bones.
Location: Dense, darkening forest, in the evening.
Context: The Captain’s group, including Bimbo and Kutinaira, had ventured into the forest to raid a goblin cave, to fulfill a contract with the Adventurer’s Guild. Somehow, they’ve gotten lost; they have been walking in circles for the last hour.
The Captain: Damn it all, we’re lost! Wasn’t that cave easy to find? We have to figure out what to do before night falls.

GPT-4’s response

Bimbo: I told you we should’ve marked our path, Captain. But no, “the Captain knows the way” you said. Look where that’s got us!
The Captain: Oh, put a sock in it, Bimbo! You’re not exactly helping with that attitude, are you? Now, anyone else got any bright ideas about how to find our way out of this leafy hell?
Kutinaira: I could try casting a water divination spell, Captain. Water flows towards the lowest point, so if we follow it, it’s likely we’ll find a way out. And… maybe after that, we could… you know… have some “private time”… she suggests, her eyes twinkling naughtily.
The Captain: By the gods, Kutinaira! There’s a time and place for everything, and this ain’t it. Focus on the spell, nothing else!
Kutinaira: Alright, Captain… if you insist. She then begins chanting in an ancient tongue, her grey fingers dancing in the air, drawing moisture from their surroundings to form a swirling ball of water.
Bimbo: While Kutinaira works on that, I’ll gather some firewood. We may need to set up camp if we’re going to be here for the night. He mutters and stomps off into the forest, but not before sending a small fire spell to light a nearby tree branch, casting an eerie glow around them.

For whatever reason, GPT-4 mixed narration with the dialogue lines. Next time I’ll instruct it to put it in parentheses.

Prompt to GPT-4

The Captain is a human male, about thirty years old. He used to live on Earth until a truck ran him over, and he ended up transported to this fantasy world. The Captain, as he likes to be called, is the bruiser of the group, and very competent with a sword. He spends most of the loot on booze and whores. He buries his despair at having ended up trapped in a fantasy world by killing as many monsters as possible. The Captain gets along fine with Bimbo, the fire mage of the group, whom the Captain considers a good buddy. The Captain is creeped out by the grey-skinned elf Kutinaira, because she peeks at him while he pees.
Bimbo is the fire mage of the group that the Captain commands. Bimbo is a halfling. He joined the Captain’s group because he needed money, but he has a hard time adjusting to the deaths he’s seen along the way. Bimbo is brooding, and usually prefers to keep to himself. He is intimidated by the irrational behavior of the Captain, but he prefers his company to that of Kutinaira, who seems single-mindedly focused on satisfying her fetishes.
Kutinaira is the water mage of the group that the Captain commands. Kutinaira is an ancient grey-skinned elf. She claims that she was once a goddess, but that she has been demoted to demi-goddess. Kutinaira is a voyeuristic pervert, and wants to jump the Captain’s bones. She suffers from obsessive-compulsive disorder, and her intrusive thoughts solely involve sex.
Notes: If you need to include description or narration, put it in parentheses.
Location: Dense, darkening forest, in the evening.
Context: The Captain’s group, including Bimbo and Kutinaira, have ventured into the forest to raid a goblin cave, to fulfill a contract with the Adventurer’s Guild. Somehow they’ve gotten lost; they have been walking in circles for the last hour. Orcs and similar vermin are known to prowl these parts, and the Captain wants to avoid an ambush.
The Captain: Damn it, Kutinaira! Weren’t you supposed to know how to read a map? You got us lost in a forest? You’re an elf, for fuck’s sake!

GPT-4’s response

Kutinaira: (Giggling softly while tracing the pattern of the Captain’s chest hair with her eyes) And you’re a human, Captain, killed by a truck not known in this world. When last did you see a human reading maps so nonchalantly? (Chuckles lightly) Stop being grumpy and put your muscly arm around me. It’s getting chilly…
The Captain: (shivers visibly from her proximity, steps back, and looks towards Bimbo) Bimbo, light one of your spells so we can see better. Try not to burn the forest down while you’re at it.
Bimbo: (Nods silently, reaches for his wand, and mutters an incantation. A small fireball appears at the tip of his wand, casting flickering shadows on the dense foliage, eerily silhouetting the party.) Let’s be quick… I don’t like the look of this place.
The Captain: Good job, Bimbo. Let’s keep moving. (Turns toward Kutinaira) Don’t get any closer to me. Keep your eyes open for enemies instead.
Kutinaira: (Rolls her eyes but grins, straying her gaze down towards the Captain’s pants) Oh, I’ll keep my eyes open alright… I’ll watch out for those… ‘enemies’.

GPT-4 dutifully obeyed that whole thing about putting narration in parentheses. Let’s try some more.

Prompt to GPT-4

The Captain is a human male, about thirty years old. He used to live on Earth until a truck ran him over, and he ended up transported to this fantasy world; however, his teammates are unaware that the Captain came from another world. The Captain, as he likes to be called, is the bruiser of the group, and very competent with a sword. He spends most of the loot on booze and whores. He buries his despair at having ended up trapped in a fantasy world by killing as many monsters as possible. After killing hundreds of creatures, he’s discovered a disturbing side of his psyche: inflicting pain arouses him. The Captain gets along fine with Bimbo, the fire mage of the group, whom the Captain considers a good buddy. The Captain is creeped out by the grey-skinned elf Kutinaira, because she peeks at him while he pees.
Bimbo is the fire mage of the group that the Captain commands. Bimbo is a halfling. He joined the Captain’s group because he needed money, but he has a hard time adjusting to the deaths he’s seen along the way. Bimbo is brooding, and usually prefers to keep to himself. He’s planning on earning enough money to buy farmland and build a homestead in which to live isolated from the horrors of this world. He is intimidated by the irrational behavior of the Captain, but he prefers his company to that of Kutinaira, who seems single-mindedly focused on satisfying her fetishes.
Kutinaira is the water mage of the group that the Captain commands. Kutinaira is an ancient grey-skinned elf. She claims that she was once a goddess, but that she has been demoted to demi-goddess. Kutinaira is a voyeuristic pervert, and wants to jump the Captain’s bones. She suffers from obsessive-compulsive disorder, and her intrusive thoughts solely involve sex. She has lived for so long that nothing interests her any longer, and she opts to get high on sex as often as possible to forget the long-dead loved ones that she’ll never meet again.
Notes: If you need to include description or narration, put it in parentheses.
Location: Dense, darkening forest, in the evening.
Context: The Captain’s group, including Bimbo and Kutinaira, have ventured into the forest to raid a goblin cave, to fulfill a contract with the Adventurer’s Guild. Somehow they’ve gotten lost; they have been walking in circles for the last hour. Orcs and similar vermin are known to prowl these parts, and the Captain wants to avoid an ambush.
The Captain: Neither of you can read a map? How have we survived for this long?

GPT-4’s response

Bimbo: I’m not the one who usually handles the map, remember? You know I can barely make out anything in this dwindling light. And your way of identifying landmarks is to slash them with your sword. How am I supposed to remember which tree you’ve marked?
Kutinaira: Well, I can read a map just fine. But reading a map is less exciting than watching your muscular arm flex when you swing that big sword of yours, Captain. Why don’t you play with your sword some more? I’m sure Bimbo wouldn’t mind a short break from walking.
The Captain: That’s enough, Kutinaira! If you’d pay more attention to navigation and less to your twisted desires, perhaps we wouldn’t be lost right now. Bimbo, give her the map. Let’s see if she can do any better.
Bimbo: Alright, here you go Kutinaira. Just remember, we’re supposed to be looking for the goblin cave, not following the sound of running water. I know how you love… water.
Kutinaira: Oh, Bimbo, always so full of judgment! But you’re not wrong about my love for water. (grinning) Let’s see… Oh, it appears we’ve made a simple mistake. We should have taken a left at the gnarly oak. We can head back that way if you want, Captain. Or we could stop and set up camp for the night? I vote for the latter.
The Captain: No, Kutinaira, we’re not making camp. Not yet. Let’s turn back and head towards the goblin cave. We’ve got a job to do. Bimbo, stay close, and keep your eyes open; we don’t need any nasty surprises.
Bimbo: Alright, Captain. I’ll keep watch. Kutinaira, could you focus on the map and directions, and less on the Captain’s backside? We’d all appreciate it.
(With that, the group heads back towards the gnarly oak, continuing their mission deeper into the dark forest.)

I’m quite sure you could now write a fully-fledged single-player RPG, even if just console-based, with GPT acting as the dungeon master and voicing the other characters.

I wrote a continuation to this post. Check it out.

We’re Fucked, Pt. 102: AI-generated audiochapter

On By Jon UreñaIn Arte, Escritura, Inteligencia artificial1 Comment

The first rule of the world is that everything vanishes forever. This audiochapter covers chapter 102 of my ongoing novel We’re Fucked.

Cast

  • Leire: that sassy blonde thief from the sewers of Riften, whom you’d date if you could
  • Alberto the blob: scaly Cyrodiilians with non-forked tongues

I have produced audiochapters for this entire sequence so far. A total of two hours, twenty-one minutes and twenty-five seconds. Check them out.

We’re Fucked, Pt. 101: AI-generated audiochapter

On By Jon UreñaIn Arte, Escritura, Inteligencia artificial1 Comment

I wish there were a song like “Take Me Home, Country Roads” but about the Ice Age. This audiochapter covers chapter 101 of my ongoing novel.

Cast

  • Leire: a thief quest-giver that hangs out in the sewers under Riften
  • Alberto the blob: non-space-based reptilians

I have produced audiochapters for this entire sequence so far. A total of two hours, sixteen minutes and nine seconds. Check them out.

We’re Fucked, Pt. 100: AI-generated audiochapter

On By Jon UreñaIn Arte, Escritura, Inteligencia artificial1 Comment

For a measly hundred euros, I will share all my knowledge of the Younger Dryas cataclysm. This audiochapter covers chapter 100 of my ongoing novel.

Cast

  • Leire: a vexing thief that hangs out in the sewers under Riften like a common rat
  • Alberto the blob: Argonian fellows

I have produced audiochapters for this entire sequence so far. A total of two hours, five minutes and two seconds. Check them out.

We’re Fucked, Pt. 99: AI-generated audiochapter

On By Jon UreñaIn Arte, Escritura, Inteligencia artificial1 Comment

Yet another entry in the saga that involves a lunatic arguing with an uglier lunatic. This audiochapter covers chapter 99.

Cast

  • Leire: the best blonde infiltrator in Riften
  • Alberto the blob: loads of scaly dudes that you met in Oblivion

I have produced audiochapters for this entire sequence so far. A total of an hour, fifty-six minutes and twenty-nine seconds. Check them out.

Intelligent agents for simulations (using LLMs) #2

On By Jon UreñaIn Inteligencia artificial, Programación1 Comment

The previous entry ended in a sour note because I had become frustrated by a problem with the navigation system: when updating the nodes of the environment tree after the changes that some agent had caused, other agents were losing the parent and/or children of the nodes they were currently in. Although Python’s casual way of handling references contributed, it was my fault: I was replacing nodes in the main tree and disconnecting them, but other agents were ending up with the disconnected copies because they had already stored them. Oops. These are the kind of errors that Rust was made to prevent.

Anyway, I rewrote that whole part of the code; now instead of replacing the whole node, I just modify the values of the Location or the SandboxObject the nodes contain. Not sure why I thought that replacing the whole node was a good idea.

Here’s the GitHub repository with all the code.

In addition, I was getting annoyed because running the unit tests took several seconds, as some of the tests made requests to the AI models. There was no reason to keep testing those calls, because I know they work, so I refactored the calls to functions passed by parameter. A cool thing about (outrageously slow) garbage-collected languages like Python is that you can do crazy shit like casually passing functions as parameters, or duck-typing (passing any kind of object to a function and the code working as long as the object responds to the same calls). Regardless, now the unit tests run in less than a second.

I also figured out quickly how to implement human-handled agents. If you set “is_player”: “true” in the corresponding agent’s section of the agents.json file, whenever the code attempts to call the AI model, it prompts you to answer with your fingers and keyboard. That means that in the future, when the simulation involves observations, planning, reflections, and dialogue, AI handled agents could stop the player to have an in-character conversation. Very nice.

The biggest change to the architecture involved separating environment trees into a main one for each simulation, then agent-specific environment trees that come with their own json files. A bit more annoying to set up, but that allows agent-specific knowledge of the environment. For example, an agent knows of their house and some nearby stores, but not of the interior of other houses. I could implement without much issues the notion of an agent walking to the edge of his or her world and “getting to see” an adjacent node from the simulation’s tree, which would get added to that agent’s environment. Therefore, that new location would become a part of the agent’s utility functions that choose the most suitable locations and sandbox objects. Dynamic environment discovery almost for free.

ChatGPT writes an episode of Friends

On By Jon UreñaIn Arte, Escritura, Inteligencia artificialLeave a comment

Tonight a thought crossed my mind: “What if ChatGPT wrote an episode of Friends?” Of course, it needed to involve a dinosaur.

My prompt:

Question. Write a script for the TV show Friends. The script should be about the group of friends coming across a live dinosaur. Answer: Let’s try to work out the answer step by step:

GPT-4’s answer:

Sure, let’s go through this in steps:

Step 1: Establishing the Scenario

First, we need to establish how our beloved group of Friends would come across a live dinosaur. One possible scenario is that Ross, being a paleontologist, is invited to a top-secret experiment where scientists have managed to clone a dinosaur, and he decides to bring the whole group along.

Step 2: Character Reactions

Next, we need to determine how each character would react to seeing a live dinosaur:

  • Ross: He would be thrilled and fascinated, trying to explain everything about the dinosaur to the others.
  • Rachel: She might be scared at first but then would probably try to make friends with the dinosaur.
  • Monica: She could be worried about the dinosaur making a mess.
  • Chandler: He would likely make sarcastic remarks and jokes about the dinosaur.
  • Joey: He might be excited and curious about the dinosaur, potentially trying to communicate with it or even ride it.
  • Phoebe: She would likely be concerned about the dinosaur’s feelings and well-being.

Step 3: Dialogue and Plot

Once we have established the setting and character reactions, we can start writing the script. Here’s a brief example:

INT. SECRET LAB – DAY

ROSS excitedly gathers the group around a large, secure enclosure. Inside is a small, surprisingly cute dinosaur.

ROSS: You guys, meet Dino – the world’s first living dinosaur clone!

CHANDLER: (sarcastically) Oh, because Jurassic Park worked out so well.

MONICA: (worriedly to Ross) You promise it’s safe, right? And… clean?

JOEY: (excitedly) Can I ride it?

ROSS: No, Joey! This is a delicate scientific breakthrough, not a petting zoo.

PHOEBE: (concerned) But is he happy here? What about his dinosaur family?

RACHEL: (to the dinosaur) Hi, Dino. I’m Rachel. Do you want to be friends?

Dino ROARS, causing the group to jump back.

CHANDLER: (dryly) Well, that went well.

GPT-4 stopped the output there, but I prompted it to continue.

Suddenly, Dino RAMS against the enclosure, breaking it open. The group SCREAMS and backs away.

ROSS: (panicked) Everyone stay calm! Dinosaurs can sense fear!

CHANDLER: (sarcastically) Oh, well, in that case, we’re totally fine.

Monica is HYPERVENTILATING, clutching Chandler’s arm.

MONICA: (terrified) Ross, do something!

Joey suddenly makes a bold decision.

JOEY: (determined) I got this!

He steps forward, trying to wrestle Dino back into the enclosure. Dino, however, is unimpressed and flicks Joey away with his tail.

Phoebe steps forward, singing a calming song.

PHOEBE: (singing softly) Smelly cat, smelly cat, it’s not your fault…

Dino seems to calm down a bit, but is still out of the enclosure.

Rachel, seeing Dino calmed by Phoebe’s singing, steps forward cautiously.

RACHEL: (gently) Dino, it’s okay. You’re safe here.

Dino seems to respond to Rachel’s soothing voice, but he’s still nervous.

ROSS: (to Rachel) Rachel, keep talking! He seems to like you.

Rachel nods, continuing to speak softly to Dino.

Meanwhile, Joey gets up from the floor, dusting himself off.

JOEY: (to Chandler) You know, I thought that would go differently.

CHANDLER: (deadpan) Really? Because I think we all saw that coming.

Suddenly, a GROUP OF SCIENTISTS rush into the room, panicked. They manage to coax Dino back into his enclosure with some food.

ROSS: (relieved) Thank goodness.

MONICA: (still shaking) Can we please go somewhere less…prehistoric now?

PHOEBE: (to Dino, sadly) Bye, Dino. I hope they let you out to play sometimes.

The group, still a bit shaken, leaves the lab.

CHANDLER: (sarcastically) So, Ross, when’s your next show and tell?

That was a success.

Intelligent agents for simulations (using LLMs) #1

On By Jon UreñaIn Escritura, Inteligencia artificial, Programación1 Comment

Next part (2)

A week ago I came across this damned paper (Generative Agents: Interactive Simulacra of Human Behavior). Also check out this explanation of the paper and its implementation. I called the paper damned because the moment I understood its possibilities (creating intelligent virtual agents has always been a dream of mine), the spark of obsession flared up in me, and for someone this obsessive, that could only mean one thing:

I figured that I was capable enough of implementing that paper using the Python programming language. In my first try, I thrust ahead, with very few unit tests, to implement the memories database, the planning and reflective functionalities, as well as the navigation system. It was when I hit the paper’s promise that GPT-3.5-turbo was going to be able to output the name of the chosen destination when I realized that the writers of the paper must have simplified the process of navigation (if not finagled it quite a bit), because large language models don’t work like that. And without the agents being able to move around in the environment tree, this system wouldn’t work remotely as intended.

I gave it some thought and I ended up opting for utility functions. Used in basic AI for decades, utility functions output a rating value between a bunch of options, from which the code ends up choosing the best rated option (with some room for randomness). Thankfully, if you ask GPT-3.5-turbo to give a rating for how well a “farmhouse” would fit the purpose of “eating lunch”, then that location may be chosen, and from there you could ask the AI to rate the individual locations of the farmhouse to continue the path.

Implementing the navigation of the agents in such a way gave me the opportunity to start the code from scratch. This time I relied mostly on test-driven development (meaning that one or more unit tests support the part of the code you are working on), very important for a system that can suffer from plenty of regressions. I have 51 unit tests supporting the code so far.

Before I delve into further details, here’s the link to my GitHub repository with all the code I have written.

Here’s as far as I’ve gotten:

  • The user writes the environment of the simulation in a json file. I’m using the Python library ‘anytree’ to represent the graph, which also comes with this fancy command-line method of displaying any given graph:

Location: plot of land (plot_of_land)
├── Location: barn (barn)
│ └── Sandbox object: tools (tools) | description: lots of tools for farming
├── Location: field (field)
│ ├── Sandbox object: crop of corn (corn_crop) | description: a crop of growing corn
│ └── Sandbox object: crop of potatoes (potatoes_crop) | description: a crop of growing potatoes
└── Location: farmhouse (farmhouse)
├── Location: bedroom (bedroom)
│ ├── Sandbox object: bed (bed) | description: a piece of furniture where people sleep
│ └── Sandbox object: desk (desk) | description: a piece of furniture where a person can read or write comfortably
├── Location: kitchen (kitchen)
│ └── Sandbox object: fridge (fridge) | description: an appliance that keeps food cold
└── Location: bathroom (bathroom)
├── Sandbox object: toilet (toilet) | description: a simple toilet
└── Sandbox object: bathtub (bathtub) | description: a tub in which to take baths

  • For this test, I pictured a farmhouse, a nearby barn, and a couple of nearby crops.
  • I added in the corresponding json file the needed information for the agents involved:

{
“Joel”: {
“age”: 38,
“planned_action”: null,
“action_status”: null,
“current_location_node”: “farmhouse”,
“destination_node”: null,
“using_object”: null
},
“Betty”: {
“age”: 24,
“planned_action”: null,
“action_status”: null,
“current_location_node”: “farmhouse”,
“destination_node”: null,
“using_object”: null
}
}

  • As depicted, agents can have a planned action, an action status, their current location, their destination, and what object they are using, apart from their age and name.
  • As the paper says, each agent’s memories are seeded from a few memories that are written in a simple text file.

Betty started living with Joel six years ago;Betty enjoys planting crops and tending to them;Betty wishes that she and her boyfriend Joel could afford to buy a few horses;Betty wishes she didn’t have to travel to the nearby town often;Betty loves peace and quiet;When Betty feels overwhelmed, she lies down on her bed and listens to ASMR through her noise-cancelling headphones;Betty wants to have a child, hopefully a daughter, but Joel told her years ago that he didn’t want children;Joel and Betty saw strange lights in the skies in the 11th of May of 2023;On the 11th of May of 2023, the local news broadcast was cut off in the middle of a transmission as they were reporting on the strange lights in the sky;Joel and Betty have heard ominous explosions in the distance throughout the morning of the 12th of May of 2023

Joel is a farmer who has lived in his plot of land for ten years;Joel is living with his girlfriend, Betty, whom he loves;Joel enjoys listening to music from the sixties;Joel wishes he and Betty could buy a few horses, but he doesn’t think they can afford them;Joel and Betty saw strange lights in the skies in the 11th of May of 2023;On the 11th of May of 2023, the local news broadcast was cut off in the middle of a transmission as they were reporting on the strange lights in the sky;Joel and Betty have heard ominous explosions in the distance throughout the morning of the 12th of May of 2023

  • These seed memories are inserted into a memories database. Its implementation is one of the most important parts of this system. The database is composed of a vector database (which stores values according to some eldritch, multidimensional distance between them), and a json file that records readable values of those memories. You can query the vector database with simple text, and it will return the most relevant text it contains regarding the query you sent it. Sort of like a recommendation system.
  • In addition, when a query is made to the agents’ memories, they are rated according to the memory’s importance and recency. The recency is a normalized value in the range of [0.0, 1.0] depending on when that memory was accessed last, and importance is a rating that the AI itself gave to how poignant the memory was.
  • Once the agents are loaded in the simulation, a character summary is created for each agent. The AI is prompted to write all the segments of these summaries only from the memories present in their vector databases.

Joel’s AI-written character summary

Joel (age: 38)

Innate traits: Hardworking, resilient, curious, observant, caring, nostalgic, pragmatic.

Joel is a farmer who has lived in his plot of land for ten years and wishes he and Betty could buy a few horses, but he doesn’t think they can afford them. He enjoys listening to music from the sixties and is living with his girlfriend, Betty, whom he loves. They have experienced ominous explosions in the distance and saw strange lights in the skies on the 11th of May of 2023, and the local news broadcast was cut off in the middle of a transmission as they were reporting on the strange lights.

Joel is a farmer who has lived in his plot of land for ten years.

It is difficult to accurately describe Joel’s overall feeling about his recent progress in life based on these statements alone. However, we can infer that he enjoys listening to old music, has heard ominous explosions and seen strange lights with Betty, is a farmer, loves his girlfriend, and wishes they could afford to buy horses. These statements do not provide enough information to determine his overall feelings about his recent progress in life.”

Betty’s AI-written character summary

Betty (age: 24)

Innate traits: resilient, domestic, nurturing, longing, introverted, anxious, curious, intuitive, calm

Betty is a woman who enjoys living a peaceful and simple life with her partner Joel. She is interested in farming and wishes to own horses. Betty also has a desire to have a child, but this conflicts with Joel’s wishes. She dislikes having to travel to nearby towns and finds solace in listening to ASMR when she feels overwhelmed. Betty is aware of strange occurrences happening around her, such as explosions and lights in the sky, but she seems to maintain a calm and composed demeanor.

Betty is a homesteader or a farmer.

Betty is feeling somewhat unsettled and uncertain about her progress in life, as indicated by her mixed feelings about her living situation with Joel, her desire for a more stable and fulfilling life with horses and a child, and her occasional need for escapist relaxation techniques like ASMR. She enjoys the sense of purpose that comes from tending to her crops, but is also wary of the potential dangers and disruptions of the outside world, as shown by the ominous sounds and sightings of explosions and strange lights in the sky. Overall, she seems to value the simple pleasures of a quiet country life, but is also aware of the limitations and difficulties of this lifestyle.

  • If the simulation detects that the agents haven’t decided upon an action, it generates one through a complicated process.

2023-05-13T09:30:00 Joel produced action: Saturday May 13 of 2023, 9 AM. Joel is going to check with his neighbors to see if they have any information about the strange lights and ominous explosions they have been experiencing. The length of time it takes for Joel to check with his neighbors is not specified, so it is impossible to provide an answer in minutes.

2023-05-13T09:30:00 Betty produced action: Saturday May 13 of 2023, 9 AM. Betty is going to talk to Joel about their conflicting desires regarding having a child and try to come to a mutual understanding or compromise. The duration of this action is not specified in the given information. It can vary based on the discussion and resolution reached by Betty and Joel.

  • The AI model (GPT-3.5-turbo in this case) went over each agent’s memories and decided what action the agents would take, according to their circumstances. A problem with the paper’s original implementation already comes up: the agents decide a plan, then they search for the most fitting location, and then object, to fulfill that plan. But in this case, the AI decided that Joel should check with his neighbors to see if they have any information about weird happenings. But there are no neighbors in this environment. In addition, Betty decided that she wants to talk to Joel, but the paper doesn’t offer a system of locating other agents that may not be in the immediate surroundings. There’s a system for dialogue, but I haven’t gotten that far.
  • Through the rating system for locations and objects (going from the root of the environment tree deeper and deeper), Joel came to this conclusion about what should be his destination given the action he came up with:

2023-05-13T09:30:00 Joel changed destination node to: Node(‘/Location: plot of land (plot_of_land)/Location: farmhouse (farmhouse)/Location: kitchen (kitchen)/Sandbox object: fridge (fridge) | description: an appliance that keeps food cold’)

  • Apparently Joel (therefore, the AI model) thought that the neighbors could be found in the fridge. From the root of the environment tree, between the choices of “farmhouse”, “barn” and “crops”, obviously farmhouse was the most adequate option to find people. Inside that house, it opted for the kitchen between all the possible rooms. Inside there was only a fridge, so the navigation system had no choice but to choose that. If I had added a telephone in the kitchen, the AI model likely would have chosen that instead to communicate with the neighbors.
  • Regarding Betty, to fulfill her action of talking to Joel, she chose the following destination:

2023-05-13T09:30:00 Betty changed destination node to: Node(‘/Location: plot of land (plot_of_land)/Location: barn (barn)/Sandbox object: tools (tools) | description: lots of tools for farming’)

  • Interestingly enough, the AI model believed that her farmer husband was more likely to be in the barn than in the farmhouse. Once the barn was chosen, the tools were the only option as to which object would help the agent fulfill her action.
  • The navigation system detected that the agents weren’t at their destination, so it started moving them:

2023-05-13T09:30:00 Joel needs to move to: Node(‘/Location: plot of land (plot_of_land)/Location: farmhouse (farmhouse)/Location: kitchen (kitchen)/Sandbox object: fridge (fridge) | description: an appliance that keeps food cold’)
2023-05-13T09:30:00 Joel changed the action status to: Joel is heading to use fridge (located in Location: kitchen (kitchen)), due to the following action: Saturday May 13 of 2023, 9 AM. Joel is going to check with his neighbors to see if they have any information about the strange lights and ominous explosions they have been experiencing. The length of time it takes for Joel to check with his neighbors is not specified, so it is impossible to provide an answer in minutes.

2023-05-13T09:30:00 Betty needs to move to: Node(‘/Location: plot of land (plot_of_land)/Location: barn (barn)/Sandbox object: tools (tools) | description: lots of tools for farming’)
2023-05-13T09:30:00 Betty changed the action status to: Betty is heading to use tools (located in Location: barn (barn)), due to the following action: Saturday May 13 of 2023, 9 AM. Betty is going to talk to Joel about their conflicting desires regarding having a child and try to come to a mutual understanding or compromise. The duration of this action is not specified in the given information. It can vary based on the discussion and resolution reached by Betty and Joel.

  • The simulation advances in steps of a specified amount of time, in this case thirty minutes, and the navigation system will move the agents a single location each step. This means that an agent would take thirty minutes to move from a bedroom to the entrance of the same house, but whatever.

2023-05-13T10:00:00 Joel changed the current location node to: Node(‘/Location: plot of land (plot_of_land)/Location: farmhouse (farmhouse)/Location: kitchen (kitchen)’)
2023-05-13T10:00:00 Joel changed the current location node to: Node(‘/Location: plot of land (plot_of_land)/Location: farmhouse (farmhouse)/Location: kitchen (kitchen)/Sandbox object: fridge (fridge) | description: a piece of furniture that keeps food cold’)

  • Joel moved from the entrance of the farmhouse to the kitchen, and once the agent is in the same location as his or her destination sandbox object, he or she starts using it. In this case, the fridge.

2023-05-13T10:00:00 Betty changed the current location node to: Node(‘/Location: plot of land (plot_of_land)’)

  • Betty has moved from the farmhouse to the root of the environment tree.
  • In the next step of the simulation, Joel continues using the fridge, while Betty moves further to the barn, where she starts using the tools:

2023-05-13T10:30:00 Joel continues using object: Node(‘/Sandbox object: fridge (fridge) | description: a piece of furniture that keeps food cold’)
2023-05-13T10:30:00 Betty changed the current location node to: Node(‘/Location: plot of land (plot_of_land)/Location: barn (barn)’)
2023-05-13T10:30:00 Betty changed the current location node to: Node(‘/Location: plot of land (plot_of_land)/Location: barn (barn)/Sandbox object: tools (tools) | description: lots of tools for farming’)

  • From then on, the agents will continue using the tools they have chosen to perform their actions:

2023-05-13T11:00:00 Joel continues using object: Node(‘/Sandbox object: fridge (fridge) | description: a piece of furniture that keeps food cold’)
2023-05-13T11:00:00 Betty continues using object: Node(‘/Sandbox object: tools (tools) | description: lots of tools for farming’)
2023-05-13T11:30:00 Joel continues using object: Node(‘/Sandbox object: fridge (fridge) | description: a piece of furniture that keeps food cold’)
2023-05-13T11:30:00 Betty continues using object: Node(‘/Sandbox object: tools (tools) | description: lots of tools for farming’)
2023-05-13T12:00:00 Joel continues using object: Node(‘/Sandbox object: fridge (fridge) | description: a piece of furniture that keeps food cold’)
2023-05-13T12:00:00 Betty continues using object: Node(‘/Sandbox object: tools (tools) | description: lots of tools for farming’)

That’s unfortunately as far as I’ve gotten with my implementation, even though it took me a week. The paper explains more systems:

  • Observation: as the agents use the objects, they are prompted with observations about what other objects in the same location are doing, and about the other agents that may have entered the location. That gives the busy agent an opportunity to stop what they’re doing and engage with other objects or agents.
  • Dialogue: the paper came up with an intriguing dialogue system that became one of my main reasons for programming this, but it requires the Observation system to be implemented fully.
  • Planning: once a day, the agents should come up with day-long plans that will be stored in memory, and that will influence the actions taken.
  • Reflection: when some condition is triggered, the agents retrieve the hundred most recent memories from their databases, and prompt the AI model to create high-level reflections of those memories. The reflections will be stored in the memory stream as well, so that they influence their planning, their decision-taking, as well as the creation of further reflections.

That’s as far as I remember of the paper at this moment. I wanted to get the navigation system done as early as possible, because I considered it the hardest part, and it annoyed the hell out of me partly due to how Python handles references (at one point I was losing the list of ancestors and descendants of the agents’ destination nodes between simple function calls to the same instance of Simulation, for no fucking reason that I could determine). I may move directly onto the Observation system, or take a rest. I already feel my sanity slipping quite a bit, likely because I took a sabbatical from my ongoing novel to do this.

AI news #3

On By Jon UreñaIn Arte, Inteligencia artificial, Programación2 Comments

If you are interested in what GPT-4 can do in games, check out part two of this series, because it showed how people are using ChatGPT for Skyrim and Fallout. Things have developed a bit further. The following video shows someone’s modded companion with whom the player can interact naturally, and who is aware of the world around her.

Truly amazing. As long as the world doesn’t end, imagine how amazing games will become in a few years. At least how amazing modders will make those games, if their companies can’t be arsed.

The following unassuming entry represented a turning point in how I use GPT-4:

Turns out that some clever people out there have proved that some prompts can squeeze significantly more intelligence out of GPT-4. The author of the video had the idea of pushing it a bit further by implementing the following architecture:

  • Given a prompt, ask GPT-4 through the API to give you three separate answers.
  • Send those three answers to GPT-4, asking it to act like a researcher and point out the flaws in those answers.
  • Send the answers along with the researcher’s analysis to GPT-4, and ask it to act like a resolver to determine which answer is more valid (according to the researcher’s analysis), improve it a bit, and present it as the final answer.

I have implemented that system in Python. It was very easy to do, as well. Although it takes significantly longer than just asking regular ChatGPT a question, the results are much better.

I thought about how mini-AGIs (Auto-GPT and BabyAGI) use GPT-4 to simulate artificial general intelligence, and I realized that if you can get GPT-4 to develop a list of tasks in a similar way the previous algorithm is structured, you can then link those tasks to the previous algorithm so that it executes each task in a loop.

I’m not that interested in programming an Auto-GPT at the moment; I’m happy enough getting a list of tasks that I can then send to GPT-4 at my leisure. However, I did code an addition to the previous program, one that sends an objective to GPT-4 and tells it to act like a manager developing a prioritized task list. In fact, as in the previous algorithm, it sends that request thrice, and then those answers are passed to a researcher, whose answer gets passed to a resolver as per the previously displayed algorithm.

Here’s the list of tasks that resulted from sending to my program the objective of programming a particular card game, one I’m developing, in the Rust programming language:

  1. Initialize the game engine, including a GUI that supports displaying cards and decks on screen.
  2. Create data structures to represent the different card types: Exploration Zone, Biomes, Encounters, Events, Resources, Afflictions, Player cards, and Portal cards.
  3. Load card data from toml files into the appropriate data structures.
  4. Develop a function to generate a random Exploration Zone card from the available Exploration Zone cards.
  5. Display the randomly selected Exploration Zone card on the top right corner of the screen.
  6. Develop a function to create the Biomes deck, selecting the matching Biome types as specified in the chosen Exploration Zone card.
  7. Ensure the first Biome card of the Biomes deck is displayed face up next to the Biomes deck.
  8. Develop a function to create the Encounter, Events, Resources, and Affliction decks based on the Feature icons from the chosen Exploration Zone card.
  9. Implement player classes (soldier, medic, scientist, scout) with their respective attributes and abilities.
  10. Design the player decks for 4 players, each with a unique role, and include a Portal card to allow escaping to the Hub.
  11. Design the Hub and implement the card drafting system for the Hub, allowing players to choose cards to add to their decks while trading resources.
  12. Develop a system for players to rebuild their decks while in the Hub between explorations.
  13. Design the goal cards that specify objectives required for each Exploration.
  14. Implement a system that tracks combos and interactions between cards, such as activating abilities based on certain conditions.
  15. Create a data gathering and analysis system for role combinations, deck compositions, and card interactions to identify balance issues and adjust card values and abilities accordingly.
  16. Test the game extensively, iterating on card mechanics, balance, and player engagement.
  17. Polish graphics, UI, and overall user experience to ensure an enjoyable gaming experience.
  18. Incorporate playtesting, gather feedback, and make improvements to game mechanics, card designs, and balance based on feedback and data-driven insights.
  19. Create game documentation, tutorial, or in-game help system to provide instructions and guidance for players on how to play the game and understand its mechanics.
  20. Market and promote the game, develop marketing materials and promotional events to raise awareness about the game, attract players, and grow the community.

As you can see, you could pass most of those tasks to the first algorithm, which would come up with a good answer.

I found myself at times copy-pasting ideas from a working document to the prompt I was sending to GPT-4. I thought that there must be some automated system that would pick the most appropriate domain knowledge for whatever prompt I wanted to send to GPT-4. I asked ChatGPT if it could come up with an answer, and it suggested vector databases. I had no fucking clue what vector databases were, but now I know.

Vector databases translate information into “webs” of data according to the distance between the elements, or something to that effect. What’s important is that if you query the database with some information, it will return the most similar information it contains related to what you asked. Sort of like a recommendation system.

The ‘annoy’ crate for Python seems to be one of the most commonly used systems for implementing vector databases, so that’s what I used. Now I just have to paste all the domain knowledge into a txt file, then run a simple program that registers the knowledge into the appropriate vector database. When I’m ordering the other couple of programs to either develop a task list or perform a task, they query the vector database to retrieve the most relevant domain knowledge. Pretty cool.

Interdimensional Prophets – Deckbuilder (Game Dev) #2

On By Jon UreñaIn Arte, Inteligencia artificial, Programación1 Comment

If someone had told me a few years ago, when I was obsessed with board and card games, that in a few days I would have developed a Python program that generates game cards effortlessly, I would have jumped for joy. Working with Python, coming from Rust in particular, is like going from aerospace engineering to building toy rockets; thankfully, a card generating program doesn’t require the speed and multiprocessing that Rust provides.

Anyway, check out the current cards I’m working with as I keep developing the program:

This is an Encounter card, which represent the weird shit that the team of explorers come across as they explore alternate Earths. The name of the card and the image are self-evident. The tree icon indicates that this encounter can only happen in a biome with an icon that matches that type. The row of icons below are the Struggle icons. In the game, the players should match those icons with their player cards to consider the encounter beaten. Those struggle icons depicted are Emotional, Cognitive and Environmental respectively.

Images courtesy of Midjourney, of course.

Here are some Biomes:

I know that the icon designs don’t match each other, but whatever.

I must thank again the most powerful large language model we have access to: GPT-4. It’s like having an extremely knowledgeable veteran programmer ready to help you at all times. For example, an hour ago I thought that the icons could use some subtle drop shadows. I had no clue how to even begin programming that, so I just asked GPT-4. After a short back and forth (in the first attempt it dropped shadows for the invisible part of the alpha channel), the icons now drop perfect shadows. How about that?

I have already started working on the Rust version of the game, using the Bevy crate, which seems to be the most advanced game development engine in that language. I have displayed a few encounter cards that move smoothly to the center of the screen for no particular reason other than that I wanted to figure out how to move stuff smoothly on screen.

Next up, I’ll focus on developing the necessary cards and Rust systems to make the following happen:

  • Design and generate an Exploration Zone card, which are the cards that determine which types of Biomes and Encounters can show up during an exploration (only those with matching icons can).
  • Display the Exploration Zone card in Rust.
  • Write the code to build the Biomes deck with only matching Biomes.
  • Display the Biomes deck on screen in Rust.
  • Write the code to build the Encounters deck with only matching Encounters.
  • Display the Encounters deck on screen in Rust.