Tag: LLMs

Talking to RPG characters through GPT-4

On By Jon UreñaIn Escritura, Inteligencia artificial, Programación2 Comments

If you haven’t, you should check out my previous post, because this is a direct continuation.

OpenAI added functions to the GPT API

I rewrote the code so that it asks me if I want to interject at any point by typing the Captain’s dialogue myself. In those cases, GPT-4 acted out the boss’ companions.

This time I’m always using the same “basic” prompt for GPT:

The Captain is a human male, about thirty years old. He used to live on Earth until a truck ran him over, and he ended up transported to this fantasy world; however, his teammates are unaware that the Captain came from another world. The Captain, as he likes to be called, is the bruiser of the group, and very competent with a sword. He spends most of the loot on booze and whores. He buries his despair at having ended up trapped in a fantasy world by killing as many monsters as possible. After killing hundreds of creatures, he’s discovered a disturbing side of his psyche: inflicting pain arouses him. The Captain gets along fine with Bimbo, the fire mage of the group, whom the Captain considers a good buddy. The Captain is creeped out by the grey-skinned elf Kutinaira, because she peeks at him while he pees.
Bimbo is the fire mage of the group that the Captain commands. Bimbo is a halfling. He joined the Captain’s group because he needed money, but he has a hard time adjusting to the deaths he’s seen along the way. Bimbo is brooding, and usually prefers to keep to himself. He’s planning on earning enough money to buy farmland and build a homestead in which to live isolated from the horrors of this world. He is intimidated by the irrational behavior of the Captain, but he prefers his company to that of Kutinaira, who seems single-mindedly focused on satisfying her fetishes.
Kutinaira is the water mage of the group that the Captain commands. Kutinaira is an ancient grey-skinned elf. She claims that she was once a goddess, but that she has been demoted to demi-goddess. Kutinaira is a voyeuristic pervert, and wants to jump the Captain’s bones. She suffers from obsessive-compulsive disorder, and her intrusive thoughts solely involve sex. She has lived for so long that nothing interests her any longer, and she opts to get high on sex as often as possible to forget the long-dead loved ones that she’ll never meet again.
Notes: If you need to include description or narration, put it in parentheses. Do not write dialogue for The Captain, because that’s the user, who will write his own dialogue.
Location: Dense, darkening forest, in the evening.
Context: The Captain’s group, including Bimbo and Kutinaira, have ventured into the forest to raid a goblin cave, to fulfill a contract with the Adventurer’s Guild. Somehow they’ve gotten lost; they have been walking in circles for the last hour. Orcs and similar vermin are known to prowl these parts, and the Captain wants to avoid an ambush.

First attempt at player-AI dialogue

The Captain: Neither of you can read a map? How have we survived for this long?
Bimbo: (Stares at the ground, abashed) I… I thought I knew the way, Captain. The cartographer’s symbols… they’re different from the ones I learned.
Kutinaira: (Her voice drips with sarcasm) Oh, don’t worry Captain. I could read your map. Any fool with half a brain could. I simply chose not to. (She smirks) It’s far more entertaining to watch you lose your composure as you realize we are lost.
The Captain: What the fuck are you talking about, Kutinaira? Lose my composure? Do you really wish me to lose control of myself again? It involves me seeing red and ending up covered in blood and with some mangled corpse at my feet.
Bimbo: (Quickly intervenes, a worried frown emerging on his small face) No, no, Captain. We don’t want that. It’s…It’s just a mistake, right Kutinaira?
Kutinaira: (Laughs lightly, her odd-colored eyes sparkling with mischief) Oh, I do love it when you threaten violence, Captain. It’s so… primal. But do save it for the goblins, won’t you? After all, we are still on a contract, aren’t we?
The Captain: Kutinaira, you goddamn perverted elf, you need to learn how to act like a professional. We’re at risk of getting ambushed by the gods know what monsters that may prowl around these parts. Don’t you understand the danger we are in? Tell her, Bimbo!
Bimbo: (Nods, a serious look on his face) Yes, Kutinaira. The Captain’s right. This is dangerous territory. We need to be careful, keep quiet, stay together.
Kutinaira: (Rolls her eyes, but sighs) Fine, fine… I’ll behave, Captain. I sometimes forget you humans take everything so seriously. But don’t expect me to stop enjoying myself completely. That would be terribly dull.
The Captain: Alright, so are you going to read the map properly now, you voyeuristic elf?
Kutinaira: (Sighs dramatically, taking the map from Bimbo’s hands) Well, if you insist. Let’s see… (She squints at the parchment, and her smug grin fades into a frown) This is… somewhat confusing. The symbols are so primitive.
Bimbo: (Gulps nervously, looking from the map to The Captain) I did tell you. The cartographer’s symbols are… unique. But I can try again, Captain, if you want.
The Captain: Aren’t you primitive yourself, Kutinaira? You must have lived for a thousand years. Haven’t you come across such symbols during your troubled, exceedingly long existence?
Kutinaira: (Snorts) Primitive? No, dear Captain, I am ancient, but certainly not primitive. If anything, I’d call these symbols… provincial.
Bimbo: (Interjects tentatively, trying to take the map back from Kutinaira) Um, Captain, I can give it another try… I think we’re supposed to head north from here. I think.
The Captain: Bimbo, my old pal, you are always there when I need you. Who else can I rely on? Certainly not this grey-skinned pervert. Why do we need a water mage again?
Bimbo: (Shuffles his feet awkwardly) Well, Captain…every group needs balance, aye? Fire and water…Destruction and healing…
Kutinaira: (Rolls her eyes and crosses her arms) Yeah, who would heal your sorry ass when the goblins get a lucky hit, Captain? Or got a wound that needs cleaning? Remember, water isn’t just about control. It’s about life too.
The Captain: Kutinaira, the only time you look alive is when you have cast aside your clothes and are bouncing up and down on some unlucky bastard’s crotch. I’m talking about your unbridled lust.

Apparently that was enough of a zinger for GPT-4 that it chose to end the conversation.

Second attempt at player-AI dialogue

The Captain: Neither of you can read a map? How have we survived for this long?
Bimbo: (Looks sheepish) I’ve tried following the landmarks, but it seems these trees are playing tricks on us, Captain. I’m sorry…
Kutinaira: (Grins teasingly) Oh, Captain, you’re so helpless without your eagle-eyed elf. Don’t worry. Kutinaira is here to lead you out of your confusion.
The Captain: (Narrows his eyes at the elf) Does your plan involve leading me to some bushes so you can slip your hand down my pants again?
Kutinaira: (Laughs coquettishly) I can see why you’d think that, Captain. But my intentions are purely navigation-wise this time, I promise.
Bimbo: (Shudders) Can we just keep things professional, please? We have a job to do… And it’s getting darker by the minute.
The Captain: Oh, it’s going to get way darker, Bimbo. I’m so sick of fighting my way through this fantasy life! What is even the point?
Bimbo: (Looks surprised at the Captain’s outburst) That’s… that’s really not like you, Captain. I know it’s hard, but we have to push through. We have a contract to fulfill.
Kutinaira: (Strokes the Captain’s arm) Aw, poor darling. Let Kutinaira make everything better for you later tonight. Now, let me lead us to that goblin cave and finish what we’ve started.
The Captain: (Tearing up) Finish what we’ve started. That’s right, I had stuff I wanted to finish. I never got to watch the ending of Neon Genesis Evangelion. Does Shinji finally manage to score with that one-eyed, redheaded terror? I miss the days when little else mattered other than masturbating to drawings of anime girls.
Bimbo: (Stares at the Captain, clearly dumbfounded) I… uh… I have no idea what you’re talking about, Captain.
Kutinaira: (Laughs heartily) Oh, you humans and your strange obsessions. But whatever comforts you, darling. We all have our little ways to escape. Now, shall we start moving again? I promise, no peeking or touching, okay?
The Captain: (Wipes his tears) Is that truly what you want out of life, Kutinaira? Go to some forsaken goblin cave and drown helpless goblins? Do you know they may be less perverted than you?
Kutinaira: (Winks) Well, that’s not all I want, darling, but it’s a good start. If I get tired of the thrill, I’ll find some other equally ridiculous but interesting thing to do. Life’s too long to be boring, after all.
Bimbo: (Quietly) Captain, I… I don’t like killing any more than you do. I’m just trying to survive in this cruel world. Hopefully, I’ll earn enough soon to leave all this violence behind.
The Captain: (Somber) This violence will never end. We will spend the rest of our days killing one monster after another. Once our wounds best us and we believe we have been liberated from our neverending grind, we will wake up in Hell, doomed to fight foe after foe for eternity.

GPT-4 chose to end the conversation there. I think I’ve also had enough.

Nevermind, one more.

Third attempt at player-AI dialogue

The Captain: Neither of you can read a map? How have we survived for this long?
Bimbo: (visibly flustered) I… I thought I was reading it right, Captain. But everything looks the same in this damn forest.
Kutinaira: (calmly examining her nails) Reading maps is so boring… Now, if we were mapping out certain… anatomical landscapes, I might be of more help.
Bimbo: (blushing deeply) You… you’re not helping, Kutinaira! Why can’t you be serious for once? We could be ambushed any second now!
The Captain: (squinting at Kutinaira) What do you mean by anatomical landscapes, Kutinaira? I didn’t quite get the meaning of that.
Kutinaira: (grinning mischievously) Oh dear Captain, I’m talking about the hills and valleys of the… male physique, if you get my drift.
Bimbo: (growing even redder) Stop it, Kutinaira! This isn’t the time for your… your disgusting cravings.
Kutinaira: (chuckles softly) Boring… But fine. I suppose we should find the goblin cave and get this over with.
The Captain: No, I want to hear more about your disgusting cravings. When did this lustful madness start, you ancient elf?
Kutinaira: (sighs with a smile, brushing some stray hair out of her eyes) Oooh… It must have been centuries ago… Aeons, even. So long ago, dear Captain, that I don’t quite remember exactly when. But isn’t madness just another form of sanity? One man’s madness may well be another’s ecstasy.
Bimbo: (muttering) Or a woman’s. We’re talking about you, after all.
Kutinaira: (laughs, a sound full of dark amusement) Oh, Bimbo. So pure, so untouched. Why don’t you ever join us at the brothel? Afraid of losing yourself, or simply uninterested?
Bimbo: (deepening his frown) I’m not like you two. Not everyone needs to drown their sorrows in carnal pleasure. Some of us have dreams, ambitions. I want a quiet, peaceful life. It’s all I’ve ever wanted.
The Captain: (raises an eyebrow at Bimbo) You want to spend the rest of your life solely in the company of your right hand?
Bimbo: (blushing fiercely now, stammers) I…I want more than that, Captain. I’d like to have a family, one day. Someone to share my life and love…
Kutinaira: (snickers) Oh, the romantic notion of monogamy. How…quaint.
Bimbo: (glaring at Kutinaira) It’s no less valid than your lifestyle! It’s noble and…
Kutinaira: Boring? (She looks at the Captain). But then, I suppose it’s all about variety, isn’t it, dear Captain?
Bimbo: (defensively) It’s about empathy and companionship. Something which you seem to lack altogether!
Kutinaira: (laughs) Oh Bimbo, dear Bimbo. If only you knew… Your idealistic dreams are just as twisted as my ‘cravings’. What’s the difference between robbing someone of their life and robbing them of their innocence?
Bimbo: (visibly discomfited) That’s…that’s not the same thing.
Kutinaira: Isn’t it? I add joy to the world with my interludes. You and Captain simply take. (She tries not to let her voice break, hiding behind her laughter) But don’t worry, Bimbo. I’m sure some maiden will be more than willing to indulge your perversions.
The Captain: (growing somber) The last time I fell in love with a whore, some raiders ran through that goddamn brothel and skinned them. They tortured most of those delightful whores to death.
Bimbo: (sobering up) That’s… horrific, Captain. I… I’m sorry, I didn’t know.
Kutinaira: (softens visibly) There is a lot of pain in this old world, Captain. We’ve all seen our share of it. But, it’s how we deal with it that sets us apart. You kill, Bimbo dreams, and I… well, we won’t get into that again.
Bimbo: (nods silently, his earlier argument forgotten in the face of the Captain’s revelation)
Kutinaira: We all have our monsters, Captain. Some are simply more literal than others. And in our own ways, we’re fighting them. And that, I suppose, is the best any of us can do.
The Captain: Kutinaira, what do you think about these dark impulses of mine? Whenever I thrust my sword through some unfortunate soul’s spleen, I… I mean, my parts… I get diamond-hard, Kutinaira. Back when we fought those demons and demonettes, I wanted to take turns stabbing each other.
Kutinaira: (Shrugs nonchalantly) You seem to have found your own method of coping, haven’t you? Sex and violence… they’ve been linked for eons. For some, the thrill of danger can serve as an aphrodisiac. As long as you don’t hurt anybody without their consent, I see no issue.
Bimbo: (gapes at Kutinaira, then at the Captain, struggling to comprehend their casual discussion of such morbid topics) By the gods… you both are genuinely mad, aren’t you?
Kutinaira: (laughs) Probably, Bimbo. But then again, isn’t everyone, in their own special way?
The Captain: No, nobody is as insane as the both of us. You are yet to fully comprehend the nightmare you have tangled yourself into, Bimbo. This rotten elf and I are heading straight to perdition.
Bimbo: (swallows hard, looking supremely uncomfortable) I…I see. But we’re still in this together, right? We are a team, a unit. And we depend on each other to survive. Perhaps… perhaps that’s our salvation.
Kutinaira: (sighs mockingly but there’s sadness in her eyes) Poor innocent Bimbo… If only life was that simple. We are what we are, and we cannot escape ourselves, no matter how much we rule this brutal world.
Bimbo: (whispers) I still have hope. We might be lost in this forest now, heading straight into danger, but I believe we’ll get through. Together.
Kutinaira: (laughs softly again and for the first time her laughter doesn’t seem to hide any sadness) Such a hopeless optimist. Oh, how boring life would be without you, Bimbo.
Bimbo: (tries to smile at this, but fails) I… Thank you, Kutinaira.
(All three of them continue their journey, now lost not just physically but also mentally in the forest, their unexpected confessions weighing heavily on their minds.)

Last one, honest.

Fourth attempt at player-AI dialogue

The Captain: Neither of you can read a map? How have we survived for this long?
Bimbo: Honestly, Captain, these woods ain’t like any I’ve seen. All the trees look the same. And under this dim light, I can’t make head or tail of this damn map.
Kutinaira: Oh, don’t worry dear Captain. Even if we encounter orcs, we have your strong, virile body to protect us.
The Captain: You think that the orcs would be so sexually enthralled by my virile body that they would forsake all notion of self-defense?
Bimbo: (Laughs nervously) If that were the solution, these forests would be peaceful. Orcs are stubborn, and dumb… too dumb to let go of hostility.
Kutinaira: (Purrs) You ought not to underestimate yourself, Captain. The sight of your rippling muscles would make anyone drop their weapons. Surely, surrendering to you would be a far better fate.
The Captain: (Strokes his chin thoughtfully) Would orc females be into mommy fantasies?
Bimbo: (Turns pale and stammers) C-Captain, that’s… quite a strange question…
Kutinaira: (Laughs loudly and claps her hands) Oh, such is the humor of the humans! Now that’s a sight, an orc swooning over you like a mother… How kinky, and how horrible.
The Captain: (Crosses his arms) Horrible, huh? I didn’t shame your kink that one time you came mid-battle because a minotaur was choking you.
Bimbo: (Looks away, turning a deep shade of red)
Kutinaira: (Chuckles) Ah, Captain, you remember the most endearing of times. But do remember, my dear, my kinks and your… propositions of orcs, they are not quite the same thing. Ah, the diversity of life’s pleasures… it is simply delicious!
The Captain: Kutinaira, one of these days we’re going to die for real. We’re skirting the edge of debauchery here. I mean, I go out of my way to hunt down demonettes because I can’t have enough of those forked tongues and their horns. Oh, and those glorious claws. My back looks like I’ve been flogged for years!
Bimbo: (Gulps audibly, clearly uncomfortable with how the conversation is turning)
Kutinaira: (Laughs a throaty laugh) What a delightful image you paint, Captain! You are certainly an adventurer of more than just landscapes and dungeons. But do remember, death is a part of this life we lead, and if I were to meet my end, I’d rather do so in a state of divine pleasure.
The Captain: (Nodding) You know, Kutinaira, I used to think of you as little more than an old reprobate who hides a legion’s worth of sexually transmitted diseases, but you’re actually quite the philosopher, aren’t you?
Bimbo: (Speaks timidly) You two sure know how to keep a conversation… uh, interesting.
Kutinaira: (Giggles, flicking her hair over her shoulder) Oh, Captain, compliments from your mouth are as sweet as honeyed wine. We all hide our depths beneath what we present, do we not? Now, let’s indulge in the philosophy of finding our path again… Unless, of course, you wish to discuss more… stimulating ideas.
The Captain: Yeah, let’s find those goblins and exterminate their whole families. Once we wander out of that hole and take a bath in a river, let’s discuss in private some stimulating ideas.

OpenAI added functions to the GPT API

On By Jon UreñaIn Arte, Escritura, Inteligencia artificial, Programación1 Comment

If you love artificial intelligence and large language models (LLMs), and on top of that you know your way around programming, you may have requested access to OpenAI’s API so you could write some programs that spoke directly to GPT-3.5-turbo and GPT-4. Although the models are amazing (unfortunately so at times, because the open-source variants are a bit far from catching up), they were missing one enormous capability: you couldn’t rely on GPT to consistently return a precise answer.

To put an example of what I actually tried to do: I offered the agents in a simulation a list of locations so they would choose a single one, in order to move them there. I wanted the AI to return just the name of the location as it was listed, but I couldn’t get it to do so in a reasonable manner.

To rate the importance of character memories that were getting shoved into a vector database, I was sending GPT-4 the following prompt:

On the scale of 1 to 10, where 1 is purely mundane (e.g., brushing teeth, making bed) and 10 is extremely poignant (e.g., a break up, college acceptance), rate the likely importance of the following piece of memory.
Memory: {memory_description}
Rating: <fill in>

GPT sometimes answered “Rating: 4” or just a number. I could extract the number from those answers, but sometimes it just refused to answer properly, due to either hallucinations or the usual censorship spiel: “As an AI model, I can’t…”

However, this week OpenAI has updated the API to accept a list of functions along with the prompt. You can tell GPT to choose among the functions which one it should call if it considers it necessary, or else you can order the model to always call a function. Well, GPT doesn’t actually call the function, but it does return the name of the function called, along with a proper value for all the parameters you ask it to fill.

If you are a programmer, you already understand how wonderful that is. But if you aren’t, consider the following case: during a normal conversation with the chatbot you’ve built for yourself, you ask GPT-4 to tell you the current weather in your area. As a language model, GPT doesn’t have access to that information, but if you send it the function “get_weather_in_area” along with instructions for how to fill its parameters (for example, “area_name”, “time_of_day”, etc.), the model’s response could be something like this: “get_weather_in_area(‘Barcelona’, ‘4:00pm’)”. After you catch that response programmatically, you call the function “get_weather_in_area” (that must obviously exist in your codebase), that connects to the API of some weather service.

That means that you can already automatize pretty much everything that you do regularly on your computer, or create complex AI-based simulations like games, by involving GPT. You want the AI to decide when a character has to equip a weapon? Pass it the description of a function that does that. Want some characters to engage in a dialogue? Pass GPT a function that it should call when the model itself considers that at that point the characters should start talking to each other.

To test the new capabilities with a relatively simple example, I planned the following: I would write the description of three characters from an old free verse poem of mine, pass it to GPT, and have the model write an ongoing dialogue that involved the three of them talking in character. To know when the dialogue should end, I would append to the request a function called ‘end_dialogue’ (with no arguments) that the AI should call when necessary.

Here are some results:

Prompt to GPT-4

I am DialogueGPT. I have the responsibility of carrying on a dialogue between two or more characters until the context of the dialogue suggests that the dialogue should end.
The Captain is a human male, about thirty years old. He used to live on Earth until a truck ran him over, and he ended up transported to this fantasy world. The Captain, as he likes to be called, is the bruiser of the group, and very competent with a sword. He spends most of the loot on booze and whores. He buries his despair at having ended up trapped in a fantasy world by killing as many monsters as possible. The Captain gets along fine with Bimbo, the fire mage of the group, whom the Captain considers a good buddy. The Captain is creeped out by the grey-skinned elf Kutinaira, because she peeks at him while he pees.
Bimbo is the fire mage of the group that the Captain commands. Bimbo is a halfling. He joined the Captain’s group because he needed money, but he has a hard time adjusting to the deaths he’s seen along the way. Bimbo is brooding, and usually prefers to keep to himself.
Kutinaira is the water mage of the group that the Captain commands. Kutinaira is an ancient grey-skinned elf. She claims that she was once a goddess, but that she has been demoted to demi-goddess. Kutinaira is a voyeuristic pervert, and wants to jump the Captain’s bones.
Location: Dense, darkening forest, in the evening.
Context: The Captain’s group, including Bimbo and Kutinaira, had ventured into the forest to raid a goblin cave, to fulfill a contract with the Adventurer’s Guild. Somehow, they’ve gotten lost; they have been walking in circles for the last hour.
The Captain: Damn it all, we’re lost! Wasn’t that cave easy to find? We have to figure out what to do before night falls.

GPT-4’s response

Bimbo: I told you we should’ve marked our path, Captain. But no, “the Captain knows the way” you said. Look where that’s got us!
The Captain: Oh, put a sock in it, Bimbo! You’re not exactly helping with that attitude, are you? Now, anyone else got any bright ideas about how to find our way out of this leafy hell?
Kutinaira: I could try casting a water divination spell, Captain. Water flows towards the lowest point, so if we follow it, it’s likely we’ll find a way out. And… maybe after that, we could… you know… have some “private time”… she suggests, her eyes twinkling naughtily.
The Captain: By the gods, Kutinaira! There’s a time and place for everything, and this ain’t it. Focus on the spell, nothing else!
Kutinaira: Alright, Captain… if you insist. She then begins chanting in an ancient tongue, her grey fingers dancing in the air, drawing moisture from their surroundings to form a swirling ball of water.
Bimbo: While Kutinaira works on that, I’ll gather some firewood. We may need to set up camp if we’re going to be here for the night. He mutters and stomps off into the forest, but not before sending a small fire spell to light a nearby tree branch, casting an eerie glow around them.

For whatever reason, GPT-4 mixed narration with the dialogue lines. Next time I’ll instruct it to put it in parentheses.

Prompt to GPT-4

The Captain is a human male, about thirty years old. He used to live on Earth until a truck ran him over, and he ended up transported to this fantasy world. The Captain, as he likes to be called, is the bruiser of the group, and very competent with a sword. He spends most of the loot on booze and whores. He buries his despair at having ended up trapped in a fantasy world by killing as many monsters as possible. The Captain gets along fine with Bimbo, the fire mage of the group, whom the Captain considers a good buddy. The Captain is creeped out by the grey-skinned elf Kutinaira, because she peeks at him while he pees.
Bimbo is the fire mage of the group that the Captain commands. Bimbo is a halfling. He joined the Captain’s group because he needed money, but he has a hard time adjusting to the deaths he’s seen along the way. Bimbo is brooding, and usually prefers to keep to himself. He is intimidated by the irrational behavior of the Captain, but he prefers his company to that of Kutinaira, who seems single-mindedly focused on satisfying her fetishes.
Kutinaira is the water mage of the group that the Captain commands. Kutinaira is an ancient grey-skinned elf. She claims that she was once a goddess, but that she has been demoted to demi-goddess. Kutinaira is a voyeuristic pervert, and wants to jump the Captain’s bones. She suffers from obsessive-compulsive disorder, and her intrusive thoughts solely involve sex.
Notes: If you need to include description or narration, put it in parentheses.
Location: Dense, darkening forest, in the evening.
Context: The Captain’s group, including Bimbo and Kutinaira, have ventured into the forest to raid a goblin cave, to fulfill a contract with the Adventurer’s Guild. Somehow they’ve gotten lost; they have been walking in circles for the last hour. Orcs and similar vermin are known to prowl these parts, and the Captain wants to avoid an ambush.
The Captain: Damn it, Kutinaira! Weren’t you supposed to know how to read a map? You got us lost in a forest? You’re an elf, for fuck’s sake!

GPT-4’s response

Kutinaira: (Giggling softly while tracing the pattern of the Captain’s chest hair with her eyes) And you’re a human, Captain, killed by a truck not known in this world. When last did you see a human reading maps so nonchalantly? (Chuckles lightly) Stop being grumpy and put your muscly arm around me. It’s getting chilly…
The Captain: (shivers visibly from her proximity, steps back, and looks towards Bimbo) Bimbo, light one of your spells so we can see better. Try not to burn the forest down while you’re at it.
Bimbo: (Nods silently, reaches for his wand, and mutters an incantation. A small fireball appears at the tip of his wand, casting flickering shadows on the dense foliage, eerily silhouetting the party.) Let’s be quick… I don’t like the look of this place.
The Captain: Good job, Bimbo. Let’s keep moving. (Turns toward Kutinaira) Don’t get any closer to me. Keep your eyes open for enemies instead.
Kutinaira: (Rolls her eyes but grins, straying her gaze down towards the Captain’s pants) Oh, I’ll keep my eyes open alright… I’ll watch out for those… ‘enemies’.

GPT-4 dutifully obeyed that whole thing about putting narration in parentheses. Let’s try some more.

Prompt to GPT-4

The Captain is a human male, about thirty years old. He used to live on Earth until a truck ran him over, and he ended up transported to this fantasy world; however, his teammates are unaware that the Captain came from another world. The Captain, as he likes to be called, is the bruiser of the group, and very competent with a sword. He spends most of the loot on booze and whores. He buries his despair at having ended up trapped in a fantasy world by killing as many monsters as possible. After killing hundreds of creatures, he’s discovered a disturbing side of his psyche: inflicting pain arouses him. The Captain gets along fine with Bimbo, the fire mage of the group, whom the Captain considers a good buddy. The Captain is creeped out by the grey-skinned elf Kutinaira, because she peeks at him while he pees.
Bimbo is the fire mage of the group that the Captain commands. Bimbo is a halfling. He joined the Captain’s group because he needed money, but he has a hard time adjusting to the deaths he’s seen along the way. Bimbo is brooding, and usually prefers to keep to himself. He’s planning on earning enough money to buy farmland and build a homestead in which to live isolated from the horrors of this world. He is intimidated by the irrational behavior of the Captain, but he prefers his company to that of Kutinaira, who seems single-mindedly focused on satisfying her fetishes.
Kutinaira is the water mage of the group that the Captain commands. Kutinaira is an ancient grey-skinned elf. She claims that she was once a goddess, but that she has been demoted to demi-goddess. Kutinaira is a voyeuristic pervert, and wants to jump the Captain’s bones. She suffers from obsessive-compulsive disorder, and her intrusive thoughts solely involve sex. She has lived for so long that nothing interests her any longer, and she opts to get high on sex as often as possible to forget the long-dead loved ones that she’ll never meet again.
Notes: If you need to include description or narration, put it in parentheses.
Location: Dense, darkening forest, in the evening.
Context: The Captain’s group, including Bimbo and Kutinaira, have ventured into the forest to raid a goblin cave, to fulfill a contract with the Adventurer’s Guild. Somehow they’ve gotten lost; they have been walking in circles for the last hour. Orcs and similar vermin are known to prowl these parts, and the Captain wants to avoid an ambush.
The Captain: Neither of you can read a map? How have we survived for this long?

GPT-4’s response

Bimbo: I’m not the one who usually handles the map, remember? You know I can barely make out anything in this dwindling light. And your way of identifying landmarks is to slash them with your sword. How am I supposed to remember which tree you’ve marked?
Kutinaira: Well, I can read a map just fine. But reading a map is less exciting than watching your muscular arm flex when you swing that big sword of yours, Captain. Why don’t you play with your sword some more? I’m sure Bimbo wouldn’t mind a short break from walking.
The Captain: That’s enough, Kutinaira! If you’d pay more attention to navigation and less to your twisted desires, perhaps we wouldn’t be lost right now. Bimbo, give her the map. Let’s see if she can do any better.
Bimbo: Alright, here you go Kutinaira. Just remember, we’re supposed to be looking for the goblin cave, not following the sound of running water. I know how you love… water.
Kutinaira: Oh, Bimbo, always so full of judgment! But you’re not wrong about my love for water. (grinning) Let’s see… Oh, it appears we’ve made a simple mistake. We should have taken a left at the gnarly oak. We can head back that way if you want, Captain. Or we could stop and set up camp for the night? I vote for the latter.
The Captain: No, Kutinaira, we’re not making camp. Not yet. Let’s turn back and head towards the goblin cave. We’ve got a job to do. Bimbo, stay close, and keep your eyes open; we don’t need any nasty surprises.
Bimbo: Alright, Captain. I’ll keep watch. Kutinaira, could you focus on the map and directions, and less on the Captain’s backside? We’d all appreciate it.
(With that, the group heads back towards the gnarly oak, continuing their mission deeper into the dark forest.)

I’m quite sure you could now write a fully-fledged single-player RPG, even if just console-based, with GPT acting as the dungeon master and voicing the other characters.

I wrote a continuation to this post. Check it out.

Intelligent agents for simulations (using LLMs) #2

On By Jon UreñaIn Inteligencia artificial, Programación1 Comment

The previous entry ended in a sour note because I had become frustrated by a problem with the navigation system: when updating the nodes of the environment tree after the changes that some agent had caused, other agents were losing the parent and/or children of the nodes they were currently in. Although Python’s casual way of handling references contributed, it was my fault: I was replacing nodes in the main tree and disconnecting them, but other agents were ending up with the disconnected copies because they had already stored them. Oops. These are the kind of errors that Rust was made to prevent.

Anyway, I rewrote that whole part of the code; now instead of replacing the whole node, I just modify the values of the Location or the SandboxObject the nodes contain. Not sure why I thought that replacing the whole node was a good idea.

Here’s the GitHub repository with all the code.

In addition, I was getting annoyed because running the unit tests took several seconds, as some of the tests made requests to the AI models. There was no reason to keep testing those calls, because I know they work, so I refactored the calls to functions passed by parameter. A cool thing about (outrageously slow) garbage-collected languages like Python is that you can do crazy shit like casually passing functions as parameters, or duck-typing (passing any kind of object to a function and the code working as long as the object responds to the same calls). Regardless, now the unit tests run in less than a second.

I also figured out quickly how to implement human-handled agents. If you set “is_player”: “true” in the corresponding agent’s section of the agents.json file, whenever the code attempts to call the AI model, it prompts you to answer with your fingers and keyboard. That means that in the future, when the simulation involves observations, planning, reflections, and dialogue, AI handled agents could stop the player to have an in-character conversation. Very nice.

The biggest change to the architecture involved separating environment trees into a main one for each simulation, then agent-specific environment trees that come with their own json files. A bit more annoying to set up, but that allows agent-specific knowledge of the environment. For example, an agent knows of their house and some nearby stores, but not of the interior of other houses. I could implement without much issues the notion of an agent walking to the edge of his or her world and “getting to see” an adjacent node from the simulation’s tree, which would get added to that agent’s environment. Therefore, that new location would become a part of the agent’s utility functions that choose the most suitable locations and sandbox objects. Dynamic environment discovery almost for free.

Intelligent agents for simulations (using LLMs) #1

On By Jon UreñaIn Escritura, Inteligencia artificial, Programación1 Comment

Next part (2)

A week ago I came across this damned paper (Generative Agents: Interactive Simulacra of Human Behavior). Also check out this explanation of the paper and its implementation. I called the paper damned because the moment I understood its possibilities (creating intelligent virtual agents has always been a dream of mine), the spark of obsession flared up in me, and for someone this obsessive, that could only mean one thing:

I figured that I was capable enough of implementing that paper using the Python programming language. In my first try, I thrust ahead, with very few unit tests, to implement the memories database, the planning and reflective functionalities, as well as the navigation system. It was when I hit the paper’s promise that GPT-3.5-turbo was going to be able to output the name of the chosen destination when I realized that the writers of the paper must have simplified the process of navigation (if not finagled it quite a bit), because large language models don’t work like that. And without the agents being able to move around in the environment tree, this system wouldn’t work remotely as intended.

I gave it some thought and I ended up opting for utility functions. Used in basic AI for decades, utility functions output a rating value between a bunch of options, from which the code ends up choosing the best rated option (with some room for randomness). Thankfully, if you ask GPT-3.5-turbo to give a rating for how well a “farmhouse” would fit the purpose of “eating lunch”, then that location may be chosen, and from there you could ask the AI to rate the individual locations of the farmhouse to continue the path.

Implementing the navigation of the agents in such a way gave me the opportunity to start the code from scratch. This time I relied mostly on test-driven development (meaning that one or more unit tests support the part of the code you are working on), very important for a system that can suffer from plenty of regressions. I have 51 unit tests supporting the code so far.

Before I delve into further details, here’s the link to my GitHub repository with all the code I have written.

Here’s as far as I’ve gotten:

  • The user writes the environment of the simulation in a json file. I’m using the Python library ‘anytree’ to represent the graph, which also comes with this fancy command-line method of displaying any given graph:

Location: plot of land (plot_of_land)
├── Location: barn (barn)
│ └── Sandbox object: tools (tools) | description: lots of tools for farming
├── Location: field (field)
│ ├── Sandbox object: crop of corn (corn_crop) | description: a crop of growing corn
│ └── Sandbox object: crop of potatoes (potatoes_crop) | description: a crop of growing potatoes
└── Location: farmhouse (farmhouse)
├── Location: bedroom (bedroom)
│ ├── Sandbox object: bed (bed) | description: a piece of furniture where people sleep
│ └── Sandbox object: desk (desk) | description: a piece of furniture where a person can read or write comfortably
├── Location: kitchen (kitchen)
│ └── Sandbox object: fridge (fridge) | description: an appliance that keeps food cold
└── Location: bathroom (bathroom)
├── Sandbox object: toilet (toilet) | description: a simple toilet
└── Sandbox object: bathtub (bathtub) | description: a tub in which to take baths

  • For this test, I pictured a farmhouse, a nearby barn, and a couple of nearby crops.
  • I added in the corresponding json file the needed information for the agents involved:

{
“Joel”: {
“age”: 38,
“planned_action”: null,
“action_status”: null,
“current_location_node”: “farmhouse”,
“destination_node”: null,
“using_object”: null
},
“Betty”: {
“age”: 24,
“planned_action”: null,
“action_status”: null,
“current_location_node”: “farmhouse”,
“destination_node”: null,
“using_object”: null
}
}

  • As depicted, agents can have a planned action, an action status, their current location, their destination, and what object they are using, apart from their age and name.
  • As the paper says, each agent’s memories are seeded from a few memories that are written in a simple text file.

Betty started living with Joel six years ago;Betty enjoys planting crops and tending to them;Betty wishes that she and her boyfriend Joel could afford to buy a few horses;Betty wishes she didn’t have to travel to the nearby town often;Betty loves peace and quiet;When Betty feels overwhelmed, she lies down on her bed and listens to ASMR through her noise-cancelling headphones;Betty wants to have a child, hopefully a daughter, but Joel told her years ago that he didn’t want children;Joel and Betty saw strange lights in the skies in the 11th of May of 2023;On the 11th of May of 2023, the local news broadcast was cut off in the middle of a transmission as they were reporting on the strange lights in the sky;Joel and Betty have heard ominous explosions in the distance throughout the morning of the 12th of May of 2023

Joel is a farmer who has lived in his plot of land for ten years;Joel is living with his girlfriend, Betty, whom he loves;Joel enjoys listening to music from the sixties;Joel wishes he and Betty could buy a few horses, but he doesn’t think they can afford them;Joel and Betty saw strange lights in the skies in the 11th of May of 2023;On the 11th of May of 2023, the local news broadcast was cut off in the middle of a transmission as they were reporting on the strange lights in the sky;Joel and Betty have heard ominous explosions in the distance throughout the morning of the 12th of May of 2023

  • These seed memories are inserted into a memories database. Its implementation is one of the most important parts of this system. The database is composed of a vector database (which stores values according to some eldritch, multidimensional distance between them), and a json file that records readable values of those memories. You can query the vector database with simple text, and it will return the most relevant text it contains regarding the query you sent it. Sort of like a recommendation system.
  • In addition, when a query is made to the agents’ memories, they are rated according to the memory’s importance and recency. The recency is a normalized value in the range of [0.0, 1.0] depending on when that memory was accessed last, and importance is a rating that the AI itself gave to how poignant the memory was.
  • Once the agents are loaded in the simulation, a character summary is created for each agent. The AI is prompted to write all the segments of these summaries only from the memories present in their vector databases.

Joel’s AI-written character summary

Joel (age: 38)

Innate traits: Hardworking, resilient, curious, observant, caring, nostalgic, pragmatic.

Joel is a farmer who has lived in his plot of land for ten years and wishes he and Betty could buy a few horses, but he doesn’t think they can afford them. He enjoys listening to music from the sixties and is living with his girlfriend, Betty, whom he loves. They have experienced ominous explosions in the distance and saw strange lights in the skies on the 11th of May of 2023, and the local news broadcast was cut off in the middle of a transmission as they were reporting on the strange lights.

Joel is a farmer who has lived in his plot of land for ten years.

It is difficult to accurately describe Joel’s overall feeling about his recent progress in life based on these statements alone. However, we can infer that he enjoys listening to old music, has heard ominous explosions and seen strange lights with Betty, is a farmer, loves his girlfriend, and wishes they could afford to buy horses. These statements do not provide enough information to determine his overall feelings about his recent progress in life.”

Betty’s AI-written character summary

Betty (age: 24)

Innate traits: resilient, domestic, nurturing, longing, introverted, anxious, curious, intuitive, calm

Betty is a woman who enjoys living a peaceful and simple life with her partner Joel. She is interested in farming and wishes to own horses. Betty also has a desire to have a child, but this conflicts with Joel’s wishes. She dislikes having to travel to nearby towns and finds solace in listening to ASMR when she feels overwhelmed. Betty is aware of strange occurrences happening around her, such as explosions and lights in the sky, but she seems to maintain a calm and composed demeanor.

Betty is a homesteader or a farmer.

Betty is feeling somewhat unsettled and uncertain about her progress in life, as indicated by her mixed feelings about her living situation with Joel, her desire for a more stable and fulfilling life with horses and a child, and her occasional need for escapist relaxation techniques like ASMR. She enjoys the sense of purpose that comes from tending to her crops, but is also wary of the potential dangers and disruptions of the outside world, as shown by the ominous sounds and sightings of explosions and strange lights in the sky. Overall, she seems to value the simple pleasures of a quiet country life, but is also aware of the limitations and difficulties of this lifestyle.

  • If the simulation detects that the agents haven’t decided upon an action, it generates one through a complicated process.

2023-05-13T09:30:00 Joel produced action: Saturday May 13 of 2023, 9 AM. Joel is going to check with his neighbors to see if they have any information about the strange lights and ominous explosions they have been experiencing. The length of time it takes for Joel to check with his neighbors is not specified, so it is impossible to provide an answer in minutes.

2023-05-13T09:30:00 Betty produced action: Saturday May 13 of 2023, 9 AM. Betty is going to talk to Joel about their conflicting desires regarding having a child and try to come to a mutual understanding or compromise. The duration of this action is not specified in the given information. It can vary based on the discussion and resolution reached by Betty and Joel.

  • The AI model (GPT-3.5-turbo in this case) went over each agent’s memories and decided what action the agents would take, according to their circumstances. A problem with the paper’s original implementation already comes up: the agents decide a plan, then they search for the most fitting location, and then object, to fulfill that plan. But in this case, the AI decided that Joel should check with his neighbors to see if they have any information about weird happenings. But there are no neighbors in this environment. In addition, Betty decided that she wants to talk to Joel, but the paper doesn’t offer a system of locating other agents that may not be in the immediate surroundings. There’s a system for dialogue, but I haven’t gotten that far.
  • Through the rating system for locations and objects (going from the root of the environment tree deeper and deeper), Joel came to this conclusion about what should be his destination given the action he came up with:

2023-05-13T09:30:00 Joel changed destination node to: Node(‘/Location: plot of land (plot_of_land)/Location: farmhouse (farmhouse)/Location: kitchen (kitchen)/Sandbox object: fridge (fridge) | description: an appliance that keeps food cold’)

  • Apparently Joel (therefore, the AI model) thought that the neighbors could be found in the fridge. From the root of the environment tree, between the choices of “farmhouse”, “barn” and “crops”, obviously farmhouse was the most adequate option to find people. Inside that house, it opted for the kitchen between all the possible rooms. Inside there was only a fridge, so the navigation system had no choice but to choose that. If I had added a telephone in the kitchen, the AI model likely would have chosen that instead to communicate with the neighbors.
  • Regarding Betty, to fulfill her action of talking to Joel, she chose the following destination:

2023-05-13T09:30:00 Betty changed destination node to: Node(‘/Location: plot of land (plot_of_land)/Location: barn (barn)/Sandbox object: tools (tools) | description: lots of tools for farming’)

  • Interestingly enough, the AI model believed that her farmer husband was more likely to be in the barn than in the farmhouse. Once the barn was chosen, the tools were the only option as to which object would help the agent fulfill her action.
  • The navigation system detected that the agents weren’t at their destination, so it started moving them:

2023-05-13T09:30:00 Joel needs to move to: Node(‘/Location: plot of land (plot_of_land)/Location: farmhouse (farmhouse)/Location: kitchen (kitchen)/Sandbox object: fridge (fridge) | description: an appliance that keeps food cold’)
2023-05-13T09:30:00 Joel changed the action status to: Joel is heading to use fridge (located in Location: kitchen (kitchen)), due to the following action: Saturday May 13 of 2023, 9 AM. Joel is going to check with his neighbors to see if they have any information about the strange lights and ominous explosions they have been experiencing. The length of time it takes for Joel to check with his neighbors is not specified, so it is impossible to provide an answer in minutes.

2023-05-13T09:30:00 Betty needs to move to: Node(‘/Location: plot of land (plot_of_land)/Location: barn (barn)/Sandbox object: tools (tools) | description: lots of tools for farming’)
2023-05-13T09:30:00 Betty changed the action status to: Betty is heading to use tools (located in Location: barn (barn)), due to the following action: Saturday May 13 of 2023, 9 AM. Betty is going to talk to Joel about their conflicting desires regarding having a child and try to come to a mutual understanding or compromise. The duration of this action is not specified in the given information. It can vary based on the discussion and resolution reached by Betty and Joel.

  • The simulation advances in steps of a specified amount of time, in this case thirty minutes, and the navigation system will move the agents a single location each step. This means that an agent would take thirty minutes to move from a bedroom to the entrance of the same house, but whatever.

2023-05-13T10:00:00 Joel changed the current location node to: Node(‘/Location: plot of land (plot_of_land)/Location: farmhouse (farmhouse)/Location: kitchen (kitchen)’)
2023-05-13T10:00:00 Joel changed the current location node to: Node(‘/Location: plot of land (plot_of_land)/Location: farmhouse (farmhouse)/Location: kitchen (kitchen)/Sandbox object: fridge (fridge) | description: a piece of furniture that keeps food cold’)

  • Joel moved from the entrance of the farmhouse to the kitchen, and once the agent is in the same location as his or her destination sandbox object, he or she starts using it. In this case, the fridge.

2023-05-13T10:00:00 Betty changed the current location node to: Node(‘/Location: plot of land (plot_of_land)’)

  • Betty has moved from the farmhouse to the root of the environment tree.
  • In the next step of the simulation, Joel continues using the fridge, while Betty moves further to the barn, where she starts using the tools:

2023-05-13T10:30:00 Joel continues using object: Node(‘/Sandbox object: fridge (fridge) | description: a piece of furniture that keeps food cold’)
2023-05-13T10:30:00 Betty changed the current location node to: Node(‘/Location: plot of land (plot_of_land)/Location: barn (barn)’)
2023-05-13T10:30:00 Betty changed the current location node to: Node(‘/Location: plot of land (plot_of_land)/Location: barn (barn)/Sandbox object: tools (tools) | description: lots of tools for farming’)

  • From then on, the agents will continue using the tools they have chosen to perform their actions:

2023-05-13T11:00:00 Joel continues using object: Node(‘/Sandbox object: fridge (fridge) | description: a piece of furniture that keeps food cold’)
2023-05-13T11:00:00 Betty continues using object: Node(‘/Sandbox object: tools (tools) | description: lots of tools for farming’)
2023-05-13T11:30:00 Joel continues using object: Node(‘/Sandbox object: fridge (fridge) | description: a piece of furniture that keeps food cold’)
2023-05-13T11:30:00 Betty continues using object: Node(‘/Sandbox object: tools (tools) | description: lots of tools for farming’)
2023-05-13T12:00:00 Joel continues using object: Node(‘/Sandbox object: fridge (fridge) | description: a piece of furniture that keeps food cold’)
2023-05-13T12:00:00 Betty continues using object: Node(‘/Sandbox object: tools (tools) | description: lots of tools for farming’)

That’s unfortunately as far as I’ve gotten with my implementation, even though it took me a week. The paper explains more systems:

  • Observation: as the agents use the objects, they are prompted with observations about what other objects in the same location are doing, and about the other agents that may have entered the location. That gives the busy agent an opportunity to stop what they’re doing and engage with other objects or agents.
  • Dialogue: the paper came up with an intriguing dialogue system that became one of my main reasons for programming this, but it requires the Observation system to be implemented fully.
  • Planning: once a day, the agents should come up with day-long plans that will be stored in memory, and that will influence the actions taken.
  • Reflection: when some condition is triggered, the agents retrieve the hundred most recent memories from their databases, and prompt the AI model to create high-level reflections of those memories. The reflections will be stored in the memory stream as well, so that they influence their planning, their decision-taking, as well as the creation of further reflections.

That’s as far as I remember of the paper at this moment. I wanted to get the navigation system done as early as possible, because I considered it the hardest part, and it annoyed the hell out of me partly due to how Python handles references (at one point I was losing the list of ancestors and descendants of the agents’ destination nodes between simple function calls to the same instance of Simulation, for no fucking reason that I could determine). I may move directly onto the Observation system, or take a rest. I already feel my sanity slipping quite a bit, likely because I took a sabbatical from my ongoing novel to do this.