I recommend you to check out the previous parts if you don’t know what this “neural narratives” thing is about. In short, I wrote in Python a system to have multi-character conversations with large language models (like Llama 3.1), in which the characters are isolated in terms of memories and bios, so no leakage to other participants like in Mantella. Here’s the GitHub repo.
Early next morning, my player character, a world-weary detective, decided to bother some folks at the local university.
I wrote “we still don’t need how” instead of “we still don’t know how.” Just pointing that out because I hate making mistakes.
The detective goes back to report to the missing girl’s impatient father.
I pictured two weary men rummaging through bookcases of ancient, cursed books at the old library, while rain pelted the windows. Rain? There’s no weather in this app. Well, there is now.
Only areas have weather. Locations, which are inside areas, are aware of the weather in their containing area, but can’t be changed from there. All the prompts to the large language model that were aware of the time of the day are now also aware of the weather, and that includes dialogues. We’ll see how much the awareness of weather affects the scenes. I just hope the AI is intelligent enough to realize that it doesn’t rain inside a building.
Anyway, my player character, accompanied by an increasingly unstable father, headed to the old library.
There they met the apparently ancient librarian.
You may have noticed that the narration parts of the dialogue now actually sound like narration parts, instead of being spoken by the character. It took me a while to figure out how to do that, and it involved splitting the original text, generating voice lines for each part, and then concatenating them. But it sounds much better now.
I recommend you to check out the previous parts if you don’t know what this “neural narratives” thing is about. In short, I wrote in Python a system to have multi-character conversations with large language models (like Llama 3.1), in which the characters are isolated in terms of memories and bios, so no leakage to other participants like in Mantella. Here’s the GitHub repo.
The previous entry ended with my player character and his partner arresting Elara Thorn, the suspicious scholar.
Well, I’m stumped. The following conversation happened immediately after.
As I tried to figure out what to do, I kept the facts of the story in my head, but why would I need to do that? Am I not programming a page that could handle such stuff? So I programmed it in.
The user has to introduce those facts manually; I can’t think of any reliable way for the large language model to determine them instead. In any case, now that I keep a file of known facts, they are used when prompting the AI for concepts, interesting situations, interesting dilemmas, and goals.
Anyway, my player character and his partner have to inform the missing teenager’s father that the main culprit is going to walk free for now.
No choice but to release the charming miss Thorn from custody.
I recommend you to check out the previous parts if you don’t know what this “neural narratives” thing is about. In short, I wrote in Python a system to have multi-character conversations with large language models (like Llama 3.1), in which the characters are isolated in terms of memories and bios, so no leakage to other participants like in Mantella. Here’s the GitHub repo.
Without further preamble…
I approached one of the students.
Let’s head to the library, then.
As I was having the big multi-char convo with Elara and two of her pals, as well as the other two people on my side, Elara treated her own companions as if she didn’t know them. Of course, she doesn’t actually know them: a character is only aware of that their bio says as well as the contents of their memories. Those people hadn’t been introduced in their memories yet. That’s an interesting problem to solve.
Of course, I could just make up their relationship by writing directly in their memories, but the whole point of having this app is for the large language model to act as a Dungeon Master. So, I’m going to create a whole new page on the site, one called Connections, that offers the opportunity to generate a relationship between two characters.
Here is the prompt I’ve put together so that the large language model will come up with a compelling, meaningful connection between two characters:
Instructions:
Using the character data provided for {name_a} and {name_b}, generate a meaningful and compelling connection between them. The connection should align with their personal information and memories. Ensure that the relationship is coherent, enriches their backstories, and is consistent with their individual characteristics. The connection does not need to be intimate or romantic; it can be any significant relationship such as friendship, rivalry, mentorship, familial ties, etc.
Character {name_a}:
{character_a_information}
Character {name_b}:
{character_b_information}
Your Task:
Using the above information, write a detailed description (approximately 3-5 sentences) of the connection between {name_a} and {name_b}. The description should:
Explain how they met or became connected.
Highlight the nature of their relationship (e.g., allies, rivals, mentor and protégé, siblings).
Incorporate elements from their personalities, backgrounds, likes, dislikes, secrets, or memories.
Be compelling and add depth to both characters.
Ensure that the connection is believable and enhances the narrative potential of their interactions.
Well, pulling off that Connections page was far quicker than I anticipated, but the app is already quite mature when it comes to examples of how stuff works.
The connection between them (that gets stored as a memory for both characters) isn’t shown to the user, to keep the intrigue. You can, of course, check it out in the JSON files that store the characters, but for storytelling purposes, it’s better to keep the secrecy. Like all other forms I’ve been dealing with, this one interacts with the server via AJAX, without actually reloading the page. Very slick.
As I was checking out how the ongoing dialogue was getting stored, I realized that at some point I had mistakenly stored the characters’ names instead of their descriptions, so I don’t think that the characters were aware of how the others looked as they spoke. This fix will open up plenty of dialogue flavor.
The error was worse than I anticipated. Turns out that when I refactored the gigantic manager class CharactersManager into five or so other classes, one of them a class named Character that handled everything related to character attributes, I had copy-pasted my way into horror: I was returning the property “name” even if what the user demanded was the description, the likes, the dislikes, the personality, etc. So these last couple of days, most of the prompts involving characters, if not all, have been missing about half of the proper data. That’s why you pytest the shit out of your app. Sorry, programming gods.
Anyway, long dialogue incoming:
That’s all for now. The following isn’t related to anything, but I must say it: recently I recommended the anime adaptation of Uzumaki. Not once, but twice. Well, consider that recommendation rescinded. Those motherfuckers really pulled a fast one on us. Dandadan is still cool, though.
I recommend you to check out the previous parts if you don’t know what this “neural narratives” thing is about. In short, I wrote in Python a system to have multi-character conversations with large language models (like Llama 3.1), in which the characters are isolated in terms of memories and bios, so no leakage to other participants like in Mantella. Here’s the GitHub repo.
As I considered my app this morning, a couple of things bothered me considerably: one, the CharactersManager class that handled pretty much any API requests related to characters had ballooned to insurmountable levels. It took me an hour and a half if not more to refactor it into four classes, given how entangled its code was with everything. Now I have a Character class that, provided with an identifier, handles the loading of its own related data, as well as saving it. That simplifies things a lot.
The second matter was far more troubling for me: every time I launched a request to the server that entailed a call to a large language model, the interface went unresponsive potentially for six seconds or more. That bothered me every time. The only real solution that I know is to use AJAX, a method of javascript programming that instead of actually sending a POST request to the server, it sort of simulates one through javascript, then returns the result as JSON data. That only has pros when it comes to the final execution, but reaching that point is troublesome. That’s what I’ve spent most day working on, and fortunately I’ve managed to achieve it for the most part: there are various parts of the app that now display “Processing…” on the button that generated the request, next to a spinner, and they return to normal once the request has been processed. If it entailed generating data that would appear on the page, then it gets displayed as well.
I’ve also changed some of the interface, particularly the buttons to subsections in Characters Hub and the Actions page. I think it looks quite cool.
As I was testing things while implementing AJAX into the chat page, I had the following idiotic conversation with the player character’s partner.
I have processed about 60 voice models of the 550 or so available in the RunPod server, so I’ll be able to enjoy many intriguing conversations through this narrative system (not to mention pretty exceptional smut).
Silliness aside, here are the descriptions of the player character and his posse as they head to the dreaded university.
I asked the AI to generate three more characters that would know or even be intimately connected with the arrogant student Elara Thorn.
That’s a perfectly normal group of people.
Last night, I thought about how this whole confrontation with Elara Thorn may play out. Were we going to accuse her of something and the LLM would play along and admit some wrongdoing? I thought about the notion of a character having a secret that even I wasn’t aware of. So I programmed just that: a section in Characters Hub that generates one or more secrets for any given character. Now all characters are created with some petty secrets of their own, which is bound to enrich their spoken parts, but this Secrets section is bound to generate some pretty hardcore secrets, as per the prompt.
You are to generate the secrets of a character. Use the provided information about the character, the world's conditions, and the specific locations involved.
Instructions:
To create secrets that are compelling and truly worth being hidden for the character, consider incorporating the following aspects:
Deep Personal Impact: The secret should have a profound effect on the character's life, shaping their personality, motivations, and actions. It might relate to a traumatic event, a pivotal choice, or a forbidden desire that they are desperate to keep concealed.
High Stakes and Consequences: The revelation of the secret should carry significant consequences for the character and potentially others. This could include personal ruin, endangerment of loved ones, loss of status, or catastrophic events within the story's world.
Moral Ambiguity: Secrets that involve morally gray areas make characters more complex and intriguing. The character might have done something considered unethical or made a compromise that weighs heavily on their conscience.
Inner Conflict and Guilt: The secret should create internal turmoil. Feelings of guilt, shame, or fear of discovery can drive the character’s behavior, making them more layered and realistic.
Hidden Identities or Double Lives: Characters may be living under an assumed identity or leading a double life. This creates tension as they balance their true self with the facade they present to the world.
Traumatic Past Events: A history involving trauma, such as witnessing or being involved in a catastrophic event, can be a secret that influences their current fears and motivations.
Forbidden Relationships: Involvement in relationships that are taboo or forbidden in their society adds emotional depth and high personal stakes if the secret is revealed.
Betrayal and Loyalty: A secret involving betrayal—whether the character betrayed someone or was betrayed—adds tension, especially if relationships with other characters are at risk.
Hidden Abilities or Powers: Concealing special abilities, especially in a world where such powers might be dangerous or outlawed, adds an element of suspense and fear of exposure.
Secret Motivations or Agendas: The character might have ulterior motives that conflict with their apparent goals or the goals of their allies, creating layers of intrigue.
Connection to Antagonists: A secret tie to the antagonist or antagonistic forces—such as being related to, indebted to, or blackmailed by them—complicates the character's role in the story.
Illegal or Illicit Activities: Participation in criminal activities, whether past or ongoing, provides clear reasons for secrecy and potential consequences if uncovered.
Prophecies or Destinies: Being the subject of a prophecy or destined for a significant role can be a burden the character tries to hide to avoid unwanted attention or responsibility.
Hidden Weaknesses or Vulnerabilities: Concealing physical, emotional, or psychological weaknesses to appear strong can add depth and tension, especially if these vulnerabilities are exploitable.
Knowledge of Critical Information: Possessing knowledge that others do not—such as impending disasters, secrets about other characters, or truths about the world's reality—can be dangerous to reveal.
Forbidden Knowledge or Research: Engaging in research or possessing knowledge that is forbidden or dangerous adds stakes, especially if discovery could lead to severe punishment.
Family Secrets and Lineage: Hidden heritage, such as being the descendant of a notable or infamous figure, can shape the character's identity and the perceptions of others.
Past Failures or Mistakes: A significant failure or mistake in the character's past that haunts them, influencing their present actions and decisions.
Survivor's Guilt: Being the sole survivor of an event and feeling responsible can be a heavy burden that the character keeps to themselves.
Secret Alliances or Memberships: Belonging to a secret society, cult, or group can add complexity, especially if their goals conflict with those of others.
Hidden Assets or Treasures: Possessing or knowing the location of valuable items can make the character a target and provide motivation to keep it hidden.
Internal Struggles with Identity: Questions about one's own identity, such as gender identity, sexual orientation, or personal beliefs that are not accepted in their society.
Mental Health Issues: Concealing mental health struggles due to fear of stigma or repercussions can add realism and depth to the character.
Unfulfilled Vengeance: Harboring a secret desire for revenge that drives the character's actions without others realizing their true intent.
Divine or Supernatural Experiences: Having had an encounter with the divine or supernatural that is disbelieved or ridiculed by society, leading them to keep it secret.
To ensure these secrets are compelling and worth hiding:
Integrate with Character Development: The secret should be a key part of the character's backstory and influence their development throughout the story.
Create Tension and Suspense: The possibility of the secret being discovered should create ongoing tension, affecting interactions and decisions.
Impact Relationships: The secret should have the potential to alter relationships with other characters significantly if revealed.
Drive the Plot Forward: The secret can serve as a catalyst for events in the story, creating twists, conflicts, or revelations that keep the narrative engaging.
Provide a Strong Motivation for Secrecy: The character should have clear, understandable reasons for keeping the secret hidden, such as fear of harm, shame, or protecting others.
Offer Opportunities for Revelation: Build moments into the story where the secret might be revealed, forcing the character to make tough choices.
Reflect Universal Themes: Themes like redemption, identity, betrayal, or the nature of truth can make the secret more relatable and impactful.
Provided Information:
{places_descriptions}
Character Information:
Name: {name}
Description: {description}
Personality: {personality}
Profile: {profile}
Likes: {likes}
Dislikes: {dislikes}
Speech Patterns: {speech_patterns}
Health: {health}
Equipment: {equipment}
Memories:
{memories}
Existing Secrets: {secrets}
Example Format:
"has developed romantic feelings for her brother, feels guilty about accidentally causing her cat's death, ..."
Your Task:
Using the above instructions and information, craft compelling, meaningful secrets for the character, adding to their existing secrets.
I recommend you to check out the previous parts if you don’t know what this “neural narratives” thing is about. In short, I wrote in Python a system to have multi-character conversations with large language models (like Llama 3.1), in which the characters are isolated in terms of memories and bios, so no leakage to other participants like in Mantella. Here’s the GitHub repo.
The previous part ended with our hardened player character returning home after an encounter with an arrogant student. At three in the morning, the following happens:
Now, a fundamental problems happens. How do you find clues in a room without making it up through a conversation? You don’t. A Dungeon Master of sorts would decide on the results of the investigation. That means I need to program a whole new thing in.
Maybe a week ago, I programmed the concept of a Goal Resolution: you gave the large language model a goal and it decided whether or not it was successful, and to what degree, according to the information provided and its own whims. But I quickly realized that it was too broad a concept: when you’re trying to accomplish something, you’re not pursuing “a goal”; you’re trying to investigate, to research, to trade, etc. A few days ago I created the concept of a Research action, which is constrained solely to that notion. Here is, why not, the prompt I send to the AI so it will come up with a resolution:
You are to generate a detailed narrative describing the player's attempt at a Research action within a rich, immersive world. Use the provided information about the player, their followers, the world's conditions, and the specific locations involved. Your narrative should be engaging and realistic, reflecting the player's abilities and circumstances.
Instructions:
1. Viability and Realism Assessment:
- Carefully consider whether the player's research goal is achievable based on:
The information about the player.
The abilities and resources of any followers accompanying the player.
The characteristics of the location, including available resources, accessibility, and any restrictions.
The time of day and how it might affect the research attempt.
2. Narrative of the Research Attempt:
Write two or three descriptive paragraphs detailing the player's research endeavor.
Include specific actions taken by the player and their followers.
Describe interactions with the environment, use of equipment, and any obstacles or challenges faced.
Ensure the attempt aligns with the world's lore and the player's personal history.
3. Outcome - Impact of the Research:
Determine the results of the research action based on the assessment.
You must be specific about what the player has discovered: this is the memory that will get saved.
Detail any new knowledge gained, abilities unlocked, or crucial information discovered.
If the research was partially successful or failed, explain what was learned or why it didn't succeed fully.
4. Consequences - Cost of the Research:
- Describe the costs associated with the research action, such as:
Time consumed and its implications.
Use or loss of resources and equipment.
Physical or mental strain on the player and followers.
Potential negative repercussions, such as drawing unwanted attention or causing disturbances.
Use your judgement to determine the extent of the consequences, according to all the information provided.
Important: the consequences are an account of the costs of the research effort, looking back now that it's finished. Do not continue the narrative nor project into the future.
5. Tone and Style:
Write in the third person, past tense, maintaining an engaging and immersive narrative style.
Ensure consistency with the established world setting and the player's character.
Provided Information:
- Time of Day: {hour} {time_of_day}
- World Description: {world_description}
- Region Description: {region_description}
- Area Description: {area_description}
{location_description}
Player and Followers Information:
- Name: {player_name}
- Description: {player_description}
- Personality: {player_personality}
- Profile: {player_profile}
- Likes: {player_likes}
- Dislikes: {player_dislikes}
- Speech patterns: {player_speech_patterns}
- Health: {player_health}
- Equipment: {player_equipment}
-----
{followers_information}
- Memories:
{combined_memories}
- Research Goal: {research_goal}
Example Format:
"Seeking to unravel the mysteries of the ancient sigils, the tall, cloaked figure of [Player Name] entered the dimly lit archives of the Grand Library. Clutching the worn tome of cryptic symbols, they, alongside their ever-curious companion [Follower Name], began sifting through scrolls and manuscripts..."
Your Task:
Using the above instructions and information, craft a compelling narrative that captures the essence of the player's Research action, reflecting both the potential rewards and the inherent costs.
That worked well enough, but investigating a missing teenager’s room isn’t “research,” so I need to create a whole new page for my app.
Creating the Investigate page took me a few hours. In the end, the user has to provide an investigation goal and the facts already known about the case (I don’t trust the AI enough to be able to glean them from the memories of the involved characters). It produced the following outcome and consequences:
Oho, the plot thickens! It seems I’ll make a visit to the university in the morning to interrogate this college student. First, though, I needed to know the opinion of the missing girl’s father.
We hurried to the police station to meet my character’s partner.
I didn’t plan to, but this is a fantastic opportunity to test the Research action. Yet another example of how dynamic and unpredictable this system is.
That worked out perfectly. So this Elara Thorn and her team may have bitten more than they could chew, and maybe made some innocents disappear in the process.
Through testing both the Investigate and the Research actions, I’ve learned that the Consequences part that I ask the LLM to produce is completely useless. Great. That saves some computing power.
My player character’s partner returned to the station.
I recommend you to check out the previous parts if you don’t know what this “neural narratives” thing is about. In short, I wrote in Python a system to have multi-character conversations with large language models (like Llama 3.1), in which the characters are isolated in terms of memories and bios, so no leakage to other participants like in Mantella. Here’s the GitHub repo.
The last entry ended when I had managed to add a hole-in-the-wall to the ongoing story. Let’s enter it and allow the player character to describe it.
Every time you enter a new location, the app checks if there is a list of character generation guidelines already created for this combination of places, and if there isn’t, it generates that list. After a short while, the app produced the following:
That one about a brilliant university student sounds like a great contrast to the existing characters. I pressed the button that generates a new character, and at the end of that process, the app presented me with the following portrait:
That looks real good. Great job on the fingers, AI. However, the large language model named this character Elara Thorn, the exact name that was generated for a character in another one of my trial-run stories with this system. Perhaps at some point I’ll need to program a way of reading names from a gigantic list, then presenting the AI with about twenty random ones to choose from.
Anyway, let’s have a multi-char convo.
After such a tense conversation, I figured that the player character would reflect on it, as well as his general circumstances. I happened to have implemented a system for self-reflection: the large language model looks at the character’s memories, then writes a sort of journal entry from the first-person perspective. It gets saved along with the rest of the memories. That helps color the dialogue and in general make the character sound more intelligent. Of course, now I generate audio files of those self-reflections as well.
My hardened player character returned home. I’ll let him describe his living arrangements.
I can imagine the player character returning to work the following day, only to be introduced to some troubled citizen who will present him with a case. However, I have also programmed a way of generating story concepts, interesting situations, interesting dilemmas, and interesting goals, in case the user isn’t too sure how to continue. Let’s generate a few.
Here are a few intriguing concepts the app has generated. When crafting any of these notions, the large language model is presented with the player’s information and that of his followers, and also all the available information about his location (world, region, area, and possibly location).
I should probably turn those post-its into something else, like papers or something, because such long texts look funky in a vertical format. That’s a minor issue in any case.
Investigating a series of gruesome murders connected to the otherworldly horrors sounds good for a story, and even more if the encounter with the arrogant student earlier does hint at a larger plot involving a secret society of reckless scholars. I also like the notion of a neighbor calling on his door to ask for his help because the person’s daughter has gone missing. Also, the sort of post-apocalyptic story of the grizzled detective leading a ragtag group as they fight against the pouring eldritch horrors sounds pretty fucking dope.
How about general goals?
Some of those are interesting. The disappeared scholar could easily be the aforementioned college girl, so how would the detectives feel about investigating her disappearance? The goal about infiltrating a cult makes me think that I would need a way of altering a character’s description so they can pass undercover, because other characters are fed the participants’ description during a conversation. And again, someone is requesting the detective’s help to find their missing child.
I recommend you to check out the previous parts if you don’t know what this “neural narratives” thing is about. In short, I wrote in Python a system to have multi-character conversations with large language models (like Llama 3.1), in which the characters are isolated in terms of memories and bios, so no leakage to other participants like in Mantella. Here’s the GitHub repo.
Last time, I managed to nail creating voice lines for the dialogues of my app, relying on a RunPod server dedicated to generating audio. Now I want to go on a serious test run of app to see what it lacks.
Starting from scratch, I created a new world, a new region of that world, a new area of that region, and a new location of that area. I won’t detail the specifics, because the app itself should do it like a story would do, so if in the course of testing the system I feel that something must be implemented to cover for the shortcomings, I will do so.
The process of creating a new playthrough requires you to provide some notion of the character you want to play as. I ended up with a good depiction of the guy.
I have the initial setting and the player character (including his automatically assigned voice model). A story needs other characters, so I went to the section that shows the character creation guidelines that have been generated for this combination of places.
I turned them into post-its. Anyway, my protagonist is a police detective, so he could do with a partner. I grabbed the second guideline and made her a woman.
The app doesn’t allow you to access most of the specific data of a character except in very paricular circumstances, so in general, you have to glean the specifics of a character from their looks and the conversations you have with them.
In a story, you need some sense of where you are. There’s a system in place for the LLM to generate a description from the first-person perspective of the player character, and at this point it’s trivial to generate a voice line for it:
Let’s interact with the sole other character around (for now). As I was running a conversation with the protagonist’s partner, I ran across the first issue: when the app had to generate a voice line for a bit of ambient text, the server (the RunPod pod) returned a 404. I guess that even if a pod is technically running, it could intermittently produce 404 errors for whatever reason. I guess I’ll need to program in some retry system to cover these cases.
I did do that. Let’s continue.
The “stop your a coffee” was my blunder. Old, stupid fingers.
The couple of grizzled detectives exited the police station, back to the grim city surrounding it.
Now I want my characters to move to the mentioned location, some bar. Even though I didn’t create any other locations for this run other than the police station, there is a lingering issue with the interface: when plenty of possible locations exist, if you press the button “Search for location,” it may link locations you don’t want (like a cave, a hospital, etc.). Now that I was looking specifically for a bar, I figured that I may as well fix this issue.
It took quite a while, but now the user can only search locations by a type. In fact, if no locations are available, because they have already been used or they don’t match the area’s categories (you don’t want a fantasy bar in a cosmic horror story), the select and the button will be disabled.
Well, that was all for today. I expected to do more, but reworking that interface was arduous.
I recommend you to check out the previous parts if you don’t know what this “neural narratives” thing is about. In short, I wrote in Python a system to have multi-character conversations with large language models (like Llama 3.1), in which the characters are isolated in terms of memories and bios, so no leakage to other participants like in Mantella. Here’s the GitHub repo.
In the previous entry, I came up with the notion of producing audio voice lines for the conversations. Mantella had spoiled me in that regard: hearing those fictional characters answering you in reasonably good voices while you stared at them did wonders for immersion. And it was a bad idea to shove that possibility into my mind, because it prevented me from sleeping last night. Instead, I moved to my desk at three in the morning and started implementing it. Now, every generated character gets assigned a voice model according to their peculiarities, and each speech turn produces voice lines that the user can play through clicking the speech bubbles. It works perfectly.
I learned that it’s a terrible idea to play audio server-side, because it crashes the server. Flask, the web framework that my app is programmed in, or maybe it happens in all web systems, also doesn’t allow the client to access any file in the server, so I had to move all the audio-playing logic to Javascript.
Given this example chat I had with a new character, who had been assigned a matching voice automatically among the relatively few I’ve introduced into the system so far:
The short convo produced this audio exchange:
Like in the original Mantella system, the quality of voice models varies greatly; sometimes they sound like theater students reading a script, recorded on a home mic. Also, the process of generating the clips sometimes shears the very end of their final sentence. Still, I can hardly complain. Listening to the characters adds so much life to the conversations you can have through this app that I see myself enjoying it for a long time to come (and not only for smut).
I’m amazed that I got this running. So, how did it happen?
In the beginning, I thought that setting up my own, local XTTS server (XTTS being a model for generating voice lines) was a good idea. I struggled through every step of the way for a few hours, fighting against obscure documentation, until I finally managed to generate a sample voice line, only to find out that it sounded like ass. Why, I have no idea. So I discarded that notion and instead I looked into Mantella’s codebase, which is up at GitHub, to see how they connected to the RunPod pods to request voice generation. RunPod is a sort of online renting system of computer and server time: you can set up a pre-configured little server that all it does is generate voice lines, and as long as you can connect to it, you’re set. Only costs seventeen cents an hour, too. Once I managed to query the list of available voice models from the RunPod pod, I knew I was going to get through this thing.
So, I had a list of all possible voice models I could rely on, and it turned out to be about five hundred fifty. They are trained from game voices, so there’s a whole breadth of possible voices one can use. How to classify them? Should I create a page on my site with a simple select box, letting the user (meaning me) scroll through a list that long?
ChatGPT, even its latest Orion preview version, clarified that it knows of no online service that could classify the more than five hundred voice samples I had produced from those voice models. I would have to do it manually, but in the beginning it would be enough with having introduced twenty or so models into the system. What tags can be applied to a voice? I relied on ChatGPT to figure that out. Now that I have that list, classifying each voice model is as easy, but time consuming, as listening to that sample on a loop while adding appropriate tags. I have ended up, so far, with the following JSON file of voice models:
{
"npcmmel": [
"MALE",
"ADULT",
"CONFIDENT",
"STEADY",
"SMOOTH",
"CLEAR",
"FORMAL",
"CHARMING",
"NO SPECIAL EFFECTS"
],
"npcmlucasmiller": [
"MALE",
"ADULT",
"CALM",
"FAST-PACED",
"SMOOTH",
"CASUAL",
"KIND",
"NO SPECIAL EFFECTS"
],
"robotmsnanny": [
"FEMALE",
"YOUNG ADULT",
"STEADY",
"WARM",
"CASUAL",
"MELODIC",
"YOUTHFUL",
"NO SPECIAL EFFECTS"
],
"npcma951": [
"MALE",
"ADULT",
"ANXIOUS",
"SLOW",
"AIRY",
"SKEPTICAL",
"NO SPECIAL EFFECTS"
],
"npcfphyllisdaily": [
"FEMALE",
"ADULT",
"STOIC",
"SLOW",
"MONOTONE",
"INSTRUCTIONAL",
"CALCULATING",
"NO SPECIAL EFFECTS"
],
"femalechild": [
"FEMALE",
"CHILDLIKE",
"PLAYFUL",
"STEADY",
"AIRY",
"MELODIC",
"INNOCENT",
"NO SPECIAL EFFECTS"
],
"femaleyoungeager": [
"FEMALE",
"YOUNG ADULT",
"HOPEFUL",
"FAST-PACED",
"CLEAR",
"INTENSE",
"OPTIMISTIC",
"NO SPECIAL EFFECTS"
],
"femalevampire": [
"FEMALE",
"MIDDLE-AGED",
"ARROGANT",
"STEADY",
"SMOOTH",
"AUTHORITATIVE",
"CYNICAL",
"NO SPECIAL EFFECTS"
],
"femalekhajiit": [
"FEMALE",
"ADULT",
"CALM",
"STEADY",
"GRAVELLY",
"CASUAL",
"PHILOSOPHICAL",
"NO SPECIAL EFFECTS"
],
"femaleuniqueghost": [
"FEMALE",
"YOUNG ADULT",
"RESIGNED",
"STEADY",
"ETHEREAL",
"MELODIC",
"INNOCENT",
"GHOSTLY"
],
"femaleghoul": [
"FEMALE",
"ADULT",
"MENACING",
"STEADY",
"RASPY",
"INTENSE",
"ENERGETIC",
"NO SPECIAL EFFECTS"
],
"femaleboston": [
"FEMALE",
"ADULT",
"CALM",
"DRAWLING",
"SOFT-SPOKEN",
"WARM",
"FLIRTATIOUS",
"SULTRY",
"NO SPECIAL EFFECTS"
]
}
I wrote a function that narrows down the list of possible categories of tags: gender, age, emotion, tempo, volume, texture, style, personality, and special effects. If at some point there’s no matching voice models, it returns a random one from the previous filtering. I’ll probably program in the characters section of the site a simple button that redoes the process for any existing character, in case any other random fitting voice may work better.
That’s all, I guess. When I first got the idea about programming this conversation system with characters controlled by large language models, I knew that programming the multi-char convos would be the most difficult thing. The second most difficult thing that I pictured was actually making them talk out loud. No idea what big thing could be coming next. Anyway, back to the brothel.
EDIT: here’s a multi-char convo in audiobook form.
I recommend you to check out the previous parts if you don’t know what this “neural narratives” thing is about. In short, I wrote in Python a system to have multi-character conversations with large language models (like Llama 3.1), in which the characters are isolated in terms of memories and bios, so no leakage to other participants like in Mantella. Here’s the GitHub repo.
Today I’ve been busy fully redesigning my web app, now that it includes plenty of interesting features.
Index page:
What it allows you to do is pretty self-explanatory, which is good. The relationship between the places is World > Region > Area > Location, and each of them have specific behavior in-game.
Story hub:
This one requires some explanation: at any point, the user can demand the system to generate story concepts, interesting situations, and interesting dilemmas. The large language model (the AI) will take into account all the info about the character, his or her followers, and their memories, to generate concepts, interesting situations and dilemmas that may fit what’s been going on.
Here are some examples of the interesting dilemmas the system can generate. Each “post-it” can be removed by just clicking on it. And now that I think about it, maybe I should make them yellow too.
Characters hub:
The player character front and center. The character images are generated by courtesy of an OpenAI model, that is provided a special prompt crafted from the character’s information.
Character generation:
Originally I told the AI to generate whatever characters it pleased based on the world, region, area and possibly location, but it ended up creating very similar characters: for example, in a police station, it would create a succession of serious detectives. So I programmed an intermediate step in which the AI creates character generation guidelines focused on coming up with a breadth of possible characters for that place. Of course, it also respects genre.
Character memories:
After each chat (and some other activities), a summary is created of what just took place, and is stored in the memories of all the participants. Those memories are loaded in most activities, and they color the characters’ dialogues and decisions. You can also generate self-reflections: the character, according to their personality, meditates upon their experiences, and writes a sort of journal entry about it, that goes into their memories to color their behavior and dialogue further. And of course, they can self-reflect about their self-reflections.
Location hub:
Lots of stuff to do on this page: generate a description of the place, find a new location among the existing ones matching the genre, visit found locations, and travel to other areas. You can also pick up followers here, or dismiss them.
The page for goal resolution is relatively simple so far, and I developed it just this morning, but it does something quite interesting: given a goal, you ask the large language model to write a narrative of the attempt to fulfill that goal, to decide whether or not the goal was achieved, and then a paragraph or two of the resolution, which gets saved as a memory.
This came about when the owner of a brothel, a succubus, suggested that I should travel to some nearby caves in search of treasure. That sounded interesting, but I would have needed to roleplay the entire thing through dialogues, including the possible obstacles found along the way. So I programmed in a system that figures it out by itself. In the future, I will probably program in a system to resolve explicit confrontations between characters or some opponent, like a bunch of wolves or something.
Now for the meat of this stuff, the chat system:
At any place, the user can choose to chat with any combination of characters present at the location, from those loitering around to the player’s followers. Multi-char convos is the first thing I made sure to program properly.
Chat:
All the NPCs speak in character, according to their personal info, their memories, and the ongoing conversation. There’s no leakage of information from other characters, because personal memories aren’t shared. If it weren’t because the calls to the LLM can be slow according to the whims of those servers, I would call the current system perfect.
So, what’s in the future? I plan on programming a system to generate goals, like it generates concepts, interesting dilemmas and interesting situations. Those goals will be loaded in a page, in case you want to attempt at resolving them. I also want to program in a way of swapping player characters easily. To deepen the simulation, I also want to include the notion of character health, and some way of altering the characters’ equipment and health after the actions have been resolved. At some point, I may rely on some local audio-generation server like XTTS to generate voices for each NPC utterance. That would be cool.
This whole thing has been a lot of fun. If I didn’t gravitate so much toward using it to engage in smut, I would have probably posted plenty of the stuff I’ve produced through the system.
EDIT: I fed this post to the Deep Dive pair, and they produced the following podcast (AI-generated):
I recommend you to check out the previous parts if you don’t know what this “neural narratives” thing is about. In short, I wrote in Python a system to have multi-character conversations with large language models (like Llama 3.1) in which the characters are isolated in terms of memories and bios, so no leakage to other participants like in Mantella. Here’s the GitHub repo.
I’ve been on a medical leave for five days, a leave that ends tomorrow. I’ve buried my troubles and worries about my life and job by programming my ass off. Right now, the code does the following:
Generates worlds, regions, areas, and locations, mostly automatically through the large language model (right now Hermes 405B).
Generates characters mostly automatically, including their portraits.
Allows the user to chat with them either through the console or with a very fancy web interface programmed in Flask, initially with the invaluable help of the newly released Orion model from OpenAI (I’ve nearly burned through all the allowed chat interactions, which renew in a week).
It creates rudimentary memories for the characters (summarized conversations). It also saves the transcriptions of the dialogues.
Here’s how the web interface looks like:
Why all of these generated people’s names start with “Z” is a bit of a mystery.
I don’t know how much further I’m going to progress with this little project. The hardest part, being able to have multi-char convos, is already working quite robustly. It wouldn’t be too complicated to create web pages for navigation with the AI as a dungeon master, and displaying AI-generated photos of AI-generated locations. It would probably be quite fun to move around from location to location, talking to generated people who will keep roleplaying according to the background info they get fed about their world, region, area, and location, as well as their own matters such as description and equipment.
The Domains of the Emperor Owl 
You must be logged in to post a comment.