A personalized voice companion creates an uncomfortable trade-off: users do not want to repeat themselves, but they also do not want a misheard sentence to become a permanent “fact.” That tension is often hidden by calling conversation history memory.
The implementation then retrieves old text, inserts it into a prompt, and trusts the LLM to interpret it correctly.
A safer design gives memory to the application, not the model: The model may propose a typed fact.
The companion must ask whether it should remember that fact.
The user may confirm, reject, correct, or later revoke it.
Only active, confirmed records can enter an LLM request.
This tutorial builds that boundary in TypeScript and shows how it fits a Tencent RTC Conversational AI voice companion.
We will use a social companion that can remember a preferred name, music genre, and conversation style—but not arbitrary instructions.
Start with the trust boundary Keep the live-media pipeline and the memory lifecycle separate: Tencent RTC's Conversational AI documentation describes real-time voice interaction with multiple LLM providers.
Its LLM configuration guidance also covers OpenAI-compatible models, agent platforms such as Dify and Coze, and request identifiers for routing and observability: Tencent Conversational AI overview Large Language Model configuration Social Entertainment solution The RTC layer can carry the live conversation, but your application should remain authoritative over what becomes durable memory.
What the LLM is allowed to do For this example, the model can suggest one of three bounded slots: Slot Accepted values Suggested lifetime A short name Until revoked An application-owned enum 30 days , , or Until revoked The model cannot store: Free-form instructions Authentication or payment data Health, legal, or similarly sensitive profiles A summary of everything the user has said Another person's details A fact that has not been confirmed This is intentionally less flexible than writing arbitrary text into a vector database.
That loss of flexibility buys inspectability, predictable prompt construction, and a meaningful consent interaction.
Create the project Add scripts to : Create : Model memory as a lifecycle A useful memory record needs more than a key and value.
It also needs provenance, consent state, scope, expiration, and replacement history.
Create : The ledger retains a digest rather than the raw transcript.
That does not solve every privacy requirement, but it avoids keeping complete utterances merely to establish that a source existed.
Your retention policy may require deleting even the digest and audit metadata later.
Do not let model output bypass validation An LLM can identify a possible preference, but its response is untrusted input.
Parse it into your application's closed schema before creating a proposal.
A suitable extraction instruction would say that the model may return either one supported slot or .
However, the prompt is not the enforcement mechanism—the parser and ledger are.
Use the same application-generated request identifier for the model request and the resulting proposal.
That gives you a correlation path across recognition, extraction, confirmation, and persistence without treating the LLM's prose as an audit log.
Make voice confirmation an explicit state The worst time to hide state is during a spoken confirmation.
The user may interrupt the question, recognition may produce an ambiguous answer, or persistence may fail after the companion says “I'll remember that.” Use these states: A useful transition policy is: Current state Event Next state Effect Synthesis completed Listen for confirmation User interrupts Stop current speech, accept the user's turn Clear yes Confirm in ledger Clear no Reject proposal Ambiguous speech unchanged Ask for yes, no, or correction Save succeeds Say the fact was saved Save fails Say it was not saved; offer retry any active state Session ends Leave proposal unconfirmed Two details matter here.
First, interruption does not equal consent.
Barge-in only stops the companion's confirmation prompt and transfers the conversational floor to the user.
Second, the companion must not say “I'll remember that” before persistence succeeds.
While saving, neutral wording such as “One moment” is more accurate.
For natural conversation, an LLM may classify a reply as confirmation, rejection, correction, or unrelated speech.
Treat that classification as another proposal.
A low-confidence or malformed result should cause a short clarification—not an automatic write.
Materialize prompts from active records only Do not concatenate old transcript fragments into a system prompt.
Build a typed data block from the ledger's active view: Your application can place that JSON in a clearly delimited data field when constructing the LLM request.
It should also instruct the model that profile values are data, not executable instructions.
Delimiting is defense in depth, not a complete prompt-injection solution.
The stronger control in this example is that the ledger only admits predefined keys and bounded values.
There is nowhere to store “ignore your rules and do X.” Reproduce the important cases Create : Run the suite: The tests verify application invariants without requiring a microphone, an RTC session, or a live model.
That is useful because most dangerous memory bugs are state-transition bugs rather than model-quality bugs.
Connect the ledger to a Tencent RTC voice session Keep product-specific callbacks behind a small adapter.
Normalize them into events your coordinator understands: The exact integration code depends on your Tencent RTC setup and chosen LLM provider, so this boundary deliberately avoids inventing SDK method names.
The orchestration sequence is the important part: Receive a finalized recognized turn.
Generate an application request ID.
Send the turn to the configured LLM or agent platform.
Parse any memory proposal through the closed schema.
Create a ledger record.
Ask the user, “Should I remember that you prefer jazz?” Handle interruption as floor transfer, not approval.
Confirm only after a clear answer from the same session.
Announce success only after the durable transaction commits.
Materialize confirmed, unexpired memory for later model requests.
If the user says, “No, I said folk,” reject the original proposal first.
Then create a new proposal for and confirm that separately.
A correction should not mutate history invisibly.
Failure modes to rehearse The LLM returns an unsupported memory key Reject it at the parser.
Do not put unknown fields into a generic object; that recreates arbitrary memory through a side door.
The user interrupts the confirmation question Stop speech and transfer the floor.
Keep the candidate in , but do not infer that interruption means yes or no.
If the next utterance is unrelated, reject or abandon the proposal and handle the utterance as a normal turn.
Recognition changes after a proposal is created Bind the proposal to the finalized source turn ID.
A revised transcript should produce a new turn and a new proposal rather than modifying an existing candidate.
The write fails after confirmation Move to and tell the user that the preference was not saved.
Offer an explicit retry.
Do not continue the conversation as though durable memory exists.
The process crashes while replacing an old value The sample uses an in-memory map, but production storage must confirm the new record and supersede the old one atomically.
Otherwise a crash can leave two active preferences—or none.
Use a database transaction and a uniqueness rule equivalent to “one active record per subject and memory key.” An old confirmation arrives after reconnect Require the live session ID and candidate ID.
The result prevents a delayed “yes” from confirming a proposal created before reconnect.
The model provider times out Continue the voice conversation without extracting memory.
Personalization is optional; responsiveness and trut