Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[JS] Message history not working with dotprompt/streaming #1580

Open
danwritecode opened this issue Jan 2, 2025 · 0 comments
Open

[JS] Message history not working with dotprompt/streaming #1580

danwritecode opened this issue Jan 2, 2025 · 0 comments
Labels
bug Something isn't working js

Comments

@danwritecode
Copy link

Describe the bug
Hey All, I created a thread in Discord already (https://discord.com/channels/1255578482214305893/1310805406188044379) but will recap here.

  1. You can see below that I'm fetching history from my DB and passing it into the model as instructed, however based on the response it doesn't seem to have seen that context at all
  2. The messages array is not appending the messages, this is probably just due to issue 1
history:  [{"role":"system","content":[{"text":"\nYou are friendly and conversational.\n\n"}]},{"role":"user","content":[{"text":"\nhello my
 name is dan,\n\n"}]},{"role":"model","content":[{"text":"Hi"},{"text":" Dan, it's nice to meet you!  How can I help you"},{"text":" today?\
n"}]}]
messages:  [{"role":"system","content":[{"text":"\nYou are friendly and conversational.\n\n"}]},{"role":"user","content":[{"text":"\nwhat di
d I say my name is?,\n\n"}]},{"role":"model","content":[{"text":"As"},{"text":" an AI, I have no memory of past conversations.  To know your
 name"},{"text":", you'll need to tell me!  What is your name?\n"},{"text":""}]}]
async function handleExistingChat(request: AgentInputType, sendChunk?: (chunk: string) => void) {
  const history = await historyStore.load(request.id!)
  const prompt = ai.prompt<typeof ChatMessageInputSchema>('agent_course_create')

  console.log("history: ", JSON.stringify(history))
  
  const { response, stream } = await prompt.stream(
    { user_message: request.message },
    { messages: history }
  )

  await streamResponse(stream, sendChunk)
  
  const { text, messages } = await response

  console.log("messages: ", JSON.stringify(messages))

  await historyStore.update(request.id!, messages)
  return text
}

The doc comments on the messages array states that it appends the message generated, which I'm not seeing happening (probably due to message history not being accounted for).

    /**
     * Appends the message generated by the selected candidate to the messages already
     * present in the generation request. The result of this method can be safely
     * serialized to JSON for persistence in a database.
     * @param index The candidate index to utilize during conversion, defaults to first candidate.
     * @returns A serializable list of messages compatible with `generate({history})`.
     */
    get messages(): MessageData[];

To Reproduce

  1. Setup a dotprompt prompt
  2. Set up a message history store of some sort
  3. Use the .stream() method and follow the example that I showed above

Expected behavior
I expect the "messages" array to be accounted for when calling the .stream() method on the prompt.

Runtime (please complete the following information):

  • MacOS Version 15.1
  • Genkit 0.9.7

** Node version

  • v20.18.1
@danwritecode danwritecode added bug Something isn't working js labels Jan 2, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working js
Projects
Status: No status
Development

No branches or pull requests

1 participant