Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to Send and Display a Link from Function Call to Frontend in LiveKit and agent-playground? #1343

Open
zxh263 opened this issue Jan 7, 2025 · 0 comments
Labels
question Further information is requested

Comments

@zxh263
Copy link

zxh263 commented Jan 7, 2025

I want to ask a question about LiveKit: The result contains a link. When this function call is invoked, I want to send this link to the frontend and display it in the chat window of the agent-playground project or in a separately developed component. How can this be implemented?

@llm.ai_callable()
async def take_photo(self):
"""This function is called when the user requests to take a photo. It captures an image from the latest video frame and saves it."""
call_ctx = AgentCallContext.get_current()
message = "Okay, I'll take a photo for you."
speech_handle = await call_ctx.agent.say(message) # noqa: F841
call_ctx.chat_ctx.append(text=message, role="assistant")

    logger.info("Capturing and saving the image")
    # Access JobContext via self.job_context
    latest_image = self.job_context.proc.userdata.get("latest_image", None)
    if latest_image is None:
        logger.error("No available image frame.")
        return "No available image."

    #result = await capture_image(latest_image, "photo")
    result=await save_frame(latest_image)
    # Return the path
    return result 
@zxh263 zxh263 added the question Further information is requested label Jan 7, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

1 participant