Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New Serverless Pattern. bedrock streaming inference integration with lambda streaming function #1812

Merged
merged 2 commits into from
Nov 9, 2023

Conversation

randomradio
Copy link
Contributor

add example to demonstrate how to read data properly from bedrock streaming inference api with AWS JS SDK v3, along with streaming LLM response with lambda streaming ability using write()

Issue #, if available:

Description of changes:

  1. Example code to demonstrate how to read from InvokeModelWithResponseStreamCommand inference streaming response with javascript SDK v3.
  2. Demonstrate how to streaming inference response with lambda function, to support a streaming experience with llarge language model.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

…eaming inference api with AWS JS SDK v3, along with streaming LLM response with lambda streaming ability using write()
@benjasl-stripe benjasl-stripe self-assigned this Oct 27, 2023
@randomradio randomradio changed the title bedrock streaming inference integration with lambda streaming function New Serverless Pattern. bedrock streaming inference integration with lambda streaming function Oct 27, 2023
@boyney123 boyney123 closed this Oct 30, 2023
@boyney123 boyney123 reopened this Oct 30, 2023
@github-actions
Copy link

@randomradio looks like you are missing the example-pattern.json file in your pattern.

You can find the example-pattern template here.

The file is used on ServerlessLand and is required. Once the file is added we can review the pattern.

@jbesw jbesw merged commit 2617b04 into aws-samples:main Nov 9, 2023
1 of 2 checks passed
@jbesw
Copy link
Contributor

jbesw commented Nov 9, 2023

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants