Langsmith nested runs do not work correctly with streaming

[grantfwilkinson]

Description

I am working with the Vercel AI Chatbot example and using streamText.

I have experimental_telemetry: AISDKExporter.getSettings() setup.

And since I am using a tool that uses Langchain multi query retriever, it is creating two runs, my chat run and my query run.

If I leave them un-nested it works correctly with latency, input/output, etc.

If I nest them with https://sdk.vercel.ai/providers/observability/langsmith traceable method, it combines them but removes the input/output + the latency is always 0.1s. See run example. If its not streaming text it works correctly.

I tried updating the run end time + add output onFinish which does not work. I tried different tracing through sentry which also didnt work. I tried moving the traceable call different levels.

Wasnt sure how to best read the debug log from open telemetry but didnt see anything obvious there.

Code example

  const chatOperation = traceable(
    async () => {
      const stream = createDataStreamResponse({
        execute: async (dataStream) => {
          await langchainService.initialize(session.user!.bubbleUserId);
          const result = streamText({
            model: customModel(model.apiIdentifier),
            system: systemPrompt,
            messages: coreMessages,
            maxSteps: 5,
            experimental_activeTools: ["searchKnowledgeBase"],
            experimental_telemetry: AISDKExporter.getSettings({}),
            tools: {
              searchKnowledgeBase: {
                description:
                  "REQUIRED: You MUST use this tool FIRST for EVERY question, no exceptions.",
                parameters: z.object({
                  query: z
                    .string()
                    .describe("the exact question from the user"),
                }),
                execute: async ({ query }) => {
                  const searchOperation = traceable(
                    async () => {
                      console.log("🔍 Searching knowledge base for:", query);
                      const results = await langchainService.similaritySearch(
                        query
                      );

                      return {
                        results: results.map((doc) => ({
                          content: doc.metadata?.text || doc.pageContent,
                          score: Math.round(doc.score * 100),
                        })),
                        sourceCount: results.length,
                        hasResults: results.length > 0,
                      };
                    },
                    { name: "search-knowledge-base" }
                  )();

                  return await searchOperation;
                },
              },
            },
            onFinish: async ({ response }) => {
              if (session.user?.id) {
                try {
                  const responseMessagesWithoutIncompleteToolCalls =
                    sanitizeResponseMessages(response.messages);

                  await saveMessages({
                    messages: responseMessagesWithoutIncompleteToolCalls.map(
                      (message) => {
                        const messageId = generateUUID();
                        if (message.role === "assistant") {
                          dataStream.writeMessageAnnotation({
                            messageIdFromServer: messageId,
                          });
                        }
                        return {
                          id: messageId,
                          chatId: id,
                          role: message.role,
                          content: message.content,
                          createdAt: new Date(),
                        };
                      }
                    ),
                  });
                } catch (error) {
                  console.error("Failed to save chat:", error);
                }
              }
            },
          });
          await result.mergeIntoDataStream(dataStream);
        },
      });
      return stream;
    },
    {
      name: `chat-${id}`,
      inputs: {
        messages: coreMessages,
      },
      metadata: {
        chatId: id,
        userId: session.user.id,
        email: session.user.email,
        model: model.apiIdentifier,
      },
    }
  );

  return await chatOperation();
}

AI provider

ai:4.0.20

Additional context

No response

[lgrammel]

This is most likely an issue on the LangSmith integration side. Can you file a report with them? CC @jacoblee93

[jacoblee93]

Thanks for flagging - will dig in!

[jacoblee93]

Hey @grantfwilkinson and @lgrammel, this should be fixed in [email protected] - can you verify and close this if so?

[grantfwilkinson]

@lgrammel thank you for tagging. @jacoblee93 I tried 0.2.15 and didnt see any changes on my end. I reviewed the commit and not sure if I need to keep traceable the same format as above or if anything changed. I used the same vercel packages as your tests.

[jacoblee93]

Sorry about the mixup - will keep you posted!