Issue Report: Inconsistent Model Identification in Puter AI Playground vs. Local Implementation

[robadugna2]

Hello,

I’ve encountered an inconsistency in the responses generated by the AI model when using the Puter AI Playground compared to running the same model in my local project. Below are the details of the issue:

Steps to Reproduce:

1. In Puter AI Playground:

   • Input: "What is your model?"
   • Response: "I am Claude, an AI assistant created by Anthropic. I'm running on Puter, an open-source platform, through a driver interface called puter-chat-completion. I aim to be direct and honest about my identity and capabilities."
   • Screenshot: Link to Screenshot

2. In Local Project:

   • Input: "What is your model?"
   • Response: "I'm based on OpenAI's GPT-3 model, which is designed for understanding and generating human-like text. If you have more specific questions or need assistance, feel free to ask!"
   • Screenshot: Link to Screenshot

Expected Behavior:

The model should consistently identify itself as either Claude (Anthropic) or GPT-3 (OpenAI) across both environments, assuming the same model and configuration are being used.

Observed Behavior:

The model identifies itself as Claude (Anthropic) in the Puter AI Playground but as GPT-3 (OpenAI) in my local project, despite using the same model and configuration.

Questions/Concerns:

1. Is there a difference in the underlying model or configuration between the Puter AI Playground and local implementations?
2. Could this discrepancy be due to a misconfiguration or a difference in how the model is being accessed or initialized in the two environments?
3. Are there any additional steps or settings I should verify to ensure consistency between the Playground and my local setup?

Additional Context:

• The screenshots provided clearly show the difference in responses.
• I am using the same model and configuration in both environments, so the inconsistency is unexpected.

I would greatly appreciate any insights or guidance on resolving this discrepancy. Thank you for your time and assistance!

Best regards,
Robel Adugna

[KernelDeimos]

Hi, thanks for reporting this. I just tried to reproduce the issue with a few naive approaches but was unable to, so we'll need a bit more information about this issue. My first is about the term "Local Project" - is this referring to an app on puter.com or an app under a selfhosted Puter environment? If you're running Puter's backend locally then it could be the case that you're using a user that doesn't have permission to access the claude model.

[robadugna2]

thank u again! yes ofcourse i am running Puter's backend locally but even when i try to ask the same question twice inside of Puter.js Playground. some of the time it respond that it's claude and sometimes groq and etc..

[KernelDeimos]

When you're running your local project, open the network tab and find the last request to /call (maybe it will be among the last few) that has your request to the Puter AI driver. If you share the payload here then we'll know if this has anything to do with the fallback system since we always include what model was actually used in the response.

[robadugna2]

{
"interface": "puter-chat-completion",
"driver": "openai-completion",
"test_mode": false,
"method": "complete",
"args": {
"messages": [
{
"role": "user",
"content": "hi"
},
{
"role": "assistant",
"content": "Hello! How can I assist you today?"
},
{
"role": "user",
"content": "what is your model?"
}
],
"model": "gpt-4o-mini",
"stream": true
}
}

[KernelDeimos]

Whoops, I said payload but I also need the response.

[AJcompilenet]

Hey I literally didn't know what the hell I was doing.. I keep getting all these messages now and maybe restrictions even and I don't even know what I did wrong.. I'm sorry if I affected your guys system in any way

…

On Mon, Jan 13, 2025, 11:44 AM Eric Dubé ***@***.***> wrote: Whoops, I said payload but I also need the response. — Reply to this email directly, view it on GitHub <#1084 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BKQI5UPK7V3NQQRC3CB5OEL2KP3Q5AVCNFSM6AAAAABU4QV76KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKOBXG43TGOJSGE> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>