Skip to content

Commit

Permalink
Run syncronous functions in a separate thread
Browse files Browse the repository at this point in the history
  • Loading branch information
Shulyaka committed Dec 12, 2024
1 parent 959aaa2 commit 31c86a2
Show file tree
Hide file tree
Showing 2 changed files with 9 additions and 2 deletions.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ There are two options:

* Extend the `custom_components.powerllm.PowerLLMTool` class to implement the functionality, then call `custom_components.powerllm.async_register_tool` to register the object of the class. See the [memory tool](https://github.com/Shulyaka/powerllm/blob/master/custom_components/powerllm/tools/memory.py) for an example

* Use the `custom_components.powerllm.llm_tool` decorator for any python function. The function is recommended to have type annotations for all parameters. If a parameter name is "hass", "llm_context", or any of the `homeassistant.helpers.llm.LLMContext` attributes, then the value for that parameter will be provided by the conversation agent ("pytest-style"). All other arguments will be provided by the LLM. Refer to the [python code tool](https://github.com/Shulyaka/powerllm/blob/master/custom_components/powerllm/tools/python_code.py) as an example.
* Use the `custom_components.powerllm.llm_tool` decorator for any python function. The function is recommended to have type annotations for all parameters. If a parameter name is "hass", "llm_context", or any of the `homeassistant.helpers.llm.LLMContext` attributes, then the value for that parameter will be provided by the conversation agent ("pytest-style"). All other arguments will be provided by the LLM. Refer to the [python code tool](https://github.com/Shulyaka/powerllm/blob/master/custom_components/powerllm/tools/python_code.py) as an example. A synchronous function will be executed in a separate thread, while an async function will be run concurrently in the main event loop and therefore is not allowed to use any blocking operations.

The tools in this repository use various techniques for demonstration.

Expand Down
9 changes: 8 additions & 1 deletion custom_components/powerllm/llm_tools.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

from __future__ import annotations

import asyncio
import inspect
import logging
from collections.abc import Callable
Expand Down Expand Up @@ -289,6 +290,8 @@ class PowerFunctionTool(PowerLLMTool):
attributes, then the value for that parameter will be provided by the
conversation agent 'pytest-style'.
All other arguments will be provided by the LLM.
Async functions are not allowed to use any blocking code, while a synchronous
function will be executed in a separate thread.
"""

function: Callable
Expand Down Expand Up @@ -373,7 +376,11 @@ async def async_call(

if inspect.iscoroutinefunction(self.function):
return await self.function(**kwargs)
return self.function(**kwargs)

if hass.loop != asyncio.get_running_loop():
return self.function(**kwargs)

return await hass.loop.run_in_executor(None, lambda: self.function(**kwargs))


@callback
Expand Down

0 comments on commit 31c86a2

Please sign in to comment.