Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue creating collector with RemoteWorkspace #1422

Open
caamartin35 opened this issue Jan 15, 2025 · 0 comments
Open

Issue creating collector with RemoteWorkspace #1422

caamartin35 opened this issue Jan 15, 2025 · 0 comments

Comments

@caamartin35
Copy link

I am using this code

import pandas as pd
import datetime
import time
from evidently.collector.config import CollectorServiceConfig
from evidently.collector.config import CollectorConfig
from evidently.collector.config import IntervalTrigger
from evidently.collector.config import ReportConfig
from evidently.report import Report
from evidently.metric_preset import DataDriftPreset

from evidently.test_suite import TestSuite
from evidently.metrics import ColumnValueRangeMetric
from evidently.tests import TestNumberOfOutRangeValues
from evidently.collector.client import CollectorClient
from evidently.ui.workspace import RemoteWorkspace

COLLECTOR_URL = "http://evidently-ai-collector:8000"
UI_URL = "http://evidently-ai-ui:8000"

client = CollectorClient(COLLECTOR_URL)

def get_or_create_project_by_name(ui_api_url: str, project_name: str):
    ui_client = RemoteWorkspace(ui_api_url)
    projects = ui_client.search_project(project_name)
    if len(projects) == 0:
        project = ui_client.create_project(project_name)
    else:
        project = projects[0]
    return str(project.id)


def get_data():
    cur = ref = pd.DataFrame([{"values1": 5.0, "values2": 0.0} for _ in range(10)])
    return cur, ref

def send_data():
    size = 1
    data = pd.DataFrame([{"values1": 3.0 + datetime.datetime.now().minute % 5, "values2": 0.0} for _ in range(size)])

    client.send_data("po_mex_inputs", data)
    print("sent")

def start_sending_data():
    print("Start data loop")
    while True:
        send_data()
        
        time.sleep(1)


print("Getting project id")
project_id = get_or_create_project_by_name(UI_URL, "Power Outage Mexico Model")

report = Report(metrics=[DataDriftPreset()])

cur, ref = get_data()

print("Running report")
report.run(current_data=cur, reference_data=ref)

test = CollectorConfig(
        id="po_mex_inputs",
        trigger=IntervalTrigger(interval=60),  # trigger every 60 seconds
        report_config=ReportConfig.from_report(report),
        project_id=project_id,  # replace with your project ID
        api_url=UI_URL  # replace with your UI service URL if different
    )

client.set_reference("po_mex_inputs", get_data()[0])
client.create_collector("po_mex_inputs", test)

start_sending_data()

The project creation and everything works except when it goes to create the collector it gives this error:

Traceback (most recent call last):
  File "/home/app/.venv/lib/python3.9/site-packages/evidently/collector/app.py", line 232, in check_service_snapshots_periodically
    await check_snapshots_factory(service, service.storage)
  File "/home/app/.venv/lib/python3.9/site-packages/evidently/collector/app.py", line 131, in check_snapshots_factory
    await create_snapshot(collector, storage)
  File "/home/app/.venv/lib/python3.9/site-packages/evidently/collector/app.py", line 136, in create_snapshot
    async with storage.lock(collector.id):
  File "/home/app/.venv/lib/python3.9/site-packages/evidently/collector/storage.py", line 52, in lock
    return self._locks[id]
KeyError: 'po_mex_inputs'

This is pretty much straight from the tutorial here:
https://github.com/evidentlyai/evidently/blob/main/examples/service/collector/generate_config.py

Is there something we are missing? When I try to send the data it gives 500 errors so clearly the collector was not created correctly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant