Direct link to SCANEO

[juansensio]

When user click on the "open" button in the SCANEO section, the SCANEO labelling tool should be opened in a new tab ready to use (in the same way that when they click on "open" the workspace, the workspace is opened).

The SCANEO labelling tool runs in the user workspace and exposes a URL to redirect users from the website. This url should be fixed (depending maybe on the user id or some other information). If SCANEO is not running when the user clicks "open", is should start and redirect when ready.

[achtsnits]

the scaneo tool is obviously scanning the cli passed folder (-d) on startup - how should that work in this setup, should the scaneo instance point to the user home folder and everything is supposed to be scanned? or do you plan a file/folder picker inside the tool?

[juansensio]

It should not use the default behaviour that you are mentioning, but the EOTDL plugin (wich is still in development). The command will be something like scaneo --etodl (the UI will manage user authentication, show a list of all the user's datasets and allow labelling as expected). I'll let you know once the tool is ready.

In the meantime you can point to some of the datasets in the cached folder (like EuroSAT, for example) just to make sure the workflow described in this issue works as expected.

[achtsnits]

can you elaborate on this plugin - where will it point to, i.e. what do you mean with "user's datasets"

do you mean instead of scanning a folder it will use the eotdl library internally to connect to eotdl datasets (filtered somehow by user) via s3/boto? where will the labels be stored, how that will work?

[juansensio]

It will communicate entirely with the EOTDL API (via the library) to retrieve and store all the required information (data and labels included), so no need to store anything locally.

[achtsnits]

sounds good

so we package scaneo and the plugin in a docker image, call scaneo --eotdl on process startup and don't need to care about authentication or storage (like volume mounts) as these concerns are all handled by scaneo itself

if the user closes the browser page with the running scaneo application we have to implement some "culling" mechanism to stop the docker process at some point, so based on the previous comment we are safe that the user might not potentially lose any work, nice

looking forward to see that in action, just let us know when we can try with the scaneo plugin, please reassign the issue then, thanks

[juansensio]

Resuming this task @achtsnits currently the tool can be started with scaneo --eotdl . Is this ok or should we do the scaneo --eotdl and then the user has to select the datasets directly inside the labelling tool ?

[achtsnits]

which eotdl version is used here with scaneo as scaneo is obviously not bundling eotdl?

[juansensio]

Latest version is ok, currently at 2024.05.02

[achtsnits]

thanks

can you please open JupyterLab (you can use regular profile, no need for Premium profile), open the terminal, run "conda activate scaneo-2024.6.4", subsequently "scaneo --eotdl EuroSAT-RGB-small" and then replace the "/lab" part in the url to "/proxy/8000/"(i,e. https://hub.api.eotdl.com/user/<username>/proxy/8000/) the ScaneO tool opens but I still see error "no images found", any idea?

[juansensio]

Looks like the UI is trying to get the images from the base URL instead of https://hub.api.eotdl.com/user/username/proxy/8000/images

I'll fix this and let you know.

[earthpulse]

@achtsnits scaneo-2024.6.20 should solve this issue !

[achtsnits]

with conda activate scaneo-2024.5.2 you now have scaneo 2024.6.20 installed

[earthpulse]

It looks like the UI can communicate with the backend properly. However, it is really slow.

Loading the images for visualization takes a long time, after that labelling does not work either (I get "You are not authorized to perform this action", an error SCANEO throws if the owner of the dataset is not the annotator. Will take a closer look at this since it should work for my user and a dataset that I ingested myself).

[achtsnits]

as idea (similiar to ml-inference) - is it possible that you create sub paths in the https server of scaneo app for eotdl, i.e. on scaneo.xxx/ you keep existing behavior and on urls like scaneo.xxx/EuroSAT-RGB-small/ and similar your are refreshing app with behavior mimicking --eotdl EuroSAT-RGB-small flag

perhaps again with a configurable EOTDL_API_KEY= env variable -> we could then deploy scaneo with a fixed url without requiring the user to manually start scaneo

[earthpulse]

Intead of starting the tool with the model as a parameter to the CLI, use only --eotdl to start it in "eotdl" mode and then select the model from a list (in a dropdown for example). It could also add the model to the URL so we can open it directly in EOTDL mode with that model ready from the UI.

The goal is to start the tool once so it is always running and be able to better manage resources.

[Patrick1G]

@juansensio @achtsnits
any progress on this? we would expect full integration of scaneo at the EOTDL top level menue, without just opening the pip page for the scaneo package? Can we please priorities this to close this for good until next MS?
@jamesemwheeler @dmoglioni

[juansensio]

Hello, we are working on it and should be working as expected for MS5.

[juansensio]

Duplicated #271