From 33e3570cfc9f7a62bfe0c3ad9dc7632863d55804 Mon Sep 17 00:00:00 2001 From: LiCHOTHU Date: Tue, 2 Apr 2024 14:28:08 -0400 Subject: [PATCH] Update README.md --- README.md | 24 ++++++++++++++++++++++-- 1 file changed, 22 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 0390660..fc42195 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,27 @@ ## Problem Statement -In the realm of object manipulation, human engagement typically manifests through a constrained array of discrete maneuvers. This interaction can often characterized by a handful of low-dimensional latent actions, such as the act of opening and closing a drawer. Notice that such interaction could diverge on different types of objects but the interaction mode such as opening and closing is discrete. In this paper, we explore how the learned prior emulates this limited repertoire of interactions and if such a prior can be learned from unsupervised play-data. we take a perspective that decomposes the policy into two distinct components: a skill selector and a low-level action predictor, where the skill selector is operating within a discretely structured latent space. +In the realm of object manipulation, human engagement typically manifests through a constrained array of discrete maneuvers. This interaction can often characterized by a handful of low-dimensional latent actions, such as the act of opening and closing a drawer. Notice that such interaction could diverge on different types of objects but the interaction mode such as opening and closing is discrete. In this paper, we explore how the learned prior emulates this limited repertoire of interactions and if such a prior can be learned from unsupervised play-data. we take a perspective that decomposes the policy into tw +### sample object data collection + +run + +``` +. collect_data.sh +``` + +to collect dataset. + +### To access full object dataset from where2act + +## Training + +run + +```bash +python scripts/train_aff.py +``` +o distinct components: a skill selector and a low-level action predictor, where the skill selector is operating within a discretely structured latent space. We introduce **ActAIM2**, which given an RGBD image of an articulated object and a robot, identifies meaningful interaction modes like opening drawer and closing drawer. ActAIM2 represents the interaction modes as discrete clusters of embedding. ActAIM2 then trains a policy that takes cluster embedding as input and produces control actions for the corresponding interactions. @@ -98,7 +118,7 @@ https://github.com/pairlab/actaim2-eccv24/assets/30140814/f08ec7d0-687c-4331-96a https://github.com/pairlab/actaim2-eccv24/assets/30140814/cbf5b5db-e41b-4e59-9043-df8f54d22eda -### Here are mode qualitative results of how robot interacting with different types of articulated objects +### Here are more qualitative results of how robot interacting with different types of articulated objects Interacting with a switch and perform turning on and turning off