Physics: Ray intersections #2514

romulogcerqueira · 2024-08-05T15:15:40Z

🎉 New feature

Related to gazebosim/gz-sensors#26 and gazebosim/gz-physics#641

Summary

This PR introduces components to hold rays and ray intersections data (MultiRay and MultiRayIntersections, respectively), and updates the physics system to compute the ray castings during Update loop.

Test it

Added a test case to physics system.

Build and run ./build/bin/INTEGRATION_physics_system --gtest_filter=PhysicsSystemFixture.RayIntersections

Checklist

Signed all commits for DCO
Added tests
Added example and/or tutorial
Updated documentation (as needed)
Updated migration guide (as needed)
Consider updating Python bindings (if the library has them)
codecheck passed (See contributing)
All tests passed (See test coverage)
While waiting for a review on your PR, please help review another open pull request to support the maintainers

Note to maintainers: Remember to use Squash-Merge and edit the commit message to match the pull request summary while retaining Signed-off-by messages.

azeey · 2024-08-06T03:09:57Z

Thanks for the PR! However, as stated in https://community.gazebosim.org/t/feature-freeze-for-gazebo-ionic/2900/2, we are now in feature freeze and will not review/merge any new PRs until after Ionic is released at the end of September. I hope you'll be able to iterate on the PR then.

romulogcerqueira · 2024-08-09T13:29:36Z

Thanks for the PR! However, as stated in https://community.gazebosim.org/t/feature-freeze-for-gazebo-ionic/2900/2, we are now in feature freeze and will not review/merge any new PRs until after Ionic is released at the end of September. I hope you'll be able to iterate on the PR then.

Thanks for the message, @azeey .

One question: Will all new features be redirected to Ionic, or will we maintain backward compatibility with Harmonic, since the latter is LTS?

azeey · 2024-08-09T16:17:36Z

One question: Will all new features be redirected to Ionic, or will we maintain backward compatibility with Harmonic, since the latter is LTS?

Once Ionic is released, we'll ask folks to target PRs to Ionic branches and the core team will backport to older release on a best-effort basis. Of course, you are welcome to backport it yourself. See https://community.gazebosim.org/t/gazebo-policy-update-new-backporting-policy/2880 for more details.

huqin-RM · 2024-11-20T10:45:36Z

I want this function because I want to realize the livox 360 radar simulation. This radar is spiral scanning, and gpu_ray cannot realize this. So when will this function be implemented? @azeey @romulogcerqueira

e1161b76-f3eb-4491-bd6c-2a62b4d7aa21.2._an.1.mp4

ntfshard · 2024-12-22T18:13:09Z

It also can be useful for our use-cases I suppose. I guess this branch should be re-targeted on a last branch

huqin-RM · 2024-12-23T10:11:58Z

@romulogcerqueira Will this feature be pushed forward to the official version?

AliceCryer · 2025-01-14T04:26:33Z

I installed gazebosim with this repository to use this feature, however when attempting to use MultiRay in my plugin, colcon build cannot find the MultiRay.hh file. (I checked and the file is there, and VSCode has no issues with the import statement.) Do I need to adjust my CMakeLists.txt, or is there something specific I need to do to include the file? My code compiles fine otherwise.
Also, is there a way to return the object name that the ray has collided with? It's not a direct part of the MultiRay feature functionality, but is it possible to use the output to query which object is located at the collision point?

romulogcerqueira · 2025-01-15T18:45:19Z

Hi @azeey,

With the release of Gazebo Ionic and the increasing interest from the Gazebo community in this feature (thanks @huqin-RM @ntfshard @AliceCryer), I’d like to move forward with this PR.

Could you please advise on the best way to proceed here, given that this PR targets gz-sim8?

azeey · 2025-01-15T23:28:03Z

Per #2514 (comment), could you target the gz-sim9 branch? You could either use git rebase or git reset --soft to accomplish this. Once you push the changes, you'll then have to change the base branch on the Github interface. Let me know if you face any problems.

And thanks for your patience on this. It's on my queue to review, but I haven't had a chance yet.

Signed-off-by: Rômulo Cerqueira <[email protected]>

romulogcerqueira · 2025-01-16T11:57:51Z

Per #2514 (comment), could you target the gz-sim9 branch? You could either use git rebase or git reset --soft to accomplish this. Once you push the changes, you'll then have to change the base branch on the Github interface. Let me know if you face any problems.

And thanks for your patience on this. It's on my queue to review, but I haven't had a chance yet.

Thanks! This PR is now targeting the gz-sim9 branch and is ready for review again.

Since I'm using Harmonic, I'm also interested in the backport of this feature later.

Signed-off-by: Rômulo Cerqueira <[email protected]>

azeey

This is great work! Thanks again for the contribution. My main concern are:

Naming of components
Coordinate frames of quantities in components
Usability: the need for two components

I would also ask you to add some documentation/tutorial that briefly shows how to use this and mentions that the Bullet collision engine must be used.

azeey · 2025-01-17T19:57:27Z

include/gz/sim/components/MultiRay.hh

+};
+
+/// \brief A struct that holds the results of raycasting.
+struct RayIntersectionInfo


Can we also include the Entity that was hit?

azeey · 2025-01-17T20:10:57Z

src/systems/physics/Physics.cc

+{
+  GZ_PROFILE("PhysicsPrivate::UpdateRayIntersections");
+  // Quit early if the MultiRayIntersections component hasn't been created.
+  // This means there are no systems that need contact information


Suggested change

// This means there are no systems that need contact information

// This means there are no systems that need raycasting information

azeey · 2025-01-17T20:11:59Z

src/systems/physics/Physics.cc

+  auto worldRayIntersectionFeature =
+      this->entityWorldMap.EntityCast<RayIntersectionFeatureList>(worldEntity);


You should check if the cast was successful. If the feature isn't available in the physics engine, the pointer will be nullptr.

azeey · 2025-01-17T20:13:21Z

include/gz/sim/components/MultiRay.hh

+};
+
+/// \brief A component type that contains multiple rays from an entity.
+using MultiRay =


Does this essentially contain the requested rays to be cast? If so, can we change the name so that it's clear?

azeey · 2025-01-17T20:14:46Z

include/gz/sim/components/MultiRay.hh

+
+/// \brief A component type that contains the raycasting results from multiple
+// rays from an entity into a physics world.
+using MultiRayIntersections =


Looking at other raycasting APIs, the work "result" usually appears in the data structure used. The comment here also mentions result, so I'd recommend adding that to the name of the component.

azeey · 2025-01-17T20:18:37Z

src/systems/physics/Physics.cc

+  _ecm.Each<components::MultiRay,
+            components::MultiRayIntersections>(


Does this mean the user has to create two components on the entity to use this feature? Would it be possible to combine them into one? Or we can have the physics engine be responsible for creating the result component (components::MultiRayIntersections).

azeey · 2025-01-17T20:25:36Z

include/gz/sim/components/MultiRay.hh

+/// \brief A struct that holds the information of a ray.
+struct RayInfo
+{
+  /// \brief Starting point of the ray in world coordinates


If this component is attached to an entity, it's natural to assume the start and end points would be expressed relative to that entity. If the user wants to use world coordinates, they would then just attach the component to the world entity.

azeey · 2025-01-17T20:26:24Z

include/gz/sim/components/MultiRay.hh

+struct RayIntersectionInfo
+{
+  /// \brief The hit point in the world coordinates
+  gz::math::Vector3d point;


Similar to my comment above, wouldn't make more sense to express these quantities relative to the frame of the entity this component is attached to?

azeey · 2025-01-17T20:27:34Z

test/integration/physics_system.cc

+        for (size_t i = 0; i < results1.size(); ++i) {
+          ASSERT_EQ(results1[i].point, math::Vector3d::Zero);
+          ASSERT_EQ(results1[i].normal, math::Vector3d(0, 0, 1));
+          double exp_fraction =


nit: use const, use camelCase style

romulogcerqueira requested review from azeey and mjcarroll as code owners August 5, 2024 15:15

github-actions bot added the 🎵 harmonic Gazebo Harmonic label Aug 5, 2024

romulogcerqueira force-pushed the romulogcerqueira/ray_intersections branch 3 times, most recently from 815c56b to e9c3a9a Compare August 6, 2024 02:58

azeey mentioned this pull request Jan 7, 2025

Interface to get pointcloud from SDF #2704

Open

iche033 mentioned this pull request Jan 10, 2025

DART doesn't support ray sensor gazebosim/gazebo-classic#911

Open

romulogcerqueira force-pushed the romulogcerqueira/ray_intersections branch from e9c3a9a to a1471bd Compare January 16, 2025 11:40

romulogcerqueira requested review from mabelzhang and jennuine as code owners January 16, 2025 11:40

romulogcerqueira changed the base branch from gz-sim8 to gz-sim9 January 16, 2025 11:41

add MultiRay and MultiRayIntersections components

7eb56e4

Signed-off-by: Rômulo Cerqueira <[email protected]>

romulogcerqueira force-pushed the romulogcerqueira/ray_intersections branch 2 times, most recently from 1a98be5 to ca55c1b Compare January 16, 2025 11:50

Rômulo Cerqueira added 2 commits January 16, 2025 10:58

computes ray intersections on update loop

adfdf62

Signed-off-by: Rômulo Cerqueira <[email protected]>

add unit tests for ray intersections on physics system

1cdde99

Signed-off-by: Rômulo Cerqueira <[email protected]>

romulogcerqueira force-pushed the romulogcerqueira/ray_intersections branch 2 times, most recently from 07a158f to 9d66d09 Compare January 17, 2025 12:14

fix linter offenses

55e2798

Signed-off-by: Rômulo Cerqueira <[email protected]>

romulogcerqueira force-pushed the romulogcerqueira/ray_intersections branch from 9d66d09 to 55e2798 Compare January 17, 2025 13:41

azeey added 🏛️ ionic Gazebo Ionic and removed 🎵 harmonic Gazebo Harmonic labels Jan 17, 2025

azeey requested changes Jan 17, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Physics: Ray intersections #2514

Physics: Ray intersections #2514

romulogcerqueira commented Aug 5, 2024 •

edited

Loading

azeey commented Aug 6, 2024

romulogcerqueira commented Aug 9, 2024

azeey commented Aug 9, 2024

huqin-RM commented Nov 20, 2024 •

edited

Loading

ntfshard commented Dec 22, 2024

huqin-RM commented Dec 23, 2024

AliceCryer commented Jan 14, 2025 •

edited

Loading

romulogcerqueira commented Jan 15, 2025

azeey commented Jan 15, 2025

romulogcerqueira commented Jan 16, 2025

azeey left a comment

azeey Jan 17, 2025

azeey Jan 17, 2025

azeey Jan 17, 2025

azeey Jan 17, 2025

azeey Jan 17, 2025

azeey Jan 17, 2025

azeey Jan 17, 2025

azeey Jan 17, 2025

azeey Jan 17, 2025

	// This means there are no systems that need contact information
	// This means there are no systems that need raycasting information

		auto worldRayIntersectionFeature =
		this->entityWorldMap.EntityCast<RayIntersectionFeatureList>(worldEntity);

		_ecm.Each<components::MultiRay,
		components::MultiRayIntersections>(

Physics: Ray intersections #2514

Are you sure you want to change the base?

Physics: Ray intersections #2514

Conversation

romulogcerqueira commented Aug 5, 2024 • edited Loading

🎉 New feature

Summary

Test it

Checklist

azeey commented Aug 6, 2024

romulogcerqueira commented Aug 9, 2024

azeey commented Aug 9, 2024

huqin-RM commented Nov 20, 2024 • edited Loading

ntfshard commented Dec 22, 2024

huqin-RM commented Dec 23, 2024

AliceCryer commented Jan 14, 2025 • edited Loading

romulogcerqueira commented Jan 15, 2025

azeey commented Jan 15, 2025

romulogcerqueira commented Jan 16, 2025

azeey left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

romulogcerqueira commented Aug 5, 2024 •

edited

Loading

huqin-RM commented Nov 20, 2024 •

edited

Loading

AliceCryer commented Jan 14, 2025 •

edited

Loading