Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement highlevel unix socket listeners #3187

Open
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

CoolCat467
Copy link
Member

@CoolCat467 CoolCat467 commented Jan 15, 2025

In this pull request, we implement highlevel unix socket support. This is an alternate implementation of and resolves #1433.

I will say, at the moment, only unix SOCK_STREAM (TCP) sockets are supported with this change, but I could pretty easily implement unix datagram socket listeners as well.

Closes #279

Copy link
Contributor

@A5rocks A5rocks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Preliminary comments but I'd like to see some tests!

@Tronic if you see this then maybe you can check these changes seem sane? (alternatively, feel free to move them all to your PR and we can switch to merging that)

src/trio/_highlevel_open_unix_listeners.py Outdated Show resolved Hide resolved
src/trio/_highlevel_open_unix_listeners.py Outdated Show resolved Hide resolved
src/trio/_highlevel_open_unix_listeners.py Outdated Show resolved Hide resolved
if sys.platform != "win32":
sock.setsockopt(tsocket.SOL_SOCKET, tsocket.SO_REUSEADDR, 1)

await sock.bind(str(fspath))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm concerned about this because the original PR does some fun stuff here. Specifically, what if there are two calls to this trying to make a file, does this work? Could we add some tests for this? (along with other tests -- just noticed there are none.)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also the original PR seems to handle this path being e.g. a directory. Does this PR? (yet another test!)

src/trio/_highlevel_open_unix_listeners.py Outdated Show resolved Hide resolved
@@ -31,6 +32,8 @@
errno.ENOTSOCK,
}

HAS_UNIX: Final = hasattr(tsocket, "AF_UNIX")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO this is cleaner than the definition in _highlevel_open_unix_listeners.py

src/trio/_highlevel_socket.py Outdated Show resolved Hide resolved
def __init__(
self,
socket: SocketType,
path: str | bytes | PathLike[str] | PathLike[bytes] | None = None,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I preferred how the old version didn't require this path argument. Is this really required? I assume we can just check whether a socket is unix type with socket.family == getattr(tsocket, "AF_UNIX", None) and have the old logic to get the path.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was looking deeper into things noted in #279 and saw this comment: #279 (comment)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, but I missed something, we only have to care about that if we are renaming socket files, which this implementation does not do, so it would be fine to do the original thing.

Copy link

codecov bot commented Jan 15, 2025

Codecov Report

Attention: Patch coverage is 93.60000% with 8 lines in your changes missing coverage. Please review.

Project coverage is 99.93542%. Comparing base (5ca9662) to head (75cc5df).

Files with missing lines Patch % Lines
src/trio/_highlevel_open_unix_listeners.py 81.08108% 4 Missing and 3 partials ⚠️
.../trio/_tests/test_highlevel_open_unix_listeners.py 98.59155% 1 Missing ⚠️
Additional details and impacted files
@@                 Coverage Diff                  @@
##                 main       #3187         +/-   ##
====================================================
- Coverage   100.00000%   99.93542%   -0.06459%     
====================================================
  Files             124         126          +2     
  Lines           18460       18581        +121     
  Branches         1216        1225          +9     
====================================================
+ Hits            18460       18569        +109     
- Misses              0           7          +7     
- Partials            0           5          +5     
Files with missing lines Coverage Δ
src/trio/__init__.py 100.00000% <100.00000%> (ø)
src/trio/_highlevel_socket.py 100.00000% <100.00000%> (ø)
.../trio/_tests/test_highlevel_open_unix_listeners.py 98.59155% <98.59155%> (ø)
src/trio/_highlevel_open_unix_listeners.py 81.08108% <81.08108%> (ø)

... and 1 file with indirect coverage changes

@A5rocks
Copy link
Contributor

A5rocks commented Jan 15, 2025

Rereading #279, now I get why the original PR has the atomic replace. I believe what this PR does is fine, because no atomic replace -> atomic replace is not a breaking change (I think?), compared to no unlinking -> unlinking.

I think it would be better to not support no downtime redeploys and explicitly document this (at least for now!). Given the complexity of the issue I think @njsmith should look over this of course!

@Sxderp
Copy link

Sxderp commented Jan 15, 2025

I believe what this PR does is fine, because no atomic replace -> atomic replace is not a breaking change (I think?), compared to no unlinking -> unlinking.

It's still compat breaking. I haven't read the code, but I'm assuming in the "before" for both scenarios you error on the path already existing (Address in use, I'd guess). If you start overwriting (via a rename) that's still a change in functionality almost equivalent to unlinking before use.

I think you should figure out the expected pre and post functionality before going stable with it.

  • Do you ignore existing socket files and reuse the path (via unlinking or atomic rename)?
  • Do you ignore existing files and reuse the path (via unlinking or atomic rename)?
  • Do you unlink the socket at the end of the program?
    • PR says yes, but I think it's more common not to? Not really sure honestly. Either way, I think devs have to account for the path existing because the cleanup may not happen on a particularly hard crash.
    • Also not compatible with the atomic rename. Since the PR is taking the option to unlink at the end we find ourselves with another potential functionality change.

@CoolCat467
Copy link
Member Author

CI is failing because of jedi and I am unsure how to go about fixing it. That particular test is incredibly confusing and tells me nothing about what the error means.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

High-level helpers for working with unix domain sockets
3 participants