Switch to Hatch

[ZeroIntensity]

Description:

As of now, we're using setuptools, which is a bit old. Let's switch to Hatch and scikit-build-core.

[AraHaan]

I think a built in build system without a setup.py file at all that can build C extensions out of the box would be the best idea.

Issue is that build system would need a way to define a custom manifest for where it can build the C extensions as intended.

[ZeroIntensity]

scikit-build-core should do fine, you just set it up with CMake.

[AraHaan]

Yeah, I would like something that does not default to CMake if the user does not have it installed but simply use the logic for finding the exact same compiler used to build python itself like what setuptools used to do (as a fallback).

[ZeroIntensity]

I don't think that exists :(

[trim21]

would you mind take a look at meson？

or simple add a meson.build (I may be able to send a pr) to make it serve as a meson package so users do not need to add runtime depency to wheel but compile this library into their binary dll

[ZeroIntensity]

I'd consider meson, but scikit-build-core has the advantage of being able to integrate with Hatch.

but compile this library into their binary dll

That unfortunately won't work, because we have to use capsules for exposing the API. I'm not a fan of this either, but it wouldn't be possible to link against installed extensions: https://discuss.python.org/t/linking-against-installed-extension-modules/51710

I could try to release linkable binaries alongside an extension, but that would significantly complicate distribution.

[trim21]

I'd consider meson, but scikit-build-core has the advantage of being able to integrate with Hatch.

but compile this library into their binary dll

That unfortunately won't work, because we have to use capsules for exposing the API. I'm not a fan of this either, but it wouldn't be possible to link against installed extensions: https://discuss.python.org/t/linking-against-installed-extension-modules/51710

I could try to release linkable binaries alongside an extension, but that would significantly complicate distribution.

I don't think it's very necessary to do that if we use meson/cmake. build system should be able to compile c code to static library and handle the linkage. For example, what nanobind is currently doing.

But this will make setuptools users hard to use, so I'm not sure either.

(it maybe make it easier to distribution, which means only source tarball)

[ZeroIntensity]

But this will make setuptools users hard to use, so I'm not sure eigher.

Yeah, that's the dilemma. I wonder if it's possible to ship (and maintain) both the capsule and a linkable distribution.

[trim21]

But this will make setuptools users hard to use, so I'm not sure eigher.

Yeah, that's the dilemma. I wonder if it's possible to ship (and maintain) both the capsule and a linkable distribution.

One way is to make it header only so we do not need link?

[ZeroIntensity]

That works, but it would screw up compatibility with ctypes. You're probably better off downloading a vendored copy as found on the releases page. (I forgot to add one for 1.3.0, but you can just download 1.2.0's or generate it via the vendor script).

[eli-schwartz]

Arriving here via pypa/setuptools#4794 (comment)

would you mind take a look at meson？

or simple add a meson.build (I may be able to send a pr) to make it serve as a meson package so users do not need to add runtime depency to wheel but compile this library into their binary dll

I'd consider meson, but scikit-build-core has the advantage of being able to integrate with Hatch.

As mentioned on that setuptools ticket, Hatch (the environment manager and workflow tool) does now support being able to build packages that use arbitrary pep 517 build backends, so you are no longer forced to use hatchling in order to use Hatch. Meson should therefore be a viable option.

So it's now possible to make a more balanced evaluation of the options as hatchling is no longer the load-bearing factor.

[ZeroIntensity]

Meson's probably the way to go then. See also #47; we're planning on dropping capsules as the primary mechanism for exposing the ABI, which should make packaging easier.