-
-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Version of pipenv > 2020.11.15 will produce error AttributeError: 'JoinedStr' object has no attribute 's' #5167
Comments
Please check with |
@matteius yes, I have tried the latest version, it's the same issue. I was testing every version from latest until I discovered that 2020.11.15 works. |
@alexanderdevm Can you try these things: 2.) Trying adding 3.) If neither of those things, I am not sure if the ssh git@ version is what is causing you the issue, are you able to specify it as |
I have updated back to latest pipenv:
|
@alexanderdevm Thanks for checking that, it will have to be investigated further where the code issue may be. |
@alexanderdevm So I did a trial and was not able to reproduce it, but I didn't use your setup.py. It seems odd to me in your setup.py that the Here is my example of the ssh:// working:
|
Yeah it has to be the |
Well to my surprise, I adapted my
|
@alexanderdevm What version of pip are you on? I suspect now based on your original output that you are not on a version of pip compatible with the newer pipenv versions. Since 2022.7.4 has vendor'd pip==22.0.4 I would suggest try installing that or newer to see if you get different results. |
i will do additional testing and see if I can create a public way to repro the issue. |
I have created repo to be able reproduce the issue github: https://github.com/alexanderdevm/pipenv-bug-repro Steps:
|
@alexanderdevm The problem seems to be that the setup.py is not installable the way its written for https://github.com/alexanderdevm/test-lib-1
|
I have did additional testing and the issue is comes from the following:
if I use a variable in the name of install_requires, then it fails with the only time it installs without an issue is if I dont use any variables, and this issue has started after release |
@alexanderdevm Can you share an example setup.py that works without issues? I tried this but I still get that setuptools is trying to find it in pypi.
I believe this is an issue with Discussions that seem relevant: I did from reading that first discussion gather that
When I changed |
@alexanderdevm Actually from my last message I found that It seems that an f-string is ast parsed as an
Yet I append the
I am not convinced this is the right way to handled the |
After exploring this further, I am not convinced there is a safe way to evaluate the f-string or format string from the setup.py. The way that requirementslib is implementing the The problem is one of safety -- if we evaluated the logic of setup.py in the resolution of dependencies there could be a malicious package in the chain that is going to execute unexpected code behind the scenes. So it appears the way it is written is not very possible to return the evaluated string from |
@matteius thank you for looking into the issue. I have created a bug with requirementslib sarugaku/requirementslib#328 The next step is a fix requirementslib side? |
@alexanderdevm I think we can make that case a better error, but I don't think we can safely evaluate an f-string or any interpolated string from the Plus the setup.py the way it is written also is not compatible with |
@matteius , a suggestion is to remove variables from Its also interesting that version of pipenv 2020 and below had no issue. Actually my all time favourite version of pipenv is |
Correct I believe that the 2018 version is still on pip-tools and is going to be lacking quite a few security minded and other types of fixes. I have only been using pipenv for 9 months now, so I cannot speak to the older versions but I know we have been making a lot of enhancements and bug fixes in recent versions. I suspect from looking at the code that it was determined to be a security vulnerability the way it was written prior and it was in 2021 that the Its not entirely clear to me what happened from 2020.11.15 -> 2021.5.29 that would have went from working to not working for your example: v2020.11.15...v2021.5.29#diff-9d03f8495a67df8e88527db31ae4040a36cfa0fbb73b2fd0d6ef0174d7066045 |
@matteius I have updated all my setup.py to be without f-string for the moment and that resolves the issue . For the moment the next course of action:
|
Thanks @alexanderdevm -- I think that the best we will be able to do is throw a better user friendly error in requirementslib that explains the issue. Please see this article for more context on why we don't want to execute arbitrary code, even though pip will do it if you go to install, that seems like a side-effect of the design and not desired. This isn't the exact article I was trying to find you, but it talks about this issue too: https://jfrog.com/blog/python-wheel-jacking-in-supply-chain-attacks/ |
I just ran into the same kind of issue as described here with:
As a workaround, I found that you can wrap your setup(
...
install_requires=list(['easysnmp', f'custom_lib@{HOST}custom_lib.git#egg=custom_lib']),
...
) I initially had an issue with using
The workaround also fixes this issue. Example: TF_VERSION = '2.3.0'
setup(
...
install_requires = list([
f'tensorflow-macos>={TF_VERSION}' if sys.platform == 'darwin' and platform.machine() == 'arm64' else f'tensorflow>={TF_VERSION}']),
...
) |
Wrapping install_requires in |
See pypa/pipenv#5167 (comment) This could lead to failures like $ pipenv install Pipfile.lock (6c5d3c) out of date, updating to (dd7943)... Locking [packages] dependencies... ⠸ Resolving dependencies... Traceback (most recent call last): File "/usr/lib/python3/dist-packages/pipenv/resolver.py", line 845, in <module> main() File "/usr/lib/python3/dist-packages/pipenv/resolver.py", line 831, in main _main( File "/usr/lib/python3/dist-packages/pipenv/resolver.py", line 811, in _main resolve_packages( File "/usr/lib/python3/dist-packages/pipenv/resolver.py", line 759, in resolve_packages results, resolver = resolve( ^^^^^^^^ File "/usr/lib/python3/dist-packages/pipenv/resolver.py", line 738, in resolve return resolve_deps( ^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/pipenv/utils/resolver.py", line 1100, in resolve_deps results, hashes, markers_lookup, resolver, skipped = actually_resolve_deps( ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/pipenv/utils/resolver.py", line 888, in actually_resolve_deps resolver = Resolver.create( ^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/pipenv/utils/resolver.py", line 458, in create constraints, skipped, index_lookup, markers_lookup = resolver.get_metadata( ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/pipenv/utils/resolver.py", line 246, in get_metadata constraint_update, lockfile_update = self.get_deps_from_req( ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/pipenv/utils/resolver.py", line 325, in get_deps_from_req req_list, lockfile = get_vcs_deps(reqs=[req]) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/pipenv/utils/dependencies.py", line 125, in get_vcs_deps with temp_path(), locked_repository(requirement) as repo: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/pipenv/utils/dependencies.py", line 375, in locked_repository with requirement.req.locked_vcs_repo(src_dir=src_dir) as repo: File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/pipenv/vendor/requirementslib/models/requirements.py", line 2203, in locked_vcs_repo self._parsed_line.vcsrepo = vcsrepo ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/pipenv/vendor/requirementslib/models/requirements.py", line 903, in vcsrepo setupinfo = SetupInfo.create( ^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/pipenv/vendor/requirementslib/models/setup_info.py", line 1564, in create created.get_initial_info() File "/usr/lib/python3/dist-packages/pipenv/vendor/requirementslib/models/setup_info.py", line 1393, in get_initial_info parsed.update(self.parse_setup_py()) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/pipenv/vendor/requirementslib/models/setup_info.py", line 1118, in parse_setup_py parsed = ast_parse_setup_py(self.setup_py.as_posix()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/pipenv/vendor/requirementslib/models/setup_info.py", line 839, in ast_parse_setup_py return SetupReader.read_setup_py(Path(path), raising) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/pipenv/vendor/requirementslib/models/setup_info.py", line 180, in read_setup_py "install_requires": caller(cls._find_install_requires, setup_call, body), ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/pipenv/vendor/requirementslib/models/setup_info.py", line 172, in caller return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/pipenv/vendor/requirementslib/models/setup_info.py", line 335, in _find_install_requires return [el.s for el in value.elts] ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/pipenv/vendor/requirementslib/models/setup_info.py", line 335, in <listcomp> return [el.s for el in value.elts] ^^^^ AttributeError: 'IfExp' object has no attribute 's'
We dropped requirementslib and I believe this error should be improved now. |
Issue description
Version of pipenv >
2020.11.15
will produce errorAttributeError: 'JoinedStr' object has no attribute 's'
when it installs a package from private repo that containssetup.py
with dependencies from private repo.Expected result
Packages and its dependencies should install from private repo (bitbucket)
Actual result
Steps to replicate
Pipfile:
custom_lib -> setup.py:
Steps:
pipenv install
pipenv --support
$ pipenv --support
Pipenv version:
'2022.1.8'
Pipenv location:
'/home/SYS/comp1/.pyenv/versions/3.7.13/lib/python3.7/site-packages/pipenv'
Python location:
'/home/SYS/comp1/.pyenv/versions/3.7.13/bin/python3'
Python installations found:
3.10.5
:/home/SYS/comp1/.pyenv/versions/3.10.5/bin/python3
3.10.4
:/usr/bin/python3
3.10.4
:/usr/bin/python
3.10.4
:/bin/python3
3.10.4
:/bin/python
3.9.13
:/usr/bin/python3.9
3.9.13
:/bin/python3.9
3.7.13
:/home/SYS/comp1/.pyenv/versions/3.7.13/bin/python3
3.7.13
:/home/SYS/comp1/.pyenv/versions/3.7.13/bin/python
3.7.13
:/home/SYS/comp1/.pyenv/versions/3.7.13/bin/python3.7m
3.7.13
:/home/SYS/comp1/.pyenv/versions/3.7.13/bin/python3.7
3.7.13
:/home/SYS/comp1/.pyenv/versions/3.7.13/bin/python3
3.7.9
:/home/SYS/comp1/.pyenv/versions/3.7.9/bin/python3
3.6.11
:/home/SYS/comp1/.pyenv/versions/3.6.11/bin/python3
3.6.9
:/usr/bin/python3.6m
3.6.9
:/usr/bin/python3.6
3.6.9
:/bin/python3.6m
3.6.9
:/bin/python3.6
3.5.10
:/usr/bin/python3.5
3.5.10
:/usr/bin/python3.5m
3.5.10
:/bin/python3.5
3.5.10
:/bin/python3.5m
2.7.18
:/usr/bin/python2.7
2.7.18
:/usr/bin/python2
2.7.18
:/bin/python2.7
2.7.18
:/bin/python2
PEP 508 Information:
System environment variables:
SHELL
PYENV_HOOK_PATH
PYENV_SHELL
NVM_INC
PKG_CONFIG_PATH
LANGUAGE
LC_ADDRESS
LC_NAME
LC_MONETARY
CONNECT_FB_PASSWORD
PYENV_VERSION
KRB5CCNAME
PWD
PYENV_VIRTUALENV_INIT
LOGNAME
XDG_SESSION_TYPE
MOTD_SHOWN
HOME
LANG
LC_PAPER
LS_COLORS
SSH_CONNECTION
PYENV_DIR
CONNECT_UI_PASSWORD
NVM_DIR
CONNECT_YT_USER
LESSCLOSE
XDG_SESSION_CLASS
TERM
LC_IDENTIFICATION
LESSOPEN
LIBVIRT_DEFAULT_URI
USER
CONNECT_UI_URL
SHLVL
NVM_CD_FLAGS
LC_TELEPHONE
LC_MEASUREMENT
XDG_SESSION_ID
PAPERSIZE
XDG_RUNTIME_DIR
SSH_CLIENT
PYENV_ROOT
LC_TIME
XDG_DATA_DIRS
PATH
DBUS_SESSION_BUS_ADDRESS
NVM_BIN
SSH_TTY
LC_NUMERIC
OLDPWD
PIP_SHIMS_BASE_MODULE
PIP_DISABLE_PIP_VERSION_CHECK
PYTHONDONTWRITEBYTECODE
PIP_PYTHON_PATH
PYTHONFINDER_IGNORE_UNSUPPORTED
Pipenv–specific environment variables:
Debug–specific environment variables:
PATH
:/home/SYS/comp1/.pyenv/versions/3.7.13/bin:/home/SYS/comp1/.pyenv/libexec:/home/SYS/comp1/.pyenv/plugins/python-build/bin:/home/SYS/comp1/.pyenv/plugins/pyenv-virtualenv/bin:/home/SYS/comp1/.pyenv/shims:/home/SYS/comp1/.pyenv/versions:/home/SYS/comp1/.pyenv/bin:/home/SYS/comp1/.pyenv/plugins/pyenv-virtualenv/shims:/home/SYS/comp1/.pyenv/bin:/home/SYS/comp1/.local/bin:/home/SYS/comp1/.pyenv/versions:/home/SYS/comp1/.pyenv/bin:/shims:/home/SYS/comp1/.nvm/versions/node/v16.13.0/bin:/home/SYS/comp1/.pyenv/plugins/pyenv-virtualenv/shims:/home/SYS/comp1/.pyenv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
SHELL
:/bin/bash
LANG
:en_US.UTF-8
PWD
:/home/SYS/comp1/git/project/pytest
Contents of
Pipfile
('/home/SYS/comp1/git/project/pytest/Pipfile'):The text was updated successfully, but these errors were encountered: