Unable to create conda environment

[AlejandroSanchezCano]

I clone the repo, cd into it, run the provided command mamba env create -n openfold_env -f environment.yml and run into the following error:

Channels:
 - conda-forge
 - bioconda
 - pytorch
Platform: linux-64
Collecting package metadata (repodata.json): done
Solving environment: done


==> WARNING: A newer version of conda exists. <==
    current version: 24.1.2
    latest version: 24.3.0

Please update conda by running

    $ conda update -n base -c conda-forge conda



Downloading and Extracting Packages:

Preparing transaction: done
Verifying transaction: done
Executing transaction: / By downloading and using the CUDA Toolkit conda packages, you accept the terms and conditions of the CUDA End User License Agreement (EULA): https://docs.nvidia.com/cuda/eula/index.html
                                                                                                                                                done
Installing pip dependencies: | Ran pip subprocess with arguments:
['/gpfs/home2/asanchez/miniforge3/envs/openfooooold/bin/python', '-m', 'pip', 'install', '-U', '-r', '/gpfs/home2/asanchez/protein_language_models/openfold/condaenv.3ckl72oz.requirements.txt', '--exists-action=b']
Pip subprocess output:
Collecting git+https://github.com/NVIDIA/dllogger.git (from -r /gpfs/home2/asanchez/protein_language_models/openfold/condaenv.3ckl72oz.requirements.txt (line 3))
  Cloning https://github.com/NVIDIA/dllogger.git to /gpfs/scratch1/nodespecific/int6/asanchez/pip-req-build-i2xa_ixd
  Resolved https://github.com/NVIDIA/dllogger.git to commit 0540a43971f4a8a16693a9de9de73c1072020769
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Collecting git+https://github.com/Dao-AILab/flash-attention.git@5b838a8 (from -r /gpfs/home2/asanchez/protein_language_models/openfold/condaenv.3ckl72oz.requirements.txt (line 4))
  Cloning https://github.com/Dao-AILab/flash-attention.git (to revision 5b838a8) to /gpfs/scratch1/nodespecific/int6/asanchez/pip-req-build-ngp87hup
  Resolved https://github.com/Dao-AILab/flash-attention.git to commit 5b838a8
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'error'

Pip subprocess error:
  Running command git clone --filter=blob:none --quiet https://github.com/NVIDIA/dllogger.git /gpfs/scratch1/nodespecific/int6/asanchez/pip-req-build-i2xa_ixd
  Running command git clone --filter=blob:none --quiet https://github.com/Dao-AILab/flash-attention.git /gpfs/scratch1/nodespecific/int6/asanchez/pip-req-build-ngp87hup
  WARNING: Did not find branch or tag '5b838a8', assuming revision or ref.
  Running command git checkout -q 5b838a8
  Running command git submodule update --init --recursive -q
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [8 lines of output]
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/gpfs/scratch1/nodespecific/int6/asanchez/pip-req-build-ngp87hup/setup.py", line 10, in <module>
          import torch
        File "/gpfs/home2/asanchez/miniforge3/envs/openfooooold/lib/python3.9/site-packages/torch/__init__.py", line 202, in <module>
          from torch._C import *  # noqa: F403
      ImportError: /gpfs/home2/asanchez/miniforge3/envs/openfooooold/lib/python3.9/site-packages/torch/lib/libtorch_cpu.so: undefined symbol: iJIT_NotifyEvent
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
                                                                                                                                                failed

CondaEnvException: Pip failed

[Bedinka]

Hi !
Having this issue too, unable to crete the env, even its getting created I am unable to run on it.
`(base) dina@:~/Documents/PPI_WDLLM/openfold$ mamba env create -n openfold_env -f environment.yml
Channels:

• conda-forge
• bioconda
• pytorch
• defaults
  Platform: linux-64
  Collecting package metadata (repodata.json): done
  Solving environment: done

Downloading and Extracting Packages:

Preparing transaction: done
Verifying transaction: done
Executing transaction: | By downloading and using the CUDA Toolkit conda packages, you accept the terms and conditions of the CUDA End User License Agreement (EULA): https://docs.nvidia.com/cuda/eula/index.html

done
Installing pip dependencies: \ Ran pip subprocess with arguments:
['/home/dina/miniconda3/envs/openfold_env/bin/python', '-m', 'pip', 'install', '-U', '-r', '/home/dina/Documents/PPI_WDLLM/openfold/condaenv.preb3c5c.requirements.txt', '--exists-action=b']
Pip subprocess output:
Collecting git+https://github.com/NVIDIA/dllogger.git (from -r /home/dina/Documents/PPI_WDLLM/openfold/condaenv.preb3c5c.requirements.txt (line 3))
Cloning https://github.com/NVIDIA/dllogger.git to /tmp/pip-req-build-71c6bsk5
Resolved https://github.com/NVIDIA/dllogger.git to commit 0540a43971f4a8a16693a9de9de73c1072020769
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Collecting git+https://github.com/Dao-AILab/flash-attention.git@5b838a8 (from -r /home/dina/Documents/PPI_WDLLM/openfold/condaenv.preb3c5c.requirements.txt (line 4))
Cloning https://github.com/Dao-AILab/flash-attention.git (to revision 5b838a8) to /tmp/pip-req-build-ayvfskhx

Pip subprocess error:
Running command git clone --filter=blob:none --quiet https://github.com/NVIDIA/dllogger.git /tmp/pip-req-build-71c6bsk5
Running command git clone --filter=blob:none --quiet https://github.com/Dao-AILab/flash-attention.git /tmp/pip-req-build-ayvfskhx
WARNING: Did not find branch or tag '5b838a8', assuming revision or ref.
Running command git checkout -q 5b838a8
fatal: unable to access 'https://github.com/Dao-AILab/flash-attention.git/': Failed to connect to github.com port 443 after 135026 ms: Couldn't connect to server
fatal: could not fetch a8e8349e1ba482c43ec63bc4676fba32fe393f67 from promisor remote
error: subprocess-exited-with-error

× git checkout -q 5b838a8 did not run successfully.
│ exit code: 128
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× git checkout -q 5b838a8 did not run successfully.
│ exit code: 128
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

failed

CondaEnvException: Pip failed

(base) dina@:~/Documents/PPI_WDLLM/openfold$ conda env create -n openfold_env -f environment.yml

CondaValueError: prefix already exists: /home/dina/miniconda3/envs/openfold_env

(base) dina@:~/Documents/PPI_WDLLM/openfold$ conda env create -n openfold_env2 -f environment.yml
Channels:

• conda-forge
• bioconda
• pytorch
• defaults
  Platform: linux-64
  Collecting package metadata (repodata.json): done
  Solving environment: done

Downloading and Extracting Packages:

Preparing transaction: done
Verifying transaction: done
Executing transaction: / By downloading and using the CUDA Toolkit conda packages, you accept the terms and conditions of the CUDA End User License Agreement (EULA): https://docs.nvidia.com/cuda/eula/index.html

done
Installing pip dependencies: | Ran pip subprocess with arguments:
['/home/dina/miniconda3/envs/openfold_env2/bin/python', '-m', 'pip', 'install', '-U', '-r', '/home/dina/Documents/PPI_WDLLM/openfold/condaenv.7a4lsks6.requirements.txt', '--exists-action=b']
Pip subprocess output:
Collecting git+https://github.com/NVIDIA/dllogger.git (from -r /home/dina/Documents/PPI_WDLLM/openfold/condaenv.7a4lsks6.requirements.txt (line 3))
Cloning https://github.com/NVIDIA/dllogger.git to /tmp/pip-req-build-958chdta
Resolved https://github.com/NVIDIA/dllogger.git to commit 0540a43971f4a8a16693a9de9de73c1072020769
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Collecting git+https://github.com/Dao-AILab/flash-attention.git@5b838a8 (from -r /home/dina/Documents/PPI_WDLLM/openfold/condaenv.7a4lsks6.requirements.txt (line 4))
Cloning https://github.com/Dao-AILab/flash-attention.git (to revision 5b838a8) to /tmp/pip-req-build-xv8wg8u4
Resolved https://github.com/Dao-AILab/flash-attention.git to commit 5b838a8
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'error'

Pip subprocess error:
Running command git clone --filter=blob:none --quiet https://github.com/NVIDIA/dllogger.git /tmp/pip-req-build-958chdta
Running command git clone --filter=blob:none --quiet https://github.com/Dao-AILab/flash-attention.git /tmp/pip-req-build-xv8wg8u4
WARNING: Did not find branch or tag '5b838a8', assuming revision or ref.
Running command git checkout -q 5b838a8
Running command git submodule update --init --recursive -q
error: subprocess-exited-with-error

× python setup.py egg_info did not run successfully.
│ exit code: 1
╰─> [8 lines of output]
Traceback (most recent call last):
File "", line 2, in
File "", line 34, in
File "/tmp/pip-req-build-xv8wg8u4/setup.py", line 10, in
import torch
File "/home/dina/miniconda3/envs/openfold_env2/lib/python3.9/site-packages/torch/init.py", line 202, in
from torch._C import * # noqa: F403
ImportError: /home/dina/miniconda3/envs/openfold_env2/lib/python3.9/site-packages/torch/lib/libtorch_cpu.so: undefined symbol: iJIT_NotifyEvent
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

failed

CondaEnvException: Pip failed

(base) dina@:~/Documents/PPI_WDLLM/openfold$ conda env list

conda environments:

base * /home/dina/miniconda3
esmfold /home/dina/miniconda3/envs/esmfold
myenv /home/dina/miniconda3/envs/myenv
openfold /home/dina/miniconda3/envs/openfold
openfold_env /home/dina/miniconda3/envs/openfold_env
openfold_env2 /home/dina/miniconda3/envs/openfold_env2

(base) dina@:/Documents/PPI_WDLLM/openfold$ conda activate openfold_env
(openfold_env) dina@:/Documents/PPI_WDLLM/openfold$ scripts/install_third_party_dependencies.sh
--2024-04-16 17:22:08-- https://git.scicore.unibas.ch/schwede/openstructure/-/raw/7102c63615b64735c4941278d92b554ec94415f8/modules/mol/alg/src/stereo_chemical_props.txt
Resolving git.scicore.unibas.ch (git.scicore.unibas.ch)... 131.152.229.50
Connecting to git.scicore.unibas.ch (git.scicore.unibas.ch)|131.152.229.50|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 9119 (8,9K) [text/plain]
Saving to: ‘openfold/resources/stereo_chemical_props.txt’

stereo_chemical_props.txt 100%[=================================================>] 8,91K --.-KB/s in 0s

Last-modified header missing -- time-stamps turned off.
2024-04-16 17:22:09 (50,3 MB/s) - ‘openfold/resources/stereo_chemical_props.txt’ saved [9119/9119]

ln: failed to create symbolic link 'tests/test_data/alphafold/common/stereo_chemical_props.txt': File exists
Traceback (most recent call last):
File "/home/dina/Documents/PPI_WDLLM/openfold/setup.py", line 19, in
import torch
File "/home/dina/miniconda3/envs/openfold_env/lib/python3.9/site-packages/torch/init.py", line 202, in
from torch._C import * # noqa: F403
ImportError: /home/dina/miniconda3/envs/openfold_env/lib/python3.9/site-packages/torch/lib/libtorch_cpu.so: undefined symbol: iJIT_NotifyEvent
Download CUTLASS, required for Deepspeed Evoformer attention kernel
fatal: destination path 'cutlass' already exists and is not an empty directory.
To make your changes take effect please reactivate your environment
To make your changes take effect please reactivate your environment
(openfold_env) dina@:/Documents/PPI_WDLLM/openfold$ conda deactivate
(openfold_env) dina@:/Documents/PPI_WDLLM/openfold$ cd ..
(openfold_env) dina@:/Documents/PPI_WDLLM$ conda activate openfold_env
(openfold_env) dina@:/Documents/PPI_WDLLM$ python esm_test2.py
Traceback (most recent call last):
File "/home/dina/Documents/PPI_WDLLM/esm_test2.py", line 1, in
import torch
File "/home/dina/miniconda3/envs/openfold_env/lib/python3.9/site-packages/torch/init.py", line 202, in
from torch._C import * # noqa: F403
ImportError: /home/dina/miniconda3/envs/openfold_env/lib/python3.9/site-packages/torch/lib/libtorch_cpu.so: undefined symbol: iJIT_NotifyEvent`

[luiswyss]

I am also facing this issue.
It relates to the following open issue in the PyTorch repo: pytorch/pytorch#123097.
I have not yet been able to find a solution.

[luiswyss]

This can be solved analogously to facebookresearch/esm#677. By using cudatoolkit-dev instead of cudatoolkit and by downgrading mkl to mkl=2024.0.0.

[shams-mehdi]

This can be solved analogously to facebookresearch/esm#677. By using cudatoolkit-dev instead of cudatoolkit and by downgrading mkl to mkl=2024.0.0.

This solved the issue for me. Thanks!

[sachinkadyan7]

Specifying the mkl version fixes the issue, and a code change for that has been pushed in the mentioned commit. Hence closing.