Invalid URLs produced when version contains a ! character

[maresb]

I received this issue report in conda-lock: conda/conda-lock#764

Tracking it down, it turns out that the x264 package has a ! character in its version. (I didn't know this was possible!)

There was a change introduced in v2 with the way URLs are escaped.

v1:

micromamba create --dry-run --json --channel conda-forge --prefix=/tmp/x x264 | grep x264
...
                "url": "https://conda.anaconda.org/conda-forge/linux-64/x264-1!164.3095-h166bdaf_2.tar.bz2",
...

v2:

micromamba create --dry-run --json --channel conda-forge --prefix=/tmp/x x264 | grep x264
...
                "url": "https://conda.anaconda.org/conda-forge/linux-64/x264-1%21164.3095-h166bdaf_2.tar.bz2",
...

The problem with this is that conda balks if I create an explicit file with this URL:

@EXPLICIT
https://conda.anaconda.org/conda-forge/linux-64/x264-1%21164.3095-h166bdaf_2.tar.bz2#6c99772d483f566d59e25037fea2c4b1

conda create --name blah --file explicit.txt

InvalidMatchSpec: Invalid spec 'https://conda.anaconda.org/conda-forge/linux-64/x264-1%21164.3095-h166bdaf_2.tar.bz2 md5=6c99772d483f566d59e25037fea2c4b1': Invalid version '1%21164.3095': invalid character(s)

One could argue that this is a bug in conda for not unescaping the URL before parsing the version, but it feels to me like it would be best to fix this here if possible to maintain consistency with previous behavior.

[jaimergp]

Oops, I'll see if we can fix this in conda in the meantime.

[jaimergp]

Opened conda/conda#14481.

[jjerphan]

This is a change of behavior of libmamba, but this is not a bug. I consider this fixed downstream with conda/conda#14481; do you agree?

[maresb]

Ya, that's certainly fair @jjerphan. Hopefully Conda can cut a release soon.

BTW, any news on the next mamba release? I'm waiting for #3696 to unblock the conda-lock release.

[jjerphan]

It is on its way with first: conda-forge/mamba-feedstock#285.

[wolfv]

Hmm I am honestly not sure if the URL should be stored as URL encoded even though it's a string in JSON. Not sure if that is best practice.

The encoding should happen at the interface where the URL is used for a HTTP request.

[jjerphan]

I would accept a fix for backward-compatibility.

[maresb]

The original PR that introduced the encoding changes: #3076

Related draft fix for micromamba env export: #3745

[maresb]

Hmm I am honestly not sure if the URL should be stored as URL encoded even though it's a string in JSON. Not sure if that is best practice.

The encoding should happen at the interface where the URL is used for a HTTP request.

Ya, I'm concerned now that this issue could potentially go deeper.

Look at my conda-meta/x264-1!164.3095-h166bdaf_2.json:

{
    ...
    "size": 897548,
    "subdir": "linux-64",
    "timestamp": 1660323080555,
    "track_features": "",
    "url": "https://conda.anaconda.org/conda-forge/linux-64/x264-1%21164.3095-h166bdaf_2.tar.bz2",
    "version": "1!164.3095"
}

Even more ominous though is the presence of pkgs/x264-1%21164.3095-h166bdaf alongside pkgs/x264-1!164.3095-h166bdaf, created at the same time as this PR. (I'm not sure if this was created by conda or micromamba though.)

Unfortunately I'm out of time, so I can't do any further investigations at the moment.