You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The fsspec.open command allows urlpath to be a list. But a few lines later, urlpath is passed as a list. If it is already a list, it becomes a list of lists. I ran into problems with this:
File /work/bm1344/conda-envs/py_312/lib/python3.12/site-packages/fsspec/core.py:487, in open(urlpath, mode, compression, encoding, errors, protocol, newline, expand, **kwargs)
429 """Given a path or paths, return one ``OpenFile`` object.
430
431 Parameters
(...)
484 https://filesystem-spec.readthedocs.io/en/latest/api.html#other-known-implementations
485 """
486 expand = DEFAULT_EXPAND if expand is None else expand
--> 487 out = open_files(
488 urlpath=[urlpath],
489 mode=mode,
490 compression=compression,
491 encoding=encoding,
492 errors=errors,
493 protocol=protocol,
494 newline=newline,
495 expand=expand,
496 **kwargs,
497 )
498 if not out:
499 raise FileNotFoundError(urlpath)
File /work/bm1344/conda-envs/py_312/lib/python3.12/site-packages/fsspec/core.py:295, in open_files(urlpath, mode, compression, encoding, errors, name_function, num, protocol, newline, auto_mkdir, expand, **kwargs)
216 def open_files(
217 urlpath,
218 mode="rb",
(...)
228 **kwargs,
229 ):
230 """Given a path or paths, return a list of ``OpenFile`` objects.
231
232 For writing, a str path must contain the "*" character, which will be filled
(...)
293 https://filesystem-spec.readthedocs.io/en/latest/api.html#other-known-implementations
294 """
--> 295 fs, fs_token, paths = get_fs_token_paths(
296 urlpath,
297 mode,
298 num=num,
299 name_function=name_function,
300 storage_options=kwargs,
301 protocol=protocol,
302 expand=expand,
303 )
304 if fs.protocol == "file":
305 fs.auto_mkdir = auto_mkdir
File /work/bm1344/conda-envs/py_312/lib/python3.12/site-packages/fsspec/core.py:651, in get_fs_token_paths(urlpath, mode, num, name_function, storage_options, protocol, expand)
649 if protocol:
650 storage_options["protocol"] = protocol
--> 651 chain = _un_chain(urlpath0, storage_options or {})
652 inkwargs = {}
653 # Reverse iterate the chain, creating a nested target_* structure
File /work/bm1344/conda-envs/py_312/lib/python3.12/site-packages/fsspec/core.py:343, in _un_chain(path, kwargs)
341 kwargs = kwargs.copy()
342 for bit in reversed(bits):
--> 343 protocol = kwargs.pop("protocol", None) or split_protocol(bit)[0] or "file"
344 cls = get_filesystem_class(protocol)
345 extra_kwargs = cls._get_kwargs_from_urls(bit)
File /work/bm1344/conda-envs/py_312/lib/python3.12/site-packages/fsspec/core.py:552, in split_protocol(urlpath)
549 if len(protocol) > 1:
550 # excludes Windows paths
551 return protocol, path
--> 552 if urlpath.startswith("data:"):
553 return urlpath.split(":", 1)
554 return None, urlpath
AttributeError: 'list' object has no attribute 'startswith'
The text was updated successfully, but these errors were encountered:
wachsylon
changed the title
Why is urlpath passed as a list to open_files if it can be already a list?
Why is urlpath passed as a list to open_files if it can be already a list?
Jan 16, 2025
The
fsspec.open
command allowsurlpath
to be a list. But a few lines later, urlpath is passed as a list. If it is already a list, it becomes a list of lists. I ran into problems with this:The text was updated successfully, but these errors were encountered: