Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] [connectors] Unable to read files without extension in tar.gz file #8478

Open
3 tasks done
zhdech opened this issue Jan 8, 2025 · 0 comments
Open
3 tasks done
Assignees
Labels

Comments

@zhdech
Copy link
Contributor

zhdech commented Jan 8, 2025

Search before asking

  • I had searched in the issues and found no similar issues.

What happened

When reading tar.gz files through the sftp method, in the file_fFormat type="text" mode, if the files inside do not have an extension, seatunnel will not be able to read the file by default. It is expected that when reading the file, it does not depend on the file extension

通过sftp 方式读取tar.gz 文件时,在file_format_type = "text" 模式下,如果里面的文件没有扩展名时,seatunnel 默认读不到该文件,期望:读文件时不依赖于文件扩展名

SeaTunnel Version

dev

SeaTunnel Config

env {
  parallelism = 1
  job.mode = "BATCH"
}
source {
  SftpFile {
    host = "192.168.1.1"
    port = 22
    user = "root"
    password = "1231223"
    path = "/data/20241213060019.tar.gz"
    archive_compress_codec = "TAR_GZ"
    file_format_type = "text"
    result_table_name = "sftp"
    schema = {
      fields {
        ATTR = "string"
      }
    }
  }
}
sink {
  Console {
    source_table_name = "sftp"
    parallelism = 1
  }
}

Running Command

bin/seatunnel.sh --config /config/v2.batch.config.template

Error Exception

No error reported, unable to read file
没有报错,无法读取文件

Zeta or Flink or Spark Version

No response

Java or Scala Version

1.8

Screenshots

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant