Is there a recommended way to use Jar files such as JDBC Jar files for SQL?

[bngsudheer]

Type of question

Best Practices

Support question

Dear NiFi Kops community,

I am seeking guidance on best practices for efficiently managing shared data in the NiFi Kops environment. Currently, I am considering several approaches and would greatly appreciate your insights.

The proposed approach involves uploading the required JAR file directly onto the pod. However, I believe there might be more robust strategies to handle shared data effectively.

After some research, I am contemplating the following best practices:

Implementing a Shared Storage Solution:

• Creating a shared storage solution using Persistent Volumes (PV) and Persistent Volume Claims (PVC) via Container Storage Interface (CSI) is a recommended approach. By adopting this method, we can ensure data persistence and enable seamless sharing of data among pods.
• Data Loading Mechanism:
  To populate the shared volume with relevant data, I intend to utilize an external script that automates the data loading process.
• Volume Mounting for Pods:
  By effectively mounting the shared volume to all relevant pods, we can ensure consistent access to the shared data across the NiFi Kops deployment. Properly configuring the mounting process is essential to achieve optimal performance and data integrity.

Exploring Cloud Blob Storage Integration:

As part of our investigation into optimal shared data practices, we are also exploring the possibility of leveraging cloud-based blob storage services such as Amazon S3 or Azure Blob Storage. This approach might provide additional benefits in terms of scalability, data redundancy, and cost-effectiveness.

In conclusion, I kindly request the experienced members of the NiFi Kops community to share their valuable insights and recommendations regarding the above-mentioned strategies.

Thank you for your time and consideration.

NiFiKop version

No response

Golang version

No response

Kubernetes version

No response

NiFi version

No response

[OneCricketeer]

Unless you're frequently swapping or upgrading databases, I'd recommend seting up a private docker registry and building your own images with all necessary drivers added to the classpath.

This would then work in any container orchestrator, rather than limit yourself to k8s features