Integrate with Nutanix Storage
Nutanix Objects in Kubeflow Pipeline
You can use standard s3 boto api to upload and download objects from a Kubeflow Pipeline. See Nutanix Objects Docs for more details on creating object store and buckets.
import boto3 bucket_name="ml-pipeline-storage" object_name="models" object_store_access_key_id="<key_id>" object_store_secret_access_key="<access_key>" host="http://<Nutanix Objects Store Domain Name>" region_name='us-west-1' s3_client = boto3.client( 's3', endpoint_url=host, aws_access_key_id=object_store_access_key_id, aws_secret_access_key=object_store_secret_access_key, region_name=region_name, verify=False) response = s3_client.upload_file(f'./test_upload_data.txt', bucket_name, object_name)
Nutanix Volumes in Kubeflow Pipeline
Nutanix volumes are created with the default storage class configured in the Karbon cluster. See Default Storage Class of Nutanix Karbon for more details about creating storage classes.
vop = dsl.VolumeOp( name="Create a volume to share data between stages on Nutanix Volumes", resource_name="data-volume", size="1Gi", modes=dsl.VOLUME_MODE_RWO)
Nutanix Files in Kubeflow Pipeline
Create a storage class to dynamically provision Nutanix File shares. See Files Storage Class of Nutanix Karbon for more details on creating storage classes for dynamic NFS Share provisioning with Nutanix Files.
Once storage class is setup, you can use
VolumeOp operation to create volume on Nutanix Files.
vop = dsl.VolumeOp( name="Create a volume to share data between stages on Nutanix Files", resource_name="data-volume", size="1Gi", modes=dsl.VOLUME_MODE_RWM, storage_class="files-sc")
Was this page helpful?
Glad to hear it! Please tell us how we can improve.
Sorry to hear that. Please tell us how we can improve.