Exporting and Importing in OCI Data Integration

The objects in OCI Data Integration can be exported and imported between workspaces, that goes for design objects in projects or published tasks in applications. The export and import is performed on the project or application or items within (see the documentation here).

The export and import action can be performed on the listing page for projects for example. The action button for ‘Create project’ includes both create new project and import from an exported one. Each project can be exported by an action on the project.

When you export you specify where the data will be written; the tenancy ocid, the bucket name and the file name for the zip. The zip can include references (to data assets for example) and you can overwrite.

When you import you specify where the zip will be read; the tenancy ocid, the bucket name and the file name for the zip. You can specify the mode for the import (see documentation here);

• retain — Retain the content that exists in the target.
• replace — Replace content that exists in the target with content from the source.
• duplicate — Duplicate the content by adding a prefix, suffix, or both to the names coming from the source to the target. You can specify a suffix or prefix.

Exporting

To use the OCI CLI to export a project or an application (or an object within) to OCI Object Storage use the “oci data-integration export-request create” command;

oci data-integration export-request create --workspace-id ocid1.disworkspace.oc1.tbd --bucket-name yourbucket --object-storage-region us-ashburn-1 --object-storage-tenancy-id ocid1.tenancy.oc1..tbd --file-name ExportDIObjects.project.zip --are-references-included True --is-object-overwrite-enabled True --object-keys '["your_object_key"]'

When the export request is created the response has a key property. Use this key property to determine whether the export request is complete. The returned status field will be in SUCCESSFUL state if complete successfully. If it fails, the status will be FAILED and the field errorMessages will have the details (such as “User does not have write access to bucket ‘exports’ or the provided folder structure does not exists”).

oci data-integration export-request get --workspace-id ocid1.disworkspace.oc1.iad.tbd --export-request-key your_export_request_key

Exporting an application will export all of the tasks, schedules and task schedules.

Importing

To use the OCI CLI to import a project or an application (or an object within) to OCI Object Storage use the “oci data-integration import-request create” command, below you can see how to use RETAIN mode;

oci data-integration import-request create --workspace-id ocid1.disworkspace.oc1.iad.tbd --bucket-name yourbucket --object-storage-region us-ashburn-1 --object-storage-tenancy-id ocid1.tenancy.oc1..tbd --file-name ExportDIObjects.project.zip --import-conflict-resolution '{"importConflictResolutionType": "RETAIN"}'

Similar to the export command, when the import request is created the response has a key property. Use this key property to determine whether the import request is complete.

oci data-integration import-request get --workspace-id ocid1.disworkspace.oc1.iad.tbd --export-request-key your_import_request_key

The returned status field will be in SUCCESSFUL state if complete successfully. If it fails, the status will be FAILED and the field errorMessages will have the details.

Monitoring

As well as individual monitoring described above, you can monitor all export and import requests, there is a ‘Request type’ property to filter requests by each type.

Again the exports can be queried from the OCI CLI;

oci data-integration export-request-summary-collection list-export-requests --workspace-id ocid1.disworkspace.oc1.iad.tbd --limit 5 --sort-by TIME_CREATED

To query the imports you can use;

oci data-integration import-request-summary-collection list-import-requests --workspace-id ocid1.disworkspace.oc1.iad.tbd --limit 5 --sort-by TIME_CREATED

The exporting and importing of objects can be done at different granularity, you can export/import projects, applications, tasks, dataflows, pipelines, user defined function libraries and so on. When exporting any of these objects if there are references to other objects you can export those also by selecting the include references option.

Summary

A useful blog to read on application promotion is here this shows how application references can be updated. For example if you export and import an application you can update the references with the commands in this blog.

Hopefully some useful information for you here, let me know what you think, would love to hear. Let me now if there are other areas you are interested in.