Lifecycle Management with Applications in OCI Data Integration

Keeping your data integration applications in order during their lifecycle is straightforward with OCI Data Integration, generally development starts and projects get created containing your integration tasks, these are published into applications (the publish step is done by creating a patch containing tasks to be published and the result is published objects) and development tests run (see this blog on that process). For more formal testing a separate test application can be created based on your development application — this can be in the same workspace or separate workspace as development. In this application you can map the data assets to your test data assets, or use parameters in parameterized tasks to accomplish this (this is the best practice for doing this) or reuse the development ones if its in the same workspace. Any changes in the development application can be kept in sync, this can then be copied to production by again creating a production application and then mapping the data assets, synchronizing content and so on (rinse and repeat). The image below illustrates that flow.

Flow diagram for application lifecycle

Let’s look at this in action. Below we see our development project we will use in this illustration, we can select the tasks to publish and click on “Publish to Application”.

Publish tasks to your development application

A dialog will appear ask for the target application to publish the objects to.

Select the application to publish tasks to

When you click on publish, you can view the published tasks in the application, the patches tab will let you monitor progress if there are lots of content being published.

Here we see the published tasks

At this point you can test your work, execute the tasks, ensure they are functioning correctly and so on. When you are happy with the state of this, create an application for QA/testing. This is done by creating an application in the workspace you are using for your testing — this could be the same workspace as your development or in a different workspace entirely as illustrated in earlier image. The create application dialog has an option “Copy resources from an existing application” it is this option you can use to copy from dev to test, test to production and so on. Select the source compartment/workspace and application.

Copy resources from an existing application

You can now view the created application for our testing and see the tasks, references and patches. Notice the source application information and tooltip displayed below which was not on our development application — theres also a very important sync link.

Clicking on ‘View Details’ you can see the full info such as the compartment/workspace and application name.

If there was more work that happened in development you can synchronize the content from the development application by clicking “Sync”. This will synchronize the application with the development application, in the dialog you can view updates/additions/deletions and review the changes.

Review the synchronization changes

Let’s look at the patches and we can see that these actions have all been recorded and there is information in the patch that defines exactly what was done to my application. I can drill into the patch to see this with even more detail or if there was some issue.

See patches applied to the application

Earlier I mentioned that you may have different data assets in testing, most definitely in production, the best practice way for handling this would be to use parameterized tasks then you can just execute and pass the data asset you are using — option 1 below. You can also remap data assets as in option 2 below.

Option 1 — Use Parameterized Tasks

See Aditya’s blog on using parameterized tasks and passing different data assets for dev versus test versus production;

https://blogs.oracle.com/dataintegration/using-parameters-in-oracle-cloud-infrastructure-oci-data-integration

Option 2 — Remap Data Assets

It also possible to remap what was configured in the source application with what you want to configure in this one to test/run with is to view the ‘References’ tab. Here you can see all of the data assets referenced in the application AND change the mapping of what there were to your test or production environment.

View the references

In above ‘References’ table, I have clicked on ‘Map‘ in the mapped to column for my data asset, I will map it to a different data asset for my testing work. You should ensure this represents the same data asset but be a test or production variant.

Map your data asset for test/production

Below you can see that this is now mapped and that the connection can now also be mapped…

Data Asset is mapped

Summary

Now the connection can be mapped and then we are complete. We are ready to test! When all is good with testing, we can go through a similar cycle in creating an application for production, ensuring its synchronized with any changes and mapping/using the data assets and connections for that environment.

The same copy/sync functionality for applications works within a workspace also, so it’s possible to carry out the development to test etc within it and also have different data assets to cleanly separate your development and tests like this;

Hope you found this useful its a very quick flyover of creating applications and synchronizing content from development through testing and production.

Check out all the blogs related to Oracle Cloud Infrastructure Data Integration — https://blogs.oracle.com/dataintegration/oracle-cloud-infrastructure-data-integration. To learn more, check out the Oracle Cloud Infrastructure Data Integration Tutorials and Documentation.

Architect at @Oracle developing cloud services for data. Connect on Twitter @i_m_dave