Loading SOAP results into OCI Object Storage?
Let’s see how we can make a SOAP call and upload the result into OCI Object Storage. This leverages the python zeep library which gives an easy to use interface to SOAP based services. There are tonnes of SOAP based services, the example below lists country info, Oracle UCM also has a SOAP based client so similar pattern could be used to get data from UCM and pipe it into Oracle Cloud (see this post here which leverages the Oracle OCM Web Services). You’ll see plenty of people doing things like this in other platforms — unloading / backing up Postgres using pgdump and piping it into AWS S3, there’s plenty more.
The example below invokes the ListOfCountryNamesByName SOAP api, then prints it to standard output. You will need to pip install zeep to use this.
One of the cool parts of the OCI CLI is there are tonnes of really cool features. The one I love is the OCI Object Storage put command. With this command you can take data from the standard input stream (or local files) for example and create objects in OCI!
python3 script.py | oci os object put --bucket-name mybucket --file - --name data_from_soap.json
Gets better. The OCI CLI Object Storage put command also supports parallel upload of parts of the file with the parallel_upload_count option;
--parallel-upload-count INTEGER RANGE
The number of parallel operations to
perform. Decreasing this value will make the
process less resource intensive but it may
take longer. Increasing this value may
decrease the time taken, but the process
will consume more system resources and
network bandwidth. The maximum is 1000.
You can also configure the part size depending on the data you are uploading you may wish to do this;
--part-size INTEGER Part size (in MiB) to use when the file is
split into multiple parts and then combined.
Part size must be greater than 10 MiB and
defaults to 128 MiB.
There is a verify checksum capability but this is only available for when local file is used, not standard input stream.
If you want to control the overwrite behavior there are properties for this also.
--force If the object name already exists, overwrite
the existing object without a confirmation
--no-overwrite If the object name already exists, do not
overwrite the existing object.
Simple stuff to do seemingly complicated things. Next up in a post I will cover the OCI CLI Docker. Another piece in the jigsaw. See how you can run above in a serverless manner. Stay tuned!