Save REST Response in OCI Object Storage

Today I am going to explain how to call an external API from OCI Functions and save the response in an Object Storage bucket. Once the function is created you can use in many places, I will use OCI Data Integration Rest Task to to call the OCI function every day. This has so many use cases.

Let’s first create a function, I’ve used Python for the function implementation. This is demonstration code, but please try, let me know what can be better.

The function can be created in similar manner as other OCI Functions I will skip that topic here, if you have never used I’d suggest going to the Function QuickStart guides below.

Functions QuickStart Guides

The function I’ve created uses the python requests library to invoke the REST API, it supports no auth and OCI authentication (the Function auth signer is passed on for the REST invocation when auth is OCI in the payload).

Inputs for the payload

1. auth — only value supported is OCI, for no auth omit property
2. url- the URL to invoke
3. headers- a dict of key/value header values ie. “Accept”:”application/json”
4. target_bucket- name for the bucket where the results are written
5. target_objectname — name for the object where the results will be written
6. method- GET/POST/PUT — if omitted, GET will be used
7. body- the JSON input payload body

You will need to give the OCI Function resource principal permissions to use whatever the OCI resources you wish to use.

This example lists the objects in a bucket for example.

echo '{"auth":"OCI","url":"https://objectstorage.us-ashburn-1.oraclecloud.com/n/yournamespace/b/a_cmd_bucket/o?limit=1000&delimiter=%2F&fields=name%2Csize%2CtimeModified%2CarchivalState%2CstorageTier", "headers":{"Accept":"application/json","Content-Type":"application/json"},"target_bucket":"yourbucket", "target_objectname":"yourobjectname"}' | fn invoke yourappname direst2os

The result will be stored in the bucket you specify and the object name.

The above is a simple example using a default GET invocation, below you can see a more complex one that is using POST, it calls the OCI Monitoring API and gets CpuUtilization for OCI Autonomous Databases. The result of this is stored into OCI Object Storage.

echo '{
"auth":"OCI",
"method":"POST",
"url":"https://telemetry.us-ashburn-1.oraclecloud.com/20180401/metrics/actions/summarizeMetricsData?compartmentId=YOUR_COMPARTMENT",
"headers":{"Accept":"application/json","Content-Type":"application/json"},
"body":{
 "namespace": "oci_autonomous_database", 
 "query": "CpuUtilization[1m]{deploymentType = 'Shared'}.mean()",
 "endTime": "2022-12-16T16:19:00.000Z",  
 "startTime": "2022-12-01T15:18:00.000Z"}, 
 "target_bucket":"YOUR_BUCKET",
 "target_objectname":"YOUR_TARGET_OBJECT"}' | fn invoke distools direst2os

OCI Functions give an endpoint that you can invoke, this comes in handy when you want to invoke it via REST. For example in OCI Data Integration you can create a REST task and capture this invocation and also schedule it. Below I have created a REST task for invoking a sample REST API (https://jsonplaceholder.typicode.com/users);

Create a REST task

You will need to allow the workspace resource principal to execute OCI Functions, for this to work. The request body is defined below, I define the URL to invoke and the target bucket and object name;

Configure the payload and HTTP method and URL

I can then easily schedule this to run on any cadence I want;

Create a task schedule

What else can be done? You can parameterize the payload body and have the URL, bucket and objects name as parameters also!

Parameterize entries in the payload

Then when you run it you can call different APIs with the same task, for example below I am using the ‘todos’ APIs rather than the ‘users’ API.

Define runtime parameter values

There you go, pretty straightforward. Once this is executing if you view the bucket you will see the file updated date and content refreshed. Hope you thought this was interesting and useful. Please share you thoughts, good or bad :)