Recursive calls in Oracle Integration Flows (Scenario: Paginated API calls for large Data Sets) by Jang-Vijay Singh

October 8, 2020 Leave a comment

image

A number of use-cases can be implemented cleanly using a recursive approach. This post is not to debate the pros and cons of recursion versus looping but provides a simple approach to achieve this.

For scenarios such as the ones listed below, and possibly more, this approach is quite efficient, concise, maintainable, and most importantly, it is highly scalable. It also leaves a smaller runtime footprint with a smaller execution time per instance than a looping flow instance. This also makes error handling easier as I will describe later.

  • Polling (continuously monitoring an FTP location, a database, or an API output)
  • Paginated API’s (when the target system exposes an API with a paginated* interface such as the eBay findProducts operation)
  • Retryable flows
Paginated Calls

Many software systems store large sets of data. For instance, a vendor might have hundreds of thousands of products and product-prices listed on EBay or an eCommerce store. They might have millions of transactions in their PayPal account.

There might occasionally be legitimate scenarios to fetch all of this data. The software system can provide various interfaces to export such data in bulk. Such a bulk data export interface could be raw data files, access to a database, but also a standard API like a Rest Service.

For reasons of performance and good practice, such Rest API’s (typically a GET operation) would limit the amount of data returned in any one call by using a pagination strategy. An eCommerce store could list hundreds of thousands of products for instance, and it wouldn’t be appropriate to return such a large data set in a single call. A pagination strategy would involve setting parameters like pageNumber and pageSize. Read the complete article here.

PaaS Partner Community

For regular information on Oracle PaaS become a member in the PaaS (Integration & Process) Partner Community please register here.

clip_image003 Blog clip_image005 Twitter clip_image004 LinkedIn image[7][2][2][2] Facebook clip_image002[8][4][2][2][2] Wiki

Technorati Tags: SOA Community,Oracle SOA,Oracle BPM,OPN,Jürgen Kress

Filed under Cloud, SOA Tagged with , , , ,

About Jürgen Kress
As a middleware expert Jürgen works at Oracle EMEA Alliances and Channels, responsible for Oracle’s EMEA Fusion Middleware partner business. He is the founder of the Oracle SOA & BPM and the WebLogic Partner Communities and the global Oracle Partner Advisory Councils. With more than 5000 members from all over the world the Middleware Partner Community is the most successful and active community at Oracle. Jürgen manages the community with monthly newsletters, webcasts and conferences. He hosts his annual Fusion Middleware Partner Community Forums and the Fusion Middleware Summer Camps, where more than 200 partners get product updates, roadmap insights and hands-on trainings. Supplemented by many web 2.0 tools like twitter, discussion forums, online communities, blogs and wikis. For the SOA & Cloud Symposium by Thomas Erl, Jürgen is a member of the steering board. He is also a frequent speaker at conferences like the SOA & BPM Integration Days, JAX, UKOUG, OUGN, or OOP.

Leave a comment