Blog/ How to handle concurrent Python API calls and process them as and when they are done?

Our client 'Eightfold' is an AI-powered talent intelligence platform powered by the most extensive global talent data set. The platform is grounded in equal opportunity algorithms. It uses deep-learning AI to help employees, candidates, contractors, and citizens understand their career potential while letting companies understand the potential of their workers.


One widespread use case of the Eightfold app platform is to show recommended courses to its customers' employees through the platform's development partner 'Degreed.' As a result of the app platform's integration with Degreed, employees can review and check courses they have taken and are recommended to take as per required skills.


The integration enables the app platform to:


  • Pull out content courses from the development partner.
  • Filter and display courses based on users' skills, goals, and preferences.
  • Allow employees to click on the course link from the app platform and navigate directly to the development partner's course detail page.


The course lists need to be fetched using two API endpoints (i.e., recommended courses and content filter query). The recommended courses should be shown on priority, followed by content courses.


To get the final result for getting the recommended course list, we made 3 API calls. The first API call was to get the list. Then, the second API call was made for getting skills per course from the first API call list. Finally, the final API call was made for fetching details of filtered courses only. However, for fetching both lists, the app took over 20 seconds. That kind of processing time is way too long. Ideally, the app platform should process a batch of content concurrently to minimize the overall processing time. 


The Challenge: Processing a batch of content concurrently

When you need to make concurrent API calls for content, you wouldn't want to wait for each call to finish. Therefore, the ideal methodology is to process a batch of content concurrently. For example, details of 30 courses are called in one go instead of individually so that the overall processing time is substantially reduced.


The Solution: Concurrent future ThreadPool Python library

We can use concurrent future ThreadPool Python library to run both the API calls (recommended courses and regular content courses) paralleling to reduce the process timing. Using this method, we can decrease the total time to about 12 seconds— that's a 40% improvement.


However, the app takes longer than expected, even after adding concurrent futures. As a resolution, debugging the code further can help find the root cause of the problem. 


During debugging, we checked which functions were taking longer to perform. We found out an API call was being made in a loop to fetch skills per course. To reduce the execution time for the loop, we added the ThreadPool method. As a result, the app execution time decreased to 6-7 secs. 


Sample code of the method:

with concurrent.futures.ThreadPoolExecutor(max_workers=recommended_num) as executor:
   futures = [executor.submit(matching_schema, course) for course in filtered_matched_course_list]
for future in concurrent.futures.as_completed(futures):
   course = future.result()


Flow Chart:




1. Get content recommendations


The app platform will fetch content from Degreed via API. The recommended courses will be filtered and displayed based on the employee's skills goals, job, project or career planner required skills.


Get content recommendations illustration


Recommended course flow chart:


The below recommended course flow chart shows the flow of the function. These actions are performed to get the recommended course list.

Recommended course flow chart

Content filterQuery flow chart:


The below flow chart shows the flow of the function to fetch the list based on filter query.


Content filterQuery flow chart


2. Get course details


The app platform will fetch detailed information about a course.


Get course details


Course details API flow chart:


The below flow chart shows the flow of the function to get detailed information about a course.


Course details API flow chart


3. Get completed courses details


The app platform will fetch a list of courses completed by employees. The completed course list appears in the employee profile.


Get completed courses details


Get completed courses details flow chart



The ThreadPool method can be used for functions and loops. Yet, it is more useful for loops. As a result, we can substantially reduce the app execution time using the method. Consequently, regardless of increasing the limit of the courses fetched, it takes approximately the same time to fetch all recommended and content courses.

Ready to get started?REQUEST A QUOTE