Developing Real-Time Data Pipelines for a Financial Analytics Tool

Enhanza is a real-time financial analytics platform for small and medium enterprises in Sweden. For this project, we developed a solution that transparently and securely scales up to 1000 organizations, offering comprehensive business analysis, digital transformation, and maintenance. Read on to learn how we achieved this result.

Part I. Overview of a Project

Client and Business Goals

Back in 2020, the entrepreneur Jon Shubeli set up a business aimed at helping small and medium businesses in Sweden streamline their financial analytics and reporting. Once established as a small startup servicing a few clients, it was expected to grow soon. After a few successful cases, it was time to scale real-time data pipelines and automate this solution, and then Broscorp came into the game.

Product Description

Enhanza is a tool that makes financial and ERP data available for analytics cleanly and understandably. All in all, it helps speed up the decision-making process and cut profit losses. Enhanza aimed to simplify the BI process so anyone could gain insights from their critical business data.

Challenges

After Jon Shubeli contacted us, we determined the project scope, timeline, and the most vital functional objectives. In this project, we needed to create real-time data pipeline in finance and deal with such challenges as:

• Handling large volumes of data efficiently, while ensuring data consistency and minimizing latency.
• The issue of limited built-in analytics. This necessitated the development of custom solutions to extract, transform, and analyze data from multiple endpoints.
• Ensuring scalability to accommodate the expansion of clients without incurring additional costs.
• Developing serverless applications and linking systems to minimize manual steps and human errors.
• Customization for clients which required integrating with third-party systems and creating custom dashboards tailored to each client’s specific requirements.

Part II. Technology Solution

Our developers were tasked with improving synchronization, enabling digital transformation, providing maintenance services, and conducting comprehensive business analysis to meet the specific needs of Enhanza’s clients.

Technology Stack for Real-Time Data Pipelines

BigQuery, Python, Prefect, Google Cloud Platform, Firebase, Manual testing, and Agile management framework

Process and Implementation

Our goal was to design a turbocharged synchronization pipeline for real-time data updates from various 3rd party APIs, comprehensive business analysis, digital transformation, and maintenance that will securely scale up to 1000 companies.

Scalability & Security

Our team chose to implement a share-nothing multitenancy solution on GCP to be able to scale seamlessly without any potential security issues for our customers. We granted customers full control over their data, including refresh & delete options. Share-nothing tenants allow for flattening of infrastructure costs when the number of active users growth.

Data extraction & Loading Pipelines

To support an ever-growing number of users we needed a reliable way of status of the system monitoring. Among all existing scheduling & monitoring solutions we choose Prefect for simplicity, reliability, and easiness of development.

Hence, the Broscorp team provided a comprehensive solution that included the following features:

Improved synchronization pipeline to support near-real-time updates from Fortnox API for hundreds of clients. Fortnox is a popular Scandinavian tool designed for finance management. It solves a lot of business problems, but it lacks good built-in analytics. However, the exposed API is rich enough for building third-party integrations. Broscorp built a set of serverless functions using GCP to collect information from multiple endpoints and store data in Google Big Query in a way to make sure it can later be efficiently retrieved to build analytics on top. We overcame the limitations, while solutions are fully scalable thereby helping Enhanza expand without any additional costs to onboard the following clients.

Digital transformation and maintenance. One of the biggest challenges was to transform Enhanza digitally. Once set up as a small startup serving a few clients, it was expected to grow soon. For this reason, all of the processes such as registration, onboarding, and connection to the data sources had to be automated to minimize interaction and avoid human errors. Broscorp analyzed every single bit, which can be automated, and prepared a set of serverless applications to link systems between each other and avoid manual steps. Moreover, Broscorp provided services to maintain the whole set of tools with a comprehensive monitoring & alert system to help react to any unusual or unexpected behavior, decreasing the outage time dramatically.

Comprehensive business analysis to meet client requirements. What is more difficult is that each client often requests customized solutions despite having standard analytics and reporting. Solutions like that involved integrating with yet another third-party system and creating custom dashboards. Every single client required tailor-made business analysis and a set of features to integrate with their systems. Broscorp organized a workflow to streamline the collection of such requirements and create custom solutions.

Unlock the Essential Steps to Launching a Successful Fintech Startup in 2025. Read the Complete Guide Today!

Part III. Value Delivered

The Broscorp team conducted in-depth research of existing financial analytics systems to identify and apply the best development practices. As a result, we created an easily scalable platform that is straightforward for both small and large businesses.

Since Broscorp stepped in, the Enhanza project reached these milestones:

• Serving more than 50GB of analytics data per day;
• Reducing sync time from several hours to 1-2 minutes;
• More than 300 connected companies;
• Increased number of integrated ERP systems from 1 to 5.

Testimonials

Need to Create a User-Friendly Data-Driven Application?

At Broscorp, we develop tailored business intelligence solutions to help businesses make better strategic decisions, identify problems, increase revenue, and open up new possibilities. If you are looking for deeper insights beyond surface-level data for your financial business, you have come to the right place. We look forward to applying our best practices to your project and collaborating with you to create tailored solutions.

Want to know more about building great digital products? Please contact us through the form below.