> ## Content Index
> Fetch the complete content index at: https://research.finbox.in/llms.txt
> Use this file to discover other available public pages before exploring further.

# Infinite Loop #32: How cascaded credit architecture makes boring models win
- URL: https://research.finbox.in/newsletter/infinite-loop/infinite-loop-how-cascaded-credit-architecture-makes-boring-models-win/
- Published: 2026-09-02T11:25:04.000Z
- Updated: 2026-09-02T11:25:35.000Z
- Description: Discover how a tiered, cascaded AI architecture delivers instant decisions while reducing compute costs.
- Author: Srijan Nagar
- Tags: #InfiniteLoop, #fintech, #AI, Credit Assessment, #Sentinel, #digitallending, srijan nagar, infinite loop

There is an understandable excitement in FinTech right now to see how far generative AI can be pushed across the credit lifecycle. The idea of letting a frontier model automate every aspect of credit decisioning sounds appealing in a product demo. But when you move from prototype to high-volume production, routing every decision through a large language model hits two operational realities: margins and latency. 

- The first issue is unit economics. Small-ticket credit is built on volume rather than wide margins. Take a ₹1,000 credit line on UPI. At standard fees (around 1%), the credit issuer makes ₹10 to ₹12 in revenue on that transaction. But sending a single, detailed prompt to a top-tier AI model can easily cost more than ₹12, eating up that entire margin.  
    
A 2025 study from a US health system illustrated this well: replacing a lean, existing ML pipeline with GPT-4-class models across four routine document workflows drove estimated annual costs from thousands to between [$115,000 and $4.6 million](https://www.nature.com/articles/s41746-025-01971-x?ref=research.finbox.in), for tasks simpler models were already handling effectively.
- The second challenge is speed. Frontier models generally take anywhere from two to ten seconds to process a detailed prompt and return a response. In a background batch process, a few seconds is negligible. At a mobile checkout, it can be fatal to conversion.  
    
Speed is make-or-break at checkout. While a standard rules engine approves an applicant in milliseconds, a heavy AI model can take several seconds to respond. When most users abandon a screen after [just three seconds of waiting](https://www.thinkwithgoogle.com/%5Fqs/documents/2340/bc22e%5FThe%5FNeed%5Ffor%5FMobile%5FSpeed%5F-%5FFINAL%5F1.pdf?ref=research.finbox.in), that brief delay means losing customers before they even see their offer.

  
These cost and latency aspects point to a clear design flaw: using heavy AI where simple logic would suffice. *The fix is a layered funnel that routes easy checks through* *simple* *filters and reserves deep compute only for complex cases.* 

## How a cascaded architecture works 

This simply means applying the same triage principles credit teams have used for decades, automated through a layered architecture. 

Take, for example, an application for a ₹15,000 personal loan:   
The file first hits an instant rules layer. Running in milliseconds at negligible cost, this checks basic bureau hygiene, KYC verification, and policy adherence. Most applicants are either eligible or outside policy, meaning the majority of the funnel is resolved right here. 

Applicants who pass these baseline checks and require nuanced risk scoring move to a fast ML layer. This is a lean, purpose-built model trained on structured bureau data, transaction history, and Account Aggregator feeds. This approach has powered emerging market across Kenya, Nigeria, and the Philippines, returning accurate risk scores on hundreds of data points in seconds without relying on an LLM. 

Only the complex files, such as non-standard income documents or conflicting identity details, are escalated to a heavy AI layer. Here, the multimodal reasoning capabilities of a large model justify their cost and processing time. Because a small fraction of your volume reaches this stage, your overall cost per decision stays low while handling edge cases smoothly. 

## Other industries already paid for this lesson 

Other industries have also run into the limits of using general-purpose AI for specialised workflows: 

- In healthcare, a study found that the smaller, specialised model was noticeably more accurate at classifying patient notes. Related research on automated medical billing showed general-purpose LLMs agreeing with human coders as little as [10% to 25%](https://www.nature.com/articles/s41746-025-01971-x?ref=research.finbox.in) of the time, accompanied by high hallucination rates on technical codes.
- In consumer tech, Klarna’s experience with customer support followed a similar arc. After [aggressively shifting support volume](https://www.customerexperiencedive.com/news/klarna-reinvests-human-talent-customer-service-AI-chatbot/747586/?ref=research.finbox.in) to an OpenAI-powered assistant in 2024, leadership acknowledged the following year that an overemphasis on pure automation had hurt service quality on complex issues, prompting a re-investment in [human-in-the-loop](https://research.finbox.in/newsletter/infinite-loop/human-oversight-for-agentic-systems-isnt-one-size-fits-all-2/) workflows.

## Lenders serving the toughest markets already run it this way 

Lenders operating with limited data rely on simple, tiered scoring systems rather than all-in-one AI. A [2026 World Bank (IFC) study](https://www.ifc.org/content/dam/ifc/doc/2026/cracking-the-credit-code-alternative-data-and-ai-for-financial-inclusion.pdf?ref=research.finbox.in) of 448 alternative-credit firms confirms this strategy works at scale. 

Instead of using heavy AI for everything, most firms use a stepped approach: basic rules for eligibility, hybrid models for standard reviews, and full automation only when data confidence is high. 

For example, Zambian fintech Eshandi auto-approves micro-loans only when their mobile money score hits a strict 90% confidence threshold, a method they’ve used to disburse nearly one million loans to women. Across all 448 firms, alternative data (like utility bills and transaction history) does the core work, while AI is used selectively for specific tasks like transaction classification and fraud detection.   
  
These operations show that a disciplined triage system — cheap, fast models handling the core volume, with heavy AI reserved for unstructured exceptions — is what holds up under scale. 

## Modern credit stacks are built in layers 

Researchers published a paper formalising this concept under the name [FrugalGPT](https://arxiv.org/abs/2305.05176?ref=research.finbox.in). Their research demonstrated that cascading queries through smaller models first and escalating only when confidence scores fall below a threshold can match the accuracy of frontier models while reducing operational costs by up to 98%. 

In risk operations, it is simply good funnel management. Handle routine decisions instantly and inexpensively and save deep reasoning, whether machine or human, for the files that warrant it. 

Just as importantly, simpler AI layers don’t require a massive "rip and replace" of your existing loan origination stack. They act as targeted overlays, plugging into legacy systems to handle specific workflows. 

As McKinsey has noted, [this kind of composability](https://www.mckinsey.com/capabilities/quantumblack/our-insights/seizing-the-agentic-ai-advantage?ref=research.finbox.in#/) is a deliberate design principle for enterprise AI: agents, tools, or models should be able to "plug into" an architecture [without requiring a system rework](https://research.finbox.in/newsletter/infinite-loop/delaying-ai-adoption-for-legacy-readiness-is-a-losing-bet/).   
  
Ultimately, this is about efficiency and control. By routing each decision to the right model at the right cost, you build an operation that is not only more affordable but far more resilient to the complexities of modern lending. 

Until next time,  
  
Srijan  
Co-founder  
FinBox