NB: This note is part of a series of my personal notes on API Security. To understand what API Security is, please read my Introduction Note on the subject
Understanding Throttling and Rate Limiting
Rate limiting is a method for restricting how many requests or actions a user or system can make in a given amount of time. It helps in preventing misuse, excessive resource utilization, and ensuring fair usage of services or APIs. Rate limitation contributes to the stability and security of the system by placing limits on the amount of requests.
Rate limitation and throttling are similar, but throttling concentrates on regulating the rate of data transfer or processing. In networking and computing contexts, it is frequently employed. Throttling limits the speed at which data is transferred or processed, minimizing congestion and making sure the system runs at its maximum efficiency. This method is frequently used to prevent overburdening servers, networks, or other hardware, which could result in failures or performance degradation.
My Experience
Sometime ago, I witnessed some exploitations in of the FinTech projects i was working on that just rolled out for public Beta, it was a mobile app, the exploiters captured the Requests from the app to the server using tools like "HTTP Toolkit ↗️" while they register and perform some initial transactions.
They found one service endpoint that performs a financial transaction and launched their attack on that. Initially, the endpoint performs series of monetary operations in the following order:
- verifies authentication
- validates input data
- checks user's financial qualification
- proceeds to perform the business transaction if the user is qualified, which requires an external service interaction.
- upon a successful response from the external service, updates the user's financial qualification (basically deducting some qualification points).
From the steps above, you might have guessed what the flaw in that flow is, right? 😞 well, i should've updated their financial qualification before proceeding to the external service, and if that one later fails, revert the update. This is a major security step taken in Blockchain smart contracts to prevent reentrancy attacks, i knew about that because i've worked in a couple of blockchain projects and have written smart contracts myself, but i never thought it'd be a problem in traditional Web 2.0
The exploiters made multiple requests to the endpoint with same payload, at the exact same time (few milliseconds difference tho).
The result of this was that they were able to pass the financial qualification check at the same time before it was updated after the business transaction with the external service. When the execution has got to the point of deducting qualification points, they had already performed 5 business transactions at the qualification of 1, the deduction code ran and now resulted in them having a minus points balance.
// Deduction the cost of the operation
pointsBalance -= operationCost
/**
* at a balance of 10, with an operation cost of 5,
* if we are to deduct 5 five times, we'll be left with -15
* */ My Solution - An AdonisJs Implementation
To fix this, the first thing i did was to change the order of the operations, updating their financial qualification first before proceeding to the external business logic. This appears to be a perfect solution but it isn't.
I created a script file and added a for loop, making requests to the same endpoints in in the loop, the requests happen almost at the same time too. with a point balance of 5 and an operationCost of 5, the multiple requests would pass that check, and proceeded to the next block of code which updates the balance, at the same time. That was when i knew i needed some kind of rate limiting.
I didn't know AdonisJs already has an inbuilt feature for throttling and rate limiting, so i tried creating a middleware to record time of a user request and the next time they should make another request (10 secs interval)
export default class TimeLockedApiCall {
public async handle({ auth, request, response }: HttpContextContract, next: () => Promise<void>) {
// code for middleware goes here. ABOVE THE NEXT CALL
const user = auth.use('api').user!
console.log(DateTime.now().toString())
if (user.nextApiCallAfter) {
// Update the Next Call Allowed Time to Next 5 Seconds
if (DateTime.now() >= user.nextApiCallAfter) {
await user.merge({ nextApiCallAfter: DateTime.fromMillis(Date.now() + 10000) }).save()
} else {
// Put the User On Pending Fraud Check
console.log('Fraud Check Activated')
return response.status(429).json({
status: 429,
message: 'Fraudulent Activity Detected, User Has Been Put On Pending Fraud Check',
})
}
} else {
await user.merge({ nextApiCallAfter: DateTime.fromMillis(Date.now() + 10000) }).save()
}
await next()
}
}Long story short, the code above didn't work, yeah, shame 😅. the script could still bypass that time check because they're running a few milliseconds difference. I then started looking around, i discovered that AdonisJs has what i'm looking for and it's been right there in their documentation.
export const { httpLimiters } = Limiter.define('global', ({ auth, request }) => {
const user = auth.use('api').user!
return Limiter.allowRequests(1)
.every('10 sec')
.usingKey(String(user.id))
.limitExceeded((error) => {
error.message = 'Fraudulent Activity Detected, User Has Been Put On Pending Fraud Check'
error.status = 429
// Put the User On Pending Fraud Check
console.log('Fraud Check Activated')
// A key-value pair of headers to set on the response
// console.log(error.headers)
})
})The Adonis limiter comes with a throttle middleware when configured, which then follows what is defined above.
