Keepers usually are bots operated to interact in very specific ways with decentralized protocols to help “keep” them in a healthy state. The most famous example are liquidation bots on Maker or Aave, that will liquidate (close) users’ loans that have a health factor too low, a situation which could damage the underlying systems.

More info on liquidations on Aave here.

The need for keepers comes from the fact that blockchains are passive environments, where nothing happens without interactions with the network, when transactions are executed and validated. Therefore, to update the state of a protocol on a blockchain and drive subsequent actions, an external transaction is required. Some of these transactions are necessary for protocols to function as expected but cannot be performed automatically. These transactions are usually executed by keepers, which are most of the time rewarded, directly or indirectly, as they bear a cost. Naturally, some actors have been building bots that are optimized to maximize these rewards.

Note that being a keeper is most of the time totally permissionless, and anyone interacting with the Ethereum network can call these functions and earn those rewards.

Keepers are involved in different places in the Angle Protocol.

Angle lets traders, called Hedging Agents (HA) in the protocol, open leveraged positions (perpetuals) on the assets accepted as collateral. When they do, their exposure to the underlying assets’ price changes is increased, as are their gains and losses. Like with any centralized exchange, it is the protocol’s job to make sure the participants don’t loose more than what they have put in so that the protocol remains solvent.

We remember from the introduction that the protocol is passive: it doesn’t know by itself when one should be liquidated or not. To make sure these risky positions are being closed when they should, the protocol incentivizes external entities, keepers, to call the liquidatePerpetuals(list of IDs) function of the PerpetualManagerFront contract, which checks whether the specified positions should be liquidated, liquidate them if they should, and reward the callers if perpetuals were liquidated.

The maintenance margin (more info here) for USDC/EUR and DAI/EUR positions is at 0.625%. This means that if the margin represents less than 0.625% of the total position, any keeper can call the liquidatePerpetuals(list of IDs)function with the correct perpetual ID, which will liquidate the position and send rewards to the keeper.

The reward for liquidating a given perpetual is set as follows for USDC (the idea is the same for FEI and FRAX):

!https://miro.medium.com/max/1400/1*SH0xqpnVlmHwRloJ4X4yHg.png

Hence a keeper liquidating n perpetuals in a single transaction gets:

!https://miro.medium.com/max/1400/1*6mZe46_5hbPCrunZTDxyrA.pn

The Liquidation Ratio is a parameter set by governance that quantifies how

much of a liquidated perpetual’s remaining cash out amount should go to the keeper. This parameter is currently set to 60% for all the collateral types of the protocol, and it can be found for each stablecoin/collateral pair in the corresponding pool page of the analytics.

Note here that the reward of the keeper depends on the cash out amount of the perpetual (which for a perpetual being liquidated is inferior to the maintenance margin). If the price decreases by too much, the cash out amount may become null meaning that keepers cannot receive incentives for liquidating. This means that keepers are better off liquidating perpetuals as soon as they get below their maintenance margin.

TL;DR

• Keepers call the liquidatePerpetuals([list of perpIDs]) of the PerpetualManagerFront contract → liquidate perpetuals if they need to be and reward the keeper. Keepers take action if they make a profit, e.g if the gas cost for performing such liquidations is inferior to the reward given by the protocol

Another situation in which positions need to be closed occurs when too many positions are opened and the protocol is over hedged. How this happens and why it needs to be controlled in the protocol is explained in this research Similarly than for liquidations, keepers calling the forceClosePerpetuals(list of IDs) function receive a similar type of reward depending on the result of their call. To get those rewards, the call of the keeper should put the protocol below the target hedge ratio (more in the research article).

!https://miro.medium.com/max/1400/1*3Cz6TS6hEV8vEfPfi4OheA.png

Where the Estimated Cost of Attack is the cost of burning enough agTokens to be in a position to call forceClosePerpetuals(list of IDs) again and make a profit. It can be detailed as:

!https://miro.medium.com/max/1400/1*GfgErqgCZOO1lffGPrkqKA.png

As detailed in the research article, keepers could indeed be tempted to burn stablecoins to increase the hedge ratio above the limit hedge ratio and be in a situation where they could profit from the reward of calling force close. Capping the reward by the estimated cost of the manipulation prevents that. KeeperClosingRatio is a linear by part function of the hedge ratio after the force close with respect to the target hedge ratio and determines how much of the closing fees should go to the keeper.

Note that keepers can force close any perpetual that they want, and the optimal behavior for them to maximize their reward is not obvious. Based on the reward function from above, they should try to:

• avoid closing too many because gas costs would be too important

• remain close to the target hedge ratio where the KeeperClosingRatio is more important

• close perpetuals which will provide them with sufficient transaction fees (computed based on the position size)

TL;DR

• Keepers call the forceClosePerpetuals(list of IDs) of the PerpetualManagerFront contract → force close perpetuals if they need to be and reward the keeper. The optimal behavior for the keeper is a complex problem and keepers should act if the reward is above the gas cost Resources:

  • Developers Doc

  • PerpetualManagerFront contract

  Updating Strategies

  Angle invests most of its idle funds in yield generating strategies. These strategies have specific debt ratios, meaning that each of them can invest only a specific share of the protocol’s funds. Over-time, they are supposed to generate gains and have greater balances than what the protocol originally deposited.

  As always, the protocol is passive and relies on keepers to enforce the update of debt ratios, deposit/withdraw funds from strategies depending on what’s in the protocol, and report the gains made by strategies.

  For example, if a lot of funds have been deposited into the protocol recently, some keepers would need to call the harvest() function on the strategy's contract in order for the strategies to take these funds out of the protocol and invest them.

  For now, there are no rewards upon calling harvest() , and team periodically calls the function. Governance token rewards could be set as incentive if needed.

  TL;DR

  • Keepers call the 

Resources:

• Developers Doc

• RewardsDistributor contract

In Angle, some fees can depend on the collateral ratio of the protocol to encourage/discourage some behaviors. This dependency is introduced through governance and takes the form of piecewise linear functions of the collateral ratio, where the fees change at some specific thresholds of collateral ratio.

For example, if the protocol is under-collateralized, deposit fees for SLPs could be set lower, and withdrawals fee higher, incentivizing them to deposit funds and not withdraw right now.

The fees functions are set upstream by governance. Computing the collateral ratio on-chain is very expensive, it would therefore not be viable to update it for every transaction in the protocol. The protocol therefore relies on keepers to compute it and update the fees accordingly. The associated functions for doing so are updateUsersSLP() and updateHA(). No rewards are associated with calling these functions.

• Keepers call the updateUsersSLP() or updateHA() functions of the FeeManager contract → fees for users, SLPs and HAs are updated to reflect the changes in collateral ratio in the protocol.

Note that this dependency of fees to the collateral ratio hasn’t been introduced yet. Only the slippage for SLPs currently depends on the collateral ratio.

Resources

• Developers Doc

• FeeManager contract

For those operating keepers on the Angle protocol, it is worth noting that as keeper transactions within yield pure rewards with no capital requirements. Contrary to Maker for instance where liquidations are based on auction mechanisms where keepers need to commit capital, in the case of Angle, liquidating a perpetual only requires notifying the protocol that a given perpetual should be liquidated. It is therefore possible that some keeper transactions of the protocol end up being front-ran by bots seeing the immediate profit.

One way to circumvent this issue is to use the flashbots RPC as detailed here.

Keepers can be a difficult concept to understand, but is not very complicated. Keepers are people operating bots, monitoring the state of decentralized protocols. When they detect specific signals, they send transactions to update this state and get rewarded in exchange. This need comes from the passive state of blockchains.

In Angle, keepers are needed to:

1. Liquidate and force-close Hedging Agents’ positions

2. Update Strategies gains and debt ratios

3. Update the staking distributions

4. Update the fees if they depend upon the collateral ratio

Note that functions for (2), (3), and (4) rewards have not been calibrated relative to the cost yet, and that there are currently performed by the team. However, anyone is free to call of these functions already if they want to, and we expect to adjust the rewards in the next few months.