August 13, 2010

In Part 1 of this article, we explored the value of mitigating any risk of your asset/liability model making incorrect forecasts by paying close attention to the certification that it was created properly by your model provider.

In this article, we’ll discuss what Bill McGuire, Ph.D., suggested you can do to make sure your credit union has implemented the model correctly. President/CEO of McGuire Performance Solutions, Scottsdale, Ariz., McGuire provided these ideas during his company’s Webinar, “ALM Model Verifications: Complying with 2010-1A IRR Regulatory Guidance and Uncovering Value.”

Under the new interest rate risk advisory published this year by the Board of Governors of the Federal Reserve System, the Federal Deposit Insurance Corp., the National Credit Union Administration, the Office of the Comptroller of the Currency, the Office of Thrift Supervision, and the Federal Financial Institutions Examination Council State Liaison Committee, a certain amount of the onus for making sure an asset/liability model functions correctly lies with the vendor, McGuire noted. For example, the vendor has to prove that the math works in the model. Still, vendor ALM model certification is not a substitute for a model verification/validation of your “as implemented” ALM model, he emphasized.

By taking a hard look at both vendor certification and CU implementation verification, “we get model risk to a manageable level, hopefully we get to zero,” McGuire said.

How often should your credit union verify its ALM model? While there is no official annual regulatory requirement, McGuire suggested completing a comprehensive ALM process verification every two or three years if underlying conditions are stable (e.g. no change in model used, no major turnover in staff, no merger activity or other material balance sheet change, etc.).

If your credit union is on a three-year cycle, it is often a good idea to do a quick technical verification/validation of the model at 18 months to check ongoing accuracy,” he added. “With the money saved by not conducting annual model verification, management needs to invest in analyses of the model assumptions that most affect accuracy - loan prepayments/paydowns and core deposit behaviors. I have received very favorable comments on such a cycle from examiners as it controls model risk and enhances model precision over time.”

A first step in verifying your model implementation is understanding the various building blocks of a model—such as data, category definitions (such as 30-year mortgages) reporting tools and the model governance environment. The idea in verification is to look for risk in all the building blocks, McGuire said.

“The model must be verified—can the model forecast accurately? And then model forecasts must be validated—does the model forecast accurately?” he said. “It’s a very important distinction.”

Components of ALM Model Verification

The structure for model risk assessment is more clearly defined by the new IRR advisory, McGuire says. Specifically, ALM model implementations need to be reviewed and vetted in multiple areas including, but not limited to, the following:

• The model must match your credit union’s specific interest rate risk analysis needs. Can it do what you need it to do? Can the model get its hands around all aspects of your credit union’s data complexity? Can the model help you look at a wide variety of test scenarios? “An institution can have ‘too much’ or ‘too little’ ALM model,” McGuire said. “Beware of the model’s true all-in cost, including specialized staff.”

• Check on model data, technical design and general set-up. This aspect of model verification “drills down into the ALM model and verifies that you have the data coming in correctly and that you’re matching categories to that data correctly,” McGuire said. “This gets down to the heart of (the way you are using) the model.”

• Make sure that the model’s behavioral assumptions about what your members would do—such as prepay—are not conservative or aggressive, but correct. McGuire noted that testing the model using historical data to see how well it predicts what actually happened to see if it forecasts accurately (also known as “back testing”) is downplayed a little bit in the new advisory, but back testing behavioral assumptions is specifically mentioned.

• How you use the model is also part of verification. What IRR testing do you do and how is that reported to the board? McGuire asked, noting that rate shocks (instantaneous and permanent parallel shifts in all interest rates) are the oldest and most common form of net interest income and net economic value interest rate tests.

Don’t neglect to consider your model governance—whether the ALCO and model users are following appropriate modeling procedures. According to McGuire, a good practice is to have a detailed user manual where every modeling step, data file, input/assumption source, and other model related activities is centrally documented. This supports back up users if they need to run the model, plus is a great cross-training tool.

“You’ll also want to have a remediation program,” McGuire added. “Let’s say I broach an interest rate limit. What am I supposed to do and how fast am I supposed to do it? How does the board get involved? What special reporting is involved? Knowing how the credit union will respond to an out-of-compliance situation is important.

In all, verification must be detailed, comprehensive across the model, and provide specific upgrade/enhancement recommendations, McGuire added.

Lisa Hochgraf is a CUES editor.