Three parts of a good independent model review
Originally published 8/24/2009 © 2021 Olson Research Associates, Inc.
1) Make sure you have the latest copy of our 3rd party validation.
We’ve had an independent third-party review our modeling calculation software to verify its ability to properly calculate cash flows. I call this the “1 + 1 = 2” test. In other words, does the calculator work? Can it compute an amortization schedule; can it calculate the correct amount of interest? Can it represent a bullet maturity? Can it compute the specified amount of prepaying principal? etc.
It’s quite costly and time consuming to go through this review process. And we don’t often change our modeling calculation software. Therefore we don’t put forward the effort every year but rather every three to five years depending on when changes have been made to the system.
You can obtain a copy of our latest third-party review report by contacting your analyst, or by emailing us at info@olsonresearch.com.
2) Review the bank’s prior quarter A/L BENCHMARKS Executive Report page 50, “Model Back Test”.
A key component of model validation is a back test. Interest rate risk measures in particular rely heavily on forecasts. The back test is designed to capture how accurate the forecast was one year ago compared to actual results. In other words, we compare the bank's actual performance to the projected performance shown in the prior year's A/L BENCHMARKS reports.
Naturally there will be discrepancies as there is no way to hit all of your performance targets perfectly. However it is good discussion to see if, in hindsight, management’s estimates of performance were reasonable. Specifically where did the bank miss the mark and more importantly, are the differences reasonable? Obviously differences in both balances and rates can contribute to differences in earnings. Review key items to see if the outages make sense. Back Test Report
Review how differences in forecasted balances might affect interest income or expense. Are projected yields higher or lower than actual performance; if so why? Can we correct these differences by adjusting next quarter's forecast? Review the differences in volume and rate to see if they are consistent with the outages in interest income or expense. Which had the greatest impact on yield or cost: differences in balances or rates? Also consider these questions: What happened to balances? Why are the rates off? Is this due to competition or general economic conditions? Have the inputs been reviewed each quarter? Document the answers from management as part of the model validation process.
3) Review the bank’s current quarter A/L BENCHMARKS Executive Report pages 32-40, “Forecast”.
Each quarter, in addition to providing a call report, loan detail, and investment detail, the bank may also provide us with a forecast or projection. This projection is usually something similar to the bank’s current budget. Once you receive your current A/L BENCHMARKS reports, you should verify that the forecast shown on pages 32-40 fairly represents the bank’s budget (as supplied via the Service Kit). Remember – the forecast shown probably will not match the bank’s budget exactly. We use detailed data downloads and assumptions (like prepayment assumptions for instance) to create the projected cash flows and income. Most bank’s only use income and expense estimates in a budgeting tool or spreadsheet. The numbers on page 32-40 won’t match, but they should be close.
