p. 1

comparison of incurred but not reported ibnr methods sponsored by society of actuaries health section prepared by cabe chadick fsa maaa wes campbell asa maaa finn knoxseith asa maaa october 2009 © 2009 society of actuaries all rights reserved the opinions expressed and conclusions reached by the authors are their own and do not represent any official position or opinion of the society of actuaries or its members the society of actuaries makes no representation or warranty to the accuracy of the information.

[close]

p. 2

comparison of ibnr methodologies foreword lewis ellis inc l&e would like to acknowledge those who participated in the development of this research paper especially the society of actuaries project oversight group pog who provided advice and guidance during the course of the project we would also like to thank the research coordinators from the society of actuaries for their efforts in organizing and coordinating the project the members of the pog were eric smithback david nelson john governale peter howard bernie rabinowitz martin staehlin michael bears pat kinney john stenson chair the project coordinators from soa were steven siegel barbara scott the researchers from lewis ellis inc were cabe chadick wes campbell finn knoxseith brian stentz © 2009 society of actuaries all rights reserved lewis ellis page 2

[close]

p. 3

comparison of ibnr methodologies table of contents introduction 5 definitions 10 executive summary 13 summary of methods to calculate ibnr 16 tabular methods 16 case reserve methods 16 projection/exposure methods 17 development methods 18 description of typical development methods 18 crossincurral method 18 paid pmpm method 19 stochastic methods 19 neural network methods 20 practical considerations for ibnr issues 21 incurred date definition 21 methods delineating icos versus true ibnr 22 excess or outlier claims 23 outside influences on health claim reserves and patterns 24 seasonality 24 days in a month to generate claim 24 claim cost trends 25 growing/diminishing blocks 25 premium changes 26 benefit changes 26 claim administration disruption/backlogs 27 methods and issues of managed care 28 margin and confidence intervals 28 © 2009 society of actuaries all rights reserved lewis ellis page 3

[close]

p. 4

comparison of ibnr methodologies followup studies 29 internal consistency with insurer financials 31 documentation and data quality 31 checklist for actuaries calculating claim reserves 32 basic review of ibnr methods tested 33 definition of accuracy applied for ibnr testing purposes 35 data simulation 36 summary 36 description of crystal ball use 37 lag set simulation 37 determination of number of scenarios for statistical significance 40 description of stress testing scenarios 41 results and conclusions 44 bibliography 54 appendices development method details a algebraic definitions of ibnr methods b description of paid pmpm method c stochastic method literature our tested implementation and stochastic software tools d detailed results of ibnr tests e managed care issues f ibnr margin g advance methods neural network h confidence intervals for regressions i © 2009 society of actuaries all rights reserved lewis ellis page 4

[close]

p. 5

comparison of ibnr methodologies introduction abstract recognizing the lack of comparative information on techniques used for estimating incurred but not reported ibnr reserves the heath section council of the society of actuaries commissioned a research project to assess the accuracy of commonly used ibnr estimation methods over a wide range of scenarios lewis ellis inc l&e was awarded the contract to perform this research to conduct the study a stochastic model was constructed to compare and score estimates produced by the ibnr methods that were selected for testing the testing was done over a significant number of iterations and alternative business situations findings from the model testing include the following 1 when lag methods were tested the more robust average lag methods and hybrid methods produced better results than straight average methods hybrid methods also produced fairly low mean errors and standard deviations 2 for the methods and scenarios tested the one exhibiting the most consistency in terms of relatively lower variance and mean error was the paid pmpm method the method also often provided useful claim reserve estimates in recent incurral months as well as handling claims administration disruptions relatively better 3 traditional lag methods exhibited the least amount of variation in mean ibnr error when tested under an alternative business situation that entailed material shifts in per capita claims costs at various time points prior to the valuation date 4 the two business situations tested that were the most problematic to the desired goal of reserve sufficiency were a very recent upward increases in claims costs and b rate spirals the following comments describe limitations of the study and practical considerations when calculating ibnr estimates and highlight specific areas where caution should be applied when interpreting the results 1 advances in computing power and increasing software sophistication are making new and nontraditional methods more accessible to practicing actuaries yet more sophisticated ibnr calculation methods while possibly more accurate often require additional assumptions and supporting data as well as adjustments e.g seasonality adjustment to paid pmpm method to derive reasonable results these methods may also require advanced technical knowledge and the purchase of specialized software as compared to those methods generally in use by practicing health care actuaries 2 the loss ratio method tested produced some of the more accurate results with fairly low standard deviations but there are several important cautions in the interpretation of these results and the appropriateness of use of this method 3 most health care actuaries use a variety of methods to estimate ibnr and the preferred method may be a combination of two or more of the tested methods or other ones for instance lag methods tend to be the most common methods used by health actuaries however the results © 2009 society of actuaries all rights reserved lewis ellis page 5

[close]

p. 6

comparison of ibnr methodologies consistently indicate these methods have the highest standard deviations thus the use of a second method is suggested in order to obtain reasonable results in the more recent incurral months 4 understanding a health plan s particular facts and circumstances current environment may be the most important assumption to specifically document prior to deciding on the most appropriate method for a specific analysis the following four operational parameters are highlighted as particularly difficult to characterize and predict a seasonality of claim payments/processing b impact of large claims on the data c impact of negative claim amounts on lag factors and loss ratios and d credibility of the underlying trend in cost that results from the developed ibnr estimate finally it is important to note that ultimately the best method or combination of methods to use in a particular situation may be dependent upon factors and actuarial judgment that cannot be tested through a scientific model in this sense the report should not be construed as recommending or endorsing a best method rather the report is intended to provide useful information about the advantages and disadvantages of many commonly used methods and advance knowledge in this area for the practicing health care actuary © 2009 society of actuaries all rights reserved lewis ellis page 6

[close]

p. 7

comparison of ibnr methodologies background and purpose with increased scrutiny on financial reporting and ongoing solvency concerns health actuaries are faced with the need more than ever to deliver reliable estimates of claim costs and reserves a number of methods exist for calculating incurred but not reported ibnr reserves however practical techniques have not been substantially updated in a number of years further significant changes in technology and speed of claim payment may make new techniques more applicable currently when faced with a choice of which method may be most accurate and appropriate in a given situation there is a dearth of information to guide health actuaries as such the health section council hsc of the soa identified a need for comparative information on the accuracy of various ibnr methods the hsc sought proposals and awarded a contract for research that would result in a practical guide on distinguishing between ibnr methods and to inform decision-making on the best method for a given circumstance the hsc was also interested in creation of a basic software tool to accompany the practical guide the focus of the research is on medical insurance and as such the intended audience is health practitioners lewis ellis inc l&e was awarded the contract to perform this research the project objectives specified that l&e should test a number of methods on a data set it provided and rate the methods for accuracy and applicability for a number of situations the methods were supposed to include those commonly used by health actuaries to calculate ibnr such as the completion factor method as well as variations of it and others statistical model methods were to be potentially considered for evaluation but the methods considered for comparison should also be such that a practitioner could apply them without the need for special hardware or software packages in order to easily compare the methods l&e was asked to either use a commonly applied measure of accuracy or develop a new measure to score the ibnr methods reliances there are many well-written and well-read pieces of actuarial literature see bibliography on health liabilities and their calculation this literature has defined terms methods and other relevant ideas so well and predominantly among practicing actuaries that we have taken liberties to replicate these definitions methods etc albeit with intended references disclaimer users of this report should possess a certain level of familiarity with actuarial techniques and health care so as not to misinterpret the data presented any use or distribution of this report should be made in its entirety in addition any third party with access to this report acknowledges as a condition of receipt that l&e does not make any representations or warranty as to the accuracy or completeness of the material any third party with access to these materials cannot bring suit claim or action against l&e under any theory of law related in any way to this material the report accompanying model and other documentation contained herein do not represent an official position statement or endorsement on behalf of the society of actuaries or its members for a particular method or combination of methods for calculation of incurred but not reported reserves ibnr nor © 2009 society of actuaries all rights reserved lewis ellis page 7

[close]

p. 8

comparison of ibnr methodologies should the material be construed to do so it is the product of a research effort commissioned by the society of actuaries health section to add to the library of resource information for the estimation of ibnr reserves and further knowledge in that area the material is neither intended to preclude the use of other methodologies for estimation of ibnr reserves for any purpose nor provide an official statement or position on the use application or preferability of other methodologies as compared to the methodologies described herein defining ibnr ibnr is an acronym short for incurred but not reported that is probably used the most by actuaries and non-actuaries alike to refer to a certain balance sheet liability of an insurer or hmo another common proxy for this acronym is claim reserves which we will also use in this report however ibnr has a more formal definition see below in which it is merely one part of an insurer s claim liabilities for the purpose of this report we are not going to use the formal narrow definition of ibnr in this report we will broadly consider ibnr or claim reserves to represent a broad collection of insurer or hmo balance sheet entries technically representing a more formal definition of liabilities but not the formal definition of reserves whereas liabilities formally relate to events that have already occurred but for which a payment has not yet been made and thus costs have already accrued reserves are formally related to entries for amounts that the company has become obligated to pay but for which the reimbursable event has not yet occurred and thus these costs will accrue in the future our broad loose use of the acronym ibnr will encompass the following formal claim liability parts 1 incurred but not reported claim liabilities liabilities for claims that are anticipated but have not been reported to the health plan as of the valuation date in this report if we want to refer to this narrow formal definition we will use the phrase true ibnr 2 in course of settlement icos claim liabilities liabilities for claims reported and received but not yet adjudicated and paid as of the valuation date 3 due and unpaid d&u liabilities liabilities for claims that have been reported adjudicated and processed but for which final payment has not been recorded as of the valuation date an example is a claim that has been adjudicated but as of the valuation date is being held until the next date on which the health plan processes claim checks 4 outstanding accounting feeds liabilities which have been acknowledged as payments but for which no check has yet been cut as of the valuation date the most common examples are payments agreed to be made to pharmacy benefit managers who process pharmacy claims at the point of sale and then bill the health plan monthly or bi-monthly for the claims this liability definition overlaps with the d&u definition with the distinction possibly being a system-tosystem interface or batch processing claims versus payments made directly to a third party e.g claimant 5 disputed or resisted claims liabilities for claims that are in dispute such as those for which a known litigation situation exists 6 margin liability for a reasonable and prudent level of conservatism to cover adverse claim deviation such a margin can be an explicit amount or implicitly provided for in the various actuarial calculations © 2009 society of actuaries all rights reserved lewis ellis page 8

[close]

p. 9

comparison of ibnr methodologies we will also use ibnr instead of the phrase unpaid claim liabilities which is used by the managed care and health maintenance organization industry as well as use ibnr instead of incurred but not paid ibnp which many actuaries use as a more formal description of all claim liabilities for the purpose of this report ibnr does not include 1 loss adjustment expenses laes liabilities for the administrative costs associated with the adjudication of unpaid claims 2 present value of amounts not yet due pvanyd this reserve covers claims that were incurred on or before the valuation date which have not accrued as of the valuation date 3 active life reserves active life reserves represent the combination of contract reserves and unearned premium reserves the term active life differentiates the reserve nature from reserves related to insureds in claims status unearned premium reserves are typically an asset entry and represents premium that has been collected and entered in the ledger but are actually allocated to a time period after the valuation date a contract reserve is a reserve set up when a portion of the premium collected in the early years is meant to help pay for higher claim costs arising in later years 4 premium deficiency reserves a premium deficiency reserve is a reserve that is established when future premiums and current reserves are not sufficient to cover future claim payments and expenses for the remainder of a contract period 5 provider liabilities provider liabilities represent the reporting entity s obligation to make future payments to providers under some form of risk-sharing arrangement 6 provider insolvency reserves additional liabilities that may arise if a capitated provider becomes insolvent © 2009 society of actuaries all rights reserved lewis ellis page 9

[close]

p. 10

comparison of ibnr methodologies definitions we consider the following to be common definitions for phrases used in conjunction with health claim reserves and ibnr active life reserves 1 the combination of contract reserves and unearned premium reserves the term active life differentiates these reserves from reserves related to contracts in claims status or for disabled lives age-to-age development factors factors to complete paid claims to date in which the factors are based on the product of multiple representative ratios i.e the respective lag month for which the factor is generated through lag month of full completion of one lag month s cumulative paid claims to the prior lag month s cumulative paid claims age-to-ultimate development factors factors to complete paid claims to date in which the factors are based on the ratio of the lag month s cumulative paid claims to the fully complete ultimate claims claims adjudication 2 the process by which the insurer s claim processing area determines that an insured is eligible for payment and establishes an amount payable in relationship to the contractual benefits claims liability 3 a claims liability is established when an event has occurred that creates an obligation to pay benefits but complete payment has not yet been made as of the valuation date contract reserves 4 a contract reserve is established when some portion of the premium collected in a contract s early durations is intentionally designed to help pay for anticipated higher claims costs in later durations claims due and unpaid a reserve for claims which have been approved but for which payment checks have not been sent exposure unit of insured lives or premiums that contributed to exposure to a risk of a claim excess/shock claim larger amount claims that have on average longer lag and may cause significant changes in total ibnr 1 2 lloyd john c health reserves 3-5 lloyd health reserves 3-5 3 lloyd health reserves 3-5 4 lloyd health reserves 3-5 © 2009 society of actuaries all rights reserved lewis ellis page 10

[close]

p. 11

comparison of ibnr methodologies incurred date 5 the date on which an obligation to pay was established relative to a claim for medical coverages this is the date on which services were rendered by a provider or the initial date of service for a sequence of events which together constitute a claim in course of settlement icos the reserve amount attributed to claims already known and identified by the insurer but not yet adjudicated settled and paid by the insurer this reserve amount contrasts with claim liabilities which have not yet been identified by the insurer a.k.a true ibnr also can be described as reserves for claims that are on file in the company at the time the valuation is done but have not yet been approved or paid ibnp 6 the incurred but not paid ibnp liability reserve amount sometimes used formally in lieu of the more informal acronym ibnr ibnp is another name or acronym for an insurer s total claim reserve ibnp is the sum total of all claim reserve parts e.g icos true ibnr etc lag 7 the time between the incurred date of a claim and the date on which a claim payment is made this time can be broken into two parts i the time between the incurred date and the date upon which it is received for consideration by the insurer and ii the time between the insurer s initial consideration date and the claim payment date loss adjustment expense 8 the liability associated with the expense of processing claims that will be paid after the valuation date present value of amounts not yet due pvanyd also called disabled life reserves in the case of insurance providing ongoing periodic benefits past the reserve valuation date because of a benefit trigger that occurred on or before the valuation date e.g monthly loss-of-time benefits for disability due to accident or sickness monthly reimbursements or payments for long-term care expenses etc the common statutory definition is the reserve for claims unaccrued which may at the option of the insurer be discounted at interest this reserve is rarely considered part of the health ibnr pmpm per member per month which is a statistic measure of the average monthly per capita incurred claims where the per capita basis is membership e.g number of insureds including primary insureds spouses and children resisted claims reserve reserves for those claims in dispute and/or where the obligation to pay such claim is not reasonably clear as of the statement date seasonality 9 tendency for the amount of incurred claim liabilities to vary in a consistent and predictable manner within the space of a calendar year in health insurance seasonality may be due to seasonal variations in morbidity the rate at which individuals seek medical care benefit design such as when a 5 6 lloyd john c health reserves 3-5 lynch robert method for calculating ibnp health reserves with low variance 7 7 lloyd health reserves 4 8 lloyd health reserves 3-5 9 lynch robert g patent description method for calculating ibnp health reserves with low variance oct 6 2005 © 2009 society of actuaries all rights reserved lewis ellis page 11

[close]

p. 12

comparison of ibnr methodologies health plan has an annual deductible or annual out-of-pocket cost-sharing maximum or calendar effects for example variation in monthly claims caused by the differing number of days in each month or differences in the number of working days in a month valuation date 10 the date upon which an insurer s financial reports are compiled by closing the insurer s general ledger creating a set of cash accounting entries and creating reserve estimates typical valuation dates are year-end quarter-end or sometimes every month s end 10 lloyd john c health reserves 4 lewis ellis page 12 © 2009 society of actuaries all rights reserved

[close]

p. 13

comparison of ibnr methodologies executive summary the purpose of this section is to summarize the results of our testing and analysis of ibnr calculation methods later sections of the report explain the methods themselves with the wide variety of policy types adjudication practices lag times and other variables it is impossible to say that a given method is the perfect method in all cases even in our testing different methods for a particular block performed better under different scenario tests we have however analyzed our results in an effort to provide some basic guidelines to help you determine the best method for your particular circumstances we have based our conclusions on only a subset of the scenario tests that we created and tested we have chosen to omit the rate shift claim shift and disruption scenarios due to extreme variance of the results we have assumed that in situations like this the actuary would be forced to make manual adjustments in some way and could never simply run an ibnr calculation with the data as-is as a result we have focused on those scenarios that are more typical and that can for the most part be handled without any adjustments 1 lag methods tend to be the most common methods used by health actuaries however our results consistently show them to have the highest standard deviation even the more advanced lag methods while their performance is definitely better than the simple methods still generate a large standard deviation another problem with lag methods is that they require a significant amount of claims completion for a given month before they begin to develop any sort of accuracy this can be seen in our results by simply looking at the mean error for the hmo medicare block versus the medicare supplement block for a block that completes fairly quickly like the hmo medicare block the mean error is in the low single digits however for the medicare supplement block the mean error is around 28 percent thus the use of a second method is required in order to achieve accurate results in the more recent incurral months our hybrid loss ratio method illustrates one such method where we combine the loss ratio method and the 9-month average lag method based on the credibility of the completion factor this is definitely a viable option but there are other methods that produce accurate results without having to calculate and combine results from two different methods 2 for rapidly completing blocks such as the rx block we tested in virtually all scenarios the results of the paid pmpm method appeared the closest to the mean and had the lowest standard deviation this method was designed to have a lower variance than other methods and our testing proves that claim under certain conditions outside of the loss ratio method which had certain inherent biases in our implementation thereof the paid pmpm method had the lowest standard deviation in virtually all of our tests on all blocks however it did have its own share of problems first the method does not handle seasonality very well even the fairly simple scenario with a seasonality pattern based on the number of workdays in a month causes the mean error and standard deviation to increase the more complex seasonality scenarios show a much larger change in the mean error in defense of the method its documentation states that the data must be normalized for seasonality however we explain in this report the general difficulty of adjusting data for seasonality the other major issue with this method is that large claims especially on blocks with quicker run-out tend to cause an overestimation of the ibnr while this can be © 2009 society of actuaries all rights reserved lewis ellis page 13

[close]

p. 14

comparison of ibnr methodologies handled by simply removing the large claims and dealing with them separately this forces two separate ibnr calculations seasonality of the claims can have a material impact on the mean error for example seasonality exhibited by claim concentration early in a calendar year resulted in typical material positive errors i.e reserve sufficiencies for the recent incurral months for a number of the ibnr calculation methods when applied to a year-end valuation date seasonality exhibited by claim concentration later in the calendar year e.g large deductibles exhibited the opposite typical results across most ibnr calculation methods we observe the following regarding the stochastic simulation approach a the mean results show very low mean errors regardless of the incurral month set e.g all recent credible months examined b the mean results show some of the relatively lowest mean errors as well as lowest standard deviation of error for the recent incurral months i.e under 50 percent completion if you desire methods that are immune to premium rate shifts e.g the 15 percent rate shift scenario there are a number of options available e.g paid pmpm method traditional lag methods whose accuracy should not be impacted by premium stability or lack thereof applying an ibnr method e.g benktander that relies on premium for its ibnr calculation requires adjustment and likely actuarial judgment to offset expected material ibnr error by applying the method mechanically without adjustment with the claim shift scenarios the variations of the traditional lag method saw the least if almost no changes in mean errors as compared to the base scenario this is reasonable given that the observed payments as of the valuation date are assumed to already reflect these claim shifts and the lag factors continue or complete this observed shift other ibnr methods predicated on prior pmpm or similar claim levels don t adequately recognize the claim shift a very dangerous scenario to the desired goal of reserve sufficiency is a recent unpredicted i.e it is not readily apparent to make a manual adjustment upward shift in per capita claim levels another dangerous scenario to reserve accuracy or sufficiency is a rate spiral however the traditional lag methods appear to be relatively unimpacted by our illustrative rate spiral scenario there is a material risk to ibnr accuracy when removing large claims from the data set from which the ibnr calculation method e.g traditional lag paid pmpm etc is applied the paid pmpm method appears to handle most of the claims processing disruption scenarios relatively better than other ibnr calculation methods care should be taken in extrapolating conclusions among the block types for example the selffunded block type and the hmo inpatient block type had the highest standard deviation of error results observed across most ibnr methods we believe this is indicative of the variance in the completion distribution particularly recent incurral months from which we extrapolated from our single sample data for each block type other block type samples had more consistent completion patterns in our sample the loss ratio method produced some of the more accurate results with fairly low standard deviations however caution should be taken in the use of these results for two main reasons first there is a bias in our data set simulations that results in the loss ratio method have a higher than expected degree of accuracy by using loss ratios to develop our data set and limiting the lewis ellis page 14 3 4 5 6 7 8 9 10 11 12 13 © 2009 society of actuaries all rights reserved

[close]

p. 15

comparison of ibnr methodologies range of those loss ratios the results are incurred claim totals that on average will hit the target loss ratio exactly second the accuracy decreases when trends in the data don t move smoothly in one direction the two varying trend scenarios illustrate this issue the best the mean error is still in the single digits but this is due to the first issue we just mentioned directly comparing the base or excessive trend scenario to the two varying trend scenarios shows how much worse the accuracy gets when the predictability of the trends is removed overall the most consistent method was the paid pmpm method it was routinely among the methods with the lowest variance and mean error it is also useful in the most recent incurral months eliminating the multiple calculation issues it requires slightly more complex calculations needing trend adjustments and making use of linear regression but it is not too difficult to accomplish in a simple spreadsheet if lag methods are still the desired solutions the more advanced average lag methods and the hybrid methods are much better alternatives to the straight average methods the hybrid methods especially produce fairly low mean errors and standard deviations the choice however really comes down to what method is best given your unique circumstances this document was designed to serve as a practical guide for the calculation of claim reserves focusing on the different methodologies available for ibnr calculations our goal was to discuss major issues affecting today s actuaries as they attempt to establish claim reserves but by no means do we consider our analysis exhaustive hopefully this guide will serve as a starting point for any actuary establishing a new reserving process or wishing to re-examine an existing process © 2009 society of actuaries all rights reserved lewis ellis page 15

[close]