Introduction

Medication-related osteonecrosis of the jaw (MRONJ) is a rare complication of treatment with antiresorptive therapies—particularly, bisphosphonates or denosumab—as well as angiogenesis inhibitors and other cancer therapeutics.1 Despite the higher per-patient risk of MRONJ among cancer patients receiving high-dose antiresorptive therapy, large-scale study of MRONJ in fracture-prevention populations is essential because antiresorptive medications are prescribed to vastly large numbers of patients for osteoporosis and fracture prevention, making even very low absolute risks highly consequential at the population level and clinically relevant to treatment decision-making.2 Since MRONJ is uncommon among patients receiving bisphosphonates or denosumab for fracture prevention, large observational studies are needed to accurately ascertain these events as comprehensive electronic medical record (EMR) review is rarely feasible at scale. However, to date, such studies have been problematic as administrative data using International Classification of Diseases (ICD), ninth (ICD-9) and tenth revision (ICD-10) codes to identify MRONJ events have yielded poor positive predictive values (PPVs), with overall PPVs <33% when compared with electronic medical record (EMR) review.3–6

Several important limitations of prior validation studies leave substantial gaps in the understanding of the PPV of ICD-9 and ICD-10 codes for MRONJ ascertainment in individuals prescribed antiresorptives from fracture prevention. First, some prior studies included patients receiving antiresorptives for bone complications of cancer, despite the substantially different clinical context of oncology-related high-dose antiresorptive therapy, which is associated with markedly higher absolute MRONJ risk and potentially different individual risk profile, distinct treatment patterns, and potentially different coding practices than antiresorptive use for fracture prevention.7,8 Second, several reports,3,6 including a systematic review,9 are primarily restricted to postmenopausal women, limiting generalizability to diverse populations. Third, some prior studies did not require the osteonecrosis of the jaw (ONJ) even to be related to a medication exposure.3 Fourth, no prior studies have included similar analyses in those with a filled prescription for denosumab for osteoporosis, despite evidence that denosumab use is associated with higher MRONJ rates than bisphosphonates.10 Fifth, because administrative coding practices, diagnostic terminology, and healthcare delivery structures vary substantially across countries and healthcare systems, validation findings from European registry-based studies may not generalize to US administrative data; notably, prior U.S.-based studies have demonstrated even lower PPVs (58–7.1%)3,4 than those reported in European cohorts.5,6 Finally, only ICD-10 coding, and not ICD-9 coding, has a specific code for MRONJ (eg, M8710: osteonecrosis due to drugs). ICD-10 coding was mandatory in the U.S. only since 2015. To date, no published reports in the U.S. include long-term time intervals to determine the PPV of this particular code for MRONJ events.

Data regarding the performance characteristics of ICD-9 and ICD-10 diagnosis codes for identifying MRONJ other than PPV are even more limited. In a Danish cohort, ICD-10 codes recorded within 180 days of a dentoalveolar procedure demonstrated a sensitivity of 77.2% (95% CI, 68.4–84.6%) for identifying MRONJ among cancer patients8 Notably, prior validation studies have employed claims-positive sampling designs that permit estimation of PPV but preclude direct assessment of sensitivity, specificity, and negative predictive value (NPV).3–6 To our knowledge, no prior report describes full validation metrics for claims-based ICD-9 or ICD-10 code algorithms for MRONJ among individuals prescribed antiresorptives for fracture prevention.

Therefore, the primary purpose of this report was to determine the utility of ICD-9 and ICD-10 codes for MRONJ diagnosis among individuals prescribed antiresorptives for fracture prevention to inform the methodology of future epidemiological research studies. To do this, we calculated the PPVs of a broad list of candidate ICD-9 and ICD-10 codes for MRONJ events in a diverse group of older men and women who had received a bisphosphonate or denosumab for fracture prevention and not for cancer-related reasons. The first cohort was comprised of Veterans, the majority of whom were men, with continuous and comprehensive Veterans Health Administration (VHA) dental benefits,11–15 who received care through the VHA, the largest integrated health system within the U.S. between October 1, 1999, and December 31, 2022, with follow-up through December 31, 2023. The second cohort was patients in HealthPartners2 (HP), the largest consumer run nonprofit healthcare company in the US,16 the majority of whom were women with the same inclusion criteria and follow-up periods. Within the VHA cohort, we additionally estimated the sensitivity, specificity, and NPV of candidate ICD-9 and ICD-10 codes to provide a comprehensive assessment of their diagnostic performance for MRONJ ascertainment.

Materials and Methods Administrative Databases and Study Cohorts

The VHA Corporative Data Warehouse (CDW) and the Virtual Data Warehouse for HP were queried for all persons aged 50 and older who had received at least one filled prescription for an antiresorptive therapy for fracture prevention between October 1, 1999, to December 31, 2022. Antiresorptive therapies for fracture prevention included oral (alendronate, risedronate, ibandronate) or intravenous (IV) bisphosphonate (zoledronic acid, ibandronate) or subcutaneous denosumab 60 mg injection. Those with a filled prescription at any time during the study period for a bisphosphonate not FDA-approved in the U.S. for osteoporosis (ie, etidronate, pamidronate or tiludronate) and those whose second dose of zoledronic acid was <273 days from the initial dose (a dosing interval usually used for cancer-related complications and not osteoporosis) were excluded from the study cohort. Participants were also excluded if, prior to their first antiresorptive prescription, they had a medical diagnosis representing a non-fracture prevention indication for antiresorptive therapy: Paget’s disease of bone, osteogenesis imperfecta, hypophosphatasia and phosphate wasting syndromes, osteopetrosis, secondary metastatic cancer, or multiple myeloma (Table 1).

Table 1 International Classification of Diseases (ICD) Ninth (ICD-9) and Tenth (ICD-10) Revision Codes for Medical Conditions Representing Indications for Antiresorptive Therapy Other Than Fracture Prevention and Used for Baseline Exclusion and Observation Censoring

This study was determined to be exempt by the VHA Central Institutional Review Board (IRB) (IRB number 1690785) under common rule 4 (Title 45, Code of Federal Regulations, Part 46, Section 104, subsection (d), category 4) This study was approved by the IRB of HP (IRB number A21-275). Patient data were handled in accordance with institutional privacy and confidentiality standards at each participating site. All study procedures were conducted in compliance with the ethical principles of the Declaration of Helsinki.

Candidate ICD Code Selection

 We examined a broad, deliberately inclusive ICD code set incorporating dental and maxillofacial conditions that may mimic or be miscoded as MRONJ in administrative data. Potential codes for MRONJ examined included: osteonecrosis of the jaw (ICD-9: 733.45); osteonecrosis of the jaw due to drugs (ICD-10: M87.180); idiopathic aseptic necrosis of bone, other site (ICD-9: 733.49; ICD-10: M87.08); osteonecrosis due to drug, unspecified bone (ICD-10: M87.10); osteonecrosis due to previous trauma, other site (ICD-10: M87.28); other secondary osteonecrosis, other site (ICD-10: M87.38); other osteonecrosis, other site (ICD-10: M87.88); osteonecrosis, unspecified (ICD-10: M87.9); inflammatory condition of jaw (ICD-9: 526.4; ICD-10: M27.2); other specified diseases of jaws (ICD-9: 526.89; ICD-10: M27.8); alveolitis or osteitis (ICD-9: 526.5; ICD-10: M27.3); exostoses/torus (ICD-9: 526.81); and periapical abscess with sinus (ICD-9: 522.7; ICD-10: K04.6).

We largely based our selection of these codes on ICD codes that have previously been used in studies examining the validity of ICD codes for MRONJ.3–6,9 Because the ICD-9 code 733.45 for osteonecrosis of the jaw was not introduced until October 1, 2007, cases of MRONJ in the first 8 years of our study would have been coded using other ICD-9 codes, especially inflammatory condition of the jaw (526.4).17 Accordingly, we evaluated a broader set of ICD-9 jaw diagnosis codes and their ICD-10 equivalents. As in several prior reports,3–6 we evaluated codes for alveolitis or osteitis and periapical abscess with sinus, as these conditions may present with pain, exposed bone, delayed socket healing, localized swelling, and drainage that overlap with suspected MRONJ. Unlike prior validation studies,3–6,9 we included codes for anatomic orofacial variants (eg exostosis/torus) that may be miscoded as or warrant further evaluation for MRONJ as exploratory screening.

MRONJ Ascertainment

Study cohort participants were observed from their first filled prescription for an antiresorptive until they developed a non-fracture prevention indication for antiresorptive therapy (Table 1), received a dose of denosumab of 120 mg (a dose usually used for cancer-related complications and not osteoporosis), received a non-antiresorptive bone health pharmacotherapy (ie, teriparatide, abaloparatide, or romosozumab), died, were lost to follow-up, or the end of the study period for an MRONJ event. Each study participant was eligible for review for an MRONJ event only during this observability period for classification as having MRONJ or not having MRONJ. MRONJ ascertainment was distinct for the VHA and HP cohorts.

VHA Cohort

Because the substantial number of VHA individuals prescribed bisphosphonate or denosumab for fracture prevention precluded individual EMR review, we developed a Natural Language Processing (NLP) algorithm to identify MRONJ cases, as described elsewhere (Submitted). In brief, the NLP algorithm was developed from human-annotated EMR note documents to ascertain MRONJ events in persons who had received a bisphosphonate or denosumab for fracture prevention. Two annotators with expertise in osteoporosis management (BL, REE) independently annotated notes using as a gold standard the American Association of Oral and Maxillofacial Surgeons’ (AAOMS) Position Paper on Medication-related osteonecrosis of the jaws definition,2 with substantial input from dentists with expertise in treating MRONJ (TBD, PJG) in adjudicating any discrepancies, supporting the content validity of this methodology. Twenty-three total unique features (ie, MRONJ, exposed bone, jaw radiation) comprised the final code book for annotators, and note review continued until each feature reached a priori target thresholds of >80% on the F-measure. A total of 870 clinical notes were human-annotated until thematic saturation was reached, with no additional language categories identified. These human-annotated notes were not restricted to any ICD-9 or ICD-10 claims.

An MRONJ designation required an in-person visit documentation from an orofacial specialty provider (Doctor of Dental Surgery (DDS), Oral and Maxillofacial Surgeon (OMFS), or Otolaryngologist) with that documentation reflecting at least one of the following: (1) MRONJ and exposed bone; (2) MRONJ with explicit staging (stage 1, 2, or 3); (3) MRONJ with a corroborative MRONJ treatment (ie, anabolic agent, PENTO protocol, hyperbaric oxygen, debridement, resection or jaw reconstruction); (4) MRONJ with an affirming contextual modifier (except stage 0 only); (5) exposed bone and a stage of 1–3 even in the absence of MRONJ explicitly being mentioned; (6) a sequestrum on exam or imaging; (7) or a sequestrectomy procedure. Throughout this process, established sources were consulted whenever possible to standardize terms, including the VHA formulary of medications and the National Library of Medicine’s Unified Medical Language System.18 Finally, a decision tree-based machine-learning model used NLP-extracted features to classify MRONJ as MRONJ or Not MRONJ for each note from an orofacial specialist (dentistry, oral maxillofacial surgery, or otolaryngology). If a Veteran had no notes eligible for NLP review from an orofacial specialist, then that Veteran was excluded from the VHA cohort.

Compared with manual EMR review and adjudication by two independent reviewers on a held-out 200 note validation set, the NLP algorithm achieved an excellent document-level performance metrics for MRONJ identification: PPV (ie, precision) of 0.97, a sensitivity (ie, recall) of 0.95, and an F-measure of 0.96. After each note received a designation of MRONJ or Not MRONJ, Veteran-level classifications were assigned hierarchically: (1) if there was any note designating MRONJ, that Veteran was determined to have MRONJ by the NLP-algorithm.

HP Cohort

At HP, the study population were those individuals who were prescribed a bisphosphonate or denosumab who also had HP medical and dental insurance. MRONJ case detection at HP with NLP was not feasible as NLP algorithms are not easily transferable across health systems because variation in EMR architecture, documentation patterns, and local clinical workflows can materially alter performance, and NLP development requires substantial annotation, validation, and technical resources that limited developing an NLP at multiple institutions. However, given variation in coding practices, patient populations, and clinical workflows across healthcare systems, assessment of ICD-9 and ICD-9 code PPVs in additional cohorts prescribed antiresorptive for fracture prevention remains valuable to more fully characterize administrative code performance, even when limited to isolated PPV estimation and acknowledging the potential verification bias introduced by methodological differences in MRONJ ascertainment. Therefore, we proceeded with a claims-positive sampling design in the HP cohort. Ascertainment of possible MRONJ cases in HP relied on identification of all encounters in the EMR associated with any candidate ICD-9 or ICD-10 codes. These all underwent individual EMR review by an experienced clinician with substantial expertise in diagnosing and treating osteoporosis (JS). Analogous to the NLP development definition of MRONJ in the VHA, the definition of an MRONJ case by manual EMR review at HP was based on the AAOMS Position Paper on Medication-Related Osteonecrosis of the Jaws-2022 Update.2

No comprehensive or random EMR review of patients without any candidate ICD code was performed in HP because rarity of MRONJ events would have required review of an infeasibly large number of charts to identify false negatives (ie, MRONJ cases without any candidate ICD code).

Statistical Analyses

Baseline characteristics were described for all confirmed MRONJ cases, stratified by cohort (VHA, HP). To assess the performance of each candidate ICD code, the PPV of each candidate code was calculated, stratified by cohort (VHA, HP) and coding era (ICD-9, ICD-10). The 95% confidence interval (CI) for each PPV was calculated using the exact binomial method.

In the VHA cohort, further performance metrics of sensitivity, specificity, and NPV were estimated for each candidate ICD code, stratified by coding era (ICD-9, ICD-10), with exact binomial 95% CIs reported. Analogous metrics were not computed for HP because we lacked estimates for true negatives and false negatives given the claims-positive sampling design used for MRONJ ascertainment in KP. For computation of ICD-9 metrics, only Veterans observable from October 1, 1999, to September 30, 2015, (ie, Veterans who filled their first bisphosphonate or denosumab prescription between these dates) were in the VHA ICD-9 cohort. For computation of ICD-10 metrics, only Veterans observable from October 1, 2015, to December 31, 2023, (ie, Veterans who were not censored on or prior to September 30, 2015) were in the VHA ICD-10 cohort. These date cut-offs are based on the dates that ICD-9 and ICD-10 codes were available in the U.S. Analyses were conducted.

Two post-hoc analyses were performed. First, the performance metrics of the combination of ICD-10 codes for either osteonecrosis of the jaw due to drugs (M87.180) or osteonecrosis due to drug, unspecified bone (M87.10) was assessed, stratified by cohort (VHA, HP) using the same statistical methods above. Second, temporal variation in ICD code performance was evaluated by plotting PPVs and 95% CIs for candidate diagnosis code groupings that had corresponding ICD-9 and ICD-10 codes across VHA and HP: osteonecrosis of the jaw (ICD-9: 733.45; ICD-10: M87.180); inflammatory condition of jaw (ICD-9: 526.4; ICD-10: M27.2); other specified diseases of jaws (ICD-9: 526.89; ICD-10: M27.8); alveolitis or osteitis (ICD-9: 526.5; ICD-10: M27.3); and periapical abscess with sinus (ICD-9: 522.7; ICD-10: K04.6). Analyses were performed in R version 4.5.2 (R Foundation for Statistical Computing, Vienna, Austria) in the VHA cohort and with Stata version 18 (StataCorp LLC, College Station, TX, USA) in the HP cohort.

Results

At the VHA, there were 138,791 Veterans age ≥50 years old with at least one filled prescription for an oral or IV bisphosphonate or denosumab for fracture prevention during the study period without any exclusion criteria and who had at least one note from an orofacial specialist – making them eligible for NLP review. Among these Veterans, 46,408 had VHA continuous and comprehensive dental benefits and were included in the VHA cohort. Among Veterans aged ≥50 years old with at least one filled prescription for an oral or IV bisphosphonate for fracture prevention during the study period and with VHA continuous and comprehensive dental benefits, 96% had at least one note from an orofacial specialist that would qualify them for VHA cohort inclusion. In the VHA cohort, 47 MRONJ cases were identified by NLP. Of those Veterans with MRONJ, 5 individuals were only observable prior to October 1, 2015, and only contributed to the VHA ICD-9 cohort; 11 were observable only on or after October 1, 2015, and only contributed to the VHA ICD-10 cohort; 31 Veterans with MRONJ were observable across the ICD-9 and ICD-10 coding periods. In sum, there were 36 Veterans with MRONJ and 34,701 Veterans without MRONJ in the VHA ICD-9 cohort; there were 42 Veterans with MRONJ and 34,831 Veterans without MRONJ in the VHA ICD-10 cohort.

At HP, 4,625 patients age ≥50 years old with at least one prescription for an oral or IV bisphosphonate or denosumab had HP dental insurance and were included in the HP cohort. Among these HP cohort members, 10 patients (0.22%) had one or more encounters with one or more candidate ICD diagnosis code and were identified by clinician EMR review as having MRONJ.

At the VHA, the cohort was predominantly male (82%) and White (77%). The mean age of MRONJ diagnosis was 72 years. Among all VHA MRONJ cases (n=47), 46 Veterans had at least one filled prescription for a bisphosphonate and 1 for denosumab. Of those Veterans without MRONJ (n=46,361), 45,930 had at least one filled prescription for a bisphosphonate and 1,149 for denosumab. At HP, the cohort was largely female (89%), and White (80%). The mean age of MRONJ diagnosis was 76 years. Of the 10 patients with MRONJ, 10 had at least one filled prescription for a bisphosphonate and 0 for denosumab. Of those without MRONJ, 67 had at least one filled prescription for a bisphosphonate and 1 for denosumab.

The ICD-9 (733.49) and ICD-10 codes (M87.08) for idiopathic aseptic necrosis of the bone, other site, and ICD-10 codes for osteonecrosis due to previous trauma, other site (M87.28) and other secondary osteonecrosis, other site (M87.38) had combined number of persons from both VHA and HP of less than ten, and therefore, diagnostic performance characteristics were not calculated due to sparse data. Diagnostic performance characteristics (PPV, sensitivity, specificity, and NPV) for individual ICD-9 codes for MRONJ diagnosis in the VHA ICD-9 cohort are shown in Table 2 and PPVs for individual ICD-9 codes for MRONJ diagnosis in the HP cohort are shown in Table 3. The highest PPV for an ICD-9 code at both health care systems was for 733.45 (osteonecrosis of the jaw); the estimated PPV was 0.43 (95% CI: 0.10, 0.82) and 0.50 (95% CI: 0.07, 0.93) at the VHA and HP, respectively.

Table 2 Diagnostic Performance Characteristics of Individual International Classification of Diseases, Ninth Revision (ICD-9) Codes for Medication-Related Osteonecrosis of the Jaw (MRONJ) Diagnosis from the Veterans Health Administration (VHA) Cohort

Table 3 Diagnostic Performance Characteristics of Individual International Classification of Diseases, Ninth Revision (ICD-9) Codes for Medication-Related Osteonecrosis of the Jaw (MRONJ) Diagnosis from the HealthPartners (HP) Cohort

Diagnostic performance characteristics (PPV, sensitivity, specificity, and NPV) for individual ICD-10 codes, and the combination of codes M87.180 or M87.10, for MRONJ diagnosis in the VHA ICD-10 cohort are shown in Table 4 and PPVs for individual ICD-10 codes, and the combination of codes M87.180 or M87.10, for MRONJ diagnosis in the HP cohort are shown in Table 5. The highest PPV for an ICD-10 code at both health care systems was for M87.10 (osteonecrosis due to drug, unspecified bone); the estimated PPV was 0.27 (95% CI: 0.06, 0.61) and 0.67 (95% CI: 0.09, 0.99) at the VHA and HP, respectively.

Table 4 Diagnostic Performance Characteristics of Individual and Select Combination International Classification of Diseases, Tenth Revision (ICD-10) Codes for Medication-Related Osteonecrosis of the Jaw (MRONJ) Diagnosis from the Veterans Health Administration (VHA) Cohort

Table 5 Diagnostic Performance Characteristics of Individual and Select Combination International Classification of Diseases, Tenth Revision (ICD-10) Codes for Medication-Related Osteonecrosis of the Jaw (MRONJ) Diagnosis from the HealthPartners (HP) Cohort

In the VHA, the most sensitive ICD-9 codes were osteonecrosis of the jaw (733.45) and inflammatory condition of jaw (526.4), with estimated sensitivity of 0.08 (95% CI: 0.02, 0.23) for each of these codes. The most sensitive individual ICD-10 code was osteonecrosis of the jaw due to drugs (M87.180) with sensitivity of 0.19 (95% CI: 0.9, 0.34), only minimally improved by considering either this code or osteonecrosis due to drugs, unspecified bone with sensitivity of 0.21 (95% CI: 0.10, 0.37). The estimated specificity was ≥0.97 and NPV was ≥0.99 for all examined ICD codes (Table 2 and Table 4).

No significant temporal variation in ICD code performance, as measured by PPV, was observed in either health system before versus after October 1, 2015, for codes corresponding to osteonecrosis of the jaw, inflammatory condition of jaw, other specified diseases of jaws, alveolitis or osteitis, or periapical abscess with sinus (Figure 1). However, confidence intervals were wide for osteonecrosis of the jaw codes in the VHA and all codes in HP, limiting inference (Figure 1).

Figure 1 Temporal variation in positive predictive value (PPV) performance for International Classification of Disease (ICD) codes across Veterans Health Administration (VHA) and HealthPartners (HP) cohorts. Osteonecrosis of the jaw (ICD-9: 733.45; ICD-10: M87.180); inflammatory condition of jaw (ICD-9: 526.4; ICD-10: M27.2); other specified diseases of jaws (ICD-9: 526.89; ICD-10: M27.8); alveolitis or osteitis (ICD-9: 526.5; ICD-10: M27.3); periapical abscess with sinus (ICD-9: 522.7; ICD-10: K04.6).

Abbreviations: ICD, International Classification of Diseases; NE, Not estimable.

Discussion

We evaluated the performance of individual ICD-9 and ICD-10 claims codes for identifying MRONJ from two health care systems, one a public (VHA) and the other a private payor (HP). No examined ICD code achieved a PPV greater than 0.70, underscoring the substantial false-positive burden and limited utility of these codes for standalone MRONJ identification. PPV performance was generally consistent across VHA and HP cohorts; however, limited numbers of claims for some ICD codes—particularly in HP—resulted in wide confidence intervals and imprecise PPV estimates. The ICD-9 diagnosis codes 526.5 and 526.81 and ICD-10 diagnosis codes M87.88 and K04.6 have a near-zero PPV in both health systems, and, therefore, appear to be too non-specific for use in algorithms to ascertain MRONJ. Although all examined ICD codes demonstrated specificity ≥0.97 and NPV ≥0.99 for MRONJ, these favorable metrics should not be interpreted as evidence of strong overall diagnostic performance; because MRONJ is a rare outcome in the studied population, high NPV is expected and high specificity may overstate practical diagnostic performance, particularly when sensitivity and PPV are low. Identification of MRONJ using ICD claims alone remains challenging due to poor overall performance of ICD codes to identify MRONJ.

Our findings are consistent with prior validation studies of ICD-9 and ICD-10 administrative codes among post-menopausal women in the U.S., which have similarly demonstrated low PPV for MRONJ identification.3,4 For ICD-9 codes, Wright et al3 reported that the best-performing code was 733.45 (osteonecrosis of the jaw), with a PPV of 0.25 (95% CI: 0.01–0.81), followed by 526.4 (inflammatory condition of jaw) with a PPV of 0.13 (95% CI: 0.04–0.27).3 A similar PPV performance hierarchy was observed in both the VHA and HP cohorts, with somewhat higher point estimates for code 733.45 (0.43–0.50), but equally wide confidence intervals reflecting the imprecision of these estimates. Both of these prior reports3,4 included the same ICD-9 codes we examined but did not include 526.89 (other specified disease of jaw), 526.81 (exostosis/torus), or 522.7 (periapical abscess with sinus); notably in our cohorts the PPVs for these codes were all near-zero. In addition, Wright et al3 included the code 5269 (unspecified disease of the jaw), which we did not examine in our study. For ICD-10 codes, our study is only the second to evaluate ICD-10 code performance for MRONJ identification in the U.S.; the prior study reported similarly poor performance, with an overall PPV of 0.05 (95% CI: 0.01–0.14).4 Our study differed from these prior reports3,4 as we included both men and women – including a predominately male cohort (VHA), confined our analyses to those using bisphosphonates or denosumab for fracture prevention, and included the ICD-10 for osteonecrosis of the jaw due to drug (M87.10) with eight years of data (through 2023); this code has only been included in one study in the U.S., and that was only through 2017.4 Our study provides an additional five years of availability of this code for analysis. Finally, within one of the two cohorts (the VHA cohort), we additionally estimated the sensitivity, specificity, and NPV of candidate ICD-9 and ICD-10 codes to provide a comprehensive assessment of their diagnostic performance for MRONJ ascertainment. To our knowledge, these additional performance metrics of either ICD-9 or ICD-10 codes have not previously been reported in a U.S. cohort.

To our knowledge, ours is the first report in the U.S. to examine the PPV of the ICD-10 code M87.10 (osteonecrosis due to drug, unspecified bone) for MRONJ. Our findings that M87.10 was the ICD-10 with the highest PPV of all the examined codes at both health systems is in agreement with a prior report from European claims data.5,6 However, in our cohorts, the PPV for MRONJ with this code was lower than for our European counterparts reporting a PPV of 0.83 (95% CI: 0.36–1.00) in Swedish5 and 1.00 (95% CI: 0.25–1.00) in Danish populations.6 The higher estimate of PPV in our study (0.67; 95% CI: 0.09,0.99) was from HP, which was predominantly female; these prior European reports were confined to women.5,6 Given the wide confidence intervals surrounding the PPV estimates for M87.10 in both our cohorts and the prior European studies, the observed differences may reflect imprecision rather than true differences in the underlying PPV. International studies evaluating World Health Organization (WHO) ICD-10 code K10.2 (corresponding to U.S. ICD-10-CM code M27.2) among antiresorptive users have reported higher PPV than observed for M27.2 in our study (PPV 0.74; 95% CI: 0.66–0.81).9 The lower PPV of M27.2 observed in our study compared with international reports for K10.2 may reflect differences in coding system structure, imperfect correspondence between WHO ICD-10 and U.S. ICD-10-CM code sets, and variation in coding practices across healthcare systems. Combining the ICD-10 codes for osteonecrosis of the jaw due to drugs (M87.180) and osteonecrosis due to drugs, unspecified bone (M87.10), did not appreciably improve PPV or sensitivity, suggesting that even algorithms incorporating the best-performing codes are unlikely to provide adequate accuracy for MRONJ ascertainment in U.S. administrative databases.

This study has several important strengths. We included both men and women and individuals of all races and ethnicities who had filled at least one prescription for an oral or IV bisphosphonate or denosumab for fracture prevention in the absence of cancer. Prior validations studies were more limited, with only one report restricting analyses to those with a filled prescription for a bisphosphonate for osteoporosis,9 and none specifically evaluating those prescribed denosumab. In the VHA cohort, 1150 patients had at least one filled prescription for denosumab, among whom one developed MRONJ. To our knowledge, this is the first ICD code validation study apply an NLP algorithm to ascertain MRONJ events from the EMR instead of manual record review. This approach enabled review of the largest number of EMRs reported to date for evaluation of PPVs across a broad range of candidate ICD codes for MRONJ, and also allowed reporting of sensitivity, specificity, and NPV of these codes as review was not restricted to those with claims for candidate ICD codes.

Within the VHA cohort, MRONJ ascertainment was restricted to Veterans with continuous and comprehensive VHA dental benefits. These Veterans either receive dental care without cost (eg, those with a 100% permanent and total disability rating or service-connected dental condition) or with reduced copays, making receipt of dental care outside the VHA less likely and thereby reducing the likelihood of missed MRONJ cases Supporting this, the VA reported that in 2020, 33% of eligible Veterans received more than 3.6 million dental procedures during 1.3 million visits.19 At HP, all individuals included in these analyses likewise had dental benefits. Inclusion of this second cohort from a private payer system-which, in contrast to the predominately male VHA cohort, was mostly female-allowed assessment of code performance across two distinct healthcare settings. Notably, both systems demonstrated similar relative PPV performance across ICD codes, with the same codes showing the highest and similar codes showing the lowest PPVs, strengthening confidence in our findings.

This study also has limitations. To start, this is an observational study, and as such is subject to residual confounding.20 Further, there is the potential for verification bias introduced by methodological differences in MRONJ ascertainment in the VHA and HP cohorts. At HP, because MRONJ ascertainment relied on manual medical record review and MRONJ events are rare, the number of EMRs that would have required review to identify a meaningful number of false negatives (ie, MRONJ cases without an ICD-9 or ICD-10 claim) was prohibitively large, therefore, we were unable to evaluate for the sensitivity, specificity, or NPV of ICD codes in the HP cohort as ICD-negative cases were not reviewed. Limited event counts for many ICD codes led to wide confidence intervals around PPV estimates in the VHA and HP, and sensitivity in the VHA, reducing estimate precision and limiting inference regarding the comparative performance of these codes.

Conclusion

In conclusion, identification of MRONJ using ICD claims remains challenging because of the low sensitivity and PPVs of available ICD codes, underscoring the urgent need for more reliable and specific operational definitions for MRONJ ascertainment in large population databases. Future efforts should focus on developing and validating hybrid ascertainment algorithms that integrate diagnosis codes with additional clinical information, such as dental procedure codes, dental records, and/or natural language processing of clinical notes. Alternatively, establishment of dedicated MRONJ registries or databases across multiple participating health care systems may provide a more reliable platform for epidemiologic study of this rare outcome.

Data Sharing Statement

Data underlying primary scientific publications from this study may be available to the public and other VA and non-VA researchers from the corresponding author (Laura D Carbone) with an appropriate data use agreement (DUA) under certain conditions and consistent with policies which prioritize protecting Veteran’s privacy and confidentiality to the fullest extent possible.

Funding

This work was supported by grant R01AG079118 from the National Institute on Aging (NIA) and the Office of the Director at the National Institutes of Health. The views expressed in this work are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs, the United States government, or the National Institutes of Health.

Disclosure

Dr Thomas Dodson reports personal fees from AO Foundation and AAOMS, outside the submitted work. Dr Joan Lo reports grants from National Institute on Aging, National Institutes of Health, outside the submitted work. The authors report no other conflicts of interest in this work.

References

1. Bansal H. Medication-related osteonecrosis of the jaw: an update. Natl J Maxillofac Surg. 2022;13(1):5–12. doi:10.4103/njms.NJMS_236_20

2. Ruggiero SL, Dodson TB, Aghaloo T, Carlson ER, Ward BB, Kademani D. American Association of Oral and Maxillofacial Surgeons’ position paper on medication-related osteonecrosis of the jaws-2022 update. J Oral Maxillofac Surg. 2022;80(5):920–943. doi:10.1016/j.joms.2022.02.008

3. Wright NC, Curtis JR, Arora T, et al. The validity of claims-based algorithms to identify serious hypersensitivity reactions and osteonecrosis of the jaw. PLoS One. 2015;10(7):e0131601. doi:10.1371/journal.pone.0131601

4. Stevens LAS, Spangler L, Yochum L, Ding Y, Wang FT. Assessment of a claims-based ICD-9 and ICD-10 medication-related osteonecrosis of the jaw algorithm in the United States. Clin Epidemiol. 2021;13:1019–1026. doi:10.2147/CLEP.S325080

5. Bergdahl J, Jarnbring F, Ehrenstein V, et al. Evaluation of an algorithm ascertaining cases of osteonecrosis of the jaw in the Swedish National Patient Register. Clin Epidemiol. 2013;5:1–7. doi:10.2147/clep.S37664

6. Gammelager H, Svaerke C, Noerholt SE, et al. Validity of an algorithm to identify osteonecrosis of the jaw in women with postmenopausal osteoporosis in the Danish National Registry of Patients. Clin Epidemiol. 2013;5:263–267. doi:10.2147/CLEP.S45226

7. Yang G, Williams R, Wang L, et al. Medication-related osteonecrosis of the jaw in cancer patients: result from the OneFlorida Clinical Research Consortium. J Bone Miner Res. 2022;37(12):2466–2471. doi:10.1002/jbmr.4708

8. Ehrenstein V, Gammelager H, Schiodt M, et al. Evaluation of an ICD-10 algorithm to detect osteonecrosis of the jaw among cancer patients in the Danish National Registry of Patients. Pharmacoepidemiol Drug Saf. 2015;24(7):693–700. doi:10.1002/pds.3786

9. Kim HY, Kim JW, Kim SJ, Lee SH, Lee HS. Uncertainty of current algorithm for bisphosphonate-related osteonecrosis of the jaw in population-based studies: a systematic review. J Bone Miner Res. 2017;32(3):584–591. doi:10.1002/jbmr.3017

10. Brunner C, Arvandi M, Marth C, et al. Incidence of medication-related osteonecrosis of the jaw in patients with breast cancer during a 20-year follow-up: a population-based multicenter retrospective study. J Clin Oncol. 2024:Jco2400171. doi:10.1200/jco.24.00171

11. Jurasic MM, Gibson G, Orner MB, Wehler CJ, Jones JA. Validation of a subjective caries risk assessment tool. J Dent. 2021;113:103748. doi:10.1016/j.jdent.2021.103748

12. Jurasic MM, Gibson G, Orner MB, Wehler CJ, Jones JA, Cabral HJ. Topical fluoride effectiveness in high caries risk adults. J Dent Res. 2022;101(8):898–904. doi:10.1177/00220345221081524

13. Jurasic MM, Gibson G, Wehler CJ, Orner MB, Jones JA. Caries prevalence and associations with medications and medical comorbidities. J Public Health Dent. 2019;79(1):34–43. doi:10.1111/jphd.12292

14. U.S. Department of Veterans Affairs VHA, Chief Business Office. Health Eligibility Center. Dental Benefits for Veterans; 2014.

15. Sanders B. Sanders, Brownley, and 11 senators introduce legislation to establish dental care pilot program for veterans with chronic health conditions; 2023. Available from: https://www.sanders.senate.gov/press-releases/news-sanders-brownley-and-11-senators-introduce-legislation-to-establish-dental-care-pilot-program-for-veterans-with-chronic-health-conditions/. Accessed June27, 2024.

16. HealthPartners. HealthPartners Webpage. Available from: www.healthpartners.com. Accessed February3, 2025.

17. Abrahamsen B. Bisphosphonate adverse effects, lessons from large databases. Curr Opin Rheumatol. 2010;22(4):404–409. doi:10.1097/BOR.0b013e32833ad677

18. Bodenreider O. The Unified Medical Language System (UMLS): integrating biomedical terminology. Nucleic Acids Res. 2004;32(Database issue):D267–70. doi:10.1093/nar/gkh061

19. Affairs UDoVs. FY 2022 Budget Submission: Medical Programs and Information Technology Programs; 2021. Available from: https://www.va.gov/budget/docs/summar/fy2022VAbudgetVolumeIImedicalProgramsAndinformationTechnology.pdf. Accessed May13, 2026.

20. Psaty BM, Koepsell TD, Lin D, et al. Assessment and control for confounding by indication in observational studies. J Am Geriatr Soc. 1999;47(6):749–754. doi:10.1111/j.1532-5415.1999.tb01603.x