ORIGINAL RESEARCH

Ultrasound-Based Nomogramfor Distinguishing Malignant Tumorsfrom Nodular Sclerosing Adenosesin Solid Breast LesionsTing Liang, MD, Shuzhen Cong, MD, Zongjian Yi, MSc, Juanjuan Liu, MD, Chunwang Huang, MD,Junhui Shen, MD, Shufang Pei, MD, Gaowen Chen, MD, Zaiyi Liu, MD

Objectives—Nodular sclerosing adenoses (NSAs) and malignant tumors (MTs)may coexist and are often classified into the same Breast Imaging Reporting andData System (BI-RADS) category. We aimed to build and validate anultrasound-based nomogram to distinguish MT from NSA for building a precisesequence of biopsies.

Materials and Methods—The training cohort included 156 patients (156 masses)with NSA or MT at one study institution. We used best subset regression todetermine the predictors for building a nomogram from ultrasonic characteristicsand patients’ age. Model performance and clinical utility were evaluated usingBrier score, concordance (C)-index, calibration curve, and decision curve analy-sis. The independent validation cohort consisted of 162 patients (162 masses)from a separate institution.

Results—Through best subset regression, we selected 6 predictors to developnomogram: age, calcification, echogenic rim, vascularity distribution, tumor size, andthickness of breast parenchyma. Brier score and C-index of the nomogram in thetraining cohort were 0.068 and 0.967 (95% confidence interval [CI]: 0.941–0.993),respectively. In addition, calibration curve demonstrated good agreement betweenprediction and pathological result. In the validation cohort, the nomogram stillobtained a favorable C-index score of 0.951 (95% CI: 0.919–0.983) and fine calibra-tion. Decision curve analysis showed that the model was clinically useful.

Conclusions—If multiple NSA and MT masses are present in the same patientand are classified into the same BI-RADS category, our nomogram can be usedas a supplement to the BI-RADS category for accurate biopsy of the mass mostlikely to be MT.

Key Words—ultrasonography; nomogram; breast; cancer; nodular sclerosingadenosis

Introduction

B reast nodular sclerosing adenosis (NSA) is a commonproliferative lesion without atypia.1 Some NSAs can mimicmalignant tumors (MTs) for both clinical palpation and

presenting as solid masses on imaging, but they undergo differentclinical treatments.2 Thus, NSA should be thoroughly evaluated bymedical imaging to distinguish it from MT.

Received September 19, 2020, from the The Sec-ond School of Clinical Medicine, Southern Medi-cal University, No.253 Gongye Middle Avenue,Guangzhou, Guangdong, People's Republic ofChina (T.L., G.C., Z.L.); Department of Ultra-sound, Affiliated Hospital of Guangdong Medi-cal University, No.57 People's Avenue South,Zhanjiang, Guangdong, People's Republic ofChina (T.L.); Department of Ultrasound,Guangdong Provincial People's Hospital,Guangdong Academy of Medical Sciences,No.106 Zhongshan Er Road, Guangzhou,Guangdong, 510080, People's Republic of China(S.C., J.L., C.H., S.P.); Department of Radiol-ogy, Guangdong Provincial People's Hospital,Guangdong Academy of Medical Sciences,No.106 Zhongshan Er Road, Guangzhou,Guangdong, 510080, People's Republic of China(Z.Y., Z.L.); and Department of Anesthesiology,Affiliated Hospital of Guangdong Medical Uni-versity, No.57 People's Avenue South,Zhanjiang, Guangdong, People's Republic ofChina (J.S.). Manuscript accepted for publica-tion December 14, 2020.

Address correspondence to Zaiyi Liu,No.106 Zhongshan Er Road, Guangzhou510080, Guangdong, People’s Republic of China.

E-mail: zyliu@163.com

AbbreviationsACR BI-RADS, American College of Radiol-ogy Breast Imaging Reporting and Data Sys-tem; BEP, background echotexture ofparenchyma; BIC, Bayes information crite-rion; C-index, concordance index; DCA,decision curve analysis; ICCs, intraclass cor-relation coefficients; MRI, magnetic reso-nance imaging; MT, malignant tumor; NSA,nodular sclerosing adenosis; RPF,anteroposterior thickness ratio of breastparenchyma to mammary fat; RPT,anteroposterior thickness ratio of breastparenchyma to tissue before pectoralis fascia;TBP, anteroposterior thickness of breastparenchyma; US, ultrasound

doi:10.1002/jum.15612

© 2021 American Institute of Ultrasound in Medicine | J Ultrasound Med 2021; 9999:1–12 | 0278-4297 | www.aium.org

Table 1. The Details of Age and Ultrasonic Characteristic in the Training and Validation Cohorts

Training Cohort

P Value

Validation Cohort

P ValueNSA MT NSA MT

(n = 79) (n = 77) (n = 82) (n = 80)

Age, mean � SD(year) 38.063 � 8.67 50.480 � 10.214 0.0000 40.573 � 7.999 50.238 � 12.373 0.0000>50 6 (8%) 32 (42%) 9 (11%) 35 (44%)41--50 24 (30%) 32 (42%) 35 (24%) 24 (30%)<41 49 (62%) 13 (16%) 38 (21%) 21 (26%)

Echo Pattern 0.3110 0.1420Hypoechoic 79 (100%) 76 (99%) 80 (98%) 74 (93%)Isoechoic 0 0 0 2 (2%)Hyperechoic 0 0 1 (1%) 1 (1%)Complex cystic and solid 0.0000 1 (1%) 1 (1%) 3 (4%)

Shape 0.0000 0.0000Oval 29 (37%) 4 (5%) 45 (55%) 1 (1%)Round 0 0 2 (2%) 1 (1%)Irregular 50 (63%) 73 (95%) 35 (43%) 78 (98%)

Margin 0.0000 0.0000Circumscribed 28 (35%) 3 (4%) 46 (56%) 2 (3%)Not circumscribed 51 (65%) 74 (96%) 36 (44%) 78 (97%)

Orientation 0.0000 0.0000Parallel 78 (99%) 61 (79%) 76 (93%) 57 (71%)Not parallel 1 (1%) 16 (21%) 6 (7%) 23 (29%)

Posterior Feature 0.1380 0.0230No posterior feature 60 (76%) 51 (66%) 61 (74%) 47 (59%)Enhancement sound 7 (9%) 7 (9%) 8 (10%) 10 (13%)Shadowing 9 (11%) 12 (16%) 9 (11%) 12 (15%)Combined pattern 3 (4%) 7 (9%) 4 (5%) 11 (14%)

Calcification 0.0000 0.0000In a mass 19 (24%) 51 (66%) 8 (10%) 60 (75%)None 60 (76%) 26 (34%) 74 (90%) 20 (25%)

Echogenic Rim 0.0000 0.0000None 75 (95%) 29 (38%) 81 (99%) 44 (55%)Enhanced 4 (5%) 48 (62%) 1 (1%) 36 (45%)

Vascularity Distribution 0.0000 0.0000Absent 39 (48%) 10 (13%) 58 (71%) 11 (14%)Vessels in rim 9 (11%) 2 (3%) 7 (9%) 3 (4%)Internal 31 (39%) 65 (84%) 17 (21%) 66 (83%)

Vascularity Grade 0.0000 0.0000Grade I 39 (49%) 10 (13%) 58 (71%) 11 (14%)Grade II 7 (9%) 5 (6%) 1 (1%) 4 (5%)Grade III 31 (39%) 39 (50%) 21 (26%) 47 (59%)Grade IV 2 (3%) 23 (30%) 2 (2%) 18 (23%)

Tumor Size (cm)mean � SD 1.357 � 0.600 2.055 � 0.865 0.0000 1.205 � 0.600 2.333 � 1.147 0.0000<0.5 0 0 2 (2%) 00.5--2 71 (90%) 45 (58%) 74 (90%) 34 (42%)>2 8 (10%) 32 (42%) 6 (7%) 46 (58)

TBP (cm) 0.4650mean � SD 0.708 � 0.222 0.819 � 0.304 0.0100 0.749 � 0.277 0.785 � 0.348RPTmean � SD 0.521 � 0.177 0.526 � 0.320 0.9126 0.506 � 0.188 0.420 � 0.166 0.0020

RPFmean � SD 1.290 � 0.638 1.080 � 0.637 0.0410 1.362 � 0.766 0.975 � 0.513 0.0000

BEP 0.0030 0.0010Homogenous 46 (58%) 62 (81%) 54 (66%) 70 (88%)Heterogenous 33 (42%) 15 (19%) 28 (34%) 10 (12%)

(Continues)

Liang et al—Model for malignant tumor and sclerosing adenosis

2 J Ultrasound Med 2021; 9999:1–12

Mammography, magnetic resonance imaging(MRI), and ultrasound (US) are the main imaging tech-niques for evaluating breast masses and preliminarilyestimating their histological properties. Mammographycannot effectively detect a mass in dense breast tissue.3

MRI is expensive and tends to have a high false-positiverate for MT.4 In contrast, US does not use ionizing radi-ation and is affordable and widely available. Therefore,US is the preferred imaging technique for diagnosis ofbreast masses, especially in eastern countries.5

Previous ultrasonic studies have suggested thatthe application of some ultrasonic techniques, includ-ing grayscale, elastography and color Doppler, is ben-eficial for differentiating malignant from benignmasses.6–8 However, both malignant and benignlesions may have some features in common and maydiffer only in degree or intensity. Consequently, inte-gration of multiple ultrasonic features to improve theexamination performance is paramount. The Ameri-can College of Radiology Breast Imaging Reportingand Data System (ACR BI-RADS) lexicon is the mostroutinely used evaluation system. By combining allthe tumor-related features, this lexicon provides theprobability of malignancy of a lesion and determinesthe subsequent clinical management. Unfortunately,NSAs can mimic MTs because they both possessirregular, hypoechoic characteristics on imaging andare usually classified into the same BI-RADS cate-gory.9,10 Therefore, in many cases, NSAs are diag-nosed as MTs by radiologists. Such as in ourinstitution, we tried to differentiate NSAs from MTs,but we often failed and classified them as BI-RADScategory 3–4 (more than half were category 4),followed by biopsies (Training Cohort in Table 1).

Pathological examination has revealed that MTsand NSAs can exist in the same breast tissue.11 In

practice, it is common that multiple masses of NSAsand MTs coexist in the same patient and they are notidentified precisely by the conventional BI-RADS cate-gory. Vacuum-assisted breast biopsy (VABB) hasbecome a major tool for evaluating breast lesions dueto its effectiveness and high accuracy of nonoperativediagnosis.12 However, complications that may be cau-sed by VABB include bleeding, pain, light-headednessand bruising, with bleeding or hematoma being themost common.13 When these multiple masses requiredbiopsy, obtaining biopsy samples on all the masses atonce may increase the risk of the above complicationsand the medical expense. In this particular case of mul-tiple masses, it is essential to develop a separate toolfor distinguishing MTs from NSAs, enabling the accu-rate selection of the most suspicious mass for eachbiopsy for building a precise sequence of biopsies.

Nomograms, created by combining selected tar-get variables, can be used to predict breast tumordiagnosis and prognosis for individual manage-ment.14,15 To the best of our knowledge, there is nonomogram based on US features to distinguish MTsand NSAs. In this study, we created and verified anomogram for the differentiation of MTs and NSAsbased on patients’ age and US findings to enableaccurate selection of the most suspicious mass foreach biopsy so as to build a precise sequence of biop-sies and improve our understanding of ultrasonicdiagnosis of MTs and NSAs.

Materials and Methods

Participants and study designEthical approval was obtained by Research EthicsCommittee of Guangdong Provincial People’s

Table 1. Continued

Training Cohort

P Value

Validation Cohort

P ValueNSA MT NSA MT

(n = 79) (n = 77) (n = 82) (n = 80)

BI-RADS Category 0.00003 28 (36%) 0 51 (62%) 1 (1%) 0.00004 51 (64%) 50 (65%) 30 (37%) 32 (40%)5 0 27 (35%) 1 (1%) 47 (59%)

Notes: NSA, nodular sclerosing adenosis; MT, malignant tumor; BEP, the background echotexture of nearby parenchyma; TBP, thickness ofbreast parenchyma; RPT, thickness ratio of breast parenchyma to tissue before pectoralis fascia; RPF, thickness ratio of breast parenchymato mammary fat.

Liang et al—Model for malignant tumor and sclerosing adenosis

J Ultrasound Med 2021; 9999:1–12 3

Hospital for this retrospective study, and the informedconsent requirement was waived. We obtained detailsof the histological characteristics of breast massesfrom pathology reports. The training cohort wasformed at one study institution, whereas the valida-tion cohort consisted of patients from a separate insti-tution. From January 1 to May 30, 2019, 156 femalepatients (156 masses) and 162 patients (162 masses)who met the inclusion and exclusion criteria formedthe training cohort (age range: 22–80 years; meanage: 44.19 � 11.30 years) from one institution andthe independent validation cohort (age range: 24–83years; mean age: 45.35 � 11.44 years) from a sepa-rate institution, respectively. All patients underwentUS-guided VABB or surgical excision. The masseswere marked on the skin, and their ultrasonic imageswere retained in the electronic system.

The inclusion criteria for patients in both cohortswere: (1) masses were clinically suspicious of breast

cancer and underwent biopsy or surgical excision;(2) breast masses were identified as NSAs or MTsthrough histological examination; and (3) solid breastmasses were defined as having little or no anechoiccomponents on the basis of ultrasonic images. Theexclusion criteria were: (1) lesions were metastatictumors; (2) patients had been exposed to systemichormone therapy or adjuvant chemotherapy; and(3) masses with ultrasonic images that have colordeviations, noises and no glands adjacent to themasses (Appendix 1).

The work flow of our study is shown in Figure 1.

Ultrasonic image acquisition and interpretation,agreement of ultrasonic featuresUltrasonic images were captured using a 14-MHz lin-ear transducer (Toshiba Aplio 500) and a 13-MHz lin-ear transducer (Hitachi Preirus). Images of the masseswere acquired in a standard manner and contained at

Figure 1. Work flow of important steps in our study. ICCs, intraclass correlation coefficients; C-index, concordance index.

Liang et al—Model for malignant tumor and sclerosing adenosis

4 J Ultrasound Med 2021; 9999:1–12

least 2 orthogonal planes (the radial and antiradialplanes or transverse and longitudinal planes). Asdirected by the ACR BI-RADS fifth edition classifica-tion scheme and a previous report,16 two breast radiol-ogists (Reader one with 10-years’ experience andReader two with 5-years’ experience) each read all theimages retrospectively and kept strict records of14 ultrasonic features about the masses as follows:

echo pattern, shape, orientation, margin, posterior fea-tures, tumor size, calcifications, echogenic rim, vascu-larity distribution, vascularity grade, backgroundechotexture of parenchyma (BEP), anteroposteriorthickness of breast parenchyma (TBP), anteroposteriorthickness ratio of breast parenchyma to tissue beforepectoralis fascia (RPT), and anteroposterior thicknessratio of breast parenchyma to mammary fat (RPF).

Figure 2. Optimal variables were determined using the best subset method. (A) A separate least squares regression was fitted for each pos-sible model of the candidate predictors. The best model was determined according to the minimal BIC. The red mark of the curve repre-sents the minimal BIC, containing 6 variables. (B) The red mark of the plot represents the 6 optimal variables: age, calcification, echogenicrim, vascularity distribution, tumor size, and thickness of breast parenchyma. BIC, Bayes information criterion.

Table 2. Six Predictors in Model

β Odd Ratio (95%CI) P value

Age −0.161 0.145 (0.060–0.352) <0.0001Calcification 0.422 3.546 (1.053–11.943) 0.0411Echogenic Rim −3.27 0.038 (0.008–0.184) <0.0001Vascularity Distribution −1.06 0.120 (0.029–0.504) 0.0038Tumor Size −1.373 0.253 (0.090–0.714) 0.0094TBP −3.035 0.402 (0.176–0.918) 0.0305

Notes: β is the regression coefficient. Thickness of breast parenchyma (TBP) is the diameter of breast parenchyma adjacent to thebreast mass.

Liang et al—Model for malignant tumor and sclerosing adenosis

J Ultrasound Med 2021; 9999:1–12 5

RPT and RPF were the adjusted parameters of TBPand were obtained after TBP was corrected by thick-ness of tissue before pectoralis fascia and thickness offat, respectively. Detailed descriptions of the featuresare presented in Appendix 2. Readers one and twowere blind to the histological outcome but not topatients’ ages. Interobserver and intraobserver agree-ments of the 14 ultrasonic features were analyzed. Toevaluate intraobserver agreement, Reader one repeatedhis analysis of ultrasonic features 1 month followingthe initial analysis.

Statistical analysisStatistical analysis was conducted with R softwarewith RStudio version 1.1.463 (RStudio) and SPSSversion 22.0 (IBM Corp.). The packages in R used inthis study are listed in Appendix 3. The statistical sig-nificance levels were 2-sided, and P < 0.05 wasdeemed to be statistically significant.

Demographic comparison of the NSA and MT groupsand comparison of the training and validation cohortsIn the training and validation cohorts, the differencesin ultrasonic and clinical features were evaluatedbetween the NSA and MT groups. We assesseddifferences in clinicopathological and ultrasonic

characteristics and age between the 2 cohorts. Con-tinuous variables were compared using the Mann–Whitney U test or t test. Categorical variables werecompared using Fisher’s exact test.

Agreement and selection of ultrasonic featuresKappa analysis was used to assess categorical vari-ables, and the intraclass correlation coefficients(ICCs) were calculated to estimate continuous vari-ables. The criteria of κ values and ICCs are shown inAppendix 4. The ultrasonic features with good agree-ment (κ > 0.60, and ICC > 0.75) were selected asthe candidate features for the development of thenomogram. For the records of each candidate ultra-sonic feature, any disagreements between the2 readers were resolved by final consensus followingdiscussion.

Development of an individualized predictionnomogramBest subset regression was used to select the deter-mined predictors from candidate ultrasonic featuresplus patients’ age for multivariable logistic regressionanalysis. Best subset regression selects the best modelfrom all possible subsets according to somegoodness-of-fit criteria.17 The method was conducted

Figure 3. Ultrasound-based nomogram for preoperative distinction of nodular adenosis from malignant tumor. We developed anultrasound-based nomogram in the training cohort, including patients’ age, calcification, echogenic rim, vascularity distribution, tumor size(cm), and thickness of breast parenchyma (cm).

Liang et al—Model for malignant tumor and sclerosing adenosis

6 J Ultrasound Med 2021; 9999:1–12

in 2 steps. In Step 1, we fitted a separate least squaresregression for each possible model of the candidatepredictors. In Step 2, the best model was identifiedaccording to the minimal Bayes information criterion.Subsequently, their β coefficients, odds ratios andP values were calculated. To predict the individualprobability of MT in a quantitative manner, we devel-oped a US-based nomogram based on the multivari-able logistic analysis in the training cohort.

Performance of the nomogram in the training cohortThe Brier score can address the statistical consistencybetween predicted risks and the actual outcomes(range: 0–0.25). A lower score indicates the higheroverall accuracy of the model.18,19 The calibration ofthe nomogram was evaluated using the calibrationcurve, which was a graphic representation of the rela-tionship between the observed outcome frequenciesand the predicted probabilities, with a bootstrappedsample of the study group. In a well-calibrated model,the predictions should fall on a 45-degree diagonalline.20 Concordance (C)-index was measured to eval-uate the performance of the model. A C-index of 0.5indicates the absence of discrimination, whereas a C-index of 1.0 indicates perfect separation of patientswith different outcomes.20 Although C-index is equiv-alent to area under the receiver operating characteris-tic (ROC) curve (AUC), C-index may be calculatedwithout specificity, sensitivity, positive predictivevalue, or negative predictive value.21

Validation of the US-based nomogram in thevalidation cohortThe logistic regression formula formed in the trainingcohort was applied to all patients in the validationcohort, with total points for each patient calculated.In the validation cohort, logistic regression was con-ducted by using the total points as a factor. Finally,the C-index and calibration curve were presentedaccording to the regression analysis.

Clinical useDecision curve analysis was used to confirm the clini-cal usefulness of the US-based nomogram by calculat-ing the net benefit at different threshold probabilitiesin the validation cohort.

Results

Clinical and ultrasonic characteristicsAge and clinicopathological and ultrasonic features ofboth the training and validation cohorts are presented

Figure 4. Calibration curves of the ultrasound-based nomogram.(A) Calibration curve was based on the nomogram in the trainingcohort. (B) Calibration curve was based on the nomogram in thevalidation cohort. Calibration curves describe the calibration of themodel based on the agreement between the predicted probabilityof breast NSA and observed results of NSA. The y axis expressesthe actual NSA rate. The x axis represents the predicted NSA prob-ability. The diagonal dotted line represents a perfect prediction byan ideal model. The dashed line represents the performance of thenomogram, of which a closer fit to the diagonal dotted line repre-sents a better prediction. NSA, nodular sclerosing adenosis.

Liang et al—Model for malignant tumor and sclerosing adenosis

J Ultrasound Med 2021; 9999:1–12 7

in Table 1. The proportion of NSAs was 50.6%(79/156) and 51% (82/162) in the training and vali-dation cohorts, respectively. There was no differencein clinicopathological characteristic between the2 cohorts (P = 0.997). In the training cohort, among14 ultrasonic features plus age, there were significantdifferences between the NSA and MT groups in12 variables (P range: 0.000–0.041), except for echopattern, posterior feature and RPT. In the validationcohort, there were significant differences between theNSA and MT groups in 13 variables (P range:0.000–0.002), except for echo pattern and TBP. In allthe masses, there were significant differences betweenthe training and validation cohorts in echo pattern,shape, margin, and echo rim (P range: 0.015–0.044),but there was no difference in the other characteris-tics (P > 0.05).

Ultrasonic features selectionAfter evaluation of the 14 ultrasonic features by agree-ment analyses, the posterior features and BEP wereexcluded due to poor agreement (κ < 0.6). The κvalues and ICCs were 0.61–0.83 and > 0.80 in theother 12 features (echo pattern, shape, orientation,margin, tumor size, calcifications, echogenic rim, vas-cularity, vascularity grades, TBP, RPT, and RPF),respectively. Thus, the above 12 satisfactory reliableultrasonic features were the candidate variables fordevelopment of the nomogram.

Development of nomogramBest subset regression determined 6 predictors for dis-tinguishing MT from NSA, including age, calcification,echogenic rim, vascularity distribution, tumor size, andTBP, from the 12 candidate ultrasonic features plus

Figure 5. Decision curve analysis for the nomogram in the validation cohort. The y axis measures the net benefit. The net benefit is decidedby calculating the difference between the expected benefit and the expected harm (Net benefit = true positive rate – [false positive rate ×weighting factor], weighting factor = threshold probability/ [1 - threshold probability]). The grey line represents the assumption that alllesions were NSA (the treat-all scheme). The black line represents the assumption that all lesions were malignant (the treat-none scheme). Ifthe threshold probability was more than 5%, using the nomogram to predict NSA added more benefit than either the treat-all scheme or thetreat none scheme. NSA, nodular sclerosing adenosis.

Liang et al—Model for malignant tumor and sclerosing adenosis

8 J Ultrasound Med 2021; 9999:1–12

patients’ age (Figure 2). The β coefficient, odds ratioand P value of each predictor are presented in Table 2.The predictive nomogram that combined all 6 predic-tors in the training cohort is shown in Figure 3.

Performance of US-based nomogram in the trainingcohortThe Brier score of 0.068 suggested high accuracy ofthe US-based nomogram. The C-index for the nomo-gram was 0.967 (95% confidence interval [CI]:0.941–0.993), which indicated a perfect performanceof discrimination for NSA. The calibration curve forthe probability of NSA showed good concordancebetween the predictive model and pathological resultin the training cohort (Figure 4A).

Validation of the nomogramSubsequently, in the validation cohort, the C-index ofthe nomogram for predicting NSA was 0.951 (95%CI: 0.919–0.983), and the fine calibration curve dem-onstrated favorable agreement between predictionand pathological result (Figure 4B).

Clinical useThe decision curve analysis was plotted for the nomo-gram and is presented in Figure 5. The decision curvedemonstrated that if the threshold probability of apatient or doctor was >5%, using the US-based nomo-gram to predict MT was more beneficial than eitherthe treat-all-patients scheme or the treat-none scheme.Within this range, net benefit was comparable, withseveral overlaps, on the basis of the US nomogram.According to the nomogram, the lower the score, thehigher the malignancy of the mass, and the biopsyshould be given priority. The exact scores of thenomogram were calculated by the following formula.

Score =Calcification + Echogenic Rim

+Vascularity Distribution−14:2454×Tumor Sizeð Þ− 31:4893×TBPð Þ− 1:6667×Ageð Þ + 274:6618

Discussion

We constructed and validated an individual US-basednomogram for solid breast masses to distinguish NSA

and MT. The nomogram incorporates 6 variables:age, tumor size, calcification, echogenic rim, vascular-ity distribution, and TBP. Our study found that thenomogram facilitated discrimination between NSAand MT for individual management. Satisfactory per-formance was demonstrated in this nomogram and afavorable C-index and calibration were shown. Thelower Brier score affirmed the high overall accuracyof the model.

Between the training and validation cohorts,there were significant differences in 4 ultrasonic fea-tures. This may be due to the fact that the 2 data setswere formed by 2 separated institutions, and therewere at least 2 differences between the institutions.First, although the ultrasound machines used in thisresearch in the 2 institutions were of the same brand,there were certain differences in the parameter adjust-ments of the machines. Secondly, although the imagesused were all saved by doctors according to the guide-line, there were certain differences in the examinationmethods of the handheld probes used by doctorstrained by different institutions, resulting in some dif-ferences in the saved images. Therefore, the differ-ences between the 2 data sets were in line with theactual clinical reality. Our nomogram constructed bythe training cohort has been applied to the validationcohort and obtained good agreement between predic-tion and pathological results, indicating that thisnomogram can be applied to different institutionsand has a wide range of regional adaptability.

Medical nomograms use biological and clinicalvariables, such as tumor grade and patient age, todetermine a statistical prognostic model that gener-ates a probability of a clinical event, such as cancerrecurrence or death, for a particular individual.22 Bothobjective variables, such as age, serum alpha fetopro-tein, and subjective variables, such as tumor capsulestatus on MRI images, calcification morphology onmammography images, can be used together to con-struct a nomogram and apply it clinically.23,24 Differ-ent ages are prone to different diseases. Patients withNSA are younger than MT,25 and increasing age hasa close relationship with higher incidence of breastcancer worldwide.26,27 Therefore, although age is notincluded in the ACR US BI-RADS for risk stratifica-tion, in addition to ultrasound features, we also takeage as a candidate variable, and the best subset regres-sion finally determined age is one of the 6 predictors

Liang et al—Model for malignant tumor and sclerosing adenosis

J Ultrasound Med 2021; 9999:1–12 9

to distinguish MT from NSA. Our results showedthat the relationship between NSA and age is as closeas the relationship between MT and age. The olderthe patient, the more likely the mass is MT, and theyounger the patient, the more likely the mass is NSA.Thus, although the objective variable age is includedin our nomogram, it did not influence or skew ournomogram created subjectively.

Our nomogram is different from the traditionalBI-RADS categorization scheme. Previous studiescould not differentiate NSA and MT through ultra-sonic features and BI-RADS category.28,29 One studyhas shown that the US BI-RADS atlas together withbreast elastography may be a powerful tool to identifyNSA and differentiate it from breast cancer.25 How-ever, the predictors, such as elasticity score, the calci-fication and vascularity, have not been independentlytested by external institution.

Consistent with previous studies, calcificationand vascularity distribution, as ultrasound featuresthat independently predict NSA and MT, have beenincorporated into our nomogram.24,30,31 Specifically,more intratumor calcifications were found and moreinternal MT patterns were shown in the masses.

Generally, tumor size is not viewed as a risk fac-tor in the US BI-RADS. However, according to thebiological behaviour of the tumor, the more rapidlythat tumor size increases, the greater the likelihood ofmalignancy. Benign tumor size can remain stable formany years or increase slowly. Therefore, it is not sur-prising that tumor size was selected as an indepen-dent risk factor in the nomogram. Currently, theechogenic rim is recognized as an adjunctive featurein the BI-RADS, and it was included as a predictor inour nomogram. The echogenic rim was associatedwith malignancy in a previous study 6 because of his-tological lymphatic invasion of surrounding breast tis-sue.6,32 Enhancement of echogenic rim is morecommon in MT and less common in NSA.

The interaction between tumor cells and hoststromal cells is related to breast cancer, and theparenchymal stromal cells play a key role in the for-mation and development of breast cancer as a matrixto promote tumor growth.33,34 In recent years, breastbackground texture has emerged as an importantindex to differentiate lesions on MRI or mammogra-phy.34,35 As far as we know, our study is the first pro-posal of morphological quantitative indices for

identification of NSA and MT. TBP was selected asan independent predictor for our nomogram. Wefound that thicker breast parenchyma has a closerrelationship with MT, and other breast parenchymafeatures were not associated with MT. In our study,the robustness of BEP was low because of radiolo-gists’ strong subjectivity. The adjusted parametersRPT and RPF were not selected as suitable predictorsfor the nomogram because body mass index (BMI)greatly influences fat thickness. TBP is only an explor-atory index of the grayscale imaging level, and it can-not provide strong evidence for differentiating MTand NSA (P = 0.465, in the validation cohort). As apotential ultrasonic feature, breast backgroundechotexture should be studied more deeply toimprove the predictive strength of our tool.

There are two reasons why traditional commonpredictors, including echo pattern, shape, margin, ori-entation, and vascularity grade, were not selected forthe model. Firstly, unlike other benign lesions, NSAultrasonic features often manifest as hypoechoic,irregular, ill-circumscribed, and nonparallel, similar toMT, and NSAs appear histologically as ill-definedmasses of firm fibrous tissue surrounded by softerlobulated areas.36 Secondly, as a semi-quantitative fac-tor, vascularity grade is highly dependent on thecolor-Doppler capability of the US instrument. Webelieve that, by excluding these features, the diagnos-tic accuracy in our model will be improved.

Our study had several limitations. Firstly, thesample size was small, so a large data set is needed tovalidate the robustness of our model. Secondly, thestudy did not integrate clinical risk factors other thanage into development of the nomogram due to theinherent incompleteness of retrospective study data.Prospective studies with complete data sets (ie, BMI,serological examination, and family history) should beconducted. Finally, our feature estimation relied on asubjective analysis with inevitable bias. Therefore, it isnecessary to apply objective parameters to improvethis nomogram (ie, ultrasonic radiomics and contrastenhancement).

Conclusions

Our proposed nomogram represents an individualapproach differentiating between NSA and MT in

Liang et al—Model for malignant tumor and sclerosing adenosis

10 J Ultrasound Med 2021; 9999:1–12

solid breast masses. The nomogram demonstratedadequate effectiveness for this task. For multipleNSAs and MTs coexisting in the same patient, classi-fied in the same BI-RADS category, this nomogramcan be used as a supplement to BI-RADS to performaccurate biopsy of the mass most likely to beMT. Therefore, our nomogram can build a precisesequence of biopsies, avoid the complications causedby multiple biopsies at once and delayed diagnosisbecause of the low accuracy of selecting mass eachtime, and reduce the medical costs. The nomogrammay allow radiologists to understand the differencebetween NSAs and MTs and make an accurate diag-nosis. Nevertheless, we should recognize that thenomogram is not suitable for screening.

Acknowledgments

This study was approved by the institutional researchboard of Guangdong Provincial People’s Hospital,Guangdong Academy of Medical Sciences.

Funding

We thank all the study participants and the medicalteams. This work was supported by the NationalKey R&D Program of China (2017 YFC1309100),National Natural Scientific Foundation of China (No.81271569), and the Medical Science and TechnologyFoundation of Guangdong Province(A2018348).

Conflict of Interest:

The authors declare no competing interests.

References

1. Visscher DW, Nassar A, Degnim AC, et al. Sclerosing adenosisand risk of breast cancer. Breast Cancer Res Treat 2014; 144:205–212.

2. Neal L, Sandhu NP, Hieken TJ, et al. Diagnosis and managementof benign, atypical, and indeterminate breast lesions detected oncore needle biopsy. Mayo Clin Proc 2014; 89:536–547.

3. Yap Y-S, Lu Y-S, Tamura K, et al. Insights into breast cancer inthe East vs the West: a review. JAMA Oncology 2019; 5:1489–1496.

4. Sardanelli F, Boetes C, Borisch B, et al. Magnetic resonance imag-ing of the breast: recommendations from the EUSOMA workinggroup. Eur J Cancer 2010; 46:1296–1316.

5. Shen S, Zhou Y, Xu Y, et al. A multi-centre randomised trial com-paring ultrasound vs mammography for screening breast cancer inhigh-risk Chinese women. Br J Cancer 2015; 112:998–1004.

6. Durmus T, Stockel J, Slowinski T, Thomas A, Fischer T. Thehyperechoic zone around breast lesions - an indirect parameter ofmalignancy. Ultraschall Med 2014; 35:547–553.

7. Lee SH, Chung J, Choi HY, et al. Evaluation of screening US–detected breast masses by combined use of elastography and colordoppler US with B-mode US in women with dense breasts: a mul-ticenter prospective study. Radiology 2017; 285:660–669.

8. Stavros AT, Thickman D, Rapp CL, Dennis MA, Parker SH,Sisney GA. Solid breast nodules: use of sonography to distinguishbetween benign and malignant lesions. Radiology 1995; 196:123–134.

9. Kim YR, Kim HS, Kim H-W. Are irregular hypoechoic breastmasses on ultrasound always malignancies?: a pictorial essay.Korean J Radiol 2015; 16:1266–1275.

10. He P, Cui LG, Chen W, Yang RL. Subcategorization of ultrasono-graphic BI-RADS category 4: assessment of diagnostic accuracy indiagnosing breast lesions and influence of clinical factors on posi-tive predictive value. Ultrasound Med Biol 2019; 45:1253–1258.

11. Yu BH, Tang SX, Xu XL, et al. Breast carcinoma in sclerosingadenosis: a clinicopathological and immunophenotypical analysison 206 lesions. J Clin Pathol 2018; 71:546–553.

12. Fu S-M, Wang X-M, Yin C-Y, Song H. Effectiveness of hemostasiswith Foley catheter after vacuum-assisted breast biopsy. J ThoracDis 2015; 7:1213–1220.

13. Lin LLY, Gao Y, Lewin AA, Toth HK, Heller SL, Moy L.Overstated harms of breast cancer screening? A large outcomesanalysis of complications associated with 9-gauge stereotacticvacuum-assisted breast biopsy. Am J Roentgenol 2019; 212:925–932.

14. Luo WQ, Huang QX, Huang XW, Hu HT, Zeng FQ, Wang W.Predicting breast cancer in breast imaging reporting and data sys-tem (BI-RADS) ultrasound category 4 or 5 lesions: a nomogramcombining radiomics and BI-RADS. Sci Rep 2019; 9:11921.

15. Han L, Zhu Y, Liu Z, et al. Radiomic nomogram for prediction ofaxillary lymph node metastasis in breast cancer. Eur Radiol 2019;29:3820–3829.

16. Adler DD, Carson PL, Rubin JM, Quinn-Reid D. Doppler ultra-sound color flow imaging in the study of breast cancer: preliminaryfindings. Ultrasound Med Biol 1990; 16:553–559.

17. Zhang Z. Variable selection with stepwise and best subsetapproaches. Ann Transl Med 2016; 4:11.

Liang et al—Model for malignant tumor and sclerosing adenosis

J Ultrasound Med 2021; 9999:1–12 11

18. Nanayakkara S, Fogarty S, Tremeer M, et al. Characterising risk ofin-hospital mortality following cardiac arrest using machine learn-ing: a retrospective international registry study. PLoS Med 2018;15:e1002709.

19. Rufibach K. Use of Brier score to assess binary predictions. J ClinEpidemiol 2010; 63:938–939.

20. Kim Y, Margonis GA, Prescott JD, et al. Nomograms to pre-dict recurrence-free and overall survival after curative re-section of adrenocortical carcinoma. JAMA Surg 2016; 151:365–373.

21. Callegaro D, Miceli R, Bonvalot S, et al. Development and externalvalidation of two nomograms to predict overall survival and occur-rence of distant metastases in adults after surgical resection oflocalised soft-tissue sarcomas of the extremities: a retrospectiveanalysis. Lancet Oncol 2016; 17:671–680.

22. Balachandran VP, Gonen M, Smith JJ, DeMatteo RP. Nomogramsin oncology: more than meets the eye. Lancet Oncol 2015; 16:e173–e180.

23. Lei Z, Li J, Wu D, et al. Nomogram for preoperative estimation ofmicrovascular invasion risk in hepatitis B virus-related hepatocellu-lar carcinoma within the Milan criteria. JAMA Surg 2016; 151:356–363.

24. Wang H, Lai J, Li J, et al. Does establishing a preoperative nomo-gram including ultrasonographic findings help predict the likeli-hood of malignancy in patients with microcalcifications? CancerImaging 2019; 19:46.

25. Liu W, Li W, Li Z, et al. Ultrasound characteristics of sclerosingadenosis mimicking breast carcinoma. Breast Cancer Res Treat2020; 181:127–134.

26. Hu Y, Yang Y, Gu R, et al. Does patient age affect the PPV3 ofACR BI-RADS Ultrasound categories 4 and 5 in the diagnosticsetting? Eur Radiol 2018; 28:2492–2498.

27. Fan L, Strasser-Weippl K, Li J-J, et al. Breast cancer in China. Lan-cet Oncol 2014; 15:e279–e289.

28. Tan H, Zhang H, Lei Z, Fu F, Wang M. Radiological and clinicalfindings in sclerosing adenosis of the breast. Medicine 2019; 98:e17061–e17061.

29. Ozturk E, Yucesoy C, Onal B, Han U, Seker G, Hekimoglu B.Mammographic and ultrasonographic findings of different breastadenosis lesions. J Belg Soc Radiol 2015; 99:21–27.

30. Choi JS, Han B-K, Ko EY, Kim GR, Ko ES, Park KW. Compari-son of synthetic and digital mammography with digital breast tom-osynthesis or alone for the detection and classification ofmicrocalcifications. Eur Radiol 2019; 29:319–329.

31. Park AY, Kwon M, Woo OH, et al. A prospective study on thevalue of ultrasound microflow assessment to distinguish malignantfrom benign solid breast masses: association between ultrasoundparameters and histologic microvessel densities. Korean J Radiol2019; 20:759–772.

32. Tamaki K, Sasano H, Ishida T, et al. The correlation betweenultrasonographic findings and pathologic features in breast disor-ders. Jpn J Clin Oncol 2010; 40:905–912.

33. Fidler IJ, Poste G. The “seed and soil” hypothesis revisited. LancetOncol 2008; 9:808.

34. Li H, Mendel KR, Lan L, Sheth D, Giger ML. Digital mammogra-phy in breast cancer: additive value of radiomics of breast paren-chyma. Radiology 2019; 291:15–20.

35. Thompson CM, Mallawaarachchi I, Dwivedi DK, et al. The associ-ation of background parenchymal enhancement at breast MRIwith breast cancer: a systematic review and meta-analysis. Radiology2019; 292:552–561.

36. Taskin F, Koseoglu K, Unsal A, Erkus M, Ozbas S, Karaman C.Sclerosing adenosis of the breast: radiologic appearance and effi-ciency of core needle biopsy. Diagn Interv Radiol 2011; 17:311–316.

Liang et al—Model for malignant tumor and sclerosing adenosis

12 J Ultrasound Med 2021; 9999:1–12