Evaluation of Salivary Transcriptome Markers for the Early Detection of Oral Squamous Cell Cancer in a Prospective Blinded Trial

Jack L. Martin, MD; Neil Gottehrer, DDS; Harvey Zalesin, DDS; Paul T. Hoff, MD; Michael Shaw, PhD; James H.W. Clarkson, MD; Pam Haan, BSN; Mark Vartanian; Terry McLeod, BSN; and Stephen M. Swanick

Oral cancer is one of the most common cancers worldwide.^1-5 Ninety percent of oral cancers are squamous cell carcinomas.^1,5 The 5-year survival rate of oral squamous cell cancer (OSCC) is approximately 60% and has not improved significantly in several decades.^1,5 Survival is over 90% if detected in early stages, but, unfortunately, this disease is most often recognized in late stages.^1,5,6 Definitive diagnosis requires histologic examination, but determining the appropriate patients for biopsy is difficult, as evidenced by a high rate of negative biopsies and low rate of early detection.⁶

Given its ease of collection and painless nature of procurement, saliva is a convenient body fluid for biomarker evaluation.⁷ The salivary proteome and transcriptome have been well characterized, and a number of candidate salivary biomarkers have been suggested as aids for early detection of oral cancer.^7,8 Single biomarkers have limited diagnostic capability, and this has prompted efforts to determine multiple biomarker panels for improved discriminatory accuracy.⁹

Prior studies identified a salivary transcriptome marker footprint for OSCC.¹⁰ These markers were subsequently pre-validated in additional multiethnic cohorts.^11,12 The National Cancer Institute (NCI) – Early Detection Research Network (EDRN) also independently validated these mRNA markers, along with discriminatory proteins, in a large case-control study.⁹ Case-control studies can introduce bias related to factors such as: control subject selection, study in a setting different from the intended clinical application, and over-fitting of the models.^9,13

The primary purpose of the present study was to employ a prospective-specimen-collection, retrospective-blinded-evaluation (PRoBE) design study to develop new predictive mRNA models for the identification of OSCC in the intended-use population of patients with oral lesions suspicious for cancer. The authors also sought to validate a pre-specified multi-marker panel derived from prior NCI-EDRN case-control studies and to validate six previously identified individual salivary mRNA markers (IL1β, IL8, OAZ1, SAT1, S100P, and DUSP1) for OSCC.

Materials and Methods

Patient Selection

Patients were recruited through the Michigan State University Department of Surgery, the University of Michigan Department of Surgery, and the St. John Providence Health System in Detroit, Michigan. In addition to multicenter participation, patients were also enrolled from the primary care referral offices of these institutions to ensure that the study population was representative of general practice and included a broad spectrum of oral pathology. Enrollment began in May 2012 and was completed by May 2014. The study protocol was approved by the Institutional Review Boards at each institution. Patients provided written informed consent before entering the trial.

Inclusion criteria included age over 18 years and requirement for a clinically driven biopsy of an oral lesion suspicious for cancer. Exclusion criteria included previously diagnosed cancer other than non-melanoma skin cancer in the last 5 years or oral cancer in the last 2 years. Patients with cancer diagnosed earlier than this could be included if they were free of known disease and were not on current treatment for cancer. Also excluded were patients with prior history of hepatitis, human immunodeficiency virus infection, autoimmune disorders, or current immunosuppressive therapy. Biopsy specimens were evaluated in the clinical pathology departments of the respective institutions by pathologists with no knowledge of the biomarker results.

Saliva Collection and RNA Isolation

Saliva was collected as previously described¹⁰ prior to oral lesion biopsy and the determination of oral cancer or benign disease. Saliva was processed by previously described methods to obtain supernatant¹⁴ and was treated with SUPERase-IN™ RNase inhibitor (20 U/mL) (Life Technologies, www.lifetechnologies.com). Samples were frozen at -80°C prior to RNA isolation.¹⁴

RNA was isolated from 300 µL saliva supernatant using the MagMax™ Viral RNA Isolation Kit (Life Technologies) adapted on the KingFisher™ Flex 96 system (ThermoFisher Scientific, www.thermoscientific.com). RNA elutes were treated with DNase using the TURBO DNA-free™ Kit (Life Technologies) per manufacturer’s guidelines.

Pre-amplification and Quantitative Polymerase Chain Reaction (PCR)

Reverse transcription (RT) and pre-amplification were performed using the SuperScript® III RT-PCR System with Platinum® Taq DNA Polymerase (Life Technologies™). Briefly, 10 uL reactions were performed using 2 µL DNase-treated RNA elute and 300 nM gene-specific outer-nested primer sets for OSCC gene targets (IL8, IL1β, OAZ1, DUSP1, S100P, SAT). H3F3A mRNA, which was included in the NCI-EDRN study, was not measured because of the absence of a primer for the updated gene sequence. Five potential housekeeping genes (MT-ATP6, RPL30, RPL37A, RPL0, and RPS17) were also measured. Primer sequences were described previously.¹⁵ Resultant product cleanup was performed with ExoSAP-IT® (Affymetrix, www.affymetrix.com).

Quantitative PCR was set up and performed in triplicate for each sample. Briefly, each 10 µL reaction was prepared using 2X SsoAdvanced™ Universal SYBR® Green Supermix (Bio-Rad, www.bio-rad.com), 100 nM gene-specific inner-nested primer set for OSCC gene target or reference gene, and 2 µL diluted (1:10) cDNA pre-amplification product. Amplification was performed on the Bio-Rad CFX96 cycler using manufacturer’s suggested cycling conditions. For measurement of delta Ct values, the six pre-specified OSCC mRNAs and five housekeeping mRNAs were measured simultaneously on the QuantStudio™ 12K Flex Real-Time PCR System (Life Technologies™).

Statistical Methods

The Ct values determined by PCR of individual mRNA markers in cancer and control as well as in dysplasia and control were compared with Wilcoxon signed-rank test. Values are presented as medians and interquartile ranges.

The pre-specified multi-marker model for verification was a three-biomarker model generated from the patient-level data of the NCI-EDRN study. This model, Log(p/(1-p)) = 1.0434 + 0.4042*IL1β -0.8683*IL8 + 0.3054*S100P, reflects the best fit across the prior cohorts. A pre-specified statistical analysis plan for the verification of this model was generated without knowledge of the biopsy and biomarker results. The two co-primary endpoints for verification of this model are the specificity and the sum of sensitivity and specificity. The expected specificity from this pre-specified model is 0.55, and the expected sum of sensitivity and specificity is 0.5536 + 0.7949 = 1.3485. The pre-specified lower performance goal for specificity is 0.5536 - 0.125 = 0.4286. This endpoint was tested by forming the lower exact one-sided 97.5% confidence limit on the estimate. For the sum of sensitivity and specificity, the pre-specified performance goal requires that the lower one-sided 97.5% confidence limit must be greater than 1.

For new predictive model generation, the primary objective was to develop a model to differentiate invasive OSCC from benign disease. To develop new predictive multi-marker models, the delta Ct values were calculated by subtracting the geometric mean of the housekeeping gene Ct values. Markers were initially screened with univariate logistic regression. Those markers with a P value less than 0.2 were allowed to enter the competition for the final model. The method of analysis was by logistic regression. The models fitted were limited to three cancer genes and required at least two housekeeping genes. The final model was developed by backward elimination and verified with forward stepwise regression. The logistic model fits the following equation:

where p is the denoted test score that relates to the probability of cancer, a is the intercept, and b is the coefficient for Gene Ct values. A further robustness analysis was performed to determine the Akaike’s Information Criteria (AIC), a measure of the best fit (lowest AIC) among similar models. The receiver operating characteristic (ROC) curve was obtained for the best-fit model and its area was computed. This model was used to determine test scores to optimize sensitivity and specificity pairs.

Results

The study population is depicted in Table 1. There were 140 patients with benign disease and 28 patients with cancer; among the cancer patients, 24 had invasive cancer and four had carcinoma in situ. Two additional enrolled cancer patients had inadequate saliva after processing to allow for PCR. Importantly, women, who are often underrepresented in clinical trials, comprise a significant proportion of study subjects. As would be expected, the patients with cancer were significantly older than those with benign disease. Among patients with cancer, there were numerically higher percentages of smokers and current alcohol drinkers, but this did not reach statistical significance. The invasive carcinomas detected in this prospective cohort were predominantly T1/T2 (n = 17, 71%), with a minority of cases being T3/T4 (n = 7, 29%).

The pre-specified three-marker model developed from prior NCI-EDRN studies was applied to the present patient population. The test score with the highest sum of sensitivity and specificity at over 133% occurred between 0.45 and 0.50. All test scores between 0.40 and 0.60 have lower one-sided 97.5% confidence bounds above the value provided in the statistical analysis plan. Each of the sums of sensitivity and specificity has lower confidence limits that are greater than 1, except the 0.60 cutoff. Thus, the co-primary hypotheses are met for all cutoffs, except those below 0.40 or at 0.60 or above.

The quantitative PCR findings of the six individual mRNAs in controls and patients with invasive cancer or all cancer (including carcinoma in situ) are presented in Table 2 and Table 3 based on raw Ct values. All six candidate mRNA markers have significantly lower Ct values in patients with OSCC. These median values reflect an approximately twofold and nearly fourfold increase in mRNA concentration in patients with invasive cancer. These differences cannot be attributed to total RNA content, which was comparable in cancer and control (13.7 ± 15.7 vs. 12.0 ± 10.8 ng/ml, P = ns). Furthermore, the Ct values of all five housekeeping genes were comparable in cancer and controls, and based on the delta Ct values, all six cancer genes were significantly upregulated (P <0.001 for all). PCR results for patients with dysplasia are compared with those without dysplasia or cancer in Table 4. Values for five of the six mRNAs were also significantly different in patients with dysplasia compared with control.

Analysis of the 20 potential three-marker models for the dataset with invasive cancer alone resulted in a model having the lowest AIC that included genes DUSP1, SAT, and OAZ1, along with MT-ATP6 and RPL30 as housekeeping genes. The summary of the coefficients, AIC, and areas under the ROC curve corresponding to the model is provided in Table 5. The test value—which maximizes the sum of sensitivity and specificity from the model and corresponding sensitivity, specificity, and sum of sensitivity and specificity—is presented in Table 6. The ROC curve for the invasive carcinoma dataset is presented in Figure 1. This demonstrates that over 90% sensitivity is achieved with a specificity near 60%.

Discussion

The mortality of OSCC has not significantly improved in several decades.^1,5,6 This is to a large extent related to late diagnosis.^1-5 Nonetheless, the majority of patients presently do not report having an oral screening examination as recommended by the American Cancer Society and the American Dental Association.^1,16,17 A recent report questions the value of routine screening for oral cancer, but these data refer to screening by general practitioners in low-risk populations.¹⁶ Data in higher-risk populations suggest the potential to reduce mortality with routine screening for oral cancer.^18,19 The initial intended use of the presently reported salivary biomarkers is for decision-making on referral to a specialist for consideration of biopsy in patients at risk due to the detection of suspicious oral lesions rather than for generalized screening in low-risk populations. Several reports highlight the difficulty of the decision for biopsy based on the clinical exam alone.^1,20,21

Saliva is a readily available biofluid that is obtained painlessly, and its collection is easily incorporated into the workflow of dental and primary care offices.²² The salivary transcriptome and proteome have been well characterized.^7,9,10,14,23 Animal models indicate that there are disease-specific changes in the salivary biomarker footprint in response to systemic malignancy.²⁴ Prior to the NCI-EDRN pre-validation study, the salivary mRNA biomarkers for OSCC were identified in a discovery study and confirmed in further case-control studies in additional ethnic groups.^10-12 Discovery studies have also identified potential discriminatory salivary biomarkers for other malignancies, and ongoing studies are necessary to bring these additional potentially life-saving panels to clinical practice.^25-27

This study successfully validated six previously identified individual salivary mRNA markers for OSCC in a study in the intended-use population of patients with oral lesions suspicious for cancer. In addition, a pre-specified salivary multi-marker panel developed with data from the prior NCI-EDRN study was validated in this new population. The present study utilized a PRoBE design for validation, as suggested in the NCI-EDRN publication.⁹ This methodology meets the most rigorous standards recommended by the NCI to eliminate biases in biomarker validation¹³; therefore, the results of the present trial support the informative nature of these biomarkers and the strength of the multi-marker panel methodology. The sum of sensitivity and specificity of this prior panel exceeds the level previously proposed by the NCI to be of value in avoiding unnecessary biopsies in patients otherwise destined for biopsy of suspicious breast lesions.¹³ Nonetheless, for optimal clinical performance, models developed in the intended-use population should incorporate internal reference genes.

A three-marker panel generated from the present study data and incorporating DUSP1, SAT, and OAZ1 with housekeeping genes resulted in an area under the ROC curve of over 0.85 and a maximal sum of sensitivity and specificity of over 150%. This performance is substantially better than the minimum threshold previously proposed by the NCI for the triage of suspicious breast lesions and has the potential to aid in the detection of the vast majority of cancers in patients with suspicious oral lesions while helping to avoid unnecessary biopsies. For example, test score cut points with sensitivity over 95% and specificity over 50% would have a negative predictive value of over 99% in populations with a less than 10% incidence of malignancy. This level of risk and substantially lower levels of risk are common for oral lesions seen in daily practice based on biopsy data from large screening trials.²

Of note is the fact that the population in this trial included patients with predominantly early-stage disease. The cost of care is substantially reduced and the outcomes markedly improved when OSCC is detected in early stages.^6,28-32 Ability to return to work with earlier diagnosis adds additional economic benefit beyond the obvious human benefits.³³ Prior studies including the NCI-EDRN pre-validation study demonstrated that these salivary mRNA markers perform comparably in T1/T2 versus T3/T4 lesions.^9,11 The findings of the present study confirm and extend these observations, and further studies are warranted to assess these potential economic and clinical benefits. The present study also includes patients with carcinoma in situ, but given the small sample size, further study is necessary in this subgroup.

The finding of significant differences in these biomarkers in patients with dysplasia requires additional study, given the small number of such subjects in this prospective study. The rate of progression from oral dysplasia to carcinoma has been reported from 16% to 36% over one to two decades³⁴; therefore, if verified in further studies, these markers may be of value for identifying patients that require more frequent follow-up for conversion from dysplasia to carcinoma. Another additional potential use is for surveillance of patients already treated for OSCC. There are more than 250,000 survivors of OSCC in the United States, and given the high recurrence rate, the availability of a convenient surveillance tool is of great potential interest.⁵ Ongoing long-term studies will be necessary to address this important additional potential application.

The six reported mRNA markers are related to genes with functions that are of interest in the setting of oral cancer. IL1β has functions that relate to cell proliferation, inflammation, and apoptosis.¹⁰ IL⁸ has functions related to angiogenesis, cell cycle arrest, and immune response.¹⁰ OAZ1 and SAT regulate enzymes related to polyamine synthesis and DNA repair and are also involved in angiogenesis and cell proliferation.^35,36 S100P is involved in the regulation of a number of cellular functions, including cell cycle progression and differentiation.³⁷ DUSP1 is upregulated by oxidative stress and growth factors and may also play a role in regulation of cellular proliferation.^38,39

It is necessary to incorporate internal reference standards to ensure comparability of tests in individual patients in clinical practice. Potential housekeeping genes that are involved in vital cellular functions have been proposed, but these must be validated in each specific disease process and in each specific type of tissue sample.^40,41 A number of such proposed genes have been found to be upregulated in certain tissues in the presence of disease and thus are unsuitable as an internal standard.⁴⁰ The present study has identified several highly suitable salivary housekeeping genes for use in OSCC detection panels. Incorporating these internal controls enhances the clinical applicability.

Conclusion

Transcriptome salivary markers for the identification of OSCC have been studied in numerous cohorts, including a prior large case-control study by the NCI-EDRN. The present prospective study adheres to the most rigorous standards of biomarker development and validates six previously identified individual salivary mRNAs for OSCC detection. A prior multi-marker model is also validated, but has less discriminatory power than models generated in the intended- use population with the incorporation of housekeeping genes. The clinical implication of these findings is that there is a potential to noninvasively and painlessly identify lesions at very low risk for cancer as well as those at significantly increased risk. Further work is necessary to assure the effective implementation of this technology in clinical practice.

Disclosure

This article was supported by PeriRx LLC, Broomall, Pennsylvania; partial support was provided by Delta Dental of Michigan Research and Data Institute, Okemos, Michigan.

About the Authors

Jack L. Martin, MD
Past President
Sharpe Strumia Research Foundation
Bryn Mawr, Pennsylvania
Chief Medical Officer
PeriRx LLC
Broomall, Pennsylvania

Neil Gottehrer, DDS
Guest Faculty, University of Maryland Dental School
Baltimore, Maryland
Director of Dental Education
PeriRx LLC
Broomall, Pennsylvania
Diplomate
American Board of Periodontology
Private Practice
Periodontics
Havertown, Pennsylvania

Harvey Zalesin, DDS
Attending Oral Surgeon
St. John Macomb-Oakland Hospital
Warren, Michigan

Paul T. Hoff, MD
Attending Physician
MI Otolaryngology Surgery Associates
Ypsilanti, Michigan

Michael Shaw, PhD
Director of Research
St. John Macomb-Oakland Hospital
Warren, Michigan

James H.W. Clarkson, MD
Assistant Professor
Michigan State University
Department of Surgery
Lansing, Michigan

Pam Haan, BSN
Research Coordinator
Michigan State University
Department of Surgery
Lansing, Michigan

Mark Vartanian
Clinical Coordinator
University of Michigan
Department of Surgery
Ann Arbor, Michigan

Terry McLeod, BSN
Research Nurse
Michigan State University
Department of Surgery
Lansing, Michigan

Stephen M. Swanick
Chief Executive Officer
PeriRx LLC
Broomall, Pennsylvania

References

1. Rethman MP, Carpenter W, Cohen EE, et al. Evidence-based clinical recommendations regarding screening for oral squamous cell carcinomas. J Am Dent Assoc. 2010;141(5):509-520.

2. Brocklehurst P, Kujan O, O’Malley LA, et al. Screening programmes for the early detection and prevention of oral cancer. Cochrane Database Syst Rev. 2013;11:CD004150.

3. Warnakulasuriya S. Global epidemiology of oral and oropharyngeal cancer. Oral Oncol. 2009;45(4-5):309-316.

4. American Cancer Society. Cancer Facts & Figures 2011. https://www.cancer.org/research/cancerfactsfigures/cancerfactsfigures/cancer-facts-figures-2011. American Cancer Society website. 2011. Accessed March 30, 2015.

5. National Cancer Institute. SEER Stat Fact Sheets: Oral Cavity and Pharynx Cancer. https://seer.cancer.gov/statfacts/html/oralcav.html. National Cancer Institute website. 2012. Accessed March 30, 2015.

6. McLeod NM, Saeed NR, Ali EA. Oral cancer: delays in referral and diagnosis persist. Br Dent J. 2005;198(11):681-684.

7. Wong DT. Salivary Diagnostics: Amazing as it might seem, doctors can detect and monitor diseases using molecules found in a sample of spit. Am Sci. 2008;96(1):37-43.

8. Nagler R, Bahar G, Shpitzer T, Feinmesser R. Concomitant analysis of salivary tumor markers–a new diagnostic tool for oral cancer. Clin Cancer Res. 2006;12(13):3979-3984.

9. Elashoff D, Zhou H, Reiss J, et al. Prevalidation of salivary biomarkers for oral cancer Detection. Cancer Epidemiol Biomarkers Prev. 2012;(4):664-672.

10. Li Y, St John MA, Zhou X, et al. Salivary transcriptome diagnostics for oral cancer detection. Clin Cancer Res. 2004;10(24):8442-8450.

11. Brinkmann O, Kastratovik DR, Dimitrijevic MV, et al. Oral squamous cell carcinoma detection by salivary biomarkers in a Serbian population. Oral Oncol. 2011;47(1):51-55.

12. Wei F, Patel P, Liao W, et al. Electrochemical sensor for multiplex biomarkers detection. Clin Cancer Res. 2009;15(13):4446-4452.

13. Pepe MS, Feng Z, Janes H, et al. Pivotal evaluation of the accuracy of a biomarker used for classification or prediction: standards for study design. J Natl Cancer Inst. 2008;100(20):1432-1438.

14. Li Y, Zhou X, St John MA, Wong DT. RNA profiling of cell-free saliva using microarray technology. J Dent Res. 2004;83(3):199-203.

15. Lee YH, Zhou H, Reiss JK, et al. Direct saliva transcriptome analysis. Clin Chem. 2011;57(9):1295-1302.

16. Moyer VA. Screening for Oral Cancer: U.S. Preventive Services Task Force Recommendation Statement. Ann Intern Med. 2014;160(1):55-60.

17. American Cancer Society. American Cancer Society Guidelines for the Early Detection of Cancer. https://www.cancer.org/healthy/findcancerearly/cancerscreeningguidelines/american-cancer-society-guidelines-for-the-early-detection-of-cancer. American Cancer Society website. 2013. Accessed March 30, 2015.

18. Sankaranarayanan R, Ramadas K, Thara S, et al. Long term effect of visual screening on oral cancer incidence and mortality in a randomized trial in Kerala, India. Oral Oncol. 2013;49(4):314-321.

19. Sankaranarayanan R, Ramadas K, Thomas G, et al. Effect of screening on oral cancer mortality in Kerala, India: a cluster-randomised controlled trial. Lancet. 2005;365(9475):1927-1933.

20. Lingen MW, Kalmar JR, Karrison T, Speight PM. Critical evaluation of diagnostic aids for the detection of oral cancer. Oral Oncol. 2008;44(1):10-22.

21. Epstein JB, Güneri P, Boyacioglu H, Abt E. The limitations of the clinical oral examination in detecting dysplastic oral lesions and oral squamous cell carcinoma. J Am Dent Assoc. 2012;143(12):1332-1342.

22. Greenberg BL, Glick M, Frantsve-Hawley J, Kantor ML. Dentists’ attitudes toward chairside screening for medical conditions. J Am Dent Assoc. 2010;141(1):52-62.

23. Hu S, Arellano M, Boontheung P, et al. Salivary proteomics for oral cancer biomarker discovery. Clin Cancer Res. 2008;14(19):6246-6252.

24. Gao K, Zhou H, Zhang L, et al. Systemic disease-induced salivary biomarker profiles in mouse models of melanoma and non-small cell lung cancer. PLoS One. 2009;4(6):e5875.

25. Zhang L, Xiao H, Zhou H, et al. Development of transcriptomic biomarker signature in human saliva to detect lung cancer. Cell Mol Life Sci. 2012;69(19):3341-3350.

26. Xiao H, Zhang L, Zhou H, et al. Proteomic analysis of human saliva from lung cancer patients using two-dimensional difference gel electrophoresis and mass spectrometry. Mol Cell Proteomics. 2012;11(2):M111.012112.

27. Zhang L, Farrell JJ, Zhou H, et al. Salivary transcriptomic biomarkers for detection of resectable pancreatic cancer. Gastroenterology. 2010;138(3):949-957.

28. Lang K, Menzin J, Earle CC, et al. The economic cost of squamous cell cancer of the head and neck: findings from linked SEER-Medicare data. Arch Otolaryngol Head Neck Surg. 2004;130(11):1269-1275.

29. Hollenbeak CS, Stack BC Jr, Daley SM, Piccirillo JF. Using comorbidity indexes to predict costs for head and neck cancer. Arch Otolaryngol Head Neck Surg. 2007;133(1):24-27.

30. Kim K, Amonkar MM, Högberg D, Kasteng F. Economic burden of resected squamous cell carcinoma of the head and neck in an incident cohort of patients in the UK. Head Neck Oncol. 2011;3:47.

31. National Cancer Institute. Cancer Prevalence and Cost of Care Projections. https://costprojections.cancer.gov. National Cancer Institute website. Accessed March 30, 2015.

32. Jacobson JJ, Epstein JB, Eichmiller FC, et al. The cost burden of oral, oral pharyngeal, and salivary gland cancers in three groups: commercial insurance, Medicare, and Medicaid. Head Neck Oncol. 2012;4:15.

33. Taylor JC, Terrell JE, Ronis DL, et al. Disability in patients with head and neck cancer. Arch Otolaryngol Head Neck Surg. 2004,130(6):764-769.

34. Napier SS, Speight PM. Natural history of potentially malignant oral lesions and conditions: an overview of the literature. J Oral Pathol Med. 2008;37(1):1-10.

35. Tsuji T, Katsurano M, Ibaragi S, et al. Ornithine decarboxylase antizyme upregulates DNA-dependent protein kinase and enhances the nonhomologous end-joining repair of DNA double-strand breaks in human oral cancer cells. Biochemistry. 2007;46(31):8920-8932.

36. Casero RA Jr, Pegg AE. Spermidine/spermine N1-acetyltransferase–the turning point in polyamine metabolism. FASEB J. 1993;7(8):653-661.

37. Schäfer BW, Heizmann CW. The S100 family of EF-hand calcium-binding proteins: functions and pathology. Trends Biochem Sci. 1996;21(4):134-140.

38. Calvisi DF, Pinna F, Meloni F, et al. Dual-specificity phosphatase 1 ubiquitination in extracellular signal-regulated kinase-mediated control of growth in human hepatocellular carcinoma. Cancer Res. 2008;68(11):4192-4200.

39. Keyse SM, Emslie EA. Oxidative stress and heat shock induce a human gene encoding a protein-tyrosine phosphatase. Nature. 1992;359(6396):644-647.

40. Dheda K, Huggett JF, Bustin SA, et al. Validation of housekeeping genes for normalizing RNA expression in real-time PCR. Biotechniques. 2004;37(1):112-119.

41. Vandesomepele J, De Preter K, Pattyn F, et al. Accurate normalization of real-time quantitative RT-PCR data by geometric averaging of multiple internal control genes. Genome Biol. 2002;3(7):research0034.