A Comparison of Quantitative Internal and External Cytopathology Evaluations

Jackie Cuda^*
*Correspondence: Jackie Cuda, Department of Neuroscience DNS, University of Padova, 35128 Padova, Italy, Email:

Introduction

Determining reliable performance measures is critical for assessing a trainee in the six domains of clinical competency defined by the Accreditation Council for Graduate Medical Education and the American Board of Medical Specialties, as well as providing valuable feedback to the trainee. Although the core competencies were first introduced in 1999, they have since become an important part of the newly restructured accreditation of training programmes known as the Next Accreditation System, which includes specialty-specific milestones that address the core competencies. The milestones include outcomes that can be used as progressive measures of performance in various core competencies to assist a programme in attesting that a trainee is progressing in semiannual evaluations and eventually competent to progress to independence [1].

Description

Given the significance of milestone evaluations, assessment tools to assess a fellow's performance in core competencies are critical in determining whether the fellow can gradually assume more responsibility during their fellowship year. Despite their importance, there is little guidance on the precise evaluation metrics that should be used. The new cytopathology milestones provide a useful framework for determining physician competency in cytopathology during training, but the specifics of assessment are left up to individual programmes. This helps programme directors decide what works best in their programme based on their curriculum and available resources. The most recent ACGME cytopathology milestones, which go into effect in July 2021, include an improved supplemental guide that highlights some proposed assessment methods for each milestone [2].

Which is a useful addition to improve fellowship programme implementation. Work-based direct observation with multisource feedback (360-degree employee evaluations), review of reports, self-directed assessment, cytologyehistology correlation, procedure and on-call logs, and retrospective peer review, among other methods, are some suggested evaluation methods for the milestones. This study describes and compares the various internal and external quantitative evaluation parameters that our institution has used to assess and provide helpful advice to cytopathology fellows and other trainees [3].

The discrepancy report was generated at slightly different times throughout the year, and the case number monthly average was calculated for comparison. Furthermore, our fellows rotated at three different hospitals, and their schedules were set for the entire year at the start of their training, whereas the faculty schedules were set biannually in April and October. As a result, the months each fellow spent on nongynecologic services and the faculty with whom they partnered could result in a wide range of case volume. Other variables, such as family leave, vacation time, and elective time, may have an impact on case number variability. It should be noted that this report only included nongynecologic cytopathology cases, excluding gynecologic cases because our fellows do not typically enter reviews diagnosis.

We generated metrics based on programme average statistics to allow the programme director and fellows to understand where the metrics were in relation to prior trainees as below average or above average. Over the course of six years, the programme averaged 89.9% concordance rate, 1.5% major discrepancy rate, and 260 cases/month average number of overall cases, which we used as internal standards against which we would compare individual data. The findings were used to support quantitative mandates specified in the ACGME Programme Requirements for Graduate Medical Education in Cytopathology, which were revised effective June 13, 2020, and stated that fellows must evaluate at least 2000 cytology specimens during the fellowship year, including at least 500 gynecologic specimens.

This is especially important early in the fellowship year or early in an individual rotation to allow for change and improvement. In general, feedback for pathology training programmes has suffered from a lack of standardisation, inconsistency, and being largely subjective based on the attending pathologist on a small number of cases. Individual feedback from a specific attending physician is subject to a variety of factors and may not accurately reflect the trainee's progress over time if they do not rotate with that attending again. Thus, the LIS-driven feedback reports for quantitative performance measures have been beneficial for more global objective data. As a result, these reports have grown in popularity and are now used by residents on nongynecologic cytology rotations [4,5].

Conclusion

In conclusion, our fellowship programme has used discrepancy reports in our LIS to generate various internal quantitative metrics of a fellow's performance in order to assess progress during the fellowship year, provide concrete feedback, and demonstrate compliance with ACGME programme requirements for case numbers. These parameters provide a useful measure of diagnostic performance and facilitate comparison to peers, as well as assisting in determining areas of weakness to focus on. These findings demonstrate that fellowship programmes can develop quantitative measures, similar to those used for quality assurance measures for cytotechnologists, that can be used to provide guidance and feedback to cytopathology fellows. This can be useful in supplementing external quantitative measures of performance, such as standardised tests, by providing additional information.

Acknowledgement

None.

Conflict of Interest

There are no conflicts of interest by author.

References

1. Brescia, Giuseppe, Giorgia Saia, Federico Apolloni and Gino Marioni, et al. "A novel nasal endoscopic approach for removing displaced dental implants from the maxillary sinus." Am J Otolaryngol 38 (2017): 92-95.

Google Scholar, Crossref, Indexed at

2. Brescia, Giuseppe, Stefano Fusetti, Federico Apolloni and Gino Marioni, et al. "Displaced dental materials in the maxillary sinus: An original series. analysis and definition of a surgical decision-making process." Ann Otol Rhinol Laryn 128 (2019):177-183.

Google Scholar, Crossref, Indexed at

3. Workman, Alan D., Eric J. Granquist and Nithin D. Adappa. "Odontogenic sinusitis: Developments in diagnosis, microbiology, and treatment." Curr Opin Otolaryngol Head Neck Surg 26 (2018): 27-33.

Google Scholar, Crossref, Indexed at

4. Little, Ryan E., Christopher M. Long, Todd A. Loehrl and David M. Poetker, et al. "Odontogenic sinusitis: A review of the current literature." Laryngoscope Investig Otolaryngol 3 (2018): 110-114.

Google Scholar, Crossref, Indexed at

5. Snidvongs, Kornkiat, Matthew Lam, Raymond Sacks and Peter Earls, et al. "Structured histopathology profiling of chronic rhinosinusitis in routine practice." Int Forum Allergy Rhinol 2 (2012): 376-385:

Google Scholar, Crossref, Indexed at