Validity and reliability of palatal rugae morphometric assessment with 3D laser scanned models

Aim: To assess the reliability and validity of morphometric features on 3D digital models produced by scanning maxillary dental casts of Malaysian Malay subjects. Methods: Dental casts of 20 subjects were scanned using a 3D laser scanner (Next Engine Inc., Santa Monica, California, USA). The palatal rugae morphometric features were assessed on the resulting 3D models using 3-Matic Research 9.0 software (Materialise NV, Heverlee, Belgium). The assessments were repeated by the first and second authors to assess the intraand interexaminer reliability, respectively. Rugae morphometric features were also evaluated on the conventional plaster models to assess the validity of the 3D method. Results: Kappa values of the validity ranged from 0.807 to 0.922 for rugae shape, size category and direction. The intraclass correlation coefficient (ICC) for rugae number validity was 0.979. For intra-examiner reliability, kappa values ranged from 0.716-1.000 for rugae shape, size category and direction. The ICC for rugae number intra-examiner reliability was 0.949. Kappa values of interexaminer reliability for rugae shape, size category and direction were 0.723-885, while the ICC of rugae number was 0.896. Conclusion: Palatal rugae analyses on 3D digital models scanned by the 3D Next Engine laser scanner using 3-Matic Research 9.0 software are valid and reliable.


Introduction
Palatal rugae are transverse mucosal ridges that are located in the anterior third of the hard palate, behind the incisive papilla, and laterally extended from the median palatine raphe 1 . The morphometric characteristic of the palatal rugae have been studied by many researchers in many fields of dentistry, especially forensic odontology and orthodontics [2][3][4][5][6] .
Palatal rugae morphometric characteristics might be used for comparison between ante-mortem and post-mortem records for the purpose of human identification in the presence of ante-mortem records such as dental casts, digital images or prosthesis 7 . They also might be used in conjunction with other methods for sex or ancestry predictions 8,9 . Plaster dental casts are commonly used by dental clinicians and researchers. However, they are susceptible to damage and occupy physical space for storage 10,11 . The digital models offer a solution to the mentioned disadvantages, and they are also ready for superimposition analyses in treatment progress monitoring 5,12,13 and for sharing with colleagues through the internet 11 .
Realizing the potential roles of digital models, any method to digitize dental casts for digital manipulation must have gone through verification in terms of validity and reliability. Several studies have shown that the difference in linear measurements between the conventional and digital models are within acceptable limits [14][15][16] .
On the other hand, only limited literatures has reported the analyses on palatal rugae using 3D digital models [17][18][19] . They used different scanner and software combinations in their studies. The coefficient values were between 0.875 and 1.000 for intraand inter-observer error rates. So far, at least to our best knowledge, there was no quantitative study assessing the validity and reliability of palatal rugae using the 3D Next Engine laser (Next Engine Inc., Santa Monica, California, USA) scanned models. Thus, we aimed to benchmark the Next Engine laser scanner combined with 3-Matic Research 9.0 (Materialise NV, Heverlee, Belgium), which has been deemed reliable software with other published 3D scanner systems.

Materials and methods
Sample Calculation of the sample size was performed using a web sample size calculator 20 .
This study utilized 20 dental casts that were selected from the School of Dental Sciences archive using a convenience sampling method. The assessments focused on the first three rugae 17 on each side of the palate. There were 119 palatal rugae observed in the 20 dental casts. Each cast contained at least six rugae in both sides of the palate, except for one cast that had five rugae. Dental casts with damaged palatal areas and defects, such as bubbles or porosity, were excluded from the study. Similar to other studies 3,9 , rugae < 2 mm in length were excluded from this study. Ethical clearance was obtained from the Universiti Sains Malaysia (USM) Human Ethics Committee (USM/JEPeM/19020125).

Data collection
The dental casts were digitized using the 3D Next Engine laser scanner. The scanning processes were 1) standardized distance between the scanner and the flatter pad on which the cast was stabilized at 14 cm and 2) a a 360° scan on the cast from every angle ( Figure 1). The scanning process began with a fluorescent light produced from the scanner capturing a 2D image of the cast, followed by laser beams emitted from the scanner to capture the geometry of the cast ( Figure 1). Next Engine scan studio software (Next Engine Inc., Santa Monica, California, USA) stitched single multiple scans together, and the unwanted parts were trimmed out from the field of view to produce the final 3D image. The scanned images were saved as stereolithography (stl) files.

Identification of palatal rugae on the digital models
Each stl file was opened with Paint 3D version 6.2003.4017.0 (Microsoft, Redmond, WA, USA) software. The palatal rugae were highlighted on 3D digital models 17,21 using a pixel pen marker tool, which could be changed from 4-6 pixel thickness according to the size and curvature of the rugae to be highlighted ( Figure 2). After highlighting the rugae, the rugae shape was determined visually by the researcher.

Assessment of palatal rugae on 3D digital models
The stl file was imported into 3-Matic Research 9.0 software. Each ruga was measured using a length measuring tool in the software accurate to 0.01 mm which was able to measure the curvatures.
The 3D image was zoomed and angulated as needed using the computer mouse to achieve good visibility of rugae borders during the measuring process.
To assess the rugae size, successive single clicks by the mouse were applied starting at the medial termination point of a ruga following its curvature to the most lateral termination point. A double click was applied at the lateral end point. The measurements of rugae size were in millimetres accurate to 0.01 mm. (Figure 3). The directions of the rugae were also assessed in 3-Matic Research 9.0 software. A straight blue colour line was spontaneously drawn after taking the measurement, which connected the medial and lateral point of a measured ruga. The direction of the line from the medial to the lateral end was assessed to determine rugae direction.
The rugae that were > 2 mm in size were counted on 3-Matic Research 9.0 software to determine the total rugae number of each digital 3D cast.

Assessment of the variables on the plaster models
The rugae were highlighted directly on the plaster dental casts by the first investigator using a 0.3 mm lead pencil under an adequate light and glass magnification lens 3,9 . The shape and direction of rugae were evaluated visually by the first investigator ( Figure 4).
The rugae size were measured by adapting 0.5 mm orthodontic wire over each ruga, which was stretched again and measured using a digital calliper accurate to 0.01 mm.

Validity test
Rugae shape, size, total number and direction on 20 3D digital models were compared statistically with those values derived from the plaster dental casts.

Reliability test
The first author assessed all variables on the 3D digital models twice within a 1 month interval for intra-examiner reliability. A maximum of four 3D images were evaluated per day. The second author (MFK) assessed the same 3D digital models for inter-examiner reliability (comparison between the first and second authors' measurements).

Statistical analysis
All the data were imported into SPSS statistical software version 26.0 (IBM, Armonk, NY, USA). Kappa statistics were used to evaluate validity and intra-and inter-exam-iner reliability for rugae shape, size category, and direction. The intraclass correlation coefficient (ICC) was used to assess the validity and intra-and interobserver reliability for rugae number. A P < 0.05 was considered significant. Table 1 shows the data for validity evaluation. Kappa values ranged from 0.807 to 0.922 for the shape, from 0.853 to 1.000 for the size category and finally from 0.846 to 1.000 for the direction of rugae.  Table 2 shows the intra-examiner reliability between the two readings of the first author on the digital 3D models. Kappa values ranged from 0.812 to 1.000 for rugae shape, from 0.889 to 1.000 for rugae size category and from 0.716 to 1.000 for rugae direction.  Table 3 shows inter-examiner reliability between the two researchers. Kappa values ranged from 0.737 to 0.843 for shape, from 0.785 to 0.857 for size category and from 0.723 to 0.885 for direction.  Table 4 shows results for the ICC, which ranged between 0.896 and 0.979 for reliability and validity of number of rugae.  Figure 5. A diagram illustrating kappa values of validity, in addition to intra-and inter-examiner reliability.

Discussion
The fundamental of good research is the method should be reliable and valid 22 . The reliability of an instrument is related to its ability to replicate the same measurements under the same conditions (consistency), while the validity of this instrument is related to the reality of the resulting measurements (accuracy) 23 .
A valid tool must be reliable, but a reliable tool may not necessarily be valid. In other words, a wrong measurement may be repeated (highly reliable) but not valid. Thus, we assessed the 3D Next Engine laser scanner and 3-Matic Research 9.0 software from both aspects (i.e., reliability and validity). The focus of reliability assessment was inter-and intra-examiner error study, while validity was assessed by comparing 3D digital model measurements against measurements on plaster dental casts using a pencil, wire, magnifier and digital calliper.
The majority of studies that assessed the palatal rugae have used plaster dental casts. They highlighted the rugae under artificial light and magnification lens using a sharp graphite pencil 2,9,24,25 or a pen marker 8,26 . From our point of view, using the sharp pencil allows highlighting a ruga without contacting the adjacent rugae, and the pencil can also be easily erased if there is any mistake. After highlighting the rugae, the shape and direction were visually assessed.
The linear size of the rugae were mostly measured using a digital calliper from the most medial to the most lateral point of each ruga 27 or using a divider and ruler 9,28 . However, these methods are only able to measure linear distance, excluding the actual curvature distance. Some other studies adapted a flexible metallic wire over the curvature of the rugae to measure the real length of the rugae 8,25 . We were inclined to use the latter method to be more reasonable, as there might be a long ruga with the medial and lateral points close to each other.
The reliability of teeth and dental arch linear measurements on 3D dental casts scanned by the 3D Next Engine laser scanner was assessed by Haque et al. 29 (2019). They took the linear measurement directly on conventional casts using a digital calliper and also on digital 3D casts using analyses tools in 3-Matic Research 9.0 software. The intra-examiner reliabilities of the two methods were excellent (ICC 0.916-0.995), while the inter-examiner reliability of the digital method was good to excellent (ICC 0.816-0.990). The validity of the digital measurements ranged between 0.913 and 0.996 29 .
Few studies have evaluated the morphologic features of palatal rugae on 3D models; however, they only assessed the reliability of rugae size. The reliability of their assessments showed good to excellent ICC values [17][18][19] .
Choi et al. 18 (2018) and Barbo et al. 17 (2018) evaluated morphometric characteristics of palatal rugae on 3D digital casts. In both studies, the reliability of intra-examiner measurements were examined on 30 digital casts. Choi et al. 18 (2018) found that the ICC was 0.970. While Barbo et al. 17 (2018) found that, the intra-rater ICC for rugae measurement ranged between 0.999 and 1.000 17,18 . Another study also reported high coefficient values of intra-and inter-observer measurements using 3D models. The ICC were between 0.875 and 0.957, respectively 19 .
Perhaps the reason the previous studies [17][18][19] yielded high ICC values is because they assessed the reliability of the length, while in our study, we assessed the reliability and validity of the size category. Moreover, in our study, we assessed the real size of rugae, including their curvatures, not only the linear size from the most medial to the most lateral points of the rugae. The measurement of curvature of each ruga may be more technically sensitive than the measurement of linear ruga length.
The previously mentioned studies [17][18][19] did not evaluate the validity of their measurements in comparison to the measurements on conventional plaster models. Saadeh et al. 30 (2017) reported high correspondence, 0.96 kappa value, of rugae size category on 50 digital 3D models with those on conventional plaster models 30 .
In our study, comparable results were noted. We found that the validity by comparing the rugae morphometric assessment between the digital 3D models and conventional plaster models was almost perfect 31 and excellent 32 .
As it is clear in the results, the third rugae values were consistently high for validity and reliability. In fact, many studies have used 2D and 3D models to assess the efficacy of palatal rugae as reference points in orthodontics also found that the third rugae were the most stable points, and thus, the most convenient to be used as reference points during superimposition for orthodontic treatment progress evaluation 6,33,34 .
The outcome of this study was limited to normal and healthy subjects in which the anatomy of the palate was accessible and had less variation if compared to cleft palate, as an example. Future study should include palatal anomalies where some modification in scanning techniques may need to be explored.
In conclusion, palatal rugae analyses on 3D digital models scanned by the 3D Next Engine laser scanner combined with 3-Matic Research 9.0 software are valid and reliable. The highest values of reliability and validity in both sides of the palate were those of the third palatal rugae.