1. Introduction

In materials science and engineering, the determination of mechanical properties plays a key role in understanding the behavior of materials, optimizing their performance, and developing technological applications. One of the widely used methods for evaluating hardness and other mechanical properties is indentation, the pressing of a harder tip of known shape into a sample with a given load. Depending on the tasks, methods differ in the principle of investigation, geometry and material of the indenter used. There are different methods of indentation and hardness determination by restored indentation (the dimensions of the residual indentation are examined) and by non-restored indentation (the dependence of the applied load on the depth of embedding is recorded). For example, the Brinell method uses a spherical indenter, which makes it possible to carry out measurements for materials with heterogeneous structure. Due to the large size of the indentation, Brinell method limited in testing materials with fine structure. The Knoop method uses a diamond pyramidal indenter with a diamond-shaped cross-section, which makes it possible to carry out measurements for thin films and brittle materials. The method of instrumental indentation implies the application of a controlled small load to the surface of the material, while the load-displacement relationship is recorded and the load-displacement curve is subsequently analyzed. Vickers method of indentation is the simplest and most universal method [1,2]. Vickers indentation involves the controlled application of force using a diamond pyramidal indenter with a specified geometry to determine the material's resistance to deformation. The residual imprint diagonals are measured to calculate hardness values, which provides insight into fundamental material properties such as strength, toughness, and wear resistance [3-5].

Vickers hardness (HV) is calculated by the formula (GOST 2999-75, ISO 6507-1:2005) [6]:

where F is the applied force in Newtons, d is the average of the two residual imprint diagonals in millimeters. This approach allows for a more accurate determination of the hardness of a wide range of materials, regardless of deformation features (e.g., the occurrence of pile-ups at the edges of the prints). The Vickers hardness value provides a quantitative assessment of a material's ability to resist an applied load.

The load is maintained to the nearest fraction of a gram. So the key challenge is to accurately measure the diagonals of the residual imprint.

To calculate the imprint diagonals on the image obtained after indentation, image segmentation with the imprint brightness histogram, as well as the analysis of grayscale gradation in images were previously used [7,8]. With the development of computer technology and algorithms, such approaches began to yield to new approaches due to the increased accuracy and flexibility of the latter.

The accuracy of diagonal lengths determination is mostly influenced by: geometric correctness of the obtained print (absence of chipping and pile-ups, Fig. 1) [9], image contrast (Fig. 2), which depends on the analyzed material, as well as image focusing [10]. An image with low contrast presents a challenge to find the imprint vertices for both the automatic algorithm and the human eye. Image focusing has the greatest effect on the accuracy and choice of measurement methods. Microscopes with objectives for which the characteristic depth of field is units of micrometers are most often used to analyze imprints. In the case of microindentation the imprint depth reaches tens of micrometers and depends on the material being examined and the applied load. Thus, both the small roughness of the sample and the depths of imprints affect the image clarity [11].

 

Fig. 1. Photo examples of imprints with curved edges

 

Fig. 2. Example of different contrast images for imprints on materials: copper on the left, polycarbonate on the right

 

Due to the significant influence of focus, image analysis approaches based on sharpness (focus) functions have been developed [12,13]. Recently, there has been a significant increase in the number of proposed algorithms for finding imprints in images based on the use of neural networks [14-16]. Despite the high accuracy of the obtained results, the works [15] and [16] did not provide imprints with irregular geometry or imprints on materials with low contrast.

In this work we demonstrate a new approach to post-indentation image processing based on wavelet transforms [17,18], which improve the accuracy of imprint diagonal measurements.

2. Methods and materials

In this study, we use a Vickers-type diamond pyramidal indenter, with opposite faces making an angle of 136°. The image of the residual imprint was studied using an optical video microscope with an objective magnification of 50x and a digital camera with 4K resolution (4632×3488 pixels).

The study was performed on a NanoScan-HV microhardness tester (TISNCM, Russia) consisting of a loading module and a microscope. The microscope objective lens is mounted on a stepper motor, which allows to precisely change the height of the objective lens above the sample with a step of 0.1 µm.

Polycabronate, copper, chromium and fullerene-doped copper samples were investigated. The selected materials have low image contrast, pile-up formation during indentation, and high surface roughness.

Before and after indentation, a series of digital camera images were acquired while moving the lens from top to bottom in 0.1 µm increments. The obtained images were analyzed using different types of focusing functions and a comparative analysis of the results was performed.

To analyze focusing functions, the software developed on the basis of Python programming languages with the use of OpenCV and PyWavelets libraries was used for flexible adjustment of processing parameters, adaptation of algorithms to different types of images and subsequent extension of functionality. Implementation of the resulting algorithm into the software of the NanoScan-HV microhardness tester was carried out in C++ without using special libraries for working with wavelets, but using a camera driver with a productive standard software DirectShow API to increase the frame rate.

3. Algorithm description

Three different algorithms applied to the gray-scale image were used for the analysis: variance F_D [19], Laplace function F_L [20] and wavelet transform based function F_W. The first two functions have the following expressions:

where X, Y are the width and height dimensions of the i-th image, I_i (x, y) is the gray pixel intensity, μ_i is the average pixel intensity of the image, C(x, y) is the convolution value with the kernel:

The variance-based method (F_D) involves calculation of the gray-scale intensity variance of an image using formula (2). It is widely used in automatic focusing systems [21] due to its simplicity and ability to estimate the brightness distribution. The Laplace function-based (F_L) algorithm involves convolution of the image with the kernel (4) to extract high-frequency components, which allows capturing changes in image sharpness [22]. The method is a standard tool in the estimation of gradient image characteristics.

The described approaches are often used in the creation of focusing systems. But their biggest problem is instability to simultaneous analysis of contrast and non-contrast images, as well as the impossibility of using them for focusing on a small area of the image (for example, when it is necessary to focus exactly on the border of the received imprint). Also, such algorithms cannot be used to create pseudo 3D images, i.e. three-dimensional images obtained by composing 2D images.

To overcome the above limitations, a focusing function based on discrete wavelet transform (DWT) is proposed. In 2D wavelet transform, the original image is divided into 4 parts. The upper left quadrant is LL, the lower left quadrant is LH, the upper right quadrant is HL, and the lower left quadrant is HH. The LL segment contains information about the original compressed image, and the HL, LH and HH segments about the high-frequency components of the image. By repeatedly applying this approach to the LL segment, the depth of the transform J can be increased.

Thus, the resulting function is F_W, which is the ratio of the high-frequency part M_H to the sum of the low-frequency parts:

The Dobeshi function with basis 4 was chosen as the transforming function. At limited computational resource it is possible to use the Haar function (Dobeshi function with basis 2).

4. Results and discussion

The comparative analysis of different types of transform functions (Fig. 3) shows that the optimal basis value for the problem solving is 4. Increasing the basis has a good effect on reducing the number and magnitude of function values fluctuations in out-of-focus positions, but in the focus region the behavior of functions with different basis is similar.

 

Fig. 3. Normalized values of wavelet transform functions depending on the position of the microscope when moving the microscope to the sample (1 - value of basis 2, 2 - value of basis 4, 3 - value of basis 6)

 

Another important parameter affecting the shape of the final function is the depth of transformation. We analyzed four depths from 1 to 4 (Fig. 4). The depth of transformation affects both the smoothness of the approximated curve and its slope. As the depth of transformation increases, the shape of the curve changes less. The optimal depth of transformation is 3-4, depending on the nature of the image and available computing power.

 

Fig. 4. Normalized values of wavelet transform functions with transform depth in the range from 1 to 4 depending on the microscope position

 

To compare the performance of different types of focusing functions (F_D, F_L, F_W), we used a set of images obtained after indenting different samples. For each specimen 3 images are obtained: 1) at the top position of the microscope, 2) at the focusing point on the specimen surface defined by the operator and 3) at the bottom point (focus on the center of the imprint).

Data from an image series of four specimens were processed by the F_D, F_L, and F_W functions. The results of applying different focusing functions depending on the position of the microscope on the specimens are shown in Fig. 5. The rectangle highlights the area selected by the operator in which the focus is on the specimen surface (near the imprint boundary). This is the most favorable focusing area for the subsequent use of automatic imprint marking algorithms.

 

Fig. 5. Normalized values of focusing functions depending on the microscope position on samples of chromium (a), copper (b), polycarbonate (c), copper doped with fullerene (d): 1 - variance, 2 - Laplace function, 3 - wavelet transformation

 

The wavelet transform based focusing function F_W shows a consistently clearer and sharper peak in the focused region on all presented material types. Different algorithms reveal focused frames in simple cases (copper and fullerene-doped copper) in the same region. However, in complex cases with insufficient contrast and small print size (polycarbonate and chromium), classical F_D, F_L focusing algorithms do not give a stable positive result. The main reasons for deviations are monotone-dark frames, the presence of surface defects near the print, or the contrasting texture of the material. The F_D and F_L functions are well suited for materials that have a uniformly pale surface color and no surface defects. The presence of synchronized emissions on all features in certain frames may be justified by frame blurring, backlighting, or small obvious defects on the surface of the material coming into focus.

Due to the fact that an imprint may occupy a large part of the image (which is necessary for subsequent partitioning of such an imprint to find its area), focusing over the whole image is impractical. Standard image partitioning algorithms rely on the fact that the imprint is in the center of the image, so focusing is proposed to focus on the 4 corner areas in the image outside the imprint. The value of the focus function is calculated as the median value over the 4 segments. The segment sizes are chosen based on the typical sizes of the resulting imprints. Graphs of the resulting F_W functions before indentation, after indentation over the whole frame and after indentation over 4 areas were analyzed (Fig. 6). The graph of the F_W function over the 4 regions is sharper and the maximum coincides with the value of the maximum of the F_W function before indentation, which means that the focus falls into the sample plane. The value of the F_W maximum in the case of focusing over the whole frame after indentation is shifted by 0.5 μm to the right, which shows the imprint contribution to the function value.

 

Fig. 6. Normalized values of the focus function for the cases: E1 - before indentation, E2 - after indentation over the whole frame, E3 - after indentation over four areas

 

The data obtained using the wavelet transform can be used not only to find a focused image to determine the size of residual imprints, but also to study the deformation material behavior [23]. In addition, an interesting application is to obtain a fully focused image over the entire image plane using the inverse transform.

To obtain a fully focused image, we applied an inverse wavelet transform, with the low-frequency components selected as the average of the frames and the high-frequency components selected as the maximum. Once this image is obtained, given the position of the microscope when each frame is acquired, it becomes possible to reconstruct the surface profile. By superimposing the fully focused image on the profile, a 3D image of the imprint is obtained (Fig. 7).

 

Fig. 7. Height map and pseudo 3D plot based on wavelet transform

 

Fig. 8. Hardness values at different focus points of the frame

 

Fig. 8 shows the dependence of hardness values obtained at different points of the focused frame. The area in which the calculated values fall within the permissible range of hardness values of this material is highlighted in color. Correctly focused frame corresponds to N23. Thus, image focusing is necessary to obtain correct hardness values.

5. Conclusions

In this paper we have presented new approach based on wavelet transform for focusing the image with residual imprint after indentation, which is more stable and efficient compared to the other approaches. Pseudo 3D image of the sample surface can be useful for precise adjustment of the starting points of indentation recognition and edge detection algorithms. Our algorithm can be used as an alternative to scanning microscopy when replacing the microscope positioning system with a more accurate one (e.g., piezo table).

The proposed approach of imprint boundary detection using wavelet transform takes into account surface defects near the imprint boundary, as well as the curvilinear contour of the boundary itself. We have proposed the method to obtain a focused image exactly on the plane of the imprint boundary.

References

1. Chou K., Eff M., Cox C. et al. Optical image and Vickers hardness dataset for repair of 1080 steel using additive friction stir deposition of Aermet 100 //Data in Brief. 2022. V. 41. P. 107862. https://doi.org/10.1016/j.dib.2022.107862.

2. Maqsood M., Rafique M., Butt M.Z. et al. Impact of carbon ion implantation on the crystal structure, surface morphology, Vickers hardness and electrochemical corrosion of zirconium //Journal of Materials Engineering and Performance. 2021. V. 30. N 6. P. 4604Ð4618. https://doi.org/10.1007/s11665-021-05765-6.

3. Zhang P., Li S.X., Zhang Z.F. General relationship between strength and hardness //Materials Science and Engineering: A. 2011. V. 529. P. 62Ð73. https://doi.org/10.1016/j.msea.2011.08.061.

4. Mazhnik E., Oganov A.R. A model of hardness and fracture toughness of solids //Journal of Applied Physics. 2019. V. 126. N 12. P. 125109. https://doi.org/10.1063/1.5113622.

5. Tian Y., Li L., Li J. et al. Correlating strength and hardness of high?entropy alloys //Advanced Engineering Materials. 2021. V. 23. N 8. P. 2001514. https://doi.org/10.1002/adem.202001514.

6. Dovale-Farelo V., Tavadze P., Lang L. et al. Vickers hardness prediction from machine learning methods //Scientific Reports. 2022. V. 12. N 1. P. 22475. https://doi.org/10.1038/s41598-022-26729-3.

7. Dominguez-Nicolas S.M., Wiederhold P. Indentation image analysis for Vickers hardness testing //2018 15th International Conference on Electrical Engineering, Computing Science and Automatic Control (CCE). 2018. P. 1Ð6. https://doi.org/10.1109/ICEEE.2018.8533881.

8. Dom?nguez-Nicolas S.M., Herrera-May A.L., Garc?a-Gonz?lez L. et al. Algorithm for automatic detection and measurement of Vickers indentation hardness using image processing //Measurement Science and Technology. 2021. V. 32. N 1. P. 15407. https://doi.org/10.1088/1361-6501/abaa66.

9. Rouxel T., Jang J., Ramamurty U. Indentation of glasses //Progress in Materials Science. 2021. V. 121. P. 100834. https://doi.org/10.1016/j.pmatsci.2021.100834.

10. Gadermayr M., Maier A., Uhl A. The impact of unfocused Vickers indentation images on the segmentation performance //International Symposium on Visual Computing. 2012. P. 468Ð478. https://doi.org/10.1007/978-3-642-33191-6_46.

11. M?nchow E.A., Correa M.B., Ogliari F.A. et al. Correlation between surface roughness and microhardness of experimental composites with varying filler concentration //The Journal of Contemporary Dental Practice. 2012. V. 13. P. 299Ð304. https://doi.org/10.5005/jp-journals-10024-1141.

12. Groen F.C.A., Young I.T., Ligthart G. A comparison of different focus functions for use in autofocus algorithms //Cytometry: The Journal of the International Society for Analytical Cytology. 1985. V. 6. N 2. P. 81Ð91. https://doi.org/10.1002/cyto.990060202.

13. Bian Z., Guo C., Jiang S. et al. Autofocusing technologies for whole slide imaging and automated microscopy //Journal of Biophotonics. 2020. V. 13. N 12. P. e202000227. https://doi.org/10.1002/jbio.202000227.

14. Tanaka Y., Seino Y., Hattori K. Automated Vickers hardness measurement using convolutional neural networks //The International Journal of Advanced Manufacturing Technology. 2020. V. 109. N 5. P. 1345Ð1355. https://doi.org/10.1007/s00170-020-05746-4.

15. Li Z., Yin F. Automated measurement of Vickers hardness using image segmentation with neural networks //Measurement. 2021. V. 186. P. 110200. https://doi.org/10.1016/j.measurement.2021.110200.

16. Buitrago Diaz J.C., Ortega-Portilla C., Mambuscay C.L. et al. Determination of Vickers Hardness in D2 Steel and TiNbN Coating Using Convolutional Neural Networks //Metals. 2023. V. 13. N 8. https://doi.org/10.3390/met13081391.

17. Liu W., Chen W. Recent advancements in empirical wavelet transform and its applications //IEEE Access. 2019. V. 7. P. 103770Ð103780. https://doi.org/10.1109/ACCESS.2019.2930529.

18. He B., Zheng H., Ding S. et al. A review of digital filtering in evaluation of surface roughness //Metrology and Measurement Systems. 2021. V. 28. N 2. P. 217Ð253. https://doi.org/10.24425/mms.2021.136606.

19. Huang W., Jing Z. Evaluation of focus measures in multi-focus image fusion //Pattern recognition letters. 2007. V. 28. N 4. P. 493Ð500. https://doi.org/10.1016/j.patrec.2006.09.005.

20. Aubry M., Paris S., Hasinoff S.W. et al. Fast local laplacian filters: Theory and applications //ACM Transactions on Graphics (TOG). 2014. V. 33. N 5. https://doi.org/10.1145/2629645.

21. Pertuz S., Puig D., Garcia M. A. Analysis of focus measure operators for shape-from-focus //Pattern Recognition. 2013. V. 46. N 5. P. 1415-1432. https://doi.org/10.1016/j.patcog.2012.11.011.

22. Shen C.H., Chen H.H. Robust focus measure for low-contrast images //2006 Digest of technical papers international conference on consumer electronics. 2006. P. 69-70. https://doi.org/10.1109/ICCE.2006.1598314.

23. Fedotkin A.P., Gladkikh E.V., Rusakov A.A., Useinov A.S. Digital processing of surface images after indentation to analyze the deformation behavior of the material //ChemChemTech. 2023 V. 66 N 10. P. 102-108. https://doi.org/10.6060/ivkkt.20236610.11y. (in Russian)