Research on gearbox temperature field image fault diagnosis method based on transfer learning and deep belief network

Scientific Reports volume 13, Article number: 6664 (2023) Cite this article

365 Accesses

1 Altmetric

Metrics details

This paper applies thermal imaging technology to gearbox fault diagnosis. The temperature field calculation model is established to obtain the temperature field images of various faults. A deep learning network model combining transfer learning of convolutional neural network with supervised training and unsupervised training of deep belief network is proposed. The model requires one-fifth of the training time of the convolutional neural network model. The data set used for training the deep learning network model is expanded by using the temperature field simulation image of the gearbox. The results show that the network model has over 97% accuracy for the diagnosis of simulation faults. The finite element model of gearbox can be modified with experimental data to obtain more accurate thermal images, and this method can be better used in practice.

Vibration signal monitoring is the most commonly used method for gearbox fault diagnosis. Vibration signal monitoring has the advantages of moderate cost, strong reliability and mature technology1. The defects of this method include contact measurement, local information, serious influence of environmental conditions, serious loss of long-distance transmission signal caused by noise pollution2,3. It is an important supplement to the basis of fault determination that the introduction of temperature signal into the field of gearbox fault diagnosis.

Recent studies have shown that the temperature signal contains a large amount of information that can be used for gearbox health status detection and fault diagnosis4. Infrared thermal imaging technology which has the advantages of more comprehensive information and no contact in temperature measurement is more concerned by researchers5. Kwan et al. developed a neural network-based image processing tool that can detect abnormal temperature increases five hours before tooth fracture6. Younus et al. proposed a new method for fault diagnosis of rotating motors based on thermal image research using image histogram features. It is proved that the classification process of thermal image features by classifiers such as support vector machine can serve machine fault diagnosis7. Subsequently, they proposed an intelligent diagnostic system to classify different machine states using infrared thermal imaging8. Lim et al. compared thermal images with vibration signals and proposed a fault diagnosis method using support vector machine algorithm through infrared thermal imaging5. Emmanuel Resendiz-Ochoa et al. proposed a method for diagnosing gear wear by analyzing infrared imaging images. The method firstly calculates the statistical time domain characteristics of infrared imaging, then reduces the dimension of data, and finally performs fault diagnosis by neural network9. Most of these studies are fault diagnosis under a specific condition. On this basis, Shao et al. proposed a transfer learning method using convolutional neural networks for bearing fault diagnosis under different operating conditions10. Yongbo Li et al. extracted fault features from thermal imaging images by using the bag-of-visual-words method, and then classified rotating machinery faults by using support vector machine, realizing the fault diagnosis of rotating machinery under non-stationary operating conditions11. Bai Tangbo proposed a fault diagnosis method for rotating machinery that can solve the shortcomings of low contrast, blurred edges and high noise of infrared thermal images12.

The above research has a common difficulty in improving accuracy, that is, the data set used for training and verification is small, and it is difficult to obtain all data under various fault conditions3.

In this paper, the finite element model of gearbox is established, the temperature field distribution on the surface of steady-state gearbox is calculated, and the corresponding temperature field image is obtained. A deep learning network model for gearbox fault diagnosis is proposed, which combines the transfer learning of convolutional neural network with deep belief network. It is verified that the gear box temperature field image has a high accuracy rate for gearbox fault diagnosis.

The gearbox used in this paper is shown in Fig. 1, which is composed of an input gear pair with parallelaxes and two sets of planetary gear trains. The gear parameters are shown in Table 1.

Gearbox three-dimensional model.

The heat transfer modes include heat conduction, heat convection and heat radiation. The relative rolling and sliding of the gear, the rolling of the bearing, the power loss caused by the gear and bearing mixing oil and mixing gas are the main sources of heat generated by the gearbox in the working state.

There are many formulas for calculating the power loss of gears. In this paper, Anderson-Loewenthal calculation method is used13. The Friction torque calculation formula of low speed bearing of Palmgren is selected to calculate the heat production power of the bearing14. The mechanical design manual is used to calculate the total efficiency of the planetary gear train to calculate the total power loss15. The churning loss of shaft and the churning loss of gear and bearing on both sides are calculated by the British standard BS ISO/TR14179-1-200116. The formulas selected above are given in Table 2.

The heat transfer coefficient of different surfaces of the gearbox varies with the contact medium. The convective heat transfer coefficient (CHTC) of the gear tooth surface adopts the mathematical model proposed by Handschuh to calculate CHTC of the gear meshing surface. CHTC of the two sides of the gear and the planet carrier adopts the calculation formula of CHTC of the rotating disk and the medium17. CHTC of bearing and shaft is calculated by the formula in Zhao's paper18. CHTC between the inner surface of the box and the medium is based on the Gnielinski formula, and CHTC between the outer surface and the medium is based on the single tube criterion proposed by Churchill and Bernstein19. The formulas selected above are given in Table 3.

The parameters used to establish the steady-state temperature field model of the gearbox can be obtained from the above formula.

During normal operation, the temperature field of gearbox is gradually stable. The stable temperature field changes when the gear fails. The running state of the gearbox can be reflected by the temperature field distribution of the gearbox.

The calculation formula of steady-state temperature field model parameters is introduced above. The temperature field distribution on the surface of the gearbox can be obtained by establishing the calculation model of the gearbox from these parameters. The model is simplified as follows:

In the working state, the contact parts of the input gear pair are alternately contacted, and the heat generation is uniform. Therefore, the gear meshing part can be simplified. The calculated heat generation can be uniformly applied to the contact surface corresponding to the simplified gear model.

The meshing parts corresponding to each gear of the planetary gear train contact alternately, so the distribution of heat on the contact surface is uniform. Therefore, the simplification of the planetary gear train can be divided into the following three steps : First, calculate the overall heat flux of the gearbox. Then the planetary gear, sun gear and planet carrier of the model are removed to obtain the equivalent model. Finally, the heat flux density which transfer to shaft and the gear ring is uniformly applied to the corresponding position of the equivalent model.

Due to the high temperature of the gearbox in the input part and the first-stage planetary gear train, the characteristics of the temperature field distribution are obvious. The temperature of the output part (the second-stage planetary gear train part) is low, so the temperature field distribution characteristics are not obvious. Therefore, this paper only studies the gear fault diagnosis of the input part of the gearbox and the first-stage planetary gear train.

In this paper, the convolutional neural network model is first established. Then, a deep learning network model for gearbox fault diagnosis combining transfer learning and deep belief network is proposed, and the two models are compared.

Convolutional neural networks are widely used in image classification. Firstly, the convolutional network model is introduced. The structure of the model is shown in Fig. 2. In this paper, the convolutional neural network is used to train the data set to obtain the classification results.

The structure of convolution network model.

The deep learning network model of gearbox fault diagnosis combining transfer learning of convolutional neural networks and deep belief network (TrCNN-DBN) consists of three parts, temperature field image expansion(TFIE), convolutional neural network transfer learning (TrCNN) and deep belief network (DBN) fault classification.

TFIE rotates the input image, adjusts the brightness of the image, adds noise and blur to the image. Rotation simulates the thermal imager jitter. Adjusting the brightness of the picture, adding noise and blurring the picture are used to simulate the changes in the light of the shooting environment. Image expansion can greatly increase the data set used in training, thereby improving accuracy.

TrCNN uses the VGG16 model proposed by the Visual Geometry Group at Oxford University. In this paper, the first 19-layer convolutional neural network of the VGG16 model is used to extract the features of the image. Then the obtained data is used for the training and classification of the DBN network. The model structure is shown in Fig. 3.

The structure of TrCNN-DBN.

DBN is a probability generation model composed of Restricted Boltzmann Machine and Sigmoid Belief Network. The combination of supervised learning and unsupervised learning can better inherit the information of the upper layer of the network to the next layer compared with the fully connected layer of the convolutional neural network.

The diagnosis process is as follows:

The temperature field distribution images under specific working conditions simulated by the steady-state thermal calculation model of the gearbox are divided into two categories, namely the training set and the verification set.

The image of the training set is expanded to increase the number of samples in the training set and simulate images in different shooting environments.

The training set image with image enhancement and the verification set image are input into TrCNN for feature extraction.

The extracted feature data is input into DBN to further extract features for fault prediction.

As shown in Fig. 4, using convolutional neural network model to train the top view picture and the left view picture, the classification accuracy of the model reaches 100% after 3 epochs, and the loss is almost 0. The model classification accuracy of the main view picture trained with convolutional neural network model is 99%, and the loss is 0. 06. The classification accuracy of TrCNN-DBN on the top view picture and the main view picture is 100%, and the classification accuracy on the left view picture is 97%. The classification accuracy of the two models is similar, but the training time of convolutional neural network model is 5 times that of TrCNN-DBN. Therefore, the TrCNN-DBN model is selected for gearbox fault diagnosis, and the training results of the model TrCNN-DBN are introduced in detail.

Training and testing performance curves.

The steady-state temperature field distribution of the gearbox housing can be completely represented by the thermal images in three directions. Therefore, the thermal images in three directions are respectively enhanced and extracted, and then the obtained feature data is combined into a vector for training and fault prediction of DBN. The specific process is as follows :

It is assumed that the rotational speed, load, ambient temperature and the kinematic viscosity of the lubricating oil will not change when the gearbox works under the specificworking condition of the input shaft speed 1450 \(\mathrm{r}/\mathrm{min}\), with the ambient temperature 20 \(\mathrm{^\circ{\rm C} }\). Assuming that the heat production of a single heat generating component is higher than 20% of the normal situation, the fault diagnosis model gives an early warning of possible faults of the gearbox. The working conditions of the gearbox are shown in Table 4.

Based on these two assumptions, the surface temperature field distribution of the gearbox housing is calculated when the heat production of each heat producing part is between 100 and 120% of the normal heat production. It is considered that the gearbox is in normal working condition at this time, and the surface temperature field image of the gearbox housing at this time is obtained. Then calculate the surface temperature field distribution of the gearbox when the heat production of a heat production part is between 120 and 140% of the heat production under normal conditions and the heat production of other heat production parts is under normal conditions. It is considered that the gearbox is in the fault state of the heat production part at this time, and the surface temperature field image of the gearbox at this time is obtained. The specific gradient can be selected according to the actual situation, and each calculation result is considered as a case. The obtained images are classified according to the training set and the verification set and operated according to the specific diagnosis process of the fault diagnosis model proposed above.

This paper simulates 150 cases of normal, 150 cases of fault 1(Part c in Fig. 1 fails) and 150 cases of fault 2(Part j in Fig. 1 fails). A total of 450 images of gearbox temperature field distribution were obtained, including 150 images under normal conditions, 150 images of fault 1 and 150 images of fault 2. Each one takes 125 pictures as the training set and 25 pictures as the verification set.

T-Distributed Stochastic Neighbor Embedding (t-SNE) is an unsupervised nonlinear technology, which is mainly used for data exploration and visualization of high-dimensional data. The image input to the model, the output of the CNN layer, and the output of the DBN layer are visualized using the t-SNE method. As shown in Fig. 5, the various types of data in the original image overlap together, and different fault types have shown some separability after passing through the CNN layer. After passing through the DBN layer, each type of data is completely separated, indicating that the model has better fault characteristics.

T-SNE dimensionality reduction visualization.

The final classification results of the model are shown in Fig. 6. From the confusion matrix, it can be seen that the TrCNN-DBN model proposed in this paper has 100% classification accuracy for each simulation condition. And f-score value, the recall rate and the precision rate of each working condition are 1, as shown in Fig. 7. The relationship between the accuracy of the model, the loss and the number of iterations is shown in Fig. 4m, n. It can be seen that after 150 epochs, the loss of the model on the training set and the verification set is reduced to close to 0. The accuracy is also 100% and almost unchanged. TrCNN-DBN proposed in this paper has high fault diagnosis accuracy for the steady-state temperature field distribution image of gearbox simulated by finite element.

Confusion matrix of three-direction view.

Precision rate,recall rate and F-score of the proposed method.

In practical work, on the one hand, some parts of the gearbox are more prone to failure than other parts. When a fault occurs, the whole gearbox will be detected,so those parts that are not prone to failure can be maintained during regular testing. The thermal imaging equipment is more expensive. In order to reduce the cost, it is necessary to reduce the number of thermal imaging equipment. On the other hand, due to the installation position of the gearbox, it may not be able to get three-direction thermal images. Therefore, this paper proposes the use of single-direction thermal image for fault diagnosis.

The fault diagnosis is carried out by using the temperature field picture in one of the three directions intercepted above and the fault diagnosis model proposed above. The calculation results are shown in Fig. 8. As can be seen from the confusion matrix, the classification accuracy of the model on the top view and the main view is 100%, but the accuracy on the left view is 97%.

Confusion matrix of single-direction view.

It can be found from Fig. 4 that the accuracy of the model on the top view has been 100% and stabilized after 60 epochs. Accuracy on the main view is close to 100% after 100 epochs, but still fluctuates. The accuracy on the left view stabilized at 97% after 100 epochs. And Fig. 7 also shows that the accuracy of the model for left view fault identification is lower than that of the other two directions. The above results show that the top view direction is the temperature sensitive direction of the two simulated faults, and fault diagnosis in this direction will have higher accuracy and stability. Comparing the results with the calculation results of the three directions images, it is found that the data of the fault insensitive direction will affect the classification results of the sensitive direction.

In this paper, the finite element temperature field simulation image of the gearbox is combined with deep learning to apply to the gearbox condition monitoring (FETFS). The accuracy of TrCNN-DBN and convolutional neural network models is compared, and the classification results of TrCNN-DBN are described in detail. It is verified that the training time required by TrCNN-DBN is one fifth of that of convolutional neural network model. Moreover, TrCNN-DBN has high fault judgment accuracy for both three-way thermal images and one-way thermal images. The FETFS method can greatly expand the temperature field distribution database when the gearbox fails. TrCNN-DBN applies the transfer learning of the convolutional neural network to the fault diagnosis of the gearbox, reducing the number of parameters that need to be trained and saving time and equipment costs. The results show that TrCNN-DBN is sensitive and reliable for the diagnosis of the simulated temperature field images in three directions of single fault of the gearbox, and the accuracy of fault identification under the set working conditions and fault conditions is above 97%. In addition, it is proved that the simulation images in a single direction can be used to diagnose faults for specific faults, and the accuracy can also reach reach more than 97%.

This paper verifies that the fault identification of gearbox can be well carried out by calculating the temperature field image of the model. Finite element model updating based on experimental results is a mature technology that has been widely used20,21,22,23. Therefore, the method proposed in this paper can be tested according to the application object, and more accurate pictures can be obtained by modifying the model. Therefore, this paper is of great significance for establishing the digital twin heat transfer model of gearbox.

The authors agree to the authorship order.

The data that support the findings of this study are available from Nanjing High Speed Gear Manufacturing Co., Ltd. but restrictions apply to the availability of these data, which were used under license for the current study, and so are not publicly available. If you need to get data, you can contact the corresponding author of this article.

Configuration constant

Tooth face width, m

Correction coefficient of temperature difference

Constants of proportionality

Drag moment coefficient of liquid agitation

Drag torque coefficient

Diameter of gearbox, m

Shaft diameter, m

Bearing pitch diameter, m

Gear pitch diameter, m

Pitch diameter, m

Darcy resistance coefficient

Coefficient related to

Gear wetting factor

Coefficient of load

Froude number

Rolling traction force, N

Slidling force, N

Convection heat transfer

Immersion depth of gear in oil, m

Transmission ratio

Oil heat conductivity, W/(m K)

Gearbox length, m

Characteristic size, m

Module

Exponential constant

Gear ratio, Ng/Np

Total friction torque, N/m

Friction moment caused by load, N/m

Bearing friction torque related to lubricating oil viscosity, N/m

Rotational speed, rpm

Average Nusselt number

Power, KW

Equivalent dynamic load, N

Rolling lose, KW

Average prandtl number

Sliding loss, KW

Windage loss, KW

Oil stirring power loss, KW

Prandtl number

Radius of gear base circle, m

Gear pitch radius, m

Reynolds number

Submerged surface area, m2

Fluid kinematic viscosity, m2/s

Bearing cage speed, m/s

Lubricating oil volume, V

Rolling speed, m/s

Sliding velocity, m/s

Gear

Pinion

Solar gear

Gear ring

Planetary gear

Planet carrier

Efficiency

Air heat conductivity, W/(m K)

Fluid density, kg/m3

Constant coefficient obtained by experiment

Rotational angular velocity, rad/min

Lubricant absolute viscosity, Pa s

Transmission loss coefficien

Li, X., Cheng, J., Shao, H., Liu, K. & Cai, B. A fusion CWSMM-based framework for rotating machinery fault diagnosis under strong interference and imbalanced case. IEEE Trans. Industr. Inf. 18(8), 5180–5189. https://doi.org/10.1109/TII.2021.3125385 (2022).

Article Google Scholar

Li, Y., Gu, J. X., Zhen, D., Xu, M. & Ball, A. An evaluation of gearbox condition monitoring using infrared thermal images applied with convolutional neural networks. Sensors. 19(9), 2205. https://doi.org/10.3390/s19092205 (2019).

Article ADS PubMed PubMed Central Google Scholar

Yongbo, L. I. et al. Rotating machinery fault diagnosis based on convolutional neural network and infrared thermal imaging. Chin. J. Aeronaut. 167(02), 42–53 (2020).

Google Scholar

Laborda, A. et al. Fatigue assessment of multilayer coatings using lock-in thermography. Mater. Des. 141, 361–373 (2018).

Article CAS Google Scholar

Lim, G. M., Bae, D. M. & Kim, J. H. Fault diagnosis of rotating machine by thermography method on support vector machine. J. Mech. Sci. Technol. 28(8), 2947–2952 (2014).

Article Google Scholar

Kwan, C. M., Xu, R., Haynes, L. S., Rozlosnik, A. E. & Dinwiddie, R. B. Gearbox failure prediction using infrared camera. SPIE Int. Soc. Opt. Eng. 4360, 285–289 (2001).

ADS Google Scholar

Younus, A. M., Achmad, A. & Yang, B. S. Evaluation of thermography image data for machine fault diagnosis. Nondestruct. Test. Eval. 25(3), 231–247 (2010).

Article ADS Google Scholar

Younus, A. & Yang, B. S. Intelligent fault diagnosis of rotating machinery using infrared thermal image. Expert Syst. Appl. 39(2), 2082–2091 (2012).

Article Google Scholar

Resendiz-Ochoa, E., Saucedo-Dorantes, J. J., Benitez-Rangel, J. P., Osornio-Rios, R. A. & Morales-Hernandez, L. A. Novel methodology for condition monitoring of gear wear using supervised learning and infrared thermography. Appl. Sci. 10(2), 506. https://doi.org/10.3390/app10020506 (2020).

Article Google Scholar

Shao, H., Xia, M., Han, G., Zhang, Y. & Wan, J. Intelligent fault diagnosis of rotor-bearing system under varying working conditions with modified transfer convolutional neural network and thermal images. IEEE Trans. Industr. Inf. 17(5), 3488–3496 (2021).

Article Google Scholar

Li, Y., Wang, X., Si, S. & Du, X. A new intelligent fault diagnosis method of rotating machinery under varying-speed conditions using infrared thermography. Complexity 2019(5), 1–12 (2019).

Google Scholar

Bai, T., Zhang, L., Duan, L. & Wang, J. Nsct-based infrared image enhancement method for rotating machinery fault diagnosis. IEEE Trans. Instrum. Meas. 65(10), 2293–2301 (2016).

Article ADS Google Scholar

Anderson, N. E., Loewenthal, S. H. & Black, J. D. An analytical method to predict efficiency of aircraft gearboxes. ASME. J. Mech. Trans. Autom. 108(3), 424–432. https://doi.org/10.1115/1.3258750 (1986).

Article Google Scholar

Xiang, X. Computer simulation and calculation for heat transfer in gear case. Journal of Mechanical Transmission. 27(5): 1-4 (2003).

Mechanical Design Manual Editorial Board. Mechanical Design Manual. Vol. 6. (Machinery Industry Press, 2004).

BS ISO/TR14179-1-2001.

Honghai, X. U., Jiang, J., & Yuan, H. U. Modeling and simulation of steady temperature field analysis on high speed train driving gearbox. Lubr. Eng. 44(9), 44–49 (2019).

Zhao, N., Peng, Y. & Lin, Y. General thermal analysis of helicopter gearbox based on pca-tn. Hangkong Dongli Xuebao/J. Aerosp. Power 32(7), 1676–1682 (2017).

Google Scholar

Shiming, Y. & Wenquan, T. Heat Transfer 4th edn, 248–258 (Higher Education Press, 2006).

Google Scholar

Pan, L., Zhang Bo, Wu., Wenmin, Z. Y. & Yumei, Hu. Simulation study on gear splash lubrication and temperature field. Automob. Technol. 9, 42–48 (2019).

Google Scholar

Wu, X., Chen, L. Heat balance analysis of planetary gear train of an earth pressure balance shield machine. China Mech. Eng. 23(7), 777–782 (2012)

Daqian, P. et al. Transient temperature field of planetary gear system in electro-mechanical transmission under different working conditions. Acta Armamentarii 10, 2268–2277 (2021).

Google Scholar

Yueming, Z., Hao, Z., Shuting, J. & Tianyu, Z. Analysis of steady-state temperature field about internal gear drive meshing surface. J. Nanjing Univ. Aeronaut. Astronaut. 1, 157–162. https://doi.org/10.16356/j.1005-2615.2022.01.018 (2022).

Article Google Scholar

Download references

This work was partly supported by the National Natural Science Foundation of China (52275092) and Jiangsu industrial and information industry transformation and upgrading Project (7602006021).

School of Mechanical Engineering, Southeast University, Nanjing, 211189, China

Xi Lu & Pan Li

You can also search for this author in PubMed Google Scholar

X.L. designed the study and drafted the paper. P.L. was involved in Programming and data analysis. All authors have read and agreed to the published version of the manuscript.

Correspondence to Xi Lu.

The authors declare no competing interests.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and Permissions

Lu, X., Li, P. Research on gearbox temperature field image fault diagnosis method based on transfer learning and deep belief network. Sci Rep 13, 6664 (2023). https://doi.org/10.1038/s41598-023-33858-w

Download citation

Received: 20 October 2022

Accepted: 20 April 2023

Published: 24 April 2023

DOI: https://doi.org/10.1038/s41598-023-33858-w

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.