- Research
- Open Access
- Published:

# Response Simulation, Data Cleansing and Restoration of Dynamic and Static Measurements Based on Deep Learning Algorithms

*International Journal of Concrete Structures and Materials*
**volume 12**, Article number: 82 (2018)

## Abstract

In this study, an output-based neuro controller was built based on the idea of the adaptive neuro-fuzzy inference system (ANFIS) and its capabilities in response simulation, data cleansing and restoration capability were verified using measurement data from actual structural testing. The ANFIS is a family of the deep learning algorithm, which incorporates the benefits of adaptive control technique, artificial neural network, and the fuzzy inference system. Thus, it is expected to produce very accurate predictions even for the highly nonlinear system. Forced vibration responses of a five-story steel building were simulated by ANFIS and its accuracy was compared with the results of Recurrent Neural Network (RNN), which is a type of traditional artificial neural networks. Simulations by ANFIS were very accurate with a much lower root means square error (RMSE) than RNN. Simulated data by ANFIS showed an almost perfect match with the original. Even the small ripples in the power spectrum plot outside the dominant frequency were successfully reproduced. In addition, the ANFIS was used to increase the sampling rate of dynamic data. It was shown that missing high-frequency contents could be successfully reproduced when the ANFIS was properly trained. Lastly, The ANFIS was applied to remove the noise in the measured data from RC column cyclic load tests. The outliers were corrected effectively, but the tendency of flattening the peak values was observed.

## Introduction

Performance data collected from structural testing have been the fundamental resource for the development of structural engineering. Efforts to simulate observed responses from controlled input by mathematical modeling lead to a finding of principal laws and progress of design equations. Simple parametric modeling, although they are effective in extracting key parameters affecting the behaviors, has limitations in accurate simulation of observed responses especially for complicated or nonlinear systems.

Artificial neural networks have been used to determine the parameters or simulate the response of unknown systems from correlations between inputs and outputs of training data (Imam et al. 2015; Sathyan et al. 2018). Nowadays, advances in artificial intelligence such as the machine learning and deep learning techniques integrated with adaptive control algorithms and/or fuzzy logic inference system allow us accurate simulations of complicated and nonlinear systems. According to Fu (1970) and Gupta and Saridis (1977), the intelligent intelligence can be defined as a combination of control theory and information technology to imitate human behavior such as ‘learning’ and ‘problem solving’. It involves interpretation of input data, treatment of the ambiguity in human perception for transformation to machine knowledge (Gupta and Saridis 1977; Saridis and Lee 1979; Fu and Gonzalez 1987; Fu 1970).

Deep learning is a class of machine learning algorithms that use multiple layers of nonlinear processing neuron units for feature extraction. An adaptive neuro-fuzzy inference system (ANFIS) is a kind of deep learning algorithm that is a combination of the adaptive control technique, artificial neural network, and the fuzzy inference system. Fuzzy logic can represent the ambiguity of human perception or decision into a mathematical model (Zeung 1997). The Artificial neural network is an implemented algorithm that performs “learning”. The neuro-fuzzy system is the combination of the two theories, which enables learning in terms of human-like reasoning. Adding the adaptive control technique minimizes the errors between the original data and the estimate by the neuro-fuzzy system.

In this study, an output-based neuro controller was built based on the idea of the ANFIS and its capabilities in response simulation, data cleansing and restoration capability were verified using measured data from actual structural testing. Forced vibration responses of a five-story steel building was simulated by ANFIS and accuracy of simulations was compared with the results of the recurrent neural network, which is a family of traditional artificial neural networks. In addition, a capability of the ANFIS was extended to up-sampling of data with a lower sampling rate. Missing information in high frequency range was reproduced by ANFIS based on learned knowledge from previous training. Lastly, learning and prediction capability were applied to data cleansing. Contaminated data with noise collected from RC column cyclic load tests were corrected by means of ANFIS.

## Deep Learning Algorithm

### RNN (Recurrent Neural Network)

The deep learning is one of the algorithms developed in the field of artificial intelligence, which uses artificial neurons that mimic the mechanism of human brains. The artificial neurons consist of a weighting vector, a confluence operator, and an activation function as shown in Fig. 1 (Schalkoff 1997). The weighting vector ω_{i} contains the accumulated knowledge acquired from a training dataset. When a new input is provided, the artificial neurons test similarity with stored information and indicate the result. This process can be stated mathematically as follows: The confluence operator calculates a weighted sum of the new input, and the activation function generates the output by comparing the value with the threshold. The activation function *h*(*x*) can be expressed as Eq. 1.

where

The functions of the neural network are two-fold, which are ‘output generation’ stated above and ‘learning’. The learning is a process of determining the adjustment \( \Delta w_{i} \) to the current weighting vector \( w_{i} \) representing learned experience and knowledge up to now. The methods of learning fall into two classes: supervised learning and unsupervised learning. In supervised learning, training data are provided so that the relationship between input and output can be inferred and applied to predict output to new input. The backpropagation algorithm that is widely used for current neural network systems is a kind of supervised learning (Fahlman 1988; Becker 1988).

Figure 2 is a schematic diagram representing the structure of the backpropagation algorithm. The key of backpropagation algorithm is that the error between the desired output *y*_{d}(*t*) and the output *y*(*t*) of the neurons is fed back to the network to optimize the weights in each neuron to minimize the error.

However, since the neuron structure in Fig. 2 performs ‘learning’ from each data set, simulating time-dependent (or order-dependent) events in which current response is affected by the previous ones is not feasible. Thus, a form of recurrent neural networks (RNN) was developed for modeling of a system whose property changes with time. The issue of many recent type of researches in ANN is how to improve the performance by adopting a new structure or adding another algorithm to deal with sequential information data (Mikolov 2010; Zaremba et al. 2014; Sak et al. 2014; Barbounis et al. 2006).

Unlike the general multi-layer neural network, the RNN shown in Fig. 3 has a concept of ‘time step’. The weights U, V, and W at various time steps are not evaluated independently but share the previous weights. Because current weights are dependent on the previous state, the current output reflects the previous state.

### ANFIS (Adaptive Neuro-Fussy Inference System)

Fuzzy logic is an intelligent controller that simulates human behaviors by incorporating ‘If–then rules’ into the system, which contains human experience or knowledge (Mendel 2001; Klir and Yuan 1995; Lee 1990; Ross 2009). Fuzzy logic control (FLC) has been widely used in industries. A well-known example is a human-in-the-loop system that is applied for vehicle control, flight control, and automation of startup and shutdown of large chemical processes. Operation of such machines or factory by the human is actually of a nonlinear control based on know-hows of operators or engineers. The human-in-the-loop system is intended to mimic the human behaviors.

Since the purpose of artificial neural networks and the fuzzy logic is identically to simulate the human memory and cognition, some researches tried to solve this problem by combining the two methods. Gupta limited the inputs *x*_{a}(*t*) and weights *ω*_{a}(*t*) to [0, 1]^{n+1} to obtain the results similar to those from the fuzzy set (Gupta and Saridis 1977). Yamakawa identified the nonlinear system by calculating the weights considering the correlation between only two adjacent neurons rather than those of all neurons (Yamakawa 1993).

Based on previous researches, the nonlinear control problem was formulated as Eq. 2 in this study.

When the correct system \( g\left[ \cdots \right] \) which makes the error function *e*(*k*) = *y*(*k*) − *y*_{m}(*k*) vanishes is known, the control function *u*(*k*) and output *y*(*k*) can be derived easily as Eqs. 3 and 4. Here, *y*_{m}(*k*) indicates the output of the model.

However, correct \( g[ \cdots ] \) is not a priori in general. Thus, an approximate function \( \hat{g}[ \cdots ] \) which minimizes the error function *e*(*k*) is used instead. When the fuzzy logic is adopted, \( \hat{g}[ \cdots ] \) can be identified by a feedback-parallel model as in Fig. 4.

The adaptive neuro-fuzzy inference system (ANFIS) (Jang 1993; Brown and Harris 1994; Kurnaz et al. 2010; Feng 2006), shown in Fig. 5, integrates the fuzzy identifier and the idea of a backpropagation neural network. The approximate function \( \hat{g}[ \cdots ] \) is identified more accurately by adjusting the fuzzy identifier using a gradient *δy*/*δu* calculated based on the output of the fuzzy model. The essential advantage of ANFIS is that it can approximate arbitrary nonlinear functions. Therefore, ANFIS is considered as a universal estimator. In this study, the fuzzy model proposed by (Sugeno 1985; Silverman 2018; Bak and Son 2017) is used in ANFIS.

Figure 6 is a block diagram of the output-based neuro controller based on ANFIS, in which the state variables that cannot be measured directly are modeled as Eq. 5.

## Performance Comparison of RNN and ANFIS

As stated in Sect. 2, the deep learning technique based on the artificial neural network is capable of approximating the system properties by finding the similarity in training data. Such ‘learning’ ability can be used to simulate the response of a system to arbitrary inputs. Especially, when the relationship between the input and response of the system is of a nonlinear fashion and its responses are dependent on the responses at the previous time, the RNN and ANFIS are accepted as viable algorithms. In this section, performances of simulation (i.e. response prediction) by RNN and ANFIS were compared using actual vibration data measured from a real building. Ambient vibration measurement and forced vibration testing were performed on the building (Heo et al. 2017a, b; Lee et al. 2017). Measured acceleration data were provided for learning, and then simulations by RNN and ANFIS, respectively, were made subsequently. Simulated responses were compared with actual response measurement.

The building used for this study is a small five-story steel structure. The height and the total weight of the building were 20 m and 447 kN, respectively. As shown in Fig. 7, two shakers and four accelerometers were installed on the top floor and used for the study. Natural frequencies of the building identified from white noise excitation were 0.83 Hz for the first mode, 0.85 Hz for the second mode, and 0.96 Hz for the third mode.

Measured data of ambient vibration during strong winds and forced vibration testing were used for “learning” of the neural networks. Forced vibration testing consists of white noise excitation and harmonic excitation at 0.96 Hz which corresponds to the natural frequency of the third mode. The neural networks were configured with four inputs, four outputs, and four in-depth networks. Vibrations of shakers were provided as the input and simulation of output (acceleration response at top floor) were made by the RNN and ANFIS, respectively. The number of neurons used for neural networks were determined by trial and error. Simulations using from two layers to ten layers were made and it was decided to use four layers which is the minimum number of layers that generated sufficiently accurate results.

Figures 8, 9 shows simulated responses and their power spectra in comparison with measured ones. Note that the first two graphs are simulations for white noise excitation and the last two graphs are for 0.96 Hz harmonic excitation. The accuracy of the simulation was quantified using the root means square error (RMSE) and indicated in each graph. The RMSE is calculated using Eq. 6 for two waveforms *x*_{1} and *x*_{2}.

As can be seen from the comparison of Figs. 8, 9, the ANFIS showed better accuracy than RNN. The RMSE in the case of ANFIS was as low as 0.5 m/s^{2}, while those of RNN ranged from 0.15 to 0.57 m/s^{2}. In the case of RNN, deviations of simulated data from measured ones can be seen more explicitly in the frequency domain. The peak of PSD appeared around 0.96 Hz which is the natural frequency of the structure. However, the shape of the PSD did not exactly match with the measured ones. Especially, as can be seen in (c2) graph representing the PSD of the response in a perpendicular direction to the direction of excitation, the simulated response has one single peak while actual measurement showed two closely spaced peaks. Since the frequency contents of the simulation are different from the measured data, it can be concluded that the simulation by RNN is not complete. On the contrary, simulations by ANFIS as shown in Fig. 9 showed an almost perfect match for all cases. The PSD plot in c2 graph has two peaks and even the small ripples outside the dominant frequency were successfully reproduced.

The problem of ANN-based simulation or control is that the identification process is black-boxed and thus the cause of failure cannot be found easily. The meaning of the weight vectors connected to the neural network is hard to understand. When the results are unsatisfactory, one may try with new training data set and/or a different number of neuron layers. The intrinsic weakness of RNN is that it constructs the system (learning) mainly to find a dominant relationship in the training data. Thus, it is difficult to correspond to multiple modes and non-dominant parts. The neuro-fuzzy identifier incorporated in ANFIS has the capability of complementing this weakness by handling mathematically ambiguous parts with the fuzzy logic.

## Applications of ANFIS

It was showed in the previous section that the ANFIS has better performance than RNN. In this section, applications of ANFIS to both dynamic data and static data are represented.

### Restoration of Lost Information in Dynamic Data

In dynamic analysis or testing, the important values extracted from the data would be peak values and their frequencies. Generally, measurements during dynamic testing are collected at a much higher sampling frequency than the natural frequencies of the structure. Accordingly, contents contained in high frequency range have relatively little importance when the behavior of the structure is the main concern of the test. Thus, it is customary to decimate the original data to a low sampling frequency for system identification analyses. However, frequency contents in the high frequency range may become important when the interactions between the building and nonstructural components are investigated since the natural frequencies of some nonstructural components are much higher than those of the building structures.

It is well known that the lost information due to the decimation of the signal to a lower sampling frequency cannot be recovered. Frequency contents of signals at a higher sampling rate by simple interpolation have completely different from those of the original signal. However, the lost information in high frequency range can be reproduced by using the ANFIS. In this section, the signal restoration capability of ANFIS is demonstrated using the same dataset as in Sect. 3. A channel of measured floor acceleration at the test building was intentionally decimated to 1/20 of the original sampling rate. The ANFIS trained in Sect. 3 was reconfigured to produce up-sampled data from decimated data.

Figure 10b shows details of decimated data and reproduced data compared with original data. Here, it can be seen that reproduced signal has a similar trend with the original although a perfect match was not possible. The RMSE between the reproduced and the original data was below 0.01. Comparisons of PSD are shown in Fig. 11. The frequency contents of the reproduced signal above the Nyquist frequency of the decimated signal, 10.2 Hz, were very similar to those of the original one.

The procedure of data restoration in this Section is similar to the response simulation in Sect. 3. Difference between two applications is the structure of input and output of the ANFIS. Once the neuro-fuzzy inference system was trained properly for response simulation, lost information in high frequency range can be successfully recovered with simple modifications in input and output structures in ANFIS, applying the rules acquired by learning.

The up-sampling capability of ANFIS was applied to earthquake ground motions. As an example, the Imperial Valley earthquake (1940, El Centro, NS component) sampled at 50 Hz was decimated to 5 Hz and then up-sampled to 50 Hz using trained ANFIS in the previous example. Figures 12, 13 compared the original ground motion and reproduced one in the time domain and the frequency domain, respectively. The peaks of the waveforms look similar to each other with differences less than 10%. The RMSE value was 0.03 m/s^{2}. Low frequency range below 10 Hz, which affects the response of building structures, was restored to almost the same as the original. The high frequency band above 10 Hz showed some differences. This is seemingly caused because the ANFIS was trained using response data of a building rather than ground motions. However, considering the uncertainty of ground motions, which may vary with fault distance and soil property etc., reproduced one seems acceptable when real data are not available.

### Data Cleansing of Static Data—RC Column Cyclic Load Test

During structural testing, it is often the case that noise contaminates parts of data due to malfunctioning of the data acquisition system. In this section, an application of ANFIS for data cleansing is presented. Example data were collected from RC column cyclic load tests as shown in Fig. 14. Figure 15 shows the original data, the force and displacement relationship. As can be seen, some of the displacement data contain abnormal values, unrealistically sudden jump from neighboring values, which is seemingly caused by the electric noise of the data acquisition system.

First, the ANFIS was trained by the output-based algorithm shown in Fig. 6. Since the test was conducted by way of a displacement-controlled quasi-static test, in which the load was increased at slow rates until the displacement reached predefined values, the displacement was expected to increase or decrease monotonically except the peak points. Thus, to train such rule, the predefined displacement history for the test in which five cycles were repeated at a target displacement was used as the training data. Figure 16 is a graph comparing the original data and the corrected data by ANFIS.

Applying the median filter is a typical method when removing the noise from the signal. The median filter is a nonlinear digital filter popularly used to eliminate scattered noise from an image, video, or signal. Figure 17 compares of the original, corrected displacements by ANFIS and median filter, respectively. Results by the median filter were not smooth but had saw-tooth shapes due to the noise. The effect of the noise was reduced but was not removed completely since the median filter is a type of averaging filter. The ANFIS distinguished outliers successfully and corrected curves looks almost identical to original data excluding the outliers. However, the ANFIS also had a drawback. As shown in Fig. 17b, the ANFIS tends to flatten the peak values, which is important values in structural testing, while the median filter kept the peak values correctly. It seems like the ANFIS regarded the peak values as the noise and compensated the value. The difference was 17% at most. The median filter shows better prediction near the peak. Thus, for practical application of noise reduction, an approach integrated with the ANFIS and the median filter would produce better results.

## Conclusion

The adaptive neuro-fuzzy inference system (ANFIS) is a family of deep learning algorithm, which incorporates the benefits of adaptive control technique, artificial neural network, and the fuzzy inference system. Fuzzy logic can represent ambiguity of human perception or decision into a mathematical model. The artificial neural network is an implemented algorithm that performs “learning”. The neuro-fuzzy system is the combination of the two theories, which enables learning in terms of human-like reasoning. Adding the adaptive control technique minimizes the errors between the original data and the estimate by the neuro-fuzzy system. Thus, the ANFIS is expected to produce very accurate predictions even for a highly nonlinear system. In this study, an output-based neuro controller was built based on the idea of the adaptive neuro-fuzzy inference system (ANFIS) and its capabilities in response simulation, data cleansing and restoration capability were verified using measurement data from actual structural testing.

For verification of response simulation, forced vibration responses of a five-story steel building were simulated by ANFIS and its accuracy was compared with the results of RNN, which is a type of traditional artificial neural networks. Simulations by ANFIS were very accurate with a much lower root means square error (RMSE) than RNN. Simulated data by ANFIS showed an almost perfect match with the original. Even the small ripples in PSD outside the dominant frequency were successfully reproduced.

In addition, the ANFIS was applied to increase the sampling rate of dynamic data. To this end, the ANFIS trained for the response simulation of test building was reconfigured. Measured vibration data of the test building were decimated to 1/20 of the original sampling rate and up-sampled. It was shown that missing high frequency contents were reproduced with small differences from the original.

Lastly, learning and prediction capability of ANFIS were applied in data cleansing. Contaminated data with noise collected from RC column cyclic load tests were corrected by means of ANFIS. The outliers were corrected effectively but the tendency of flattening the peak values were observed. The ANFIS corrected the outliers effectively but the tendency of flattening the peak values was observed, the latter could be complemented by additional use of the median filter.

## References

Bak, C., & Son, H. (2017). Development of prediction model for root industry production process using artificial neural network.

*Journal of the Korean Society for Precision Engineering, 34*(1), 23–27.Barbounis, T. G., Theocharis, J. B., Alexiadis, M. C., & Dokopoulos, P. S. (2006). Long-term wind speed and power forecasting using local recurrent neural network models.

*IEEE Transactions on Energy Conversion.,**21*(1), 273–284.Becker, S., & Le Cun, Y. (1988) Improving the convergence of back-propagation learning with second order methods. In

*Proceedings of the 1988 Connectionist Models Summer School*(pp. 29–37). San Matteo: Morgan Kaufmann.Brown, M., & Harris, C. J. (1994).

*Neurofuzzy adaptive modelling and control*. Upper Saddle River: Prentice Hall.Fahlman S. E. (1988). An empirical study of learning speed in back-propagation networks.

Feng, G. (2006). A survey on analysis and design of model-based fuzzy control systems.

*IEEE Transactions on Fuzzy systems.,**14*(5), 676–697.Fu, K.-S. (1970). Learning control systems–Review and outlook.

*IEEE Transactions on Automatic Control,**15*(2), 210–221.Fu, K. S., Gonzalez, R., & Lee, C. S. G. (1987).

*Robotics: control sensing. Vis*. New York: Tata McGraw-Hill Education.Gupta, M. M., & Saridis, G. N. (1977).

*Fuzzy automata and decision processes*. New York: North Holland.Heo, S. J., Park, H. H., Mun, D. H., & Lee, S. H. (2017a). System identification of a 5 story steel frame under ambient wind loads using RTK-GPS.

*Journal of the Wind Engineering Institute of Korea,**21*(1), 29–35.Heo, S. J., Park, H. H., Mun, D. H., Lee, S. H. (2017b). System identification of five-story building on wind load using RD method. In

*Proceedings of WEIIK Symposium*(pp. 55–58).Imam, A., Anifowose, F., & Azad, A. K. (2015). Residual strength of corroded reinforced concrete beams using an adaptive model based on ANN.

*International Journal of Concrete Structures and Materials, 9*(2), 159–172.Jang, J.-S. R. (1993). ANFIS: adaptive-network-based fuzzy inference system.

*IEEE Transactions on Systems, Man, and Cybernetics,**23*(3), 665–685.Klir, G., & Yuan, B. (1995).

*Fuzzy sets and fuzzy logic*. New Jersey: Prentice hall.Kurnaz, S., Cetin, O., & Kaynak, O. (2010). Adaptive neuro-fuzzy inference system based autonomous flight control of unmanned air vehicles.

*Expert Systems with Applications.,**37*(2), 1229–1234.Lee, C.-C. (1990). Fuzzy logic in control systems: fuzzy logic controller. I.

*IEEE Transactions on Systems, Man, and Cybernetics,**20*(2), 404–418.Lee, S. H., Jeon, M. J., Mun, D. H., Heo, S. J., Park, C. Y., & Woo, S. S. (2017). The multi-mode identification of the 5-story structure using active mass damper.

*Proceedings of AIK Symposium,**37*(2), 696–697.Mendel, J. M. (2001).

*Uncertain rule-based fuzzy logic systems: introduction and new directions*. Upper Saddle River: Prentice Hall PTR.Mikolov, T., Karafiát, M., Burget, L., Černocký, J., & Khudanpur, S. (2010). Recurrent neural network-based language model. In

*Eleventh Annual Conference of the International Speech Communication Association*.Ross, T. J. (2009).

*Fuzzy logic with engineering applications*. Hoboken: Wiley.Sak. H., Senior. A., & Beaufays. F. (2014). Long short-term memory recurrent neural network architectures for large scale acoustic modeling. In

*Fifteenth Annual Conference of the International Speech Communication Association*. 2014.Saridis, G. N., & Lee, C.-S. G. (1979). An approximation theory of optimal control for trainable manipulators.

*IEEE Transactions on Systems, Man, and Cybernetics,**9*(3), 152–159.Sathyan, D., Anand, K. B., Prakash, A. J., & Premjith, B. (2018). Modeling the fresh and hardened stage properties of self-compacting concrete using random kitchen sink algorithm.

*International Journal of Concrete Structures and Materials, 12*(1), 24.Schalkoff, R. J. (1997).

*Artificial neural networks*. New York: McGraw-Hill.Silverman, B. W. (2018).

*Density estimation for statistics and data analysis*. Abringdon: Routledge.Sugeno, M. (1985).

*Industrial applications of fuzzy control*. Amsterdam: Elsevier Science Inc.Yamakawa, T. (1993). A fuzzy inference engine in nonlinear analog mode and its application to a fuzzy logic control.

*IEEE Transactions on Neural Networks.,**4*(3), 496–522.Zaremba, W,, Sutskever, I., Vinyals, O. (2014). Recurrent neural network regularization. arXiv preprint arXiv:1409.2329.

Zeung, N. B. (1997).

*Fuzzy logic control*. South Korea: Hong Reung Science Publishing Co.

## Authors’ contributions

SH modeled ANFIS controller and performed simulation using measured data. ZC performed literature survey regarding ANN. EY was a major contributor in writing the manuscript. All authors read and approved the final manuscript.

### Acknowledgements

This research received financial support from Ministry of Land, Infrastructure and Transport of Korean government, Project No. 18AUDP-B106327-04 And the National Research Foundation of Korea (NRF) grant funded by the Korean Government (MSIP) (No. NRF-2015R1A2A1A10054506). This support is gratefully acknowledged.

### Competing interests

The authors declare that they have no competing interests

### Availability of data and materials

Not applicable.

### Funding

Ministry of Land, Infrastructure and Transport of Korean government, National Research Foundation of Korea.

### Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

## Author information

## Additional information

Journal information: ISSN 1976-0485 / eISSN 2234-1315

## Rights and permissions

**Open Access** This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

## About this article

#### Received

#### Accepted

#### Published

#### DOI

### Keywords

- artificial intelligence
- deep learning
- adaptive neuro-fuzzy inference system
- response simulation
- data cleansing
- noise reduction
- data restoration