Optimization of Backpropagation Using Harmony Search for Gold Price Forecasting

Gold is a precious metal often used for investment, due to its cash-in ease and yearly value increase. This indicates that price forecasting is used to determine the prospect of future gold prices. Strong gold price forecasting is highly desired by investors to make decisions. That is why technical indicators are very important used for forecasting. By using technical indicators the information obtained can be more informative than using pure gold prices. One of the commonly used methods is Backpropagation (BP). BP has been shown to have good performance in dealing with nonlinear problems. However, due to the random determination of the parameters of neurons in the hidden layer BP requires a number of neurons in the hidden layer to get optimal results. Therefore, this study aims to analyze the optimization of Backpropagation (BP) through the Harmony Search (HS) algorithm by evaluating the use of relevant technical indicators for forecasting gold prices. In the HS-BP model, this method is used to determine input variables and neurons in the hidden layer. HS with the principle of musicians with the aim of finding the best harmony. This technique is used based on the results of the fitness function. In this research, the fitness function used is Mean Square Error (MSE). HS aims to optimize BP in such a way that the forecasting system provides the lowest MSE and improves the forecasting performance of gold prices. Based on this research, the input variables used are Moving Average, Relative Strength Index, and Bollinger Bands. Next, the selected variables and neurons are applied to the BP algorithm. Where the implementation uses gold closing price data for January 2020-2021. The results showed that the proposed method has better results in forecasting accuracy and convergence error. HS-BP provides a better level of gold price forecasting than the regular BP model.


Introduction
Gold is a precious metal known as an investment asset, which is easily traded in any condition. This is due to being the main commodity in the economic market (Chandar et al., 2016), which greatly affects the economy. Several investors are found to presently participate in gold investment, to maintain their wealth and earn future profits (Yuan et al., 2020). Due to several influential factors, the price of gold is observed to have high fluctuations, often characterized by nonlinearity and discontinuity. This is based on the interaction of gold with many factors, such as inflation, oil price, and dollar rate (Abdelkader et al., 2020). Therefore, gold price forecasting is very important for investor policy, to reduce risk and obtain profit. Several study experts are found to have investigated the predictability of the gold market using fundamental and technical analyses, time series prediction, and machine learning methods (Yuan et al., 2020), specifically the Artificial Neural Network having the ability to handle nonlinear, complex, and dynamic data, respectively (Chen et al., 2018); (Gu et al., 2017). Also, one of the most popular methods used is BP for gold price forecasting, which was developed by Chingpei lin (Lin, 2015). Based on the study, the BP model was able to perform more adequately than the Principal Component and Multiple Regression (PCR and MR) models, at MAPE value of 0.011%. However, BP structure had a critical point that should be carefully analyzed (Göçken et al., 2018). This was because the model needs input variables and neurons in the hidden layer, to obtain the forecasting similar to the actual results. The lack of BP also causes the inability to determine the factors to be included in the input variables and neurons. This indicates that the selection and determination of input variables and neurons are very important in BP model construction (Göçken et al., 2018), to obtain optimal forecasting results for network output. Therefore, this study proposes a trial and error method (Harmony Search) to obtain these influential factors (input variables and neurons).
Based on the improvisation process performed by musicians, HS method is an algorithm used to obtain effective and efficient relationships. This has the ability to determine optimal solutions in various problems, It has the ability to solve optimization problems in various problems, specifically in statistical fields (Göçken et al., 2018;Shabani et al., 2017). Optimization is the process of making something better, by trying variations on the initial concept and using the information obtained to get new results (Abo-Hammour et al., 2014); . Also, it is used to improve the performance of BP training, due to deciding and determining the input variables and neurons in the hidden layer. According to the study by Ping Jiang, a hybrid HS-BP optimization model was developed for forecasting processes, where the HS technique was used to select the optimal threshold value and weight for BP. The results showed that HS was able to optimize BP at MAPE value of 7.19% (Jiang et al., 2018). Therefore, this study aims to apply the HS-BP model for the international gold price forecasting of the London Bullion Market Association (LBMA), where HS is used to select and determine the input variables and neurons within the hidden layer. These influential factors are subsequently implemented into the BP algorithm, to obtain optimal forecasting results. Also, Mean Absolute Percentage Error (MAPE) is being used to compare the results of the proposed and regular HS-BP and BP models, respectively. The daily closing price of gold from January 15, 2020-2021 is subsequently used in this study, where the results obtained are very useful in reducing investment risks.
Backpropagation works well, especially on nonlinear, complex, and dynamic data. Meanwhile, Harmony Search with the working principle of music to find the best harmony has a simple concept, few parameters, no need for complicated mathematical calculations, and is easy to implement to overcome optimization problems in determining the relevant input variables and the number of neurons in the hidden layer. The input variables used are technical indicators because the information obtained can be more informative than using pure gold prices. If the determination of the input variable and the number of neurons is not correct, then less than optimal forecasting results can be obtained. This motivates the author to discuss Backpropagation Optimization (BP) through the Harmony Search (HS) algorithm on gold closing-price LBMA. This paper is classified as follows: section 2 describes methods such as Technical Indicators, BP, HS, and the techniques proposed by HS-BP. The methodology and datasets are presented in section 3. Section 4 describes the experimental analysis and performance of the proposed model. Finally, the conclusions of this work are summarized in section 5.

Method Technical Indicator
The input variables used in this study were technical indicators, which are an effective tool in determining the original market gold prices (Li et al., 2020); (Picasso et al., 2019). Using technical indicators can be more informative than using pure prices (Göçken et al., 2018) and it is a very practical way for gold analysts and fund managers to analyze the gold market. However, these are often incorrect in observing the market overview. This indicates that the technical indicators are preferably used to determine the potential patterns of gold price movements (Li et al., 2020), based on the successful implementation by study experts (Shynkevich et al., 2017). Although there are many indicators to be used, not all are likely to provide a good solution. To obtain profit, investors should be careful in the selection of useful technical indicators accurately. Technical indicators are applied as the input variables of Backpropagation (BP) to forecast gold prices. The underlying logic for using Harmony Search (HS) for variable selection is to evaluate the usefulness of indicators and eliminate irrelevant ones to simplify the model (Göçken et al., 2018). Therefore, the input variable used in this study is a technical indicator meanwhile, the output factor was the closing price of gold. The technical indicators used are as follows: Simple Moving Average (SMA) SMA is the moving average of gold prices over a certain period of time. Moving average is used to smoothen out the daily noise in the time series (Demir et al., 2020;Phooi M'ng, 2018).
( ) is computed as follows: where SMA is a simple moving average, n is moving average length, and is the closing gold price at period t.

Exponential Moving Average (EMA)
EMA is an exponentially moving average (Phooi M'ng, 2018) that uses a multiplier to place greater significance on more recent price trends (Demir et al., 2020).
( ) is computed as follows: where EMA is an exponential moving average, n is moving average length, and is the closing gold price at period t.

Relative Strength Index (RSI)
RSI is a technical analysis indicator that shows the historical strength and weakness of gold prices (Sezer & Ozbayoglu, 2018). RSI fluctuates between 0 and 100. If the value is over 70, the gold price is considered to be the "overbought". Meanwhile, if the value is under 30 the gold price is assumed to be the "oversold" region (Lien Minh et al., 2018). is computed as follows: Where RS is the average gain over a certain period of time / average loss over a specified period of time.

Bollinger Bands %B (BB)
BB is used to measure the volatility of gold prices if it is only used without other indicators. Just like the RSI, it can also be used to determine overbought and oversold conditions.
Where is the Bollinger bands, is the bottom value of BB, is the top value of BB, is the average of the highest, lowest, and closing prices, ( ) is the average closing price for n periods, is the number of standard deviations and ( , ) is the standard deviation for n periods (Lauguico et al., 2019).

Backpropagation
Backpropagation is a technique used in developing neural networks that offer multilayer training. Standard BP is an application of the gradient descent method. The purpose of BP is to find the optimal weight and bias to get the smallest error or Mean Square Error (MSE). Apart from using BP for training, it is not necessary to determine the exact initial design of the neural network architecture and parameters (Nur et al., 2015). The following equation is used to set the weights using the BP algorithm: where is the weight of neuron I in layer − 1and neuron j in layer I, is a positive number called the learning rate which is used to monitor learning steps which are usually small positive numbers.
BP has two passes through : a forward pass and a backward pass. In the forward pass, the information unit goes to the input layer and the effect spreads to the hidden layer and finally produces the output (Al-Milli, 2013). In the reverse path, there is a return along the path so that an error occurs. To reduce errors, it can be solved by modifying the neuron weights in each layer. BP is able to produce a local optimal weight that is close to, but its ability is too weak. Therefore the proposed method aims to increase the learning ability in BP, so we focus on changing the weights to determine the number of neurons in the hidden layer.

Harmony Search
HS is a metaheuristic method inspired by the improvisation process carried out by musicians (Shabani et al., 2017)(Shabani et al., 2017); (Chauhan et al., 2018), to obtain an optimal solution. A numerical optimization is an important tool in decision science and mathematical engineering systems analysis that can solve nonlinear problems (Omar & Arqub, 2014). HS has a simple concept, few parameters, does not need to do complex mathematical calculations, is easy to implement, and can handle discrete and continuous decision variables without the need for gradients. Therefore, HS in science and engineering has been successfully applied as an optimization method and is reported to be a competitive alternative for many rivals. In this study, the HS-BP model is proposed to select and determine the relevant input variables and the number of neurons in the hidden layer.
In generating the global optimal, the evaluation is carried out in a manner that depends on the optimization problem (Abu Arqub et al., 2012). In this case, there is an objective function to evaluate the obtained solution vector. The objective function is the function that will be optimized (minimum or maximum). The objective function can be written as follows: Where HM is the initial harmony memory and ( 1 ) is the objective function for the I-th solution vector which will then be searched for the minimal objective function. is called a solution. Each row in the HM matrix represents a solution vector. The objective function used is the mean square error which can be written as follows: (11) HMCR is the probability of whether a value in the HM component is used or not, and 1 − is the probability that a value is not taken from HM but is taken from a random number vector range of values. The equation is as follows: The HMCR parameter ranges between values 0 and 1. If the value is less than HMCR, a value of 1 is selected, otherwise a value of 0 is selected. The HMCR value is usually between 0.7 to 0.95. A value that has been selected is evaluated whether or not an adjustment is needed using PAR. This PAR parameter ranges from 0 to 1. The PAR parameters that determine the probability of adjustment are as follows: bw is the bandwidth i.e. the step size of the movement, and rand(0,1) is a random number between 0 to 1. If the value is 1 in both PAR and HMCR, that value is selected for the new harmony. Another value is selected as 0. In the adjustment stage the value of ′ will be chosen randomly, the value of ′ is in the range and has a probability of HMCR.

HS-BP Forecasting Model
In this study, the HS-BP model was proposed to select and determine the input variables and neurons within the hidden layer, respectively. The steps to obtain relevant optimal solutions are shown as follows: 3. Calculating the value of the objective function f(x), based on the early initialization of HM. 4. Determine the random value to obtain the new solution. 5. When the random value > HMCR, a new solution was randomly obtained with a range from , indicating a return to step 4. However, a new solution was obtained from HM when the rand value ≤ HMCR. 6. When the rand value > PAR, the need and search for adjustments and new solutions were not required.
However, adjustments were made when the rand value ≤ PAR. 7. When j ≤ N (many decision variables), return to step 3. Meanwhile, proceed to the following stage when j > N. 8. When the new solution vector has a smaller objective function f(x) compared to the old type, the worst HM factor should be replaced with the best variables. 9. Go back to step 4. 10. The search for a new solution was expected to stop when the maximum repetition (t) had been identified.
In using HS, it was used to train feed-forward BP with the same network architecture and objective function to evaluate the proposed technique of HS-BP. The HS parameter used is harmony memory 8 in each iteration. The learning rate (HMCR) is set to 90% probability of a value on the HM component being used. 30% (PAR) probability is required to adjust the value of the solution vector. All algorithms are coded and executed on the same computer using R Studio.

⋯
] was obtained and applied to BP method, where 1 = the units in the hidden layer, and 2 , ⋯ , = the decision variable (i.e., whether the input variable was used). When the value of the decision variable = 0 and 1, the factor was observed to be unused and used, respectively (Göçken et al., 2018). Furthermore, the optimal solution vector obtained by HS model was applied to BP algorithm. The network architecture of HS-BP model is shown in Figure 1. Based on Figure 1, HS-BP network architecture had three layers, namely (1) The input layer, which was further processed in the study, (2) The hidden layer, which contained several neurons with a tangent hyperbolic activation function for processing information from the input segment, and (3) The output layer, which was the accumulation of the hidden layer. Also, it had a linear activation function that processed information from the hidden layer. Moreover, each layer was connected by several connectors known as weights (Pradhan et al., 2020).

Forecasting Model Evaluation
The process of forecasting was evaluated to determine the best model. This indicated that there were several error measurement methods for time series, including MAPE (Mean Absolute Percentage Error) and MSE (Mean Standard Error). According to study of Getry, MAPE was a suitable forecasting accuracy technique. The calculation of this parameter is shown in Eq. 15 as follows: Besides the accuracy value, other aspects should be considered, specifically time. This was due to the determination of the training process effectiveness and efficiency. When this process has high accuracy and a long training time, the model is infeasible to be realistically performed (Radityo, 2017).

Methodology
In this study, secondary data were obtained from https://www.investing.com/indices/lbma-gold-fixing-pricehistorical-data, based on the closing, highest, and lowest prices of daily gold for LBMA, with units of US$/troy ounce. Moreover, the utilized sample was obtained from January 15, 2020-2021, with a total of 255 daily observations. In this study, the RStudio software also used HS-BP model for gold price forecasting. The input variables subsequently used the Bollinger bands (3 and 5

Results and Discussion Descriptive analysis
The original graphical representation of the closing gold prices from January 15, 2020-2021 is shown in Figure 3 as follows: Figure 3: Graph Gold Price LBMA Based on Figure 3, the highest and lowest gold prices were US$ 2,067.15 and 1,474.25/troy ounce on August 6 and March 19, 2020, respectively. The increase in gold prices was caused by the corona pandemic so the United States economy shrank an amazing 9.5% from April to June. This has led to fears of a setback if recovery efforts fail so demand for gold soars as a safe investment. Meanwhile, The decline in gold prices occurred due to the spread of the corona virus outbreak so that the US central bank (Feder Reserved / The Fed) increased interest rates by 50 bps, gold prices shot up around 3% (Rupert neate, 2020). In 2020, the closing price of gold had an average value of US$ 1,782.51/troy ounces. This indicated that the closing price movement of LBMA gold did not drastically increase/decrease, although had a fluctuating data pattern. In this study, 90 and 10% of the gold price information (230 and 25 observations) were used as a training dataset and test, respectively. This indicated that the percentage of training data was greater than the test based on the accurate formation of algorithm patterns. These data were subsequently preprocessed towards changing into a range of -1 to 1, according to the utilized activation function (hyperbolic tangent). In addition, the training and testing data are shown in Tables 2 and 3, respectively.

Harmony Search Solution Vector
This study proposed a gold price forecasting model with a hybrid design, to obtain an accurate result. Using HS algorithm, the first step was based on determining the optimal solution vector. The solution vector is optimal when the solution vector has a smaller objective function (MSE) than the previous solution vector. The solution vector consists of the relevant input variables and the neurons in the hidden layer. This was subsequently applied to BP training, as the input variables and neurons produced are shown in Table 4.  Table 4, 6 neurons and 3 variables were selected as inputs in the hidden layer. After going through the elimination of other solution vectors. This indicated that 3 variables were used for gold price forecasting in the Bollinger Band (5 days), as well as Simple and Exponential Moving Averages (3 days each). As the objective function, the MSE value of the best solution vector was 0.00987 is better than the previous solution vector.

Comparison of Regular BP and HS-BP Models
In the hidden layer, the regular BP model utilized several neurons that were determined from input and output variables. The results showed that the output and input variables were 8 and 1, indicating that the hidden layer contained 2 and 5 units, respectively. Meanwhile, the HS-BP model used 3 input variables and 6 neurons in the hidden layer. This was observed as the optimal solution vector from the harmony search. Experiments were carried out using a maximum of 5000,10000, and 15000 iterations for each model. In addition, the computational time is also calculated from each experiment. Table 5 shows the performance results of each model. The results showed that HS-BP was better than regular BP in terms of the level of convergence, training time and forecasting accuracy which showed technical ability in forecasting performance. In addition, the HS-BP model is able to contribute to producing relevant technical indicators so as to produce an optimal solution vector. In addition, HS is also able to show its ability and validity to optimize learning on BP towards accurate results in forecasting. Forecasting results for the future period Table 5 shows the result of the existing gold price forecasting for January 18-22, 2021, using the HS-BP model. This indicates the need for present data to produce a good model, due to one forecasted period having small differences between the actual closing value. Therefore, the following forecast period was not very good compared to only one duration.

Conclusion
This study focused on HS-BP modelling for LBMA's daily gold price forecast, to improve BP performances through HS algorithm, which determined the input variables and neurons to be utilized in the hidden layer. Total sum squared error, training time accuracy, and convergence were compared between the HS-BP and regular BP models. The experimental results show that the HS-BP model can choose technical indicators as relevant input variables and neurons in the hidden layer, and is able to optimize training on BP with optimal time, MSE, and accuracy. This indicated that HS-BP model had a good performance in gold price forecasting. Therefore, several parameters affecting BP architecture should be considered in future studies. Variants such as Improved Harmony Search, should also be used to enhance forecasting accuracy.