Electricity load forecasting has been a substantial problem in the electric power system management process. An accurate forecasting model is essential to avoid imprecise predictions that can negatively affect system efficiency, economy, and sustainability. Among several prediction techniques, deep learning methods, especially the Long Short-Term Memory (LSTM), have been shown to have a superior performance in predicting the electricity load consumption. However, the consequences of using these methods have not fully been explored in terms of the various hidden layer structures, the depth of the model architecture, and the impact of tuning the model hyperparameters. In this paper, a systematic experimental methodology has been conducted to investigate the impact of using deep-stacked unidirectional (Uni-LSTM) and bidirectional (Bi-LSTM) networks on predicting electricity load consumption. In particular, two stacked configurations, which include two and three LSTM layers, are compared with the single-layered LSTM for both types to show the significant importance of adding the stacked layers. Moreover, for each proposed configuration, a hyperparameter optimization tool has been implemented to obtain the best model. The results indicate that the deep-stacked LSTM layers have no significant improvement in the prediction accuracy; nevertheless, they consume almost twice the time of the single-layered models. Also, the Bi-LSTM networks outperform the Uni-LSTM networks by 76.25%, 75.49%, and 75.35% in terms of Root Mean Square Error (RMSE), with respect to one, two, and three-layer model configurations, respectively. Furthermore, regarding the prediction accuracy comparison over the total tested period, the optimized Bi-LSTM model outperforms both the optimized Uni-LSTM model by 75.98%, 89.1%, and 89.37%, and the Support Vector Regression (SVR) model by 82.54%, 92.59%, and 92.89% in terms of (RMSE), the Mean Average Percentage Error (MAPE), and Mean Absolute Errors (MAE).