Developing an appropriate model for accurate prediction of energy consumption is very essential for developing an effective energy management system for residential buildings. In view of this, the Short-term Load Forecasting (STLF) of household appliances has been performing an important role in supervising and managing energy in the residential community. In the domain of big data analytics, data-driven load forecasting approaches have realized an amazing performance in the recognition of patterns of residential electric loads and forecasting energy consumption. Nevertheless, current research emphasizes the use of powerful feature-engineering methods, which are ineffective and result in low generalization performance. Further, considering the differences in the consumption behavior of various home appliances, it is unfeasible to discover energy consumption characteristics physically in the power system. Thus, this study addresses the problems of STLF using a novel two-stream deep learning (DL) model called STLF-Net. The first stream is designed with Gated Recurrent Units (GRUs) to learn and capture the long-term temporal representations of the energy utilization data. Simultaneously, in the second stream, the short-term information and positional representations are modeled using a stack of temporal convolutional (TC) modules. The TC module is designated using dilated causal convolutions and residual connection to enable efficient feature extraction while alleviating the gradient vanishing issues. The learned representations from the two streams are fused and subsequently passed to several dense layers to generate the final hour-ahead load forecasts. Experimental assessments on two public energy consumption predictions datasets (IHEPC and AEP) demonstrated the superior performance of the STLF-Net over the recent cutting-edge data-driven approaches.