The fast expansion of cities along with the growing number of automobiles has created a severe problem with traffic congestion in today’s urban areas. The current traffic system operates poorly because it results in extended travel durations while drivers consume excessive fuel and produce environmental damage which leads to financial losses. Traditional traffic control systems operate on fixed signal schedules which use past information to operate but they fail to manage the changing traffic situations that occur in real time. The research paper provides a Big Data framework which helps smart cities predict their urban traffic patterns and control their citywide traffic flow. The system collects live traffic information from multiple different data sources which include IOT traffic sensors and GPS-equipped vehicles and traffic monitoring cameras and mobile phone applications. The system uses Apache Kafka to receive live data streams which it stores in Hadoop Distributed File System (HDFS) for its storage needs and processes using Apache Spark. The research employs three machine learning and deep learning models which include Random Forest and Support Vector Regression (SVR) and Long Short-Term Memory (LSTM) networks to achieve precise traffic flow and congestion predictions. The system uses prediction results to modify traffic signals and find better routes which enhances city traffic management while building sustainable transportation networks for smart cities.

This project proposed an integrated Big Data–based framework for urban traffic flow prediction and management within smart cities. Rapid urbanization and an immense increase in the number of vehicles necessitate a more effective solution for prevailing traffic management systems, which are highly dependent on fixed signal timings and historical data. This system overcomes these issues by collecting real-time traffic data from various heterogeneous sources such as IoT traffic sensors, GPS-enabled vehicles, traffic surveillance cameras, and mobile apps. Apache Kafka ensured fault-tolerant, high-throughput, reliable real-time data ingestion. HDFS allowed scalable, fault-tolerant storage of large volumes of traffic data. Apache Spark allowed the efficient real-time and batch processing of streams of data, along with data cleaning, aggregation, and feature extraction. The study proposed the incorporation of machine learning and deep learning techniques- Random Forest, Support Vector Regression (SVR), and Long Short-Term Memory-LSTM networks-to forecast the state of traffic flow or congestion. Among those, LSTM emerged as the best, due to its non-linearity and ability to grasp deep complex temporal dependencies that exist in traffic data. The result of prediction outcomes was holistically leveraged for enabling intelligent traffic light control systems as well as route optimization, thus ensuring decreased congestion, average vehicle speed, as well as lowered travel time. Validation of the experimental results has established the scalability, accuracy, as well as real-time capability of the proposed model for dealing with real-time traffic data. In conclusion, the project not only suggests the beneficial usage of Big Data technologies but also their integration with prediction systems, thus ensuring enhanced mobility, reduced environmental effects, as well as intelligent transportation systems for smarter cities in the future.

[1]Y. Zheng, L. Capra, O. Wolfson, and H. Yang, “Urban computing: Concepts, methodologies, and applications,” ACM Transactions on Intelligent Systems and Technology, vol. 5, no. 3, pp. 1–55, Sept. 2014. [2]B. M. Williams and L. A. Hoel, “Modeling and forecasting vehicular traffic flow as a seasonal ARIMA process: Theoretical basis and empirical results,” Journal of Transportation Engineering, vol. 129, no. 6, pp. 664– 672, Nov. 2003. [3]W. Min and L. Wynter, “Real-time road traffic prediction with spatio-temporal correlations,” Transportation Research Part C: Emerging Technologies, vol. 19, no. 4, pp. 606–616, Aug. 2011. [4]Y. Lv, Y. Duan, W. Kang, Z. Li, and F.-Y. Wang, “Traffic flow prediction with big data: A deep learning approach,” IEEE Transactions on Intelligent Transportation Systems, vol. 16, no. 2, pp. 865–873, Apr. 2015. [5]N. G. Polson and V. O. Sokolov, “Deep learning for short-term traffic flow prediction,” Transportation Research Part C: Emerging Technologies, vol. 79, pp. 1–17, June 2017. [6]Y. Li, R. Yu, C. Shahabi, and Y. Liu, “Diffusion convolutional recurrent neural network: Data-driven traffic forecasting,” in Proc. IEEE International Conference on Learning Representations (ICLR), Vancouver, Canada, 2018, pp. 1–16. [7]M. Abdel-Aty, A. Dhindsa, and S. Pande, “Real-time traffic signal control using integrated real-time traffic data,” Transportation Research Record, no. 2259, pp. 29–39, 2011. [8]T. White, Hadoop: The Definitive Guide, 4th ed. Sebastopol, CA, USA: O’Reilly Media, 2015. [9]M. Zaharia et al., “Apache Spark: A unified engine for big data processing,” Communications of the ACM, vol. 59, no. 11, pp. 56–65, Nov. 2016. [10]B. L. Smith, B. M. Williams, and R. K. Oswald, “Comparison of parametric and nonparametric models for traffic flow forecasting,” Transportation Research Part C: Emerging Technologies, vol. 10, no. 4, pp. 303–321, 2002. [11]J. Medina-Salgado, E. Sánchez-Díaz, and J. L. Llorca, “Short-term traffic flow prediction using support vector regression,” Expert Systems with Applications, vol. 39, no. 10, pp. 10416–10425, 2012. [12]L. Breiman, “Random forests,” Machine Learning, vol. 45, no. 1, pp. 5–32, 2001. H. Karau, A. Konwinski, P. Wendell, and M. Zaharia, Learning Spark: Lightning-Fast Big Data Analysis. Sebastopol, CA, USA: O’Reilly Media, 2015.

© 2025 International Journal of Engineering and Techniques (IJET).

Digital Object Identifier (DOI)

IJET assigns a unique DOI to every accepted article via Zenodo for persistent linking and citation. Your article’s DOI: https://doi.org/{{doi}}

Open DOI About Zenodo DOI

Close