This advanced machine learning project represents a complete forecasting solution for Malaysian unemployment rates, combining 15 years of government labor data with sophisticated AI models to achieve exceptional accuracy. The system demonstrates deep learning superiority over traditional methods while providing practical tools for workforce planning and policy formulation, establishing new benchmarks in unemployment prediction through rigorous statistical validation.
Web Application Features
- Real-Time Data Management using OpenDOSM APIs every 24 hours
- Interactive Forecasting Hub for better and interactive analysis
- Advanced Visualization Suite with DashPloty powered tools & libraries
- Statistical Analysis Tools
- Forecast Hub with 3 different AI models for forecasting
Technologies, Frameworks & Software Used
Project Scope
🎯Objective
Develop and evaluate time series forecasting models to predict Malaysia’s monthly unemployment rate using government labor force data, providing insights for workforce planning and policy formulation.
📊Data Source
OpenDOSM (Malaysian Department of Statistics) providing 15 years of longitudinal data across multiple economic cycles, including COVID-19 impact and recovery periods. This dataset is fetched via API every 24 hours which ensures it is constantly being fed to our system with no manual updates required.
💡Solution
Our solution delivers a sophisticated multi-model forecasting system that transforms unemployment prediction through advanced AI. The LSTM neural network achieved unprecedented 1.84% MAPE accuracy—six times better than traditional ARIMA models—using a two-layer architecture with dropout regularization and Adam optimization. We developed specialized SARIMA models achieving 3.20% MAPE and pioneered CEEMDAN-LSTM methodology for youth unemployment, reaching 2.72% MAPE for the 15-30 age group.
The complete solution integrates a professional web application built with Dash and Plotly, featuring real-time API integration with Malaysian government databases, interactive visualizations, and comprehensive analytical tools. This provides immediate practical value for policymakers, economists, and workforce planners seeking data-driven insights into Malaysia’s employment landscape.
Market Overview
Real-time unemployment indicators and interactive time series visualization across configurable periods with multiple demographic breakdowns.
Data Explorer
Interactive analysis capabilities for examining multiple unemployment metrics simultaneously with customizable chart types and filtering options.
Statistical Analysis
Comprehensive testing procedures including stationarity assessment, normality evaluation, and autocorrelation analysis with real-time validation.
Data Transformation
Preprocessing methodologies including log transformation, differencing operations, and Box-Cox optimization with immediate ADF testing feedback.
Forecasting Hub
Automated prediction generation across LSTM, SARIMA, and ARIMA models with configurable forecast horizons and demographic segmentation.
Performance Metrics
Comprehensive evaluation using MAPE, RMSE, and MAE with confidence intervals and directional accuracy across multiple model approaches.
1.84
LSTM MAPE
General unemployment
2.72
LSTM MAPE
Seasonally adjusted
3.20
SARIMA MAPE
Best traditional model
2.7
Youth MAPE
General unemployment
6x
Better
Than ARIMA baseline
12
Months
Forecast horizon