Big Data Analytics Platform and Tools for Epidemic Forecasting and Mitigation

• Set up the Apache NiFi server .
• Configure a network-shared folder named EPIFOM_DATA to allow file access. (Create network map drives in server)
• This folder contains structured subfolders such as:

• csv/
• pdf/idsp/
• json/

• Design and configure a NiFi data pipeline that:
• Monitors the EPIFOM_DATA folder and its subdirectories.
• Automatically detects and processes new files as they arrive.
• Implement scheduling in NiFi to periodically poll for new files and process them incrementally (delta processing).
• Ingest the processed data into Delta Lake for further analytics and downstream usage.