Project

General

Profile

Actions
Copy link

Activity #3744

open

Module #3693: Big Data Analytics Platform and Tools for Epidemic Forecasting and Mitigation

Module #3696: Center for Epidemic Forecsting(EPIFORM)

Spark Installation and Configuration

Added by Prasidh J S about 1 year ago. Updated about 1 year ago.

Status:
Resolved
Priority:
Normal
Assignee:
Prasidh J S
Start date:
06/02/2025
Due date:
% Done:

100%

Estimated time:
Planned Due Date:

Description

Installed Spark 3.4.4 on the cluster.

Configured Spark in two modes:

Standalone Mode: Set up Spark master and worker nodes manually.
      YARN Mode: Integrated Spark with Hadoop YARN for resource management.

Configured spark-env.sh, spark-defaults.conf, and worker nodes list.

Verified Spark job execution in both client and cluster deploy modes.

Monitored job progress via Spark UI and YARN ResourceManager UI.

Actions
Copy link
 #1

Updated by Prasidh J S about 1 year ago

  • Start date changed from 06/11/2025 to 06/02/2025
Actions
Copy link
 #2

Updated by Prasidh J S about 1 year ago

  • Status changed from New to In Progress
Actions
Copy link
 #3

Updated by Prasidh J S about 1 year ago

  • % Done changed from 70 to 100
Actions
Copy link
 #4

Updated by Prasidh J S about 1 year ago

  • Status changed from In Progress to Resolved
Actions
Copy link

Also available in: Atom PDF