Math 447/627 - Introduction to Parallel Computing

Fall 2022 - Matthias K. Gobbert
Presentations of the Class Projects

Tuesday, December 20, 2022, 01:00 p.m.

  1. 01:05-01:20
    Distributed Deep Learning via PyTorch Distributed Data Parallelization (DDP)
    Garima Kumari, Department of Information Systems
    Utilizing a GPU is the most straightforward technique to accelerate neural network training because it offers significant performance gains over CPUs for the kinds of calculations that neural networks frequently perform (matrix adds and multiplies). One GPU quickly becomes inadequate as the model or dataset grows larger. Large language models, like BERT and GPT-2, are trained on thousands of GPUs. We need the means to distribute the model and data across many GPUs and coordinate the training in order to accomplish multi-GPU training. Instead of creating distinct, independent algorithms for diverse cloud properties, the goal of this project is to establish an integrated, best estimation-based cloud retrieval system and apply it to data obtained by the MODIS spectroradiometer, which uses 10 thermal infrared bands. The model accounts for sensor configurations, background surface, and atmospheric profile, microphysical and optical models of ice and liquid cloud particles, as well as radiative transmission in a multilayered, plane-parallel atmosphere. Measurements and model errors are carefully measured from comparisons of model estimates with clear-sky observations over the ocean. Even so, the primary objective is to use Pytorch DDP to train the model and run it on both a single GPU and many GPUs. Although the primary goal of this paper is to concentrate on setting up the environment configuration on the taki cluster and running it on a single GPU to assess performance and efficiency. This work is joint with Dr. Jianwu Wang.

  2. 01:20-01:35
    Testing MPI with OpenMP
    Charan Duggirala, Department of Information Systems
    Modern mathematical equations are increasingly becoming very complex to solve and require computers with larger processing power to perform computations. Parallel computing is harnessing the power of multiple processors and efficiently performs calculations in a fraction of the time. Through the means of MPI (Message Passage Interface), we are able to divide our computational work among many processor nodes and processes thus achieving faster execution times and greater accuracy. While a process is an instance that is being executed at any point of time, we are interested in understanding threading within each process in the taki cluster in HPCF. We will be using the 2018 nodes in our taki cluster which has a total of 42 compute nodes totaling of 1512 cores and over 15 TB of pooled memory. We are currently using a hybrid combination of MPI+OpenMP API on our existing Poisson code and comparing its results with an MPI-only version. We will also understand how the master thread spawns more threads for the parallel portion of the program and evaluate the significant cost of performing threading operations.

  3. 01:35-01:50
    Parallel Implementation of Cloud Phase Prediction using Deep Learning Approaches
    Xingyan Li, Department of Information Systems
    Data-driven approaches to learning data representation such as deep learning have shown promising results in supervised learning and unsupervised learning, but the complex network architectures and the huge amount of training data make it computationally consuming. Therefore, the usage of highly functional hardware including GPUs and algorithms of distributed computation load including parallel computation is required. This study first setup environment for PyTorch-based deep learning. Then we utilizes GPU clusters of UMBC High Performance Computing Facility (HPCF) to train and evaluate VDAM, a deep learning based model for cloud property retrieval. Finally we also learns using multiple GPUs by PyTorch-based Data Parallel functions. This study is in in collaboration with my advisor Dr. Jianwu Wang.

  4. 01:50-02:05
    Parallel Performance Analysis of 1D Wave Equation Using Finite Difference Method on the Taki 2018 Cluster
    Abdullah Al Imran, Department of Mechanical Engineering
    The following report describes a 1D wave solver using parallelizing concepts from MPI. The solver aims to solve the wave equation through central finite difference discretization of its partial differential equation in space and time. Dirichlet boundary conditions have been used for this problem. The boundary conditions, grid size, and the duration of the run time are some major input parameters in this experiment. Finally, those results have been post-processed and shown in tables and plots. In a nutshell, this study focuses on the results and conclusions regarding the developed code's performance using taki cluster from HPCF facilities at UMBC, for various combinations of nodes and tasks per node.

  5. 02:05-02:20
    Prediction of Rapid Intensification of Tropical Cyclones
    Nadeem Shah, Department of Mechnical Engineering
    In this project validation of the physics structure with existing experimental aircraft data from Imaging Wind and Rain Airborne Profiler (IWRAP) radar is performed. The main objective is to understand the connections between coherent turbulent structures and convection in data and models. The WRF model is applied in three steps: 1) Preprocessing, 2) Simulation using MPI parallelization and 3) post-processing. In the Preprocessing step, the Weather Preprocessing System (WPS) is used to define WRF grid, generate map, elevation and land information for WRF. The simulation is performed to produce the model forecast by scheduling jobs using the existing taki 2018 cluster. The post processing is accomplished to visualize, analyze and validate the results. The results include the NetCDF file format which can be used to get the wind speeds, temperature, moisture etc. at 3-D gridded locations. This work is jointly performed with Dr. Stephen R. Guimond, Associate Research Professor (Univ. of Maryland Baltimore County - JCET and Physics), Goddard Space Flight Center, NASA.

Copyright © 2001-2022 by Matthias K. Gobbert. All Rights Reserved.
This page version 1.0, December 2022.