Menu Close
 

HAL Training

HAL Cluster
NCSA Hardware Accelerated Learning (HAL) Cluster

Previously the center hosted a free online training series that helped prepare people for deep learning projects on NCSA’s HAL cluster. Each session featured an expert instructor covering topics focused on familiarizing novel users with different aspects and uses of HAL and training them to build deep neural network models.  

Previous session videos

GETTING STARTED WITH HAL

Overview: This tutorial will introduce new users to HAL system, how to interact with it through Open OnDemand web interface, including Jupiter notebook and VS Code, and through command line interface via SSH. The tutorial will cover the Anaconda environment, batch job submission, data transfer, as well as an overview to the basic machine learning workflow.

  • Training Slides
  • Instructor: Volodymyr Kindratenko, Director of the Center for Artificial Intelligence Innovation
  • Date: January 25, 2023

Introduction to PyTorch

Overview: This tutorial will introduce basics of PyTorch framework necessary to build a neural network, train it and evaluate the accuracy of the model.

  • GitHub Repo
  • Instructor: Priyam Mazumdar
  • Date: February 1, 2023

Introduction to Tensor Flow

Overview: This tutorial will introduce basics of TensorFlow necessary to build a neural network, train it and evaluate the accuracy of the model.

Distributed Data Parallel Model Training in PyTorch

Overview: This tutorial walks through distributed data parallel training in PyTorch via DDP. We will start with a simple non-distributed training job, and end with deploying a training job across several GPUs in a single HAL node. Along the way, you will learn about DDP to accelerate your model training. You will also learn how to monitor GPU status to help profile code performance to fully utilize GPU computing power.

  • Instructor: Shirui Luo
  • Date: February 15, 2023

Robust Physics-Informed Neural Networks

Overview: Physics Informed Neural Networks (PINNs) have recently been found to be effective PDE solvers. This talk will focus on how traditional PINN architectures along with physics-inspired regularizers fail to retrieve the intended solution when training data is noisy and how this problem can be solved using Gaussian Process based smoothing techniques.

How to use Pretrained Models

Overview: There are several popular AI model repositories that provide access to pre-trained models via easy-to-use APIs. Hugging Face is one of the latest such repositories that hosts a number of very recent models, such as Facebook’s OPT and OpenAI’s GPT models, as well as many datasets. This tutorial will show how to use basic pre-trained PyTorch models and then will introduce how to get started with using models from the Hugging Face ecosystem.

DRYML an open source meta-library for machine learning and more

Overview: DRYML aims to empower the Machine Learning practitioner to spend less time writing boilerplate code, and more time implementing new techniques. DRYML provides a model serialization framework along with serialization implementation for many common ML frameworks and model types, a framework for defining and training models on a specific problem, and a system to compare models from different ML frameworks on the same footing.

Students will leverage their knowledge across multiple ML frameworks such as tensorflow, pytorch,  and sklearn to build new models, optimize them, and compare them across ML frameworks.

Explainability of Deep Neural Networks

Overview: Deep Neural Networks are often treated as black boxes. This talk will focus on some of the modern methods of explainability for DNNs and discuss their implementation, usage, and limitations.

Introduction to Transformer models

Overview: In this tutorial, you’ll gain an understanding of how to developing and training Transformer models. We’ll compare with Sequence models and explore how to construct and utilize a Vision Transformer. You’ll learn about essential components such as PatchEmbeddings, Attention Layers, Class tokens, and Positional Embeddings, which play crucial roles in the architecture of the Transformer. By the end of this tutorial, you’ll have a foundation to build and train your Transformer models effectively.

  • Training Slides
  • Instructor: Priyam Mazumdar – MS Student
  • Date: April 12, 2023

Weights & Biases workshop: Track, Visualize, and Improve Your Experiments

University of Illinois Only link to NCSA Recording

Overview: Andrea will take you through a W&B introduction and product walkthrough, including Experiment tracking, W&B Tables, Sweeps, Artifacts, Dashboards/Reports, and Integrations! Followed by a Colab classification competition in Kaggle with swag for top submissions.

  • Presentation
  • Weights & Biases student and research-focused resources on their Academic Page!
  • Instructor: Andrea Parker
  • Date: April 19, 2023