HAL Training

Previous session videos

The center hosted a free online training series that helped prepare people for deep learning projects on NCSA’s HAL cluster. Each session featured an expert instructor covering topics focused on familiarizing novel users with different aspects and uses of HAL and training them to build deep neural network models.  

HAL Cluster Wiki

Getting Started with HAL

Overview: This tutorial will introduce new users to HAL system, how to interact with it through Open OnDemand web interface, including Jupiter notebook and VS Code, and through command line interface via SSH. The tutorial will cover the Anaconda environment, batch job submission, data transfer, as well as an overview to the basic machine learning workflow.

  • Training Slides
  • Instructor: Volodymyr Kindratenko, Director of the Center for Artificial Intelligence Innovation
  • Date: January 25, 2023

Introduction to PyTorch

Overview: This tutorial will introduce basics of PyTorch framework necessary to build a neural network, train it and evaluate the accuracy of the model.

  • GitHub Repo
  • Instructor: Priyam Mazumdar
  • Date: February 1, 2023

Introduction to TensorFlow

Overview: This tutorial will introduce basics of TensorFlow necessary to build a neural network, train it and evaluate the accuracy of the model.

Distributed Data Parallel Model Training in PyTorch

Overview: This tutorial walks through distributed data parallel training in PyTorch via DDP. We will start with a simple non-distributed training job, and end with deploying a training job across several GPUs in a single HAL node. Along the way, you will learn about DDP to accelerate your model training. You will also learn how to monitor GPU status to help profile code performance to fully utilize GPU computing power.

  • Instructor: Shirui Luo
  • Date: February 15, 2023

Robust Physics-Informed Neural Networks

Overview: Physics Informed Neural Networks (PINNs) have recently been found to be effective PDE solvers. This talk will focus on how traditional PINN architectures along with physics-inspired regularizers fail to retrieve the intended solution when training data is noisy and how this problem can be solved using Gaussian Process based smoothing techniques.

How to use Pretrained Models

Overview: There are several popular AI model repositories that provide access to pre-trained models via easy-to-use APIs. Hugging Face is one of the latest such repositories that hosts a number of very recent models, such as Facebook’s OPT and OpenAI’s GPT models, as well as many datasets. This tutorial will show how to use basic pre-trained PyTorch models and then will introduce how to get started with using models from the Hugging Face ecosystem.

DRYML an open source meta-library for machine learning and more

Overview: DRYML aims to empower the Machine Learning practitioner to spend less time writing boilerplate code, and more time implementing new techniques. DRYML provides a model serialization framework along with serialization implementation for many common ML frameworks and model types, a framework for defining and training models on a specific problem, and a system to compare models from different ML frameworks on the same footing. Students will leverage their knowledge across multiple ML frameworks such as tensorflow, pytorch,  and sklearn to build new models, optimize them, and compare them across ML frameworks.

Explainability of Deep Neural Networks

Overview: Deep Neural Networks are often treated as black boxes. This talk will focus on some of the modern methods of explainability for DNNs and discuss their implementation, usage, and limitations.

Introduction to Transformer models

Overview: In this tutorial, you’ll gain an understanding of how to developing and training Transformer models. We’ll compare with Sequence models and explore how to construct and utilize a Vision Transformer. You’ll learn about essential components such as PatchEmbeddings, Attention Layers, Class tokens, and Positional Embeddings, which play crucial roles in the architecture of the Transformer. By the end of this tutorial, you’ll have a foundation to build and train your Transformer models effectively.

  • Training Slides
  • Instructor: Priyam Mazumdar – MS Student
  • Date: April 12, 2023

Weights & Biases workshop: Track, Visualize, and Improve Your Experiments

Overview: Andrea will take you through a W&B introduction and product walkthrough, including Experiment tracking, W&B Tables, Sweeps, Artifacts, Dashboards/Reports, and Integrations! Followed by a Colab classification competition in Kaggle with swag for top submissions.

University of Illinois Only. Link to Recording

  • Presentation
  • Weights & Biases student and research-focused resources on their Academic Page!
  • Instructor: Andrea Parker
  • Date: April 19, 2023

Introduction to Machine Learning and Neural Networks

Overview: This tutorial will provide an easy to follow introduction to machine learning with neural networks. We will go over various examples from simple dense networks to convolutional network architectures using TensorFlow framework on HAL system.

  • GitHub Notebook
  • Instructor: Aaron Saxton, Data Engineer, NCSA
  • Date: September 14, 2022

Introduction to PyTorch

Overview: This tutorial will introduce basics of PyTorch framework necessary to build a neural network, train it and evaluate the accuracy of the model.

  • Training Notebook
  • Instructor: Priyam Mazumdar, Graduate Student, NCSA
  • Date: September 28, 2022

Data Loading and Tools in PyTorch and TensorFlow

Overview: This tutorial will demonstrate how to use data loaders provided with PyTorch and TensorFlow and how to develop application-specific data loaders. Other topics will include how to make use of different tools, such as Weight&Biases, to help with model development and training.

  • Instructor: William Eustis, Undergraduate Student, NCSA
  • Date: October 5, 2022

Machine Learning First Deep Neural Network

Recording

Overview: Everyone talks about Deep Learning and using Neural Networks. The algorithms have been found in certain cases to produce better estimates than humans! They are great for data prediction and understanding more about your data. This workshop is an introduction to Deep Learning using Tensorflow and Keras. A Jupyter notebook is used for hands-on student exercises along with a set of lecture slides.

Convolution Neural Network Models

Recording

Overview: Convolutional Neural Networks (CNNs) are used primarily for anything involving images – image classification, to image segmentation, to putting bounding boxes around an object. This interactive workshop includes a Jupyter notebook using TensorFlow and Keras, along with a set of lecture slides. In addition, this Lecture introduces the concept of Inference. This is processing data through a trained model without any back propagation.

  • Instructor: Jeff Layton, NVIDIA
  • Date: March 9, 2022

Machine Learning with H2O on HAL

Recording

Overview: This tutorial will cover basic machine learning techniques, such as Linear Regression, Decision Tree, Support Vector Machine, Naive Bayes, K-Nearest Neighbors, K-Means, and Random Forest, using H2O machine learning platform on HAL system.

Instructor: Dawei Mu
Date: March 23, 2022

Data Loaders

Recording

Overview: This tutorial will demonstrate how to use data loaders provided with PyTorch and TensorFlow and how to develop application-specific data loaders.

Graph Neural Networks (GNNs)

Recording

Overview: This tutorial will introduce basic concepts in Graph Neural Networks (GNNs) and will show how to use PyTorch Geometric Library to start implementing GNNs.

  • Instructors: Minyang Tian & Michael Volk
  • Date: April 6, 2022

Inference Optimization with NVIDIA TensorRT

Recording

Overview: In many applications of deep learning models, we would benefit from reduced latency (time taken for inference). This tutorial will introduce NVIDIA TensorRT, an SDK for high-performance deep learning inference. We will go through all the steps necessary to convert a trained deep learning model to an inference-optimized model on HAL.

  • Instructors: Nikil Ravi & Pranshu Chaturvedi
  • Date: April 13, 2022

Hands-On Deep Learning for Computer Vision

Recording

Overview: This tutorial will introduce how machine learning can be accomplished with neural networks and will go over various examples from simple dense networks to convolutional network architectures using TensorFlow on HAL system.

Distributed Deep Learning on HAL

Recording

Overview: This tutorial will explain how to use multiple GPUs for accelerating deep learning on HAL using frameworks such as TensorFlow or PyTorch.

INTRO TO NATURAL LANGUAGE PROCESSING (NLP) WITH PYTORCH

Recording

Overview: This tutorial will introduce neural network architectures for processing natural language texts. We will cover Recurrent Neural Networks (RNNs) and Generative Neural Networks (GNNs), attention mechanisms, and how to build text classification models.

  • Instructor: Volodymyr Kindratenko
  • Date: October 27, 2021

OPENVINO™ TOOLKIT INTEGRATION WITH TENSORFLOW: HANDS-ON PRACTICE INTEL DEVCLOUD

Recording

Overview: This tutorial will introduce Tensorflow integration available with the OpenVINO™ toolkit and hands-on practice in the Intel® DevCloud explaining the minimal code changes needed with Tensorflow to gain greater inference performance with OpenVINO.  Suggested to pre-register for access to the Intel® DevCloud here. Access is free.

ROBUST PHYSICS INFORMED NEURAL NETWORKS

Recording

Overview: Physics Informed Neural Networks (PINNs) have recently been found to be effective PDE solvers. This talk will focus on how traditional PINN architectures along with physics-inspired regularizers fail to retrieve the intended solution when training data is noisy and how this problem can be solved using Gaussian Process based smoothing techniques.

PHYSICS INFORMED DEEP LEARNING

Recording

Overview; This tutorial will explore how to incorporate physics into deep learning models with various examples ranging from using physics informed neural networks (PINNs) for forward and inverse problems to employing physics informed DeepONets for a hybrid data and physics approach to problems.

Center for Artificial Intelligence Innovation
1205 W. Clark St.
Urbana, Illinois 61801
Email: caii_ai@lists.illinois.edu
National Center for Supercomputing Applications (NCSA)

Office of the Vice Chancellor of Research and Innovation (OVCRI) (link opens in new window)

CookieSettings CookieSettings