This is the course page for the parallel processing unit of the “Computational Economics Boot Camp.” This “camp” consists of two courses, three credits each: Math 513R and Econ 413R.
Welcome to the “Computational Economics Bootcamp” section on parallel programming! Below I have included outcome objectives, assignments & labs, class video lectures, and reference material.
- Class Text: A Python Introduction to Parallel Programming with MPI
- Video Lectures: Parallel Programming with MPI, using Python and mpi4py
This material is a prerequisite to these labs.
- comfortable with the unix command line evirnoment
- learning the bash shell
- ssh and scp (learn to use them efficiently)
- compiling and running programs
- the VI editor
- Fulton Supercomputer
- learn the SLURM scheduler on the Fulton Computers
- submit standard serial jobs as well as parallel jobs
- overview and essentials of parallel computing
- Amdahl’s law
- data parallel v. task parallel
- parallel programming with MPI (taught with mpi4py)
- intro to MPI
- Point-to-Point communication
- Collective Communication
Survey of existing parallel libraries and tools
- brief overview of PETSc
- overview of large data formats, such as HDF5
Dealing With Large Datasets
- Parallel and Normal HDF5
- Relational Datasets v HDF5
Assignments and Labs—
Assignment A (Prerequisite Assignment):
After you have obtained an account with the Fulton Supercomputer, watch the following instructional videos:
On the BYU Supercomputing Channel, watch these videos:
- Connect to FSL using SSH
- Submitting Jobs Using qsub
- BYU’s PBS Script Generator: Note that we will be using MPI. He mentions it a few times in this video. What he says about it will be useful to know.
- Job Scheduler
On my channel, watch this video:
On Monday, we will talk about these and get everyone started on Fulton.
Fulton Job Submission Assignment
To further help you get up to speed, watch these two videos to demonstrate how to efficiently use ssh (through PuTTY and winSCP) and for some guidance on using the Unix command line.
- Fulton Supercomputing and Tips for Using PuTTY and winSCP
- Unix Tutorial and Other Misc Info about using the Fulton Supercomputer
Now complete the following mini-project. You will submit your first parallel job on the supercomputer: Job Submission Assignment
You will need the following file. Click to download: testJob
At the end of the week, all exercises from the class text A Python Introduction to Parallel Programming with MPI will be due. This includes the chapters “An Overview of Parallel Computing,” “Introduction to MPI,” “Point-to-Point Communication,” “Collective Communication,” etc..
- HDF Assignment 1 Here you will learn the purpose of the HDF and HDF5 formats and deleop some hand’s on experience with them. For this assignment, you will need this file: testHDF_AssignP1.zip
Note about Labs:
When turning in labs, you should submit them using standard Python documentation style (PEP 8 — Style Guide for Python). Good style and documentation is an important aspect of writing code and is a good habit to have. Too many times have I come back to my own code and not been able to remember exactly what it did and how it did it. Remember, “Code is read much more often than it is written.” Include many comments, plan out what your program will do ahead of time (so that it’s clear what your program takes in and what it returns), and make sure that it looks nice and readable.
Class Video Lectures—
- Lecture 1: High Performance Computing with Python. Why Parallel? Why Python?
- Lecture 2: Introduction to MPI with Python and mpi4py part 1
- Lecture 3: Introduction to MPI with Python and mpi4py part 2
- Lecture 4: Point-to-Point Communication with Python and mpi4py
- Lecture 5: Collective Communication with Python and mpi4py
- A Python Introduction to Parallel Programming with MPI. A short introduction and tutorial to distributed memory computing with Python. Although HPC projects should usually be written in faster, compiled languages, like C or Fortran, Python is an excellent tool for algorithm development and proof of concept. This is especially true in parallel programming, where code becomes especially complex.
Using the Fulton Supercomputer:
- Fulton Supercomputing Lab Website
- BYU Supercomputing Youtube Channel
- Extra Video Tutorials (my Youtube Channel)
- Parallel Programming with MPI. Book website. This also contains a link to a free online manual on which the book was based.
- mpi-forum.org provides useful documents
- API reference for mpi4py