Dask Workshop

6 hours of instruction

Introduces Dask for scaling data analysis in Python. The workshop begins with an overview of the fundamentals of parallel computing in Python with explorations of technical limitations of NumPy & Pandas. After exploring core Dask data structures, participants will apply Dask arrays & dataframes in practice, using dashboard tools to monitor Dask workflows and measure performance.

PREREQUISITES

Participants should have prior experience using the Python language and, in particular, using standard Python tools for data analysis (notably NumPy, Pandas, Scikit-Learn, Jupyter). No prior exposure to Dask or to parallel computing is required.

LEARNING OBJECTIVES

1. Explain relevant parallel computing concepts in the context of data analysis pipelines.
2. ​Identify where in a data-processing pipeline parallelism is attainable or difficult.
3. Identify opportunities for parallel computation in existing Python data workflows.
4. ​Develop scalable Dask data pipelines to extend examples using Pandas/NumPy.
5. Select Dask data structures appropriate to a given compute-intensive scenario.
6. ​Construct scalable data analysis pipelines in Python using Dask from scratch.
7. ​Apply Dask dashboard tools to monitor performance of data analytics.
8. Use Dask diagnostic tools to assess and tune performance in applications.
9. Apply distinct schedulers appropriate to relevant hardware available.
10. Plan out & execute embarrassingly parallel Dask workflows on remote data.