Today, we want to call attention to a highly useful package in the torch ecosystem: torchopt. It extends torch by providing a set of popular optimization algorithms not available in the base library. As this post will show, it is also fun to use!
Use 3D to visualize matrix multiplication expressions, attention heads with real weights, and more.
What is Attention, and why is it used in state-of-the-art models? This article discusses the types of Attention and walks you through their implementations.
This article discusses GPT-2 and BERT models, as well using knowledge distillation to create highly accurate models with fewer parameters than their teachers
Fortran is a compiled language which means that once written, the source code must be passed through a compiler to produce a machine executable that can be run.
In this episode of Open Source Directions, we were joined by Thomas Wiecki once again who talked about the work being done with PyMC. PyMC3 is a Python package for Bayesian statistical modeling and Probabilistic Machine Learning focusing on advanced Markov chain Monte Carlo (MCMC) and variational inference (VI) algorithms. Its flexibility and extensibility make it applicable to a large suite of problems.
