This website is a collection of material that I gathered during his my in Machine Learning (ML) during my post-doc/Visiting Scholar appointment at Harvard University in 2018-2019. I have constantly added more materials since I started to teach the “Machine Learning” Ph.D. course at UFRGS. The idea of this compilation is to help those who are interested in learning ML. The material may help those who are not sure how to start, but also those who already know ML and want to improve his/her skills.

Thanks to the research assistants Jessyca Padilha and Gabriel Ramis for her help with the organization of the version 1.0 and 1.1, respectively.

Tutorials & Learning Path

In this section you will find resources for those who are starting their studies in Machine Learning and are not sure about which path to follow. There is no single path to learn ML. However, there are two major approaches: bottom-up or top-down (Brownlee, 2016): - Bottom-up: This is the most traditional path, where the student learning process starts with the theory about math/calculus, statistics, and computer science. This approach provides a solid foundation for those who are interested in mastering the knowledge required to become researchers of ML. For many of these researchers, the applications of ML may be of secondary importance if compared to the innovation and efficiency or the algorithms they create or develop. Computer science students usually follow this the bottom-up learning approach, and they often need several years of training before the first results start to show. The starting point of the bottom-up approach is the study of general math and calculus, statistics, and programming languages. - Top-down: Aimed at a different goal compared to the bottom-up, the top-down approach goal is to learn how to implement the ML algorithms to improve explanation and/or prediction in the researcher’s knowledge domain. This application can be in business, health, law, communication, art, etc. In such an approach, researchers are not necessarily trying to improve the existing algorithms or to advance the knowledge frontier of ML. Rather, their goal is to apply ML to solve specific problems of their discipline. The top-down approach does not require a strong background in mathematics/calculus, statistics/probability, and programming languages. Most of the references shown here are more appropriate for the top-down approach.

How to choose a learning path:

Need Help Getting Started with Applied Machine Learning?

5 Skills You Need to Become a Machine Learning Engineer

Learning Path : Your mentor to become a machine learning expert

Teach yourself Machine Learning the hard way!

Teach yourself Machine Learning the hard way! (Part 2)

How To Become A Data Scientist In 2019: A Beginner’s Guide

How Did I Learn Data Science

Advice On Building a Machine Learning Carrer and Reading Research Papers

Why you should not become a programmer or not learn programming language?

Garbade, D. M. J. (2018, September 14). Clearing the Confusion: AI vs Machine Learning vs Deep Learning Differences. Retrieved March 27, 2019, from Towards Data Science

Rules of Machine Learning:Best Practices for ML Engineering

Introduction to Data Science / Science infographic

Required Background

Math and Calculus

Kahn Academy

Deisenroth, M. P., Faisal, A. A., & Ong, C. S. (2019). Mathematics for Machine Learning. Retrieved March 21, 2019, from Mathematics for Machine Learning website

Prerequisites/Previous Necessary Knowledge

Math channel in Youtube: mathematicalmonk

Linear Algebra

Essential Math Skills for Machine Learning

Probability

Blitzstein, J. K., & Hwang, J. (2014). Introduction to Probability. CRC Press.

Bayes Theorem: The Holy Grail of Data Science

Bayesian Statistics for Data Science

My Journey to Bayesian Statistics

Neiswanger, W. (2019, January 15). Intro to Modern Bayesian Learning and Probabilistic Programming. Retrieved March 26, 2019,

Deep Bayesian networks using Monte Carlo dropouts

Predicting the likelihood of a customer to make repeat purchases using logistic regression

Bayesian Bandits explained simply

Why Learn Probability For Machine Learning

Quantum Machine Learning: Inference on Bayesian Networks

Programming Languages, Softwares

What’s the Best Statistical Software? A Comparison of R, Python, SAS, SPSS and STATA

Python And R For Data Wrangling Examples For Both, Including Speed Up Considerations

Guide To R And Python In A Single Jupyter Notebook

Python

Python Crash Course: A Hands-On, Project-Based Introduction to Programming

How to Use ggplot2 in Python

Everything You Ever Wanted To Know About Setting Up Python On Windows, Linux And Mac

How to Develop Super Learner Ensembles in Python

R

Introduction to iml: Interpretable Machine Learning in R

Chollet, F., & Allaire, J. J. (2018). Deep Learning with R. Manning Publications Company

Python And R For Data Wrangling Examples For Both, Including Speed Up Considerations

- Jamovi

Matlab

Matlab statistics toolbox

MathWorks. (2018). Detector Performance Analysis Using ROC Curves - MATLAB & Simulink. Retrieved January 15, 2019

- Jupyter

Datasets (free datasets, data preparation, webscraping)

Types Of Data Sets In Data Science, Data Mining and Machine Learning

100+ of the Best Free Data Sources For Your Next Project

Big Data And AI: 30 Amazing (And Free) Public Data Sources For 2018

Web Scraping

Coleta autom?tica de dados para ?ndices de pre?o e ajustamento de qualidade utilizando web scraping

UCI Machine Learning Respository

Git Hub

Harvard Dataverse

Registry of Open Data on AWS

The Dataset Collection

KD Nuggets

Data Gov

50 Amazing Free Data Sources You Should Know

Wikipedia’s list of Machine Learning datasets

Datasets subreddit

The Mnist Database of handwritten digits

The PASCAL Visual Object Classifica- tion Challenge

Enron: A Dataset for Email Classification 3

Dataset list

Tag Overflow

5 Steps to correctly prepare your data for your machine learning model

Algorithms

Most used algorithms

Commonly used Machine Learning Algorithms

Top 10 Machine Learning Algorithms For Beginners

10 Machine Learning Algorithms You need to Know

The 5 Sampling Algorithms That Every Data Scientist Need To Know

The 5 Feature Selection Algorithms every Data Scientist should know

Supervised Learning Algorithms

k-Nearest Neighbors

Linear Regression

Logistic Regression

Support Vector Machines (SVMs)

Decision Trees and Random Forests

Support Vector Machines (SVM) ### Deep Learning / Neural Networks Chahal, K. S., & Dey, K. (2018). A Survey of Modern Object Detection Literature using Deep Learning. ArXiv:1808.07256 [Cs].

Understanding Neural Networks

Deep Learning NLP Explained

The Most Intuitive and Easiest Guide for Artificial Neural Network

DEEP LEARNING (definitions and types)

Feature Visualization How neural networks build up their understanding of images

Neural Networks and Deep Learning

Foundations Built for a General Theory of Neural Networks

He, K., Zhang, X., Ren, S., & Sun, J. (2015). Deep Residual Learning for Image Recognition. ArXiv:1512.03385 [Cs].

Take 4+: Presentations on Elements of Neural Networks and Deep Learning / Parts 1-8

Text Classification/RNN/ or CNN

RNN Simplified- A beginner’s guide

Recurrent Neural Networks (RNNs)

Building a Neural Network in Python - Language Modeling Task

How To Code Your First LSTM Network In Keras

7 Machine Learning Tools For Non Data Scientists

How To Train A Progressive Growin GAN In Keras For Synthesizing Faces

Deep Reinforcement Learning

Understanding LSTMs

RNN Effectiveness

Begginers Guide To Understanding Convolutional Neural Network

How to Fix k-Fold Cross-Validation for Imbalanced Classification

Unsupervised Learning Algorithms

Clustering * k-Means * Hierarchical Cluster Analysis (HCA) * Expectation Maximization

Visualization and dimensionality reduction * Principal Component Analysis (PCA) * Kernel PCA * Locally-Linear Embedding (LLE) * t-distributed Stochastic Neighbor Embedding (t-SNE)

Association rule learning * Apriori * Eclat

Reinforcement Learning Algorithms

Evolutionary Algorithms in Machine Learning with Risto Miikkulainen

ComputerVision DeepLearning MachineLearning Math NLP Python ReinforcementLearning

Sutton, R. S., & Barto, A. G. (2018). Reinforcement Learning: An Introduction. MIT Press

Top 5 Natural Language Processing Python Libraries For Data Scientist

NLP In H2Os Driverless

Libraries

Xgboots

Beginners Tutorial on XGBoost and Parameter Tuning in R

How to use XGBoost algorithm in R in easy steps

Extreme Gradient Boosting

Tensor Flow

Machine Learning with TensorFlow and R

Beginner’s guide for TensorFlow: The basics of Google’s machine-learning library

Time Series Forecasting with TensorFlow.js

Keras

How to build your first Neural Network to predict house prices with Keras

Transfer Learning with Keras and Deep Learning

How to Perform Face Detection with Deep Learning in Keras

Getting started with Tensorflow, Keras in Python and R

10 Useful R Packages Data Science & ML Enthusiasts Should Know

Shogun - Large-scale machine learning matlab tool- box

Stan platform for statistical modeling

Yellowbrick: Machine Learning Visualization

This glossary defines general machine learning terms and terms specific to TensorFlow

Scikit.learn: Python machine learning modules

Rolling In Deep Learning Basics Concepts For Everyone

Top 10 Python Libraries for Data Science

Model Interpretation

Interpretable Machine Learning- A Guide for Making Black Box Models Explainable

A Unified Approach to Interpreting Model Predictions

Hulstaert, L. (2018a, February 20). Interpreting machine learning models. Retrieved March 25, 2019, from Towards Data Science

Hulstaert, L. (2018b, July 11). Model Predictions with LIME. Retrieved January 16, 2019,

Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead

One Feature Attribution Method to (Supposedly) Rule Them All: Shapley Values

Interpreting recurrent neural networks on multivariate time series

Interpreting a Model with ‘MLI’ and ‘Interpret This Model Button’

Resources:

Books, Libraries, Online Courses, Blogs, and Web Sites, Seminars, conferences and courses at Universities

Books

James, G., Witten, D., Hastie, T., & Tibshirani, R. (2013). An Introduction to Statistical Learning: with Applications in R. - source 1

James, G., Witten, D., Hastie, T., & Tibshirani, R. (2013). An Introduction to Statistical Learning: with Applications in R. - source 2

Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep learning. MIT press Cambridge

Hoff, P. D. (2009). A First Course in Bayesian Statistical Methods. Springer Science & Business Media.

Stephen Marsland

Learning Bayesian Models with R

Learning From Data

Practical Machine Learning in R and Python: Third edition

Geron, A. (2017). Hands-on machine learning with Scikit-Learn and TensorFlow: concepts, tools, and techniques to build intelligent systems. O???Reilly Media, Inc.

Athey, S. (2019). The Impact of Machine Learning on Economics. In A. K. Agrawal, J. S. Gans, & A. Goldfarb (Eds.), The Economics of Artificial Intelligence: An Agenda.

Bishop, C. M. (2016). Pattern Recognition and Machine Learning. Springer New York.

Marsland, S. (2015). Machine Learning: An Algorithmic Perspective, Second Edition. CRC Press.

Mitchell, T. M. (1997). Machine Learning. McGraw-Hill.

Murphy, K. P. (2012). Machine Learning: A Probabilistic Perspective. MIT Press.

Szeliski, R. (2010). Computer Vision: Algorithms and Applications. Springer Science & Business Media.

Trevor, H., Robert, T., & JH, F. (2009). The elements of statistical learning: data mining, inference, and prediction. New York, NY: Springer.

Articles

Cui, G., Wong, M. L., & Lui, H.-K. (2006). Machine Learning for Direct Marketing Response Models: Bayesian Networks with Evolutionary Programming. Management Science, 52(4), 597???612.

Domingos, P. (2012). A Few Useful Things to Know About Machine Learning. Communications of the ACM, 55(10), 78???87.

Breiman, L. (2001). Random forests. Machine Learning, 45(1), 5-32.

Jordan, M. I., & Mitchell, T. M. (2015). Machine learning: Trends, perspectives, and prospects. Science, 349(6245), 255???260.

Ribeiro, M. T., Singh, S., & Guestrin, C. (2016). Why should I trust you?: Explaining the predictions of any classifier. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 1135???1144. ACM.

Targeted Learning in Data Science: Causal Inference for Complex Longitudinal Studies (2018)

Sprigg, J., Becker, K., & Cosmas, A. (2014). Estimating individual promotional campaign impacts through Bayesian inference. Journal of Consumer Marketing, 31(6/7), 541-552.