[competitive-data-science] Week 1

As in any competitive field, you need to work very hard to get a prize in a competition.

Real World ML Pipeline:

• Understanding of business problem
• Problem formalization
• Data collecting
• Data preprocessing
• Modelling
• Way to evaluate model in real life
• Way to deploy model

Real World Aspect:

• Competition Problem formalization
• Choice of target metric
• Deployment issues
• Inference speed
• Data collecting
• Model complexity
• Target metric value

Competition Aspect:

• Competition Problem formalization (N)
• Choice of target metric (N)
• Deployment issues (N)
• Inference speed (N)
• Data collecting (Y/N)
• Model complexity (Y/N)
• Target metric value (Y)

Recap of main ML algorithms:

• Explanation of Random Forest
• Explanation/Demonstration of Gradient Boosting
• Example of kNN

Overview of ML methods:

• Scikit-Learn (or sklearn) library
• Overview of k-NN (sklearn's documentation)
• Overview of Linear Models (sklearn's documentation)
• Overview of Decision Trees (sklearn's documentation)
• Overview of algorithms and parameters in H2O documentation

Additional Tools:

• Vowpal Wabbit repository
• XGBoost repository
• LightGBM repository
• Interactive demo of simple feed-forward Neural Net
• Frameworks for Neural Nets: Keras, PyTorch, TensorFlow, MXNet, Lasagne
• Example from sklearn with different decision surfaces
• Arbitrary order factorization machines

Stack and packages:

• Basic SciPy stack (ipython, numpy, pandas, matplotlib)
• Jupyter Notebook
• Stand-alone python tSNE package
• Libraries to work with sparse CTR-like data: LibFM, LibFFM
• Another tree-based method: RGF (implemetation, paper)
• Python distribution with all-included packages: Anaconda
• Blog "datas-frame" (contains posts about effective Pandas usage)

Feature preprocessing:

• Preprocessing in Sklearn
• Andrew NG about gradient descent and feature scaling
• Feature Scaling and the effect of standardization for machine learning algorithms

Feature generation:

• Discover Feature Engineering, How to Engineer Features and How to Get Good at It
• Discussion of feature engineering on Quora

Feature extraction from text:

• Bag of words
• Feature extraction from text with Sklearn
• More examples of using Sklearn
• Word2vec
• Tutorial to Word2vec
• Tutorial to word2vec usage
• Text Classification With Word2Vec
• Introduction to Word Embedding Models with Word2Vec

NLP Libraries:

• NLTK
• TextBlob

Feature extraction from images:

• Pretrained models
• Using pretrained models in Keras
• Image classification with a pre-trained deep neural network
• Finetuning
• How to Retrain Inception's Final Layer for New Categories in Tensorflow
• Fine-tuning Deep Learning Models in Keras