Clever questions, answers, post on machine learning and artifical intelligence
4 Contributors
39.2K followers
· 31 posts in the last week
Machine Learning and Artificial Intelligence Questions and Answers and Resources.
Supervised Learning
Linear Regression
Logistic Regression
Naive Bayes
Support Vector Machines
Decision Trees
K-Nearest Neighbors
Machine Learning in Practice
Bias-Variance Tradeoff
How to Select a Model
How to Select Features
Regularizing Your Model
Ensembling: How to Combine Your Models
Evaluation Metrics
Unsupervised Learning
Market Basket Analysis
K-Means Clustering
Principal Components Analysis
Deep Learning
Feedforward Neural Networks
Grab Bag of Neural Network Practices
Convolutional Neural Networks
Recurrent Neural Networks
Test Your Knowledge
Best Subset Features Feature
Selection Examples
Adding Features Example
Top 200 Open and Public Dataset
A curated, daily feed of newly published datasets in machine learning
The CIFAR-10 dataset consists of 60000 32x32 colour images in 10 classes, with 6000 images per class. There are 50000 training images and 10000 test images.
Data Sciences - Top 100 DataSets - Data Visualization - Data Analytics - Big Data - Data Lakes
Problem Solving Steps in Machine Learning (ML)
1. Problem definition: what is the problem, why does the problem need to be solved, how can the problem be solved?
Activities involved in problem definition include:
• Feature Engineering
• Outlier treatment
• Data formatting
• Data cleaning
• Data Normalization
2. Data Collection: What data is needed for the project, Where is it available, How can it be obtained?
3. Data Preparation: this stage usually takes the bulk of the project time. It involves exploring, conditioning and transforming data before modeling and analysis is done.
4. Algorithm Selection: Several Machine learning algorithms for handling various kinds of data – images, audio, text or numbers - exist. These algorithms are either supervised, unsupervised or reinforcement learning algorithms.
5. Data Modeling: A ML algorithm is trained with input data and Patterns in the data are discovered. This knowledge is used in predictions when new data is given. Frameworks like PyTorch and TensorFlow have pre-trained models that can be used in solving several problems.
6. Model Validation: the outputs of a model are compared to real world observations to know if they both correspond to each other in quantity and quality. After splitting a given data into train, test and cross validation datasets, the cross validation dataset is used to validate the model. Model validation involves tuning the hyper-parameters of the model.
Types of Model validation are:
• Split Sample Validation
• Cross Validation
• Bootstrapping Validation
Cross Validation
Jack-Knife/Leave-one-out
K-fold cross-validation i.e.10-fold cross-validation
7. Model Evaluation: The model is evaluated to test its final performance. In this stage we find out which model represents our data most. Then we determine how well a chosen model will work in the future. After model training is done and validation is carried out, the test dataset is used for evaluation of the model.
Different types of Evaluations exist for Classification and Regression algorithms.
Classification
• Confusion Matrix – Accuracy, False Positive, False Negative, Recall, Precision, Specificity, negative Predicted Value.
• Receiver Operating Characteristic Curve (ROC Curve)
• Area Under Curve (AUC)
• Gini Coefficient
• Gain and Lift Charts
• KS Chart (Kolmogorov-Smirnov)
Regression
• Sum Squared Error (SSE), Mean Square Error(MSE) and Root Mean Square (RMSE)
• Relative Squared Error (RSE)
• Mean Absolute Error (MAE)
• Mean absolute Deviation (MAD)
• Relative Absolute Error (RAE)
• Coefficient of Determination – SSR, SST and SSE
• Adjusted R2
• Analysis of Residuals
8. Model Deployment: ML model is put to use in order to automate data-driven decision making in the production line. Sometimes, it involves integrating the model into a broader system which might provide an interactive interface.
