Machine Learning Advanced — Free Practice Questions

Scikit-learn's LinearRegression uses SVD internally, so it is a real method. Its cost grows roughly with the square of the feature count, which becomes prohibitive at millions of features.

The closed-form normal equation requires inverting an n×n matrix, where n is the number of features. That is roughly cubic in n — completely infeasible at this scale, whereas SGD handles one sample at a time.

Note : You have million of feature i.e. columns in your dataset and no information about number of rows is specified. If you have a training set with millions of features you can use Stochastic Gradient Descent or Mini-batch Gradient Descent, and perhaps Batch Gradient Descent if the training set fits in memory. But you cannot use the Normal Equation or the SVD approach because the computational complexity grows quickly (more than quadratically) with the number of features.

Overshoots. An unrestricted tree splits until leaves are pure, giving depth around log₂(m) for m instances. log₂(1,000,000) ≈ 19.93, so roughly 20.

True in a strict sense — a badly unbalanced tree can be deeper. The question asks for the approximate expected depth of a well-balanced binary tree, which log₂(m) gives.

The depth of a well-balanced binary tree containing m leaves is equal to log2(m). A binary Decision Tree (one that makes only binary decisions, as is the case with all trees in Scikit-Learn) will end up more or less well balanced at the end of training, with one leaf per training instance if it is trained without restrictions. Thus, if the training set contains one million instances, the Decision Tree will have a depth of log2(10^6) ≈ 20 (actually a bit more since the tree will generally not be perfectly well balanced).

Far too shallow. Depth 10 allows only 2¹⁰ ≈ 1,024 leaves, nowhere near enough to isolate a million instances.

Correct as far as it goes, and the primary use. Incomplete, though, because k-means is also applied to dimensionality reduction by replacing points with their cluster assignment or distances to centroids.

Recommendation systems use collaborative filtering or matrix factorisation. Clustering can group similar users as one input, but that is not what k-means is for.

Also correct on its own — k-means features are a recognised reduction technique. Since both this and clustering apply, the combined option is the answer.

Though you would be tempted to answer Clustering or Segmentation, K means can also be used for dimensionality reduction. For e.g. if the input feature space is 100 and if you create 10 clusters then each training instance can be represented by 10 features where each feature value is the distance of the instance from each cluster centroid. This way you use k means for dimensionality reduction as well.

Underfitting means the model is too simple to capture the pattern. Leaving the hyperparameters alone guarantees the problem persists.

The wrong direction. Gamma sets how far a single training example's influence reaches — a small gamma gives a broad, smooth boundary, which is exactly what causes underfitting. Raising it lets the boundary follow the data more closely.

If an SVM classifier trained with an RBF kernel underfits the training set, there might be too much regularization. To decrease it, you need to increase gamma

That’s a trick question: it depends on the dataset. Let’s look at two extreme examples. First, suppose the dataset is composed of points that are almost perfectly aligned. In this case, PCA can reduce the dataset down to just one dimension while still preserving 95% of the variance. Now imagine that the dataset is composed of perfectly random points, scattered all around the 1,000 dimensions. In this case roughly 950 dimensions are required to preserve 95% of the variance. So the answer is, it depends on the dataset, and it could be any number between 1 and 950. Plotting the explained variance as a function of the number of dimensions is one way to get a rough idea of the dataset’s intrinsic dimensionality.

Assumes a fixed answer. The number of components needed to reach 95% variance depends entirely on how correlated the features are.

Confuses the variance threshold with a dimension count. Retaining 95% of variance usually needs far fewer than 950 of 1,000 components, because variance concentrates in the leading ones.

Reads the 95% figure as though it were the number of dimensions. The two are unrelated — highly correlated data might reach 95% variance in 10 components, weakly correlated data might need 800.

Random Forest Classifier is similar to Bagging of Decision Trees but with a slight difference . RF additionally allows sampling of features and hence each estimator of RF may see a subset of features

They are closely related but not identical. Bagged trees sample the rows and then consider every feature at each split. A random forest additionally samples a random subset of features at each split, which decorrelates the trees and is what makes the ensemble stronger.

Nonparametric. The tree grows to fit the data, so the number of parameters — nodes and splits — is determined by the training set rather than fixed in advance.

Nonparametric in the general case. The model is defined by its support vectors, and how many there are depends on the data.

The clearest nonparametric example. It stores the entire training set and learns no fixed set of coefficients at all.

The logistic regression model is parametric because it has a finite set of parameters. Specifically, the parameters are the regression coefficients. These usually correspond to one for each predictor plus a constant. Logistic regression is a particular form of the generalized linear model. A learning model that summarizes data with a set of parameters of fixed size (independent of the number of training examples) is called a parametric model. No matter how much data you throw at a parametric model, it won’t change its mind about how many parameters it needs.

Parametric. Once you choose the architecture, the number of weights is fixed regardless of how much data you train on.

Also parametric for the same reason — a predetermined set of weights and biases, independent of dataset size.

Algorithms that do not make strong assumptions about the form of the mapping function are called nonparametric machine learning algorithms. By not making assumptions, they are free to learn any functional form from the training data. Nonparametric methods are good when you have a lot of data and no prior knowledge, and when you don’t want to worry too much about choosing just the right features. An easy to understand nonparametric model is the k-nearest neighbors algorithm that makes predictions based on the k most similar training patterns for a new data instance. The method does not assume anything about the form of the mapping function other than patterns that are close are likely to have a similar output variable.

The textbook parametric model: one coefficient per feature plus an intercept, fixed before training begins.