GATE DA / Section 6

Machine Learning

Supervised learning — regression and classification, simple and multiple linear regression, ridge regression, logistic regression, k-nearest neighbours, naive Bayes, linear discriminant analysis, support vector machines, decision trees, the bias–variance trade-off, cross-validation (leave-one-out and k-fold), multi-layer perceptrons and feed-forward neural networks; unsupervised learning — k-means and k-medoids, hierarchical clustering (single, complete and average linkage) and principal component analysis.

13 topics124 practice questions≈ 11–14 marksPractise this section →

Machine learning ties the mathematics of Sections 1–3 to practice: least squares is a projection, logistic regression is maximum likelihood, naive Bayes and LDA are Bayes' theorem, PCA is an eigendecomposition, and training a neural network is gradient descent with the chain rule. Third-party analyses put ML at roughly 11 marks in 2024 and 14 in 2025.

What the syllabus lists: Supervised learning — regression and classification problems, simple linear regression, multiple linear regression, ridge regression, logistic regression, k-nearest neighbour, naive Bayes classifier, linear discriminant analysis, support vector machine, decision trees, bias–variance trade-off, cross-validation methods such as leave-one-out (LOO) cross-validation and k-fold cross-validation, multi-layer perceptron, feed-forward neural network. Unsupervised learning — clustering algorithms, k-means/k-medoid, hierarchical clustering, top-down, bottom-up: single-linkage, multiple-linkage, dimensionality reduction, principal component analysis.

Topics

Read in order — later topics build on earlier ones

01Foundations — Regression vs Classification, Loss Functions and EvaluationThe supervised and unsupervised learning settings, regression vs classification, the training/validation/test workflow, empirical risk minimisation and common loss functions, and how models are evaluated — MSE, MAE, R², the confusion matrix, accuracy, precision, recall, F1, specificity, ROC curves and AUC — including the pitfalls of imbalanced data.0/10 solved 02Simple and Multiple Linear RegressionThe linear model and least-squares estimation — closed-form slope and intercept for simple regression, the matrix form and normal equations for multiple regression, geometric interpretation as projection, properties of residuals, R² and its link to correlation, assumptions and diagnostics, interpreting coefficients, categorical predictors and interactions, multicollinearity, and fitting by gradient descent.0/10 solved 03Ridge Regression and RegularizationWhy least squares overfits with many or correlated features, the ridge objective and its closed-form solution, how the penalty λ shrinks coefficients (SVD view and effective degrees of freedom), the bias–variance effect, standardisation and the unpenalised intercept, the Bayesian MAP interpretation, and comparison with the lasso (L1) and elastic net.0/9 solved 04Logistic RegressionModelling class probabilities with the sigmoid, odds and log-odds, interpreting coefficients as odds ratios, the linear decision boundary, maximum likelihood and the cross-entropy (log) loss, its gradient and gradient-descent training, convexity, separable data and regularisation, multinomial (softmax) regression, and threshold choice.0/10 solved 05k-Nearest NeighboursThe k-NN algorithm for classification and regression, distance metrics (Euclidean, Manhattan, Minkowski, cosine, Hamming), why feature scaling matters, choosing k and its bias–variance effect, distance-weighted voting, computational cost and speed-ups (k-d trees, approximate search), the curse of dimensionality, and the 1-NN error guarantee.0/9 solved 06Naive Bayes ClassifierThe Bayes-optimal (MAP) classifier, the naive conditional-independence assumption and why it makes learning tractable, parameter estimation by counting, worked spam example, zero counts and Laplace smoothing, Bernoulli, multinomial and Gaussian variants, log-space computation, and strengths and weaknesses.0/8 solved 07Linear Discriminant AnalysisLDA as a generative classifier with Gaussian class-conditional densities and a shared covariance matrix, the linear discriminant functions and decision boundaries, the one-dimensional case and the effect of priors, quadratic discriminant analysis when covariances differ, Fisher's LDA as supervised dimensionality reduction (between- vs within-class scatter), and comparisons with logistic regression and PCA.0/8 solved 08Support Vector MachinesMaximum-margin linear classifiers — hyperplanes, functional and geometric margins, the hard-margin optimisation problem and its support vectors, soft margins with slack variables and the parameter C, the hinge-loss view, the dual problem, the kernel trick with polynomial and RBF kernels, hyperparameter effects, and multi-class strategies.0/10 solved 09Decision TreesHow decision trees partition the feature space, greedy top-down induction, impurity measures (entropy, Gini, misclassification error), information gain and gain ratio worked step by step, splitting numeric attributes, regression trees with variance reduction, stopping rules and pruning, overfitting, and ensembles built from trees (bagging, random forests, boosting).0/10 solved 10Bias–Variance Trade-off and Cross-ValidationUnderfitting and overfitting, the bias–variance decomposition of expected squared error, how model complexity, regularisation and data size move bias and variance, learning curves, and estimating generalisation error — hold-out validation, k-fold cross-validation, leave-one-out cross-validation, stratification, nested CV, and avoiding data leakage.0/10 solved 11Multi-Layer Perceptrons and Feed-Forward Neural NetworksThe perceptron and its learning rule, linear separability and the XOR problem, multi-layer feed-forward networks, activation functions, the forward pass and output layers (linear, sigmoid, softmax) with matching losses, backpropagation via the chain rule, training with (stochastic) gradient descent, counting parameters, the universal approximation theorem, vanishing gradients, initialisation and regularisation.0/10 solved 12Clustering — k-Means, k-Medoids and Hierarchical ClusteringThe clustering problem and distance choices, k-means (Lloyd's algorithm, the within-cluster sum of squares, convergence, initialisation with k-means++, choosing k), k-medoids (PAM) and why it is robust, hierarchical clustering bottom-up (agglomerative with single, complete, average, centroid and Ward linkage) and top-down (divisive), dendrograms, and cluster validation with the silhouette.0/10 solved 13Dimensionality Reduction and Principal Component AnalysisWhy reduce dimensions, feature selection vs feature extraction, PCA from two viewpoints (maximum variance and minimum reconstruction error), the algorithm via the covariance eigendecomposition or the SVD, a fully worked 2-D example, explained variance and choosing the number of components, scores and reconstruction, standardisation, and limitations with non-linear alternatives.0/10 solved