Machine Learning Lab

Definition: Jupyter Notebook is an interactive, browser-based environment in which Python code is written and executed in separate cells, with the output shown directly below each cell.

Reasons for using it in ML:

1. Output, tables and graphs appear immediately below the code.
2. Code can be run step by step, which is useful for exploring data.
3. Code can be combined with notes and explanations (Markdown cells).
4. Results are easy to share as a single notebook file.

Example: A student loads a CSV file in one cell, views the first five rows in the next cell, and plots a graph in a third cell. IDEs such as Spyder, VS Code and PyCharm can also be used.

Definition: These are the four main Python libraries used for data handling, analysis, visualisation and Machine Learning.

Example: Pandas reads students.csv, NumPy calculates the average marks, Matplotlib draws a histogram, and Scikit-learn trains a regression model.

Aim: To import a CSV dataset using Pandas and perform basic data operations such as viewing, selecting and filtering data.

Theory: A CSV (Comma-Separated Values) file stores tabular data as plain text. Pandas reads it into a DataFrame, a table of rows and columns. Functions such as head(), shape, info() and describe() help understand the structure of the data before building a model.

Algorithm:

1. Import the Pandas and NumPy libraries.
2. Load the CSV file using read_csv().
3. Display the first rows, shape, column names, data types and summary statistics.
4. Select columns and filter rows using conditions.
5. Use NumPy functions to calculate statistics on a column.

Program:

import pandas as pd
import numpy as np

# Load the dataset
df = pd.read_csv("students.csv")

print(df.head())          # first 5 rows
print(df.shape)           # (rows, columns)
print(df.columns)         # column names
df.info()                 # data types and non-null counts
print(df.describe())      # count, mean, std, min, quartiles, max

# Selecting and filtering
print(df["Marks"])                    # one column
print(df[["Hours", "Marks"]])         # two columns
print(df[df["Marks"] > 60])          # rows where Marks > 60

# NumPy operations on a column
marks = df["Marks"].to_numpy()
print("Mean:", np.mean(marks), "Max:", np.max(marks), "SD:", np.std(marks))

Output (description): The program displays the first five records, the number of rows and columns, the data type of each column, summary statistics, the selected columns, the filtered rows with marks above 60, and the mean, maximum and standard deviation of marks. Exact values depend on the dataset.

Conclusion: The dataset was successfully imported into a DataFrame, and basic operations such as viewing, selecting, filtering and summarising data were performed using Pandas and NumPy.

Aim: To clean a dataset by handling missing values, removing duplicate rows and encoding a categorical column.

Theory: Real datasets often contain missing values (NaN), duplicate rows and text categories. Missing numerical values can be filled with the median, duplicates can be removed with drop_duplicates(), and text labels such as Pass/Fail can be converted to numbers (Pass = 1, Fail = 0) because ML algorithms work with numbers.

Algorithm:

1. Load the dataset.
2. Count missing values in each column using isnull().sum().
3. Fill missing values in the Attendance column with its median.
4. Count and remove duplicate rows.
5. Encode the Result column as 1 and 0.
6. Display the cleaned dataset.

Program:

import pandas as pd

df = pd.read_csv("students.csv")

# 1. Check missing values in each column
print(df.isnull().sum())

# 2. Fill missing Attendance with the median
df["Attendance"] = df["Attendance"].fillna(df["Attendance"].median())
#    (or drop rows with missing values: df = df.dropna())

# 3. Remove duplicate rows
print("Duplicates:", df.duplicated().sum())
df = df.drop_duplicates()

# 4. Encode the target column: Pass = 1, Fail = 0
df["Result"] = df["Result"].map({"Pass": 1, "Fail": 0})

print(df.isnull().sum())   # should now be all 0
print(df.head())

Output (description): The first output shows the number of missing values per column (for example, Attendance = 2). After cleaning, all missing-value counts become 0, the number of duplicates removed is printed, and the Result column shows 1 and 0 instead of Pass and Fail.

Conclusion: The dataset was cleaned successfully – missing values were filled, duplicates removed and the categorical column encoded – making it ready for model building.

Aim: To select relevant features using correlation and split the dataset into training and testing sets.

Theory: Correlation measures how strongly two variables are related (from −1 to +1). Features strongly correlated with the target are useful for prediction. The data is then split into a training set (to teach the model) and a test set (to evaluate it) using train_test_split().

Algorithm:

1. Calculate the correlation table of the cleaned dataset.
2. Choose features strongly correlated with the target; drop IDs and names.
3. Separate features (X) and target (y).
4. Split the data into 80% training and 20% testing.
5. Display the shapes of the training and test sets.

Program:

from sklearn.model_selection import train_test_split

# Correlation of each feature with the target (after encoding)
print(df.corr())
# Keep features with strong correlation to the target; drop IDs / names

X = df[["Hours", "Attendance"]]   # features
y = df["Result"]                  # target

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42)

print(X_train.shape, X_test.shape)

Output (description): The correlation table shows values between −1 and +1 for each pair of columns (for example, Hours has a strong positive correlation with Result). The shapes printed show that about 80% of rows are in the training set and 20% in the test set.

Key points: test_size=0.2 keeps 20% of data for testing; random_state=42 gives the same split every time.

Conclusion: Relevant features were selected using correlation, and the data was divided into training and testing sets for model building and evaluation.

Aim: To perform Exploratory Data Analysis (EDA) by visualising data using a histogram, scatter plot and bar chart.

Theory: EDA is the process of exploring data visually and statistically to find patterns, relationships and unusual values before modelling. A histogram shows the distribution of one variable, a scatter plot shows the relationship between two variables, and a bar chart compares categories.

Algorithm:

1. Import Matplotlib.
2. Plot a histogram of Marks.
3. Plot a scatter plot of Hours against Marks.
4. Plot a bar chart of Pass and Fail counts.
5. Add titles and axis labels, then display each graph.

Program:

import matplotlib.pyplot as plt

# Histogram: distribution of marks
plt.hist(df["Marks"], bins=5, edgecolor="black")
plt.title("Distribution of Marks")
plt.xlabel("Marks"); plt.ylabel("Number of students")
plt.show()

# Scatter plot: relationship between hours and marks
plt.scatter(df["Hours"], df["Marks"])
plt.title("Study Hours vs Marks")
plt.xlabel("Hours"); plt.ylabel("Marks")
plt.show()

# Bar chart: count of Pass and Fail
df["Result"].value_counts().plot(kind="bar")
plt.title("Pass (1) vs Fail (0)")
plt.show()

Output (description): Three graphs are displayed – a histogram showing how marks are spread, a scatter plot showing an upward trend between study hours and marks, and a bar chart comparing the number of students who passed and failed.

Conclusion: The data was visualised successfully. The upward pattern in the scatter plot indicates a positive relationship between study hours and marks.

Aim: To implement simple linear regression in Python and evaluate it using MAE, MSE and RMSE.

Theory: Simple Linear Regression fits a straight line Y = a + bX to the data using the least squares method. MAE is the average absolute error, MSE is the average squared error, and RMSE is the square root of MSE. Lower values indicate a better model.

Algorithm:

1. Import NumPy, Matplotlib and the required Scikit-learn classes.
2. Create the input X (as a 2D array) and output y.
3. Create a LinearRegression model and train it with fit().
4. Print the intercept and slope, and predict Y for X = 6.
5. Calculate MAE, MSE and RMSE.
6. Plot the data points and the regression line.

Program:

import numpy as np
import matplotlib.pyplot as plt
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_absolute_error, mean_squared_error

X = np.array([1, 2, 3, 4, 5]).reshape(-1, 1)   # 2D input required
y = np.array([2, 4, 5, 4, 5])

model = LinearRegression()
model.fit(X, y)
print("Intercept (a):", model.intercept_)
print("Slope (b):", model.coef_[0])
print("Prediction for X=6:", model.predict([[6]]))

y_pred = model.predict(X)
mae = mean_absolute_error(y, y_pred)
mse = mean_squared_error(y, y_pred)
rmse = np.sqrt(mse)
print("MAE:", mae, "MSE:", mse, "RMSE:", rmse)

plt.scatter(X, y, label="Actual")
plt.plot(X, y_pred, color="red", label="Regression line")
plt.xlabel("X"); plt.ylabel("Y"); plt.legend(); plt.show()

Output:

Intercept (a): 2.2
Slope (b): 0.6
Prediction for X=6: [5.8]
MAE: 0.64 MSE: 0.48 RMSE: 0.693

A graph shows the five data points and the red regression line.

Conclusion: The regression line obtained is Y = 2.2 + 0.6X, which matches the manual calculation in the Machine Learning theory paper (Q15). The model has MAE = 0.64, MSE = 0.48 and RMSE = 0.693.

Aim: To predict students' marks from study hours and attendance using multiple linear regression and evaluate the model on test data.

Theory: Multiple Linear Regression predicts a continuous output using two or more input features: Y = b₀ + b₁X₁ + b₂X₂. The model is trained on the training set and evaluated on the unseen test set using MAE and RMSE.

Algorithm:

1. Select features (Hours, Attendance) and target (Marks).
2. Split the data into training and test sets.
3. Train a LinearRegression model on the training set.
4. Print the coefficients and intercept.
5. Predict marks for the test set and calculate MAE and RMSE.

Program:

from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_absolute_error, mean_squared_error
import numpy as np

X = df[["Hours", "Attendance"]]
y = df["Marks"]
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

model = LinearRegression().fit(X_train, y_train)
print("Coefficients:", model.coef_, "Intercept:", model.intercept_)

y_pred = model.predict(X_test)
print("MAE :", mean_absolute_error(y_test, y_pred))
print("RMSE:", np.sqrt(mean_squared_error(y_test, y_pred)))

Output (description): The program prints two coefficients (one for Hours and one for Attendance), the intercept, and the MAE and RMSE on the test set. Positive coefficients show that marks increase with study hours and attendance. Exact values depend on the dataset.

Conclusion: A multiple linear regression model was built using two features and evaluated on unseen data, showing how well marks can be predicted from study hours and attendance.

Aim: To implement three classification algorithms – Logistic Regression, KNN and Naive Bayes – and compare their performance.

Theory: Logistic Regression uses the sigmoid function, KNN uses the majority class of the nearest neighbours, and Naive Bayes uses Bayes' theorem. Their performance is compared using a confusion matrix, accuracy, precision and recall. Feature scaling is applied because KNN depends on distance.

Algorithm:

1. Select features and the encoded target (Pass = 1, Fail = 0).
2. Split the data into training and test sets (with stratification).
3. Scale the features using StandardScaler.
4. Create the three models and store them in a dictionary.
5. For each model: train, predict, and print the confusion matrix, accuracy, precision and recall.
6. Compare the results.

Program:

from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
from sklearn.neighbors import KNeighborsClassifier
from sklearn.naive_bayes import GaussianNB
from sklearn.metrics import (confusion_matrix, accuracy_score,
                             precision_score, recall_score)

X = df[["Hours", "Attendance"]]
y = df["Result"]                       # 1 = Pass, 0 = Fail
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.25, random_state=42, stratify=y)

# Feature scaling (essential for KNN)
sc = StandardScaler()
X_train_s = sc.fit_transform(X_train)
X_test_s = sc.transform(X_test)

models = {
    "Logistic Regression": LogisticRegression(),
    "KNN (k=5)": KNeighborsClassifier(n_neighbors=5),
    "Naive Bayes": GaussianNB(),
}

for name, clf in models.items():
    clf.fit(X_train_s, y_train)
    p = clf.predict(X_test_s)
    print(name)
    print(confusion_matrix(y_test, p))
    print("Accuracy :", round(accuracy_score(y_test, p), 3))
    print("Precision:", round(precision_score(y_test, p), 3))
    print("Recall   :", round(recall_score(y_test, p), 3))

Output (description): For each model, the program prints a 2×2 confusion matrix in the form [[TN FP] [FN TP]] (rows = actual, columns = predicted) followed by accuracy, precision and recall. The model with the highest values performs best on this dataset.

Key points for the record: The scaler is fitted only on training data (fit_transform) and applied to test data (transform) to avoid data leakage. stratify=y keeps the same Pass/Fail ratio in both sets.

Conclusion: Three classification models were implemented and compared. The confusion matrix and metrics show which algorithm classifies students most accurately for the given data.

Aim: To classify a new data point using the K-Nearest Neighbours algorithm with K = 3.

Theory: KNN calculates the distance from the new point to all training points, selects the K nearest ones, and assigns the majority class among them.

Algorithm:

1. Create the training points and their class labels.
2. Create a KNN classifier with n_neighbors=3.
3. Train it with fit().
4. Predict the class of the new point (3, 2).

Program:

from sklearn.neighbors import KNeighborsClassifier

X = [[1, 2], [2, 3], [3, 1], [6, 5], [7, 7], [2, 1]]
y = ["Pass", "Pass", "Fail", "Fail", "Fail", "Pass"]

knn = KNeighborsClassifier(n_neighbors=3)
knn.fit(X, y)
print(knn.predict([[3, 2]]))

Output:

['Pass']

Conclusion: The new point (3, 2) is classified as Pass, which matches the manual calculation in the Machine Learning theory paper (Q25).

Aim: To build a decision tree classifier using entropy and visualise the tree.

Theory: A decision tree splits data using the feature that gives the highest information gain (lowest entropy). criterion="entropy" uses information gain, as in the ID3 algorithm, and max_depth limits the depth of the tree to reduce overfitting.

Algorithm:

1. Import the DecisionTreeClassifier, plot_tree and export_text functions.
2. Create the tree with criterion="entropy" and max_depth=3.
3. Train the tree on the training data.
4. Calculate test accuracy.
5. Print the rules as text and draw the tree diagram.

Program:

from sklearn.tree import DecisionTreeClassifier, plot_tree, export_text
from sklearn.metrics import accuracy_score
import matplotlib.pyplot as plt

tree = DecisionTreeClassifier(criterion="entropy", max_depth=3, random_state=42)
tree.fit(X_train, y_train)

print("Test accuracy:", accuracy_score(y_test, tree.predict(X_test)))
print(export_text(tree, feature_names=["Hours", "Attendance"]))   # rules as text

plt.figure(figsize=(10, 6))
plot_tree(tree, feature_names=["Hours", "Attendance"],
          class_names=["Fail", "Pass"], filled=True)
plt.show()

Output (description): The program prints the test accuracy and the tree rules, for example |--- Hours <= 4.50 → class: 0 (Fail) and |--- Hours > 4.50 → class: 1 (Pass). A coloured tree diagram is displayed, with the most informative feature at the root.

Conclusion: A decision tree was built using information gain. The tree shows that study hours is the most important feature for predicting the result, and its rules are easy to interpret.

Aim: To group data into clusters using the K-Means algorithm and visualise the clusters.

Theory: K-Means assigns each point to the nearest centroid and recalculates centroids as the mean of their points, repeating until the centroids stop changing. It is an unsupervised algorithm, so no labels are needed.

Algorithm:

1. Apply K-Means (K = 2) to the 1D data from the theory example with initial centroids 4 and 12.
2. Print the cluster labels and final centroids.
3. Apply K-Means (K = 3) to students' Hours and Marks.
4. Plot the points coloured by cluster and mark the centroids.

Program:

import numpy as np
from sklearn.cluster import KMeans
import matplotlib.pyplot as plt

# (a) 1D example from theory (ML theory paper, Q36)
data = np.array([2, 4, 10, 12, 3, 20, 30, 11, 25]).reshape(-1, 1)
km = KMeans(n_clusters=2, init=np.array([[4], [12]]), n_init=1)
km.fit(data)
print("Labels:", km.labels_)
print("Centroids:", km.cluster_centers_.ravel())

# (b) Clustering students on Hours and Marks
pts = df[["Hours", "Marks"]]
km2 = KMeans(n_clusters=3, random_state=42, n_init=10).fit(pts)

plt.scatter(pts["Hours"], pts["Marks"], c=km2.labels_)
plt.scatter(km2.cluster_centers_[:, 0], km2.cluster_centers_[:, 1],
            marker="X", s=200, c="red", label="Centroids")
plt.xlabel("Hours"); plt.ylabel("Marks"); plt.legend()
plt.title("K-Means clusters of students")
plt.show()

Output:

Labels: [0 0 0 0 0 1 1 0 1]
Centroids: [ 7. 25.]

A scatter plot shows students in three colours (clusters), with red X marks at the centroids.

Conclusion: K-Means produced the clusters {2, 3, 4, 10, 11, 12} and {20, 25, 30} with centroids 7 and 25, matching the manual calculation. The student plot shows three groups, such as low, medium and high performers.

Aim: To evaluate a decision tree model using 5-fold cross-validation.

Theory: In K-fold cross-validation, the data is split into K folds; the model is trained on K − 1 folds and tested on the remaining fold, K times. The average score is a more reliable measure of performance than a single train–test split.

Algorithm:

1. Create a decision tree model.
2. Use cross_val_score() with cv=5.
3. Print the five fold scores and their mean.

Program:

from sklearn.model_selection import cross_val_score
from sklearn.tree import DecisionTreeClassifier

model = DecisionTreeClassifier(max_depth=3, random_state=42)
scores = cross_val_score(model, X, y, cv=5)     # 5 folds, accuracy by default

print("Fold scores:", scores)
print("Mean accuracy:", scores.mean())

Output (description): Five accuracy values are printed (one per fold), followed by their mean, for example Mean accuracy: 0.93. Exact values depend on the dataset.

Conclusion: 5-fold cross-validation was performed, and the mean accuracy gives a reliable estimate of how well the model will perform on new data.

Aim: To demonstrate underfitting, good fit and overfitting by changing the maximum depth of a decision tree.

Theory: A very shallow tree is too simple and underfits (low training and test accuracy). A very deep tree memorises training data and overfits (very high training accuracy but lower test accuracy). The best depth gives high test accuracy close to training accuracy.

Algorithm:

1. Choose several depths: 1, 2, 3, 5 and unlimited (None).
2. For each depth, train a decision tree.
3. Print training and test accuracy.
4. Compare the values to identify underfitting, good fit and overfitting.

Program:

from sklearn.tree import DecisionTreeClassifier

for depth in [1, 2, 3, 5, None]:          # None = grow fully
    t = DecisionTreeClassifier(max_depth=depth, random_state=42)
    t.fit(X_train, y_train)
    print(f"depth={depth}: train={t.score(X_train, y_train):.2f}, "
          f"test={t.score(X_test, y_test):.2f}")

Output (description): One line is printed for each depth with training and test accuracy. Typically, small depths give lower accuracy on both sets, and full depth gives training accuracy near 1.00 with lower test accuracy.

Interpretation:

1. Low train and low test accuracy (very small depth) → underfitting.
2. Train accuracy near 1.00 but clearly lower test accuracy (full depth) → overfitting.
3. Highest test accuracy close to train accuracy → good fit.

Conclusion: Changing tree depth clearly showed underfitting and overfitting. The best model is the one with the depth that gives the highest test accuracy.

Introduction

The mini project applies the complete Machine Learning workflow to a real problem. Here, the aim is to predict whether a student will pass using academic data. The project combines data handling, preprocessing, visualisation, model building and evaluation.

Aim: To predict student results (Pass / Fail) using the complete ML workflow.

Steps of the Mini Project

1. Problem definition: Binary classification (Pass = 1 / Fail = 0).
2. Load data: Read the dataset using pd.read_csv() and inspect it with head(), info() and describe().
3. Preprocess: Fill missing values, remove duplicates, encode labels and scale features.
4. EDA: Draw histograms and scatter plots and study the correlation table.
5. Feature selection: Keep Hours and Attendance; drop ID columns.
6. Split: Use train_test_split(test_size=0.2).
7. Train: Build Logistic Regression, KNN, Naive Bayes and Decision Tree models.
8. Evaluate: Use the confusion matrix, accuracy, precision, recall and 5-fold cross-validation.
9. Check fit: Compare training and test accuracy to detect overfitting or underfitting.
10. Conclude: Report the best model with its scores, graphs and interpretation.

Record format: Aim, Dataset description, Algorithm / steps, Program, Output (with screenshots of plots), Result and Conclusion.

Example result: "Logistic Regression gave the highest cross-validated recall and was selected to identify at-risk students."

Conclusion

The mini project demonstrates the complete ML workflow from raw data to a tested model. It develops practical skills in Python, data preparation, model building and evaluation needed for real-world ML tasks.