diff --git a/.gitignore b/.gitignore
new file mode 100644
index 0000000..313f23e
--- /dev/null
+++ b/.gitignore
@@ -0,0 +1,6 @@
+package.json
+package-lock.json
+build/
+plugins/
+node_modules/
+.DS_Store
\ No newline at end of file
diff --git a/.vscode/settings.json b/.vscode/settings.json
new file mode 100644
index 0000000..6f3a291
--- /dev/null
+++ b/.vscode/settings.json
@@ -0,0 +1,3 @@
+{
+ "liveServer.settings.port": 5501
+}
\ No newline at end of file
diff --git a/experiment-descriptor.json b/experiment-descriptor.json
index c474242..700644d 100644
--- a/experiment-descriptor.json
+++ b/experiment-descriptor.json
@@ -2,6 +2,7 @@
"unit-type": "lu",
"label": "",
"basedir": ".",
+ "LaTeXinMD": "true",
"units": [
{
"unit-type": "aim"
diff --git a/experiment/aim.md b/experiment/aim.md
index a3f7d87..26589d6 100644
--- a/experiment/aim.md
+++ b/experiment/aim.md
@@ -1,5 +1,28 @@
+### Understanding Classification Approaches
+#### Discriminative vs Generative Classifiers
+1. **Discriminative Classifiers** (e.g., Linear Perceptrons)
+ - Learn decision boundaries directly
+ - Effective for well-separated classes
+ - Focus on class separation
+ - Simpler to implement
-Linear perceptrons allow us to learn a decision boundary that would separate two classes. They are very effective when there are only two classes, and they are well separated. Such classifiers are referred to as discriminative classifiers.
+2. **Generative Classifiers** (Bayesian Approach)
+ - Model each class as a random vector
+ - Use probability distributions/density functions
+ - Consider class likelihoods
+ - More flexible in handling complex data
-In contrast, generative classifiers consider each sample as a random vector, and explicity model each class by their distribution or density functions. To carry out the classification, we compute the likelihood that a given sample belong to each of the candidate classes, and assign the sample to the class that is most likely. In other words, we need to compute P(ωi/x) for each class ωi. However, the density functions provide only the likelihood of seeing a particular sample, given that the sample belongs to a specific class. i.e., the density functions provide us p(x/ωi). The Bayes rule provides us with an approach to compute the likelihood of the class for a given sample, from the density functions and related information.
+### Bayesian Classification
+The experiment focuses on understanding how Bayesian classification:
+- Computes class probabilities using Bayes' rule
+- Combines prior knowledge with observed evidence
+- Makes decisions based on posterior probabilities
+- Handles uncertainty in classification
+
+### Key Concepts
+- Prior probabilities
+- Likelihood functions
+- Posterior probabilities
+- Decision boundaries
+- Class density functions
diff --git a/experiment/assignment.md b/experiment/assignment.md
index 54defa2..d555bc0 100644
--- a/experiment/assignment.md
+++ b/experiment/assignment.md
@@ -1,14 +1,34 @@
- 1. The covariance matrix is always
+### Question 1: Covariance Matrix Properties
+The covariance matrix is always:
- a) Square
+a) Square
+b) Positive Semidefinite
+c) Positive Definite
+d) Symmetric
+e) None of the above
- b) Positive Semidefinite
+For each of the properties you selected, describe what would happen if the covariance matrix does not satisfy that property.
- c) Positive Definite
+### Question 2: Decision Boundaries
+Describe the possible set of decision boundaries that can be generated using Gaussian density functions in a two-class problem.
- d) Symmetric
+### Question 3: Bayesian Classification Components
+Consider a two-class classification problem using Bayesian approach:
- e) None of the above.
- For each of the properties you selected, describe what would happen if the covariance matrix does not satisfy that property.
+a) Explain how the prior probabilities affect the decision boundary when:
+ - Both classes have equal priors
+ - One class has a higher prior than the other
-2. Describe the possible set of decision boundaries that can be generated using gaussian density functions in a two class problem.
+b) How does the likelihood function influence the classification decision when:
+ - The classes have similar covariance matrices
+ - The classes have different covariance matrices
+
+### Question 4: Practical Application
+Given a dataset with two classes, where:
+- Class 1: Mean = (2, 3), Covariance = [[1, 0], [0, 1]]
+- Class 2: Mean = (4, 5), Covariance = [[2, 0], [0, 2]]
+- Prior probabilities: P(Class 1) = 0.6, P(Class 2) = 0.4
+
+a) Calculate the posterior probability for a test point (3, 4)
+b) Determine the class assignment for this point
+c) Explain how the decision would change if the prior probabilities were equal
diff --git a/experiment/objective.md b/experiment/objective.md
index 7db1c3e..1da6975 100644
--- a/experiment/objective.md
+++ b/experiment/objective.md
@@ -1,11 +1,8 @@
The high level goals of the experiment are:
1. To understand the computation of likelihood of a class, given a sample.
-
2. To understand the the use of density/distribution functions to model a class.
3. To understand the effect of prior probabilities in Bayesian classification.
-
4. To understand how two (or more) density functions interact in the feature space to decide a decision boundary between classes.
-
5. To understand how the decision boundary varies based on nature of the density functions.
diff --git a/experiment/procedure.md b/experiment/procedure.md
index 81be621..689a517 100644
--- a/experiment/procedure.md
+++ b/experiment/procedure.md
@@ -1,28 +1,29 @@
-**Stage 1:**
+### Set Class Parameters
+In the **Parameters** section on the left side:
- 1. Lauch the experiment and clear the pane. Assign differnt means and covariances for each of the classes and observe the resulting densities. Use the mark-all button to observe the decision boundaries.
+ * You will see parameters for **Class 1** and **Class 2**.
+ * Each class has fields to set:
- 2. Note down your observations on the relationship between the decision boundaries and the density functions.
+ * Mean values (`Mean X`, `Mean Y`)
+ * Covariance matrix values (`Covariance XX`, `Covariance XY`, `Covariance YY`)
+ * Prior Probability (the probability of each class before seeing data)
-**Stage 2:**
+### Adjust Parameters as Desired
- 1. Repeat the above procedure for different values of prior probabilities.
+* Modify any of the values by clicking on the input box and typing new numbers.
+* The means define the center location of each class’s distribution.
+* The covariance matrix controls the shape and orientation of the distribution.
+* The prior probability defines how likely each class is, used for classification decisions.
- 2. Observe the change in the scaled density functions and decision boundaries
+### Visualize Classification
- 3. Note down your observations regarding the change of decision boundaries
+After setting the parameters, the visualization will automatically reflect these changes on the plot canvas (600x600 pixels) on the right.
-**Stage 3:**
+#### Use Buttons to Interact with the Plot
-Generate the following types of decision boundaries by varying the means and covariance matrices.
+* **Mark All:** Click the **Mark All** button to classify and mark all points in the visualization area based on the current parameters.
+* **Clear:** Click the **Clear** button to remove all marks and reset the plot display.
- 1. Straight line
-
- 2. Parallel Straight lines
-
- 3. Concentric circles
- 4. Parabola
- 5. Hyperbola
- 6. Four Quadrants
-
-Explain why these shapes are generated in each case.
+#### Interpret the Visualization
+* The plot shows the 2D distributions for Class 1 and Class 2 based on your parameter inputs.
+* Points or areas marked correspond to the classification regions according to Bayesian decision rules.
diff --git a/experiment/simulation/Exp5.tar.gz b/experiment/simulation/Exp5.tar.gz
deleted file mode 100644
index 669608a..0000000
Binary files a/experiment/simulation/Exp5.tar.gz and /dev/null differ
diff --git a/experiment/simulation/Exp5.zip b/experiment/simulation/Exp5.zip
deleted file mode 100644
index 4c81840..0000000
Binary files a/experiment/simulation/Exp5.zip and /dev/null differ
diff --git a/experiment/simulation/Exp5/Exp5.jar b/experiment/simulation/Exp5/Exp5.jar
deleted file mode 100755
index d40a73b..0000000
Binary files a/experiment/simulation/Exp5/Exp5.jar and /dev/null differ
diff --git a/experiment/simulation/Exp5/Exp5.jnlp b/experiment/simulation/Exp5/Exp5.jnlp
deleted file mode 100755
index 2f00f92..0000000
--- a/experiment/simulation/Exp5/Exp5.jnlp
+++ /dev/null
@@ -1,19 +0,0 @@
-
-
-
- Virtual Labs: Pattern Recognition
- amit
-
- Exp5:Bayesian Classification
- Exp3
-
-
-
-
-
-
-
-
-
-
-
diff --git a/experiment/simulation/Exp5/README.txt b/experiment/simulation/Exp5/README.txt
deleted file mode 100644
index b147ff2..0000000
--- a/experiment/simulation/Exp5/README.txt
+++ /dev/null
@@ -1,15 +0,0 @@
-For linux users:
-1. Extract the downloaded .tar.gz folder.
-2. Click on the .jar file of the experiment.
-
-For windows users:
-1. Extract the downloaded .zip folder.
-2. Go to the Java console.
-3. Go to the Security tab.
-4. Check the "Enable Java content in the browser".
-5. Click on the button "Edit Site List".
-6. In the new dilogue box that appears, click on "Add".
-7. Type and add the following url to the exception list : "http://test.virtual-labs.ac.in".
-8. Click on the "OK" button of the dialogue box.
-9. Click on the "OK" button of the Java console.
-10. Click on the .jar file of the experiment.
\ No newline at end of file
diff --git a/experiment/simulation/Exp5/lib/Jama-1.0.2.jar b/experiment/simulation/Exp5/lib/Jama-1.0.2.jar
deleted file mode 100644
index 824d133..0000000
Binary files a/experiment/simulation/Exp5/lib/Jama-1.0.2.jar and /dev/null differ
diff --git a/experiment/simulation/Exp5/lib/jcommon-1.0.16.jar b/experiment/simulation/Exp5/lib/jcommon-1.0.16.jar
deleted file mode 100644
index 4cd6807..0000000
Binary files a/experiment/simulation/Exp5/lib/jcommon-1.0.16.jar and /dev/null differ
diff --git a/experiment/simulation/Exp5/lib/jfreechart-1.0.13.jar b/experiment/simulation/Exp5/lib/jfreechart-1.0.13.jar
deleted file mode 100644
index 83c6993..0000000
Binary files a/experiment/simulation/Exp5/lib/jfreechart-1.0.13.jar and /dev/null differ
diff --git a/experiment/simulation/Exp5/lib/junit.jar b/experiment/simulation/Exp5/lib/junit.jar
deleted file mode 100644
index 674d71e..0000000
Binary files a/experiment/simulation/Exp5/lib/junit.jar and /dev/null differ
diff --git a/experiment/simulation/Exp5/lib/libsvm.jar b/experiment/simulation/Exp5/lib/libsvm.jar
deleted file mode 100644
index d6bf594..0000000
Binary files a/experiment/simulation/Exp5/lib/libsvm.jar and /dev/null differ
diff --git a/experiment/simulation/README.md b/experiment/simulation/README.md
new file mode 100644
index 0000000..e90f8e8
--- /dev/null
+++ b/experiment/simulation/README.md
@@ -0,0 +1,6 @@
+the files used for simulation are:
+- index.html
+- script.js
+- style.css
+
+rest of the files are NOT USED for this version
\ No newline at end of file
diff --git a/experiment/simulation/index.html b/experiment/simulation/index.html
index f6d1f4d..4d63581 100644
--- a/experiment/simulation/index.html
+++ b/experiment/simulation/index.html
@@ -1,16 +1,95 @@
-
+
+
+
diff --git a/experiment/simulation/script.js b/experiment/simulation/script.js
new file mode 100644
index 0000000..ac2a043
--- /dev/null
+++ b/experiment/simulation/script.js
@@ -0,0 +1,624 @@
+document.addEventListener('DOMContentLoaded', function() {
+ // Canvas setup
+ const canvas = document.getElementById('plot-canvas');
+ const ctx = canvas.getContext('2d');
+ let width = canvas.width;
+ let height = canvas.height;
+ // Make canvas responsive
+ function resizeCanvas() {
+ const visualizationDiv = document.querySelector('.visualization');
+ const maxWidth = visualizationDiv.clientWidth - 40; // Account for padding
+
+ if (maxWidth < 600) {
+ // Keep aspect ratio
+ const ratio = canvas.height / canvas.width;
+ canvas.width = maxWidth;
+ canvas.height = maxWidth * ratio;
+ width = canvas.width;
+ height = canvas.height;
+
+ // Redraw everything
+ clearCanvas();
+ if (points1.length > 0 || points2.length > 0) {
+ drawPoints();
+ }
+ } else if (window.innerWidth >= 900 && canvas.width < 600) {
+ // Reset to original size on larger screens
+ canvas.width = 600;
+ canvas.height = 600;
+ width = canvas.width;
+ height = canvas.height;
+
+ // Redraw everything
+ clearCanvas();
+ if (points1.length > 0 || points2.length > 0) {
+ drawPoints();
+ }
+ }
+ }
+
+ // Call resize on load and window resize
+ resizeCanvas();
+ window.addEventListener('resize', function() {
+ // Debounce the resize event to prevent too many redraws
+ clearTimeout(window.resizeTimer);
+ window.resizeTimer = setTimeout(function() {
+ resizeCanvas();
+ }, 250);
+ });
+
+ // Button elements
+ const generateBtn = document.getElementById('generate-btn');
+ const markAllBtn = document.getElementById('mark-all-btn');
+ const clearBtn = document.getElementById('clear-btn');
+
+ // Data storage
+ let points1 = [];
+ let points2 = [];
+ let plotBounds = { xMin: -10, xMax: 10, yMin: -10, yMax: 10 };
+
+ // Input validation function
+ function validateInputs() {
+ const inputs = [
+ 'mean1x', 'mean1y', 'cov1xx', 'cov1xy', 'cov1yy', 'prior1',
+ 'mean2x', 'mean2y', 'cov2xx', 'cov2xy', 'cov2yy', 'prior2'
+ ];
+
+ let isValid = true;
+ let errorMessage = "";
+
+ // Check if all inputs are valid numbers
+ for (const id of inputs) {
+ const value = document.getElementById(id).value;
+ if (isNaN(parseFloat(value))) {
+ isValid = false;
+ errorMessage = `${id} must be a valid number`;
+ break;
+ }
+ }
+
+ // Check if priors sum to 1
+ if (isValid) {
+ const prior1 = parseFloat(document.getElementById('prior1').value);
+ const prior2 = parseFloat(document.getElementById('prior2').value);
+
+ if (Math.abs(prior1 + prior2 - 1) > 0.001) {
+ isValid = false;
+ errorMessage = "Prior probabilities must sum to 1";
+ }
+ }
+
+ // Check if covariance matrices are positive definite
+ if (isValid) {
+ const cov1xx = parseFloat(document.getElementById('cov1xx').value);
+ const cov1xy = parseFloat(document.getElementById('cov1xy').value);
+ const cov1yy = parseFloat(document.getElementById('cov1yy').value);
+
+ const cov2xx = parseFloat(document.getElementById('cov2xx').value);
+ const cov2xy = parseFloat(document.getElementById('cov2xy').value);
+ const cov2yy = parseFloat(document.getElementById('cov2yy').value);
+
+ // Check if matrices are positive definite
+ if (cov1xx <= 0 || cov1yy <= 0 || cov1xx * cov1yy <= cov1xy * cov1xy) {
+ isValid = false;
+ errorMessage = "Covariance matrix for Class 1 is not positive definite";
+ }
+
+ if (cov2xx <= 0 || cov2yy <= 0 || cov2xx * cov2yy <= cov2xy * cov2xy) {
+ isValid = false;
+ errorMessage = "Covariance matrix for Class 2 is not positive definite";
+ }
+ }
+
+ if (!isValid) {
+ alert(errorMessage);
+ }
+
+ return isValid;
+ }
+
+ // Function to get parameters from inputs
+ function getParams() {
+ return {
+ mean1: [parseFloat(document.getElementById('mean1x').value), parseFloat(document.getElementById('mean1y').value)],
+ cov1: [
+ [parseFloat(document.getElementById('cov1xx').value), parseFloat(document.getElementById('cov1xy').value)],
+ [parseFloat(document.getElementById('cov1xy').value), parseFloat(document.getElementById('cov1yy').value)]
+ ],
+ prior1: parseFloat(document.getElementById('prior1').value),
+
+ mean2: [parseFloat(document.getElementById('mean2x').value), parseFloat(document.getElementById('mean2y').value)],
+ cov2: [
+ [parseFloat(document.getElementById('cov2xx').value), parseFloat(document.getElementById('cov2xy').value)],
+ [parseFloat(document.getElementById('cov2xy').value), parseFloat(document.getElementById('cov2yy').value)]
+ ],
+ prior2: parseFloat(document.getElementById('prior2').value)
+ };
+ }
+
+ // Function to generate multivariate normal samples
+ function generateMultivariateNormal(mean, cov, numSamples) {
+ const samples = [];
+
+ // Calculate Cholesky decomposition of covariance matrix
+ const L = choleskyDecomposition(cov);
+
+ for (let i = 0; i < numSamples; i++) {
+ // Generate two independent standard normal variables
+ const z1 = boxMullerTransform();
+ const z2 = boxMullerTransform();
+
+ // Apply transformation
+ const x = mean[0] + L[0][0] * z1 + L[0][1] * z2;
+ const y = mean[1] + L[1][0] * z1 + L[1][1] * z2;
+
+ samples.push([x, y]);
+ }
+
+ return samples;
+ }
+
+ // Box-Muller transform for generating standard normal random variables
+ function boxMullerTransform() {
+ const u1 = Math.random();
+ const u2 = Math.random();
+
+ const z = Math.sqrt(-2.0 * Math.log(u1)) * Math.cos(2.0 * Math.PI * u2);
+ return z;
+ }
+
+ // Cholesky decomposition of a 2x2 matrix
+ function choleskyDecomposition(cov) {
+ const L = [[0, 0], [0, 0]];
+
+ L[0][0] = Math.sqrt(cov[0][0]);
+ L[0][1] = 0;
+ L[1][0] = cov[0][1] / L[0][0];
+ L[1][1] = Math.sqrt(cov[1][1] - L[1][0] * L[1][0]);
+
+ return L;
+ }
+ // Function to map data coordinates to canvas coordinates
+ function mapToCanvas(x, y) {
+ const xRange = plotBounds.xMax - plotBounds.xMin;
+ const yRange = plotBounds.yMax - plotBounds.yMin;
+
+ const xCanvas = ((x - plotBounds.xMin) / xRange) * width;
+ const yCanvas = height - ((y - plotBounds.yMin) / yRange) * height;
+
+ return [xCanvas, yCanvas];
+ }
+
+ // Function to map canvas coordinates to data coordinates
+ function mapFromCanvas(xCanvas, yCanvas) {
+ const xRange = plotBounds.xMax - plotBounds.xMin;
+ const yRange = plotBounds.yMax - plotBounds.yMin;
+
+ const x = plotBounds.xMin + (xCanvas / width) * xRange;
+ const y = plotBounds.yMin + ((height - yCanvas) / height) * yRange;
+
+ return [x, y];
+ }
+ // Function to clear the canvas
+ function clearCanvas() {
+ ctx.clearRect(0, 0, width, height);
+ drawAxes();
+ }
+
+ // Function to draw the x and y axes
+ function drawAxes() {
+ ctx.strokeStyle = "#999";
+ ctx.lineWidth = 1;
+ ctx.beginPath();
+
+ // Draw x-axis
+ const [x0, yAxis] = mapToCanvas(0, 0);
+ const [xWidth, _] = mapToCanvas(plotBounds.xMax, 0);
+ ctx.moveTo(0, yAxis);
+ ctx.lineTo(width, yAxis);
+
+ // Draw y-axis
+ const [xAxis, y0] = mapToCanvas(0, 0);
+ const [__, yHeight] = mapToCanvas(0, plotBounds.yMax);
+ ctx.moveTo(xAxis, 0);
+ ctx.lineTo(xAxis, height);
+
+ ctx.stroke();
+
+ // Draw tick marks and labels
+ ctx.font = '10px Arial';
+ ctx.fillStyle = '#555';
+ ctx.textAlign = 'center';
+
+ // X-axis ticks
+ for (let x = Math.ceil(plotBounds.xMin); x <= Math.floor(plotBounds.xMax); x++) {
+ if (x === 0) continue; // Skip the origin
+ const [xPos, yPos] = mapToCanvas(x, 0);
+ ctx.beginPath();
+ ctx.moveTo(xPos, yAxis - 3);
+ ctx.lineTo(xPos, yAxis + 3);
+ ctx.stroke();
+ ctx.fillText(x.toString(), xPos, yAxis + 15);
+ }
+
+ // Y-axis ticks
+ ctx.textAlign = 'right';
+ for (let y = Math.ceil(plotBounds.yMin); y <= Math.floor(plotBounds.yMax); y++) {
+ if (y === 0) continue; // Skip the origin
+ const [xPos, yPos] = mapToCanvas(0, y);
+ ctx.beginPath();
+ ctx.moveTo(xAxis - 3, yPos);
+ ctx.lineTo(xAxis + 3, yPos);
+ ctx.stroke();
+ ctx.fillText(y.toString(), xAxis - 5, yPos + 4);
+ }
+
+ // Origin label
+ ctx.textAlign = 'right';
+ ctx.fillText('0', xAxis - 5, yAxis + 15);
+ }
+
+ // Function to draw data points
+ function drawPoints() {
+ // Draw class 1 points
+ ctx.fillStyle = 'rgba(30, 144, 255, 0.7)';
+ for (const point of points1) {
+ const [x, y] = mapToCanvas(point[0], point[1]);
+ ctx.beginPath();
+ ctx.arc(x, y, 4, 0, Math.PI * 2);
+ ctx.fill();
+ }
+
+ // Draw class 2 points
+ ctx.fillStyle = 'rgba(220, 20, 60, 0.7)';
+ for (const point of points2) {
+ const [x, y] = mapToCanvas(point[0], point[1]);
+ ctx.beginPath();
+ ctx.arc(x, y, 4, 0, Math.PI * 2);
+ ctx.fill();
+ }
+ }
+
+ // Function to calculate multivariate normal density
+ function mvnPDF(x, mean, cov) {
+ const dim = 2;
+ const dx = [x[0] - mean[0], x[1] - mean[1]];
+
+ // Calculate inverse of covariance matrix
+ const det = cov[0][0] * cov[1][1] - cov[0][1] * cov[1][0];
+ const invCov = [
+ [cov[1][1] / det, -cov[0][1] / det],
+ [-cov[1][0] / det, cov[0][0] / det]
+ ];
+
+ // Calculate exponent term: -0.5 * (x-μ)^T * Σ^(-1) * (x-μ)
+ let exponent = 0;
+ for (let i = 0; i < dim; i++) {
+ for (let j = 0; j < dim; j++) {
+ exponent += dx[i] * invCov[i][j] * dx[j];
+ }
+ }
+ exponent *= -0.5;
+
+ // Calculate coefficient: 1 / (sqrt((2π)^n * |Σ|))
+ const coefficient = 1 / (Math.sqrt(Math.pow(2 * Math.PI, dim) * det));
+
+ return coefficient * Math.exp(exponent);
+ }
+
+ // Function to visualize density functions
+ function visualizeDensities() {
+ if (!validateInputs()) return;
+
+ const params = getParams();
+ clearCanvas();
+ drawPoints();
+
+ // Create density heatmap
+ const resolution = 50;
+ const xStep = (plotBounds.xMax - plotBounds.xMin) / resolution;
+ const yStep = (plotBounds.yMax - plotBounds.yMin) / resolution;
+
+ // Calculate max density for normalization
+ let maxDensity1 = 0;
+ let maxDensity2 = 0;
+
+ // Pre-compute densities
+ const densities1 = [];
+ const densities2 = [];
+
+ for (let i = 0; i < resolution; i++) {
+ densities1.push([]);
+ densities2.push([]);
+
+ const x = plotBounds.xMin + i * xStep;
+
+ for (let j = 0; j < resolution; j++) {
+ const y = plotBounds.yMin + j * yStep;
+
+ const density1 = mvnPDF([x, y], params.mean1, params.cov1);
+ const density2 = mvnPDF([x, y], params.mean2, params.cov2);
+
+ densities1[i].push(density1);
+ densities2[i].push(density2);
+
+ maxDensity1 = Math.max(maxDensity1, density1);
+ maxDensity2 = Math.max(maxDensity2, density2);
+ }
+ }
+
+ // Draw density contours
+ const numContours = 5;
+ const contourLevels = [];
+
+ for (let i = 1; i <= numContours; i++) {
+ contourLevels.push(i / (numContours + 1));
+ }
+
+ // Draw Class 1 contours
+ ctx.strokeStyle = 'rgba(30, 144, 255, 0.8)';
+ ctx.lineWidth = 2;
+
+ for (const level of contourLevels) {
+ const threshold = maxDensity1 * level;
+ drawContour(densities1, threshold, 'blue');
+ }
+
+ // Draw Class 2 contours
+ ctx.strokeStyle = 'rgba(220, 20, 60, 0.8)';
+
+ for (const level of contourLevels) {
+ const threshold = maxDensity2 * level;
+ drawContour(densities2, threshold, 'red');
+ }
+
+ return { densities1, densities2 };
+ }
+
+ // Function to draw a contour at a specific threshold level
+ function drawContour(densities, threshold, color) {
+ const resolution = densities.length;
+ const xStep = (plotBounds.xMax - plotBounds.xMin) / resolution;
+ const yStep = (plotBounds.yMax - plotBounds.yMin) / resolution;
+
+ ctx.strokeStyle = color === 'blue' ? 'rgba(30, 144, 255, 0.8)' : 'rgba(220, 20, 60, 0.8)';
+ ctx.lineWidth = 2;
+ ctx.beginPath();
+
+ let started = false;
+
+ for (let i = 0; i < resolution - 1; i++) {
+ const x = plotBounds.xMin + i * xStep;
+
+ for (let j = 0; j < resolution - 1; j++) {
+ const y = plotBounds.yMin + j * yStep;
+
+ const d00 = densities[i][j] - threshold;
+ const d01 = densities[i][j + 1] - threshold;
+ const d10 = densities[i + 1][j] - threshold;
+ const d11 = densities[i + 1][j + 1] - threshold;
+
+ // Check if contour passes through this cell
+ if ((d00 * d01 <= 0) || (d00 * d10 <= 0) || (d01 * d11 <= 0) || (d10 * d11 <= 0)) {
+ const centerX = x + xStep / 2;
+ const centerY = y + yStep / 2;
+ const [canvasX, canvasY] = mapToCanvas(centerX, centerY);
+
+ if (!started) {
+ ctx.moveTo(canvasX, canvasY);
+ started = true;
+ } else {
+ ctx.lineTo(canvasX, canvasY);
+ }
+ }
+ }
+ }
+
+ ctx.stroke();
+ }
+
+ // Function to visualize decision boundary
+ function visualizeDecisionBoundary() {
+ if (!validateInputs()) return;
+
+ const params = getParams();
+
+ // Clear the canvas and redraw axes
+ clearCanvas();
+
+ // Higher resolution for smooth decision boundary
+ const resolution = 300; // Increased resolution for sharper boundaries
+ const xStep = (plotBounds.xMax - plotBounds.xMin) / resolution;
+ const yStep = (plotBounds.yMax - plotBounds.yMin) / resolution;
+
+ // Create two separate canvas layers
+ // 1. Background layer for class regions
+ const bgCanvas = document.createElement('canvas');
+ bgCanvas.width = width;
+ bgCanvas.height = height;
+ const bgCtx = bgCanvas.getContext('2d');
+
+ // 2. Boundary layer for decision boundary
+ const boundaryCanvas = document.createElement('canvas');
+ boundaryCanvas.width = width;
+ boundaryCanvas.height = height;
+ const boundaryCtx = boundaryCanvas.getContext('2d');
+
+ // Create ImageData for both layers
+ const bgImageData = bgCtx.createImageData(width, height);
+ const boundaryImageData = boundaryCtx.createImageData(width, height);
+
+ // For each pixel in the canvas
+ for (let canvasY = 0; canvasY < height; canvasY++) {
+ for (let canvasX = 0; canvasX < width; canvasX++) {
+ // Convert canvas coordinates to data coordinates
+ const [x, y] = mapFromCanvas(canvasX, canvasY);
+
+ // Calculate posterior probabilities using Bayes' rule
+ const likelihood1 = mvnPDF([x, y], params.mean1, params.cov1);
+ const likelihood2 = mvnPDF([x, y], params.mean2, params.cov2);
+
+ const posterior1 = likelihood1 * params.prior1;
+ const posterior2 = likelihood2 * params.prior2;
+
+ // Calculate index in image data array
+ const index = (canvasY * width + canvasX) * 4;
+
+ // Set background color based on which class is more likely
+ if (posterior1 > posterior2) {
+ // Class 1 region - light blue
+ bgImageData.data[index] = 200; // R
+ bgImageData.data[index + 1] = 230; // G
+ bgImageData.data[index + 2] = 255; // B
+ bgImageData.data[index + 3] = 100; // Alpha
+ } else {
+ // Class 2 region - light red
+ bgImageData.data[index] = 255; // R
+ bgImageData.data[index + 1] = 200; // G
+ bgImageData.data[index + 2] = 200; // B
+ bgImageData.data[index + 3] = 100; // Alpha
+ }
+
+ // Check if point is on decision boundary (approximately equal posteriors)
+ // Using a more precise threshold for boundary detection
+ if (Math.abs(posterior1 - posterior2) < 0.005 * Math.max(posterior1, posterior2)) {
+ boundaryImageData.data[index] = 0; // R
+ boundaryImageData.data[index + 1] = 0; // G
+ boundaryImageData.data[index + 2] = 0; // B
+ boundaryImageData.data[index + 3] = 255; // Alpha - fully opaque
+ }
+ }
+ }
+
+ // Apply the background classes
+ bgCtx.putImageData(bgImageData, 0, 0);
+ ctx.drawImage(bgCanvas, 0, 0);
+
+ // Draw the axes
+ drawAxes();
+
+ // Apply the boundary as a separate layer
+ boundaryCtx.putImageData(boundaryImageData, 0, 0);
+
+ // Apply a blur to make the boundary more visible
+ boundaryCtx.globalCompositeOperation = 'source-over';
+ boundaryCtx.filter = 'blur(0.5px)';
+ boundaryCtx.drawImage(boundaryCanvas, 0, 0);
+
+ // Reset the filter and draw the boundary with increased thickness
+ boundaryCtx.filter = 'none';
+ boundaryCtx.globalCompositeOperation = 'source-over';
+ ctx.drawImage(boundaryCanvas, 0, 0);
+
+ // Draw data points on top
+ drawPoints();
+
+ // Add legend with increased visibility
+ ctx.font = 'bold 14px Arial';
+ ctx.textAlign = 'left';
+
+ // Add a semi-transparent background for the legend
+ ctx.fillStyle = 'rgba(255, 255, 255, 0.7)';
+ ctx.fillRect(10, 10, 130, 80);
+
+ // Class 1 legend
+ ctx.fillStyle = 'rgba(30, 144, 255, 0.8)';
+ ctx.beginPath();
+ ctx.arc(25, 25, 6, 0, Math.PI * 2);
+ ctx.fill();
+ ctx.fillStyle = 'black';
+ ctx.fillText('Class 1', 40, 29);
+
+ // Class 2 legend
+ ctx.fillStyle = 'rgba(220, 20, 60, 0.8)';
+ ctx.beginPath();
+ ctx.arc(25, 50, 6, 0, Math.PI * 2);
+ ctx.fill();
+ ctx.fillStyle = 'black';
+ ctx.fillText('Class 2', 40, 54);
+
+ // Decision boundary legend
+ ctx.fillStyle = 'black';
+ ctx.fillRect(22, 70, 12, 3);
+ ctx.fillText('Decision Boundary', 40, 74);
+ }
+
+ // Event listeners for buttons
+ generateBtn.addEventListener('click', function() {
+ if (!validateInputs()) return;
+
+ const params = getParams();
+ const numSamples = 100;
+
+ // Generate random samples for each class
+ points1 = generateMultivariateNormal(params.mean1, params.cov1, numSamples);
+ points2 = generateMultivariateNormal(params.mean2, params.cov2, numSamples);
+
+ // Determine plot bounds based on data
+ let xMin = Infinity, xMax = -Infinity, yMin = Infinity, yMax = -Infinity;
+
+ for (const point of [...points1, ...points2]) {
+ xMin = Math.min(xMin, point[0]);
+ xMax = Math.max(xMax, point[0]);
+ yMin = Math.min(yMin, point[1]);
+ yMax = Math.max(yMax, point[1]);
+ }
+
+ // Add some padding to bounds
+ const xPadding = (xMax - xMin) * 0.2;
+ const yPadding = (yMax - yMin) * 0.2;
+
+ plotBounds = {
+ xMin: Math.max(-10, xMin - xPadding),
+ xMax: Math.min(10, xMax + xPadding),
+ yMin: Math.max(-10, yMin - yPadding),
+ yMax: Math.min(10, yMax + yPadding)
+ };
+
+ clearCanvas();
+ drawPoints();
+ });
+
+ markAllBtn.addEventListener('click', function() {
+ visualizeDecisionBoundary();
+ });
+
+ clearBtn.addEventListener('click', function() {
+ points1 = [];
+ points2 = [];
+ plotBounds = { xMin: -10, xMax: 10, yMin: -10, yMax: 10 };
+ clearCanvas();
+ });
+ // Initialize the canvas
+ clearCanvas();
+
+ // Add touch support for mobile devices
+ canvas.addEventListener('touchstart', function(e) {
+ e.preventDefault(); // Prevent scrolling when touching the canvas
+ const touch = e.touches[0];
+ const rect = canvas.getBoundingClientRect();
+ const x = touch.clientX - rect.left;
+ const y = touch.clientY - rect.top;
+ // Process the touch as you would a click
+ // Add code here if needed to handle the touch event
+ });
+
+ // Load example configurations if available
+ if (typeof boundaryExamples !== 'undefined') {
+ // Create example buttons
+ const configContainer = document.createElement('div');
+ configContainer.className = 'example-buttons';
+ configContainer.innerHTML = '
Load Examples:
';
+
+ for (const key in boundaryExamples) {
+ const button = document.createElement('button');
+ button.textContent = boundaryExamples[key].name;
+ button.onclick = function() { loadExample(key); };
+ configContainer.appendChild(button);
+ }
+
+ // Add to the control panel
+ const controlPanel = document.querySelector('.experiment-controls');
+ controlPanel.appendChild(configContainer);
+ }
+});
diff --git a/experiment/simulation/style.css b/experiment/simulation/style.css
new file mode 100644
index 0000000..81bae0b
--- /dev/null
+++ b/experiment/simulation/style.css
@@ -0,0 +1,355 @@
+* {
+ margin: 0;
+ padding: 0;
+ box-sizing: border-box;
+ font-family: 'Arial', sans-serif;
+}
+
+body {
+ background-color: #f5f7fa;
+ color: #333;
+ line-height: 1.6;
+ font-size: 16px;
+ -webkit-font-smoothing: antialiased;
+ -moz-osx-font-smoothing: grayscale;
+ overflow-x: hidden;
+}
+
+.container {
+ max-width: 1200px;
+ margin: 0 auto;
+ padding: 20px;
+ width: 100%;
+}
+
+header {
+ text-align: center;
+ margin-bottom: 30px;
+ padding: 20px;
+ background-color: #3498db;
+ color: white;
+ border-radius: 8px;
+}
+
+h1, h2, h3 {
+ font-weight: 600;
+}
+
+h1 {
+ font-size: 28px;
+}
+
+h2 {
+ font-size: 22px;
+ margin-bottom: 15px;
+ color: #2c3e50;
+}
+
+h3 {
+ font-size: 18px;
+ margin-bottom: 10px;
+ color: #34495e;
+}
+
+.main-content {
+ display: flex;
+ flex-wrap: wrap;
+ gap: 20px;
+ margin-bottom: 30px;
+ width: 100%;
+}
+
+.control-panel {
+ flex: 1;
+ min-width: 300px;
+ background-color: white;
+ padding: 20px;
+ border-radius: 8px;
+ box-shadow: 0 2px 10px rgba(0, 0, 0, 0.1);
+}
+
+.experiment-controls {
+ display: flex;
+ flex-direction: column;
+ gap: 15px;
+}
+
+.class-params {
+ padding: 15px;
+ background-color: #f0f4f8;
+ border-radius: 8px;
+ margin-bottom: 15px;
+}
+
+.param-group {
+ display: flex;
+ align-items: center;
+ margin-bottom: 10px;
+}
+
+.param-group label {
+ width: 150px;
+ font-weight: 500;
+}
+
+input[type="text"] {
+ width: 100%;
+ padding: 8px;
+ border: 1px solid #ddd;
+ border-radius: 4px;
+ font-size: 14px;
+ -webkit-appearance: none;
+ -moz-appearance: none;
+ appearance: none;
+}
+
+.buttons {
+ display: flex;
+ flex-wrap: wrap;
+ gap: 10px;
+ margin-top: 15px;
+}
+
+button {
+ padding: 10px 15px;
+ background-color: #3498db;
+ color: white;
+ border: none;
+ border-radius: 4px;
+ cursor: pointer;
+ transition: background-color 0.3s;
+ font-size: 16px;
+ min-width: 100px;
+ -webkit-tap-highlight-color: transparent;
+ touch-action: manipulation;
+}
+
+button:hover {
+ background-color: #2980b9;
+}
+
+button:active {
+ transform: translateY(1px);
+}
+
+#clear-btn {
+ background-color: #e74c3c;
+}
+
+#clear-btn:hover {
+ background-color: #c0392b;
+}
+
+.visualization {
+ flex: 2;
+ min-width: 400px;
+ background-color: white;
+ padding: 20px;
+ border-radius: 8px;
+ box-shadow: 0 2px 10px rgba(0, 0, 0, 0.1);
+ display: flex;
+ justify-content: center;
+ align-items: center;
+ overflow: hidden;
+}
+
+#plot-canvas {
+ background-color: #fafafa;
+ border: 1px solid #eee;
+ max-width: 100%;
+ height: auto;
+ display: block;
+}
+
+.instructions {
+ background-color: white;
+ padding: 20px;
+ border-radius: 8px;
+ box-shadow: 0 2px 10px rgba(0, 0, 0, 0.1);
+ margin-top: 20px;
+}
+
+.configurations {
+ display: grid;
+ grid-template-columns: repeat(auto-fill, minmax(300px, 1fr));
+ gap: 20px;
+ margin-top: 15px;
+}
+
+.config {
+ background-color: #f0f4f8;
+ border-radius: 8px;
+ padding: 15px;
+}
+
+.config h3 {
+ color: #3498db;
+ margin-bottom: 10px;
+ border-bottom: 1px solid #ddd;
+ padding-bottom: 5px;
+}
+
+.config pre {
+ font-family: 'Courier New', monospace;
+ background-color: #fff;
+ padding: 10px;
+ border-radius: 4px;
+ overflow-x: auto;
+ font-size: 13px;
+ line-height: 1.4;
+}
+
+.stages {
+ display: flex;
+ flex-direction: column;
+ gap: 20px;
+}
+
+.stage {
+ padding: 15px;
+ background-color: #f0f4f8;
+ border-radius: 8px;
+}
+
+ul {
+ padding-left: 20px;
+ margin-bottom: 10px;
+}
+
+p {
+ margin-bottom: 10px;
+}
+
+/* Responsive styles */
+@media (max-width: 1100px) {
+ .container {
+ padding: 15px;
+ }
+
+ h1 {
+ font-size: 24px;
+ }
+
+ .control-panel {
+ min-width: 280px;
+ }
+}
+
+@media (max-width: 900px) {
+ .main-content {
+ flex-direction: column;
+ }
+
+ .visualization {
+ min-height: 400px;
+ max-width: 100%;
+ margin: 0 auto;
+ }
+
+ #plot-canvas {
+ max-width: 100%;
+ width: 100%;
+ height: auto;
+ }
+
+ .class-params {
+ padding: 12px;
+ }
+}
+
+@media (max-width: 768px) {
+ header {
+ padding: 15px;
+ margin-bottom: 20px;
+ }
+
+ .container {
+ padding: 12px;
+ }
+
+ .control-panel, .visualization {
+ padding: 15px;
+ }
+
+ .param-group {
+ margin-bottom: 8px;
+ }
+}
+
+@media (max-width: 600px) {
+ .container {
+ padding: 10px;
+ }
+
+ header {
+ padding: 12px;
+ margin-bottom: 15px;
+ }
+
+ h1 {
+ font-size: 20px;
+ }
+
+ h2 {
+ font-size: 18px;
+ margin-bottom: 12px;
+ }
+
+ h3 {
+ font-size: 16px;
+ margin-bottom: 8px;
+ }
+
+ .control-panel, .visualization {
+ padding: 12px;
+ }
+
+ .param-group {
+ flex-direction: column;
+ align-items: flex-start;
+ }
+
+ .param-group label {
+ width: 100%;
+ margin-bottom: 5px;
+ }
+
+ .buttons {
+ justify-content: space-between;
+ }
+
+ button {
+ padding: 8px 12px;
+ font-size: 14px;
+ min-width: auto;
+ flex: 1;
+ }
+}
+
+@media (max-width: 480px) {
+ h1 {
+ font-size: 18px;
+ }
+
+ .container {
+ padding: 8px;
+ }
+
+ .control-panel, .visualization {
+ padding: 10px;
+ }
+
+ .class-params {
+ padding: 10px;
+ margin-bottom: 12px;
+ }
+
+ .param-group {
+ margin-bottom: 6px;
+ }
+
+ input[type="text"] {
+ padding: 6px;
+ font-size: 14px;
+ }
+}
diff --git a/experiment/theory.md b/experiment/theory.md
index 42394b9..7616b9e 100644
--- a/experiment/theory.md
+++ b/experiment/theory.md
@@ -1,15 +1,43 @@
-Consider the following quote from a 2000 article in the Economist on the Bayesian Approach:
+### Introduction to Bayesian Approach
+The Bayesian approach provides a mathematical framework for updating beliefs based on new evidence. This concept is beautifully illustrated in the following example from a 2000 Economist article:
-*"The essence of the Bayesian approach is to provide a mathematical rule explaining how you should change your existing beliefs in the light of new evidence. In other words, it allows us to combine new data with their existing knowledge or expertise. The canonical example is to imagine that a precocious newborn observes his first sunset, and wonders whether the sun will rise again or not. He assigns equal prior probabilities to both possible outcomes, and represents this by placing one white and one black marble into a bag. The following day, when the sun rises, the child places another white marble in the bag. The probability that a marble plucked randomly from the bag will be white (i.e., the child's degree of belief in future sunrises) has thus gone from a half to two-thirds. After sunrise the next day, the child adds another white marble, and the probability (and thus the degree of belief) goes from two-thirds to three-quarters. And so on. Gradually, the initial belief that the sun is just as likely as not to rise each morning is modified to become a near-certainty that the sun will always rise."*
+> "The essence of the Bayesian approach is to provide a mathematical rule explaining how you should change your existing beliefs in the light of new evidence. In other words, it allows us to combine new data with their existing knowledge or expertise. The canonical example is to imagine that a precocious newborn observes his first sunset, and wonders whether the sun will rise again or not. He assigns equal prior probabilities to both possible outcomes, and represents this by placing one white and one black marble into a bag. The following day, when the sun rises, the child places another white marble in the bag. The probability that a marble plucked randomly from the bag will be white (i.e., the child's degree of belief in future sunrises) has thus gone from a half to two-thirds. After sunrise the next day, the child adds another white marble, and the probability (and thus the degree of belief) goes from two-thirds to three-quarters. And so on. Gradually, the initial belief that the sun is just as likely as not to rise each morning is modified to become a near-certainty that the sun will always rise."
-In terms of classification, the Bayes theorem allows us to combine prior probabilities, along with observed evidence to arrive at the posterior probability. More or less, conditional probabilities represent the probability of an event occurring given evidence. To better understand, Bayes Theorem can be derived from the joint probability of A and B (i.e. P(A,B)) as follows:
+### Bayes' Theorem in Classification
+In classification, Bayes' theorem allows us to:
+- Combine prior probabilities with observed evidence
+- Calculate posterior probabilities
+- Make decisions based on updated beliefs
-
+#### Mathematical Formulation
+Bayes' Theorem can be derived from the joint probability of events A and B:
+
-where *P(A|B)* is referred to as the posterior; *P(B|A)* is known as the likelihood, *P(A)* is the prior and *P(B)* is generally the evidence and is used as a scaling factor. Therefore, it is handy to remember Bayes Rule as:
+where:
+- $P(A|B)$ is the posterior probability
+- $P(B|A)$ is the likelihood
+- $P(A)$ is the prior probability
+- $P(B)$ is the evidence (scaling factor)
-
+#### Simplified Form
+For practical applications, we often use this form:
+
-These terms will be discussed a little later.
+### Key Components
+1. **Prior Probability** ($P(A)$)
+ - Initial belief before seeing new evidence
+ - Based on previous knowledge or experience
+
+2. **Likelihood** ($P(B|A)$)
+ - Probability of observing the evidence given the hypothesis
+ - Measures how well the evidence supports the hypothesis
+
+3. **Posterior Probability** ($P(A|B)$)
+ - Updated belief after considering new evidence
+ - Used for making final decisions
+
+4. **Evidence** ($P(B)$)
+ - Total probability of observing the evidence
+ - Acts as a normalizing constant