updates

content/CSE510/CSE510_L8.md

@@ -0,0 +1,78 @@
+# CSE510 Deep Reinforcement Learning (Lecture 8)
+
+## Convolutional Neural Networks
+
+Another note in computer vision can be found here: [CSE559A Lecture 10](../CSE559A/CSE559A_L10#convolutional-layer)
+
+Basically, it is a stack of different layers:
+
+- Convolutional layer
+- Non-linearity layer
+- Pooling layer (or downsampling layer)
+- Fully connected layer
+
+### Convolutional layer
+
+Filtering: The math behind the matching. 
+
+1. Line up the feature and the image patch. 
+2. Multiply each image pixel by the corresponding feature pixel. 
+3. Add them up. 
+4. Divide by the total number of pixels in the feature.
+
+Idea of a convolutional neural network, in some sense, is to let the network "learn" the right filters for a specific task.
+
+### Non-linearity Layer
+
+> [!TIP]
+>
+> This is irrelevant to the lecture, but consider the following term:
+>
+> "Bounded rationality"
+
+- Convolution is a linear operation
+- Non-linearity layer creates an activation map from the feature map generated by the convolutional layer
+- Consisting an activation function (an element-wise operation)
+- Rectified linear units (ReLUs) is advantageous over the traditional sigmoid or tanh activation functions
+
+### Pooling layer
+
+Shrinking the Image Stack
+
+- Motivation: the activation maps can be large
+- Reducing the spacial size of the activation maps
+  - Often after multiple stages of other layers (i.e., convolutional and non-linear layers)
+- Steps:
+  1. Pick a window size (usually 2 or 3).
+  2. Pick a stride (usually 2).
+  3. Walk your window across your filtered images.
+  4. From each window, take the maximum value.
+
+Pros:
+
+- Reducing the computational requirements
+- Minimizing the likelihood of overfitting
+
+Cons:
+
+- Aggressive reduction can limit the depth of a network and ultimately limit the performance
+
+### Fully connected layer
+
+- Multilayer perceptron (MLP)
+- Mapping the activation volume from previous layers into a class probability distribution
+- Non-linearity is built in the neurons, instead of a separate layer
+- Viewed as 1x1 convolution kernels
+
+For classification: Output layer is a regular, fully connected layer with softmax non-linearity
+
+- Output provides an estimate of the conditional probability of each class
+
+> [!TIP]
+>
+> The golden triangle of machine learning:
+>
+> - Data
+> - Algorithm
+> - Computation
+

content/CSE510/_meta.js

@@ -10,4 +10,5 @@ export default {
     CSE510_L5: "CSE510 Deep Reinforcement Learning (Lecture 5)",
     CSE510_L6: "CSE510 Deep Reinforcement Learning (Lecture 6)",
     CSE510_L7: "CSE510 Deep Reinforcement Learning (Lecture 7)",
+    CSE510_L8: "CSE510 Deep Reinforcement Learning (Lecture 8)",
 }