updates

content/CSE510/CSE510_L17.md

@@ -55,6 +55,27 @@ Close-loop planning:
 
 - At each state, iteratively build a search tree to evaluate actions, select the best-first action, and the move the next state.
 
+Use model as simulator to evaluate actions.
+
+#### MCTS Algorithm Overview
+
+1. Selection: Select the best-first action from the search tree
+2. Expansion: Add a new node to the search tree
+3. Simulation: Simulate the next state from the selected action
+4. Backpropagation: Update the values of the nodes in the search tree
+
+#### Policies in MCTS
+
+Tree policy:
+
+Decision policy:
+
+- Max (highest weight)
+- Robust (most visits)
+- Max-Robust (max of the two)
+
+#### Upper Confidence Bound on Trees (UCT)
+
 
 
 #### Continuous Case: Trajectory Optimization