Background Flashcards

10 cards. The model everything else is built on.

Every card shows its question first. Reveal one at a time to test yourself, or reveal all of them to use the deck as a revision sheet.

Search this deck

AllConceptDistinctionEquationIntuition

Reveal allHide all

10 cards

• Concept

  What does a policy represent in reinforcement learning?

  Concept

  A policy defines how an agent chooses actions from the information available to it.

  $$a_t\sim \pi (a_t\mid o_t)$$

  The policy $\pi$ maps observation $o_t$ to a distribution over possible actions.

  Show answer

• Equation

  What does this mean?

  $$a_t\sim \pi (a_t\mid o_t)$$

  Equation

  At time $t$, the agent chooses action $a_t$ according to policy $\pi$, conditioned on its current observation $o_t$.

  $\pi$

  the decision rule the agent follows

  $o_t$

  everything the agent can see at this step

  $\sim$

  sampled from, so the policy may be stochastic

  Show answer

• Concept

  What changes when reinforcement learning moves from one agent to multiple agents?

  Concept

  An agent's outcome can now depend on what other agents do. Other agents may also be learning, changing the environment each agent experiences.

  Show answer

• Concept

  What is a joint action, and why is it important in MARL?

  Concept

  A joint action is the combination of all agents' actions at a particular time.

  $${\mathbf{a}}_t=(a_t^1,\dots ,a_t^n)$$

  The environment responds to this combination, so the outcome may depend on how the agents' actions fit together.

  Show answer

• Equation

  What does this represent?

  $${\mathbf{a}}_t=(a_t^1,\dots ,a_t^n)$$

  Equation

  The joint action at time $t$, containing the actions selected by all $n$ agents.

  $a_t^i$

  agent $i$'s own action, chosen from its own observation

  ${\mathbf{a}}_t$

  the tuple the environment actually responds to

  Show answer

• Distinction

  What is the difference between the environment state and an agent's observation?

  Distinction

  The state represents the underlying condition of the environment. An observation contains only the information available to a particular agent.

  Show answer

• Equation

  What does this tell us about agent $i$'s information?

  $$o_t^i=O_i(s_t)$$

  Equation

  Agent $i$'s observation $o_t^i$ is generated from the underlying state $s_t$ through its observation function $O_i$.

  The agent may therefore see only part of the complete state.

  Show answer

• Concept

  What does partial observability mean in a multi-agent environment?

  Concept

  An agent does not directly observe all information about the environment that may be relevant to its decision.

  Show answer

• Intuition

  Why can agents in the same environment make decisions from different information?

  Intuition

  Each agent may have its own observation function, location, sensors, or access to information. They share the environment without necessarily sharing the same view of it.

  Show answer

• Distinction

  What does giving all agents the same reward accomplish, and what does it not guarantee?

  Distinction

  A shared reward aligns the agents toward the same objective.

  It does not guarantee that their individual actions will be coordinated, or reveal how each agent contributed to the outcome.

  Show answer

Back to all decks · Read the Background material