Skip to content
MARL in Cooperative Environments
Edit this page

2.5Communication Constraints

5 min read

Communication channels constrain which information agents can exchange and how reliably it arrives.

In this section you will

  • Reason about a finite message capacity
  • Reason about a limited communication range
  • Reason about messages that are dropped
  • Reason about messages that arrive corrupted

A discrete message space has a size, and that size is a hard limit on how many distinct things can ever be said.

Figure 1
∣M∣=Kor, for a b-bit message,∣M∣=2b\bigl|\tone{comm}{\mathcal{M}}\bigr| = K \qquad\text{or, for a } b\text{-bit message,}\qquad \bigl|\tone{comm}{\mathcal{M}}\bigr| = 2^{b}
KK
how many distinct messages exist
bb
bits available per message; each bit doubles the space
Capacity counts distinctions, not detail. K messages can express exactly K situations.

Drag the slider to see what a bit buys.

Distinct messages against Bits per message. 1: 2, 2: 4, 3: 8, 4: 16, 5: 32, 6: 64, 7: 128, 8: 256 06412819225612345678Bits per messageDistinct messages

Capacity is best thought about backwards, starting from the distinctions the team actually needs. With one bit, agent 1 has exactly two things it can say:

MessageMeaning
0no help needed
1help needed

That is a real protocol and it is often enough. But notice what it cannot do: it cannot say what kind of help. Two bits can.

MessageMeaning
00no help needed
01bring an ingredient
10start cooking
11ready to serve

A message may reach only agents that are close enough.

In the Dec-POMDP this is part of the observation model rather than a separate mechanism: whether agent jj observes mtim^\ag_t depends on the state, through the distance between them.

Figure 2
agent j receives mti  ⟺  dist(i, j)≤d⏟the range\text{agent } j \text{ receives } \tone{comm}{m^\ag_t} \iff \mathrm{dist}\bigl(\ag,\ j\bigr) \leq \ubrace{observe}{d}{the range}
mtim^i_t
the message sent by agent i at step t
dist(i,j)\mathrm{dist}(i,j)
the current distance between sender and receiver
dd
the maximum communication range
Message receipt depends on the state-dependent distance between agents.

Two consequences worth noticing. Agents cannot rely on being heard, so a protocol built on “everyone knows what I said” fails as soon as someone walks away. And range makes communication state-dependent: the same message sent from a different position reaches a different audience.

A sent message may simply not arrive.

Agent 1 receives its own observation and produces two things: an environment action that goes down into the kitchen, and a message that goes sideways to agent 2. Agent 2 combines its own observation with the received message to choose its own environment action, which also goes into the kitchen. The message channel is marked as unreliable, so the message may not arrive. o1o2Agent 1Agent 2m1may not arrivex1x2The kitchennext state, and one team reward

The same picture with an unreliable channel. Agent 1 sends; agent 2 may receive nothing at all.

The standard way to model this is with a probability: with probability plossp_{\text{loss}}, the message the receiver observes is replaced by the empty message.

Figure 3
received={mtiwith probability 1−ploss∅with probability ploss\text{received} = \begin{cases} \tone{comm}{m^\ag_t} & \text{with probability } 1 - p_{\text{loss}} \\[2pt] \tone{conflict}{\varnothing} & \text{with probability } p_{\text{loss}} \end{cases}
∅\varnothing
the empty message: the receiver gets nothing, exactly as if none had been sent
Loss turns a message into a silence, and silence is a message the sender did not choose.

Loss is worse than it first appears, because ∅\varnothing is a legitimate message that means “say nothing”. So a receiver cannot distinguish

  • the sender chose to stay silent, from
  • the sender spoke and the channel ate it.

A protocol in which silence carries meaning is therefore fragile under loss. If no message means “carry on as planned”, then a dropped “stop” reads as approval.

Weaker than loss and sometimes more damaging: the message arrives, changed.

For a continuous message the usual model adds a random perturbation.

Figure 4
received=mti+ϵ,ϵ∼N(0,σ2)\text{received} = \tone{comm}{m^\ag_t} + \tone{conflict}{\epsilon}, \qquad \epsilon \sim \mathcal{N}(0, \sigma^{2})
ϵ\epsilon
noise added by the channel
σ2\sigma^{2}
how much of it
The message arrives, slightly wrong. Nothing tells the receiver by how much.

Noise punishes a particular kind of protocol: one where nearby messages mean very different things. If a learned vector uses tiny differences to distinguish “bring a tomato” from “bring an onion”, noise destroys it. A protocol whose meanings are far apart in message space survives.

Knowledge check

A team's protocol uses silence to mean 'continue as planned'. What goes wrong when 10% of messages are lost?

Select one answer.

  • Capacity ∣M∣=K|\mathcal{M}| = K, or 2b2^b for bb bits, counts distinctions. A narrow channel forces the team to merge situations, not to describe them more briefly.
  • Range makes communication state-dependent: who hears you depends on where everyone is.
  • Loss replaces a message with ∅\varnothing, which is indistinguishable from a chosen silence. Protocols in which silence means something are fragile.
  • Noise perturbs the message, punishing protocols whose meanings sit close together in message space.
  • The principle: a protocol must remain useful under its own channel’s constraints, not just on a perfect one.