Agents and Expected Rewards

March 18, 2023 Cognitive Insights No Comments

This is a story from expectation to motivation. Any motivation is eventually the expectation of ‘reward,’ be it a concrete one or more abstract; be it deeply value-based or more superficial.

The simple case looks like conditioning.

You may read first about what agents are. An agent acts upon a reward by learning from it and changing its behavior towards heightening future expected rewards.

Within a non-complex agent, this is theoretically like conditioning: a (seemingly) mechanical process that can be described from the outside without considering anything inside ― no complex reasoning or other mental processing.

Who says, ‘conditioning theories,’ says Pavlov and automatically thinks of his dogs.

But these dogs were also complex agents.

They were no abstract drawings on a sheet of paper.

To any agent with some complexity – like Pavlov’s dogs or you or me – the true reward is the interpretation, not the treat or the sound as a substituting (‘unconditioned’) stimulus.

Therefore, one can rightly ask what functions as an expected reward in any concrete case. Naturally, the reward may be rewarding because of the underlying (deeper) value. This is an everyday issue, yet much more complex (and more than simply ‘subjective’) than is generally thought. A lot of this goes on at the subconceptual level, as does most of what is related to motivation.

Reward is in the eye of the observer.

In real life, the reward is part of a concrete observation, which is the whole of feedback an agent gets after one or more of his actions. Thus, a reward can be seen as part of a pattern on its way toward pattern recognition and completion. [There’s more to say about rewards in this vein.]

It may be difficult to discern what should be proper feedback to serve as a reward. Is it the treat from the dog’s boss or the attention that comes with it? This difference may seem inconsequential. However, in many cases, it’s crucial, and seeing the difference may be the start of solving many problems in the human sphere ― even at the geopolitical level.

We may be consciously looking at the treats – or the social status, or the loads of money – while the brain is looking at the level of meaningfulness. Actually, it is only looking at that level.

A little story of dopamine

Brain researchers used to think of dopamine as one of the happiness neuromodulators, the brain’s little treat or prize (to itself?) for doing something good.

However, we see now that dopamine has a different function. It flows more on expectation than on fulfillment. It is, therefore, not the motivator, but part of the motivation itself. Meanwhile, it prepares the agent’s brain to learn what is good or bad for the organism.

Dopamine symbolizes the expectation of the reward, not the reward itself. One way to use this insight is by leveraging on the anticipation a bit more when you want to teach your dog (or child or…) to behave in a desired manner.

Keep some suspense in a friendly way.

The other side of the coin: immediate gratification

In immediate gratification, there is much less expectation. Dopamine flows in short peaks, not leaving time to savor expectations. May that make people more egotistic? What about gaming, and the use of game-theoretic means to try to motivate people? Is this not degrading ― even culturally if many people are subjected to this?

Delayed gratification can lead to more depth, especially if frustration is meanwhile skillfully avoided. The brain/mind gets more time and incentive for forming broader patterns. This way, the focus can spontaneously turn toward deeper values.

That is not the end. But at least it is on the road toward end values (not needing any other underlying). May the final one be global compassion?