Analogs to Reinforcement

The problem of delayed reinforcement

Suppose a parent decides to increase the politeness level of her child. She wants him to say "Please" more frequently than once or twice a day. She decides that each time her child says, "Please", before a request, she will let him stay up an extra 10 minutes later that night. Desiring not to appear manipulative, she doesn't tell him of her plan.

Will she be successful in getting her child to say "Please" more often?
yes
no

What is the most likely reason her plan will fail?
Saying "Please" is not a behavior that can be increased through reinforcement.
Staying up later is not a reinforcer for most children.
The child will not make the connection between saying, "Please", and getting the reinforcement hours later.

The problem here is that too long of a delay occurs between behavior and the consequences of behavior. You may recall that the reinforcement contingency is defined as the immediate, response-contingent presentation of a reinforcer resulting in an increased frequency of that response.

Immediate means less than 60 seconds. Research with animals and humans has found that reinforcement delayed more than 60 seconds has little or no reinforcing effect.

How could the mother get her child to make a connection between saying, "Please", and getting the reinforcer of 10 extra minutes at bedtime for each "Please"?
Since extra time at bedtime cannot be immediately delivered after each response, there is no way that this reinforcer can have an effect on the boy's behavior.
The mother can tell her son, "Every time you say, 'Please', before a request, I'll let you stay up an extra 10 minutes."
She can take her son to the bedroom after every "Please" and tell him that he can stay up 10 minutes right now.

By informing her son of the relationship between his behavior and the reinforcer for that behavior, the mother has stated a rule and her son's behavior is now said to be rule-governed.

So, can behavior be affected by consequences that are not immediate, that is, delayed more than 60 seconds?
yes
no

Rule-governed behavior

Why do we need to understand rule-governed behavior?

The reason that we need to introduce and understand the concept of rule-governed behavior is that much of our behavior is controlled by delayed consequences.

Which of the following behaviors is not controlled by delayed consequences?
studying for an exam
registering for classes
working at your place of employment
paying for clothes at Wal-Mart

Do we really need to talk about rules governing this behavior? Isn't the connection between studying and getting a passing grade so obvious that it doesn't need to be verbalized?

It's only obvious because we know the rule. 

What if passing an exam on Monday was entirely dependent upon eating broccoli on Thursday? Would you make the connection?
yes
no

So, rules are necessary when consequences do not follow immediately. Otherwise a connection will not be made.

Definitions

A rule

A rule is a description of a three-term behavioral contingency (antecedents-behavior-consequences). 

In looser terms, it is a statement that a specific behavior will pay off in a particular situation.

Rule-governed behavior

Rule-governed behavior is behavior under the control of a rule.

Rule-governed vs. contingency control

Behavior is either rule-governed or contingency controlled. We have defined rule-governed. What is contingency control? 

Contingency control is defined as the direct control of behavior by a contingency without the involvement of rules. In other words, contingency control is when some behavior is controlled by its immediate consequences and no rules are involved.

Suppose Bobby whispers something funny to his teammates on his pee wee hockey team and they all laugh. His whispering is strengthened by the direct-acting and immediate consequences in that situation, the laughter of his teammates. Is this rule-governed or contingency controlled?
rule-governed
contingency controlled

Suppose now that Bobby's coach wants to decrease Bobby's disruptive whispering. He says to Bobby, "If you listen carefully and don't whisper when I'm talking to the team, we'll have an extra 5 minutes of scrimmage at the end of practice." Bobby frequently repeats his coach's words to himself during practice, manages to stay quiet, and he gets reinforced at the end of practice. This is rule-governed behavior. Why?
More than a 60 second delay exists between Bobby's behavior and the reinforcer of extra scrimmage.
Bobby is stating a rule to himself, "If I do this, I'll get that."
Both a and b

Direct-acting vs. Indirect-acting

We can also refer to these 2 kinds of control over behavior direct-acting and indirect-acting contingencies.

Contingency control is a direct-acting contingency because the outcome of the response reinforces or punishes the behavior. 

Wait a minute. Doesn't the outcome of behavior also control the behavior in rule-governed behavior? "If I study extra hard, I will get an A on the exam."

Not according to Malott. Outcomes control behavior only when the outcome is immediate. So in rule-governed behavior, it's not the outcome that controls the behavior.

What, then, does control the behavior in rule-governed behavior?
the rule stating the contingency
secondary reinforcers associated with language
the outcome of the behavior

Thus, rule-governed behavior is an indirect-acting contingency defined as a contingency that controls the response, but not because the outcome reinforces or punishes that response.

It sure seems that we are splitting verbal hairs here. However, I think that I can prove that in rule-governed behavior, it is the statement of the rule that controls the behavior, not what really happens (the outcome).

Many people take the herbal supplement Echinacea in order to prevent illness. When exposed to some virus, they take a dose of Echinacea and more often than not don't get sick. 

So, is it the outcome (not getting sick) that controls this behavior? 

What if researchers in Germany discover that Echinacea has absolutely no disease-preventive properties? If the results of this research don't reach the general public in the U.S., will millions still take Echinacea?
yes
no

If people continue to take Echinacea (behavior) even though it does not work (outcome), this proves that it's not the outcome, but rather the rule that is controlling their behavior.

If it turns out that Echinacea really does work in preventing disease, then the behavior does in fact produce the outcome. Would the Echinacea-takers' behavior be contingency-controlled or rule-controlled?
contingency-controlled
rule-controlled

In terms of behaviors and outcomes, there are 2 separate issues. First, does the behavior produce an outcome, immediate or delayed? And, second, what controls the behavior, an immediate consequence (contingency-controlled) or a rule (rule-controlled).

A quick terminology refresher: Rule-controlled behavior is ___________ and contingency-controlled behavior is ___________.
an indirect-acting contingency; a direct-acting contingency
a direct-acting contingency; an indirect-acting contingency

How to determine whether something is rule-controlled?

•  If the outcome of a behavior is not immediate, that is, occurs within 60 seconds of the behavior, it is probably rule-governed.

If a parent says to her child, "I'll give you a quarter every time you say "Thank you" when I give you something." How long will it take to get a change in the child's behavior?
Saying "Thank you" will gradually increase in frequency as it is reinforced more and more often.
The child will immediately say "Thank you" more often.

• Rules often lead to immediate behavior change, whereas direct-acting contingency controlled behavior often takes several trials to work.
•  Can an animal do it? The importance of verbalization. 

Rules involve the verbal statement of some contingencies. Since animals can't verbalize, they can't operate according to rules.

Two types of not direct acting contingencies

A contingency between a behavior and its outcome is direct-acting when the outcome is immediate. If the outcome is delayed, then the contingency is not direct acting.

There are two types of not direct acting contingencies:

Indirect-acting contingency

This is rule-governed behavior.

An ineffective contingency

If an outcome is delayed and the individual doesn't know the rule linking his behavior and its outcome, will the outcome be effective in controlling his behavior?
yes
no

It's an Ineffective Contingency if You Don't Know the Rule

So, to sum up, there are 2 types of Behavioral Contingencies: direct acting (effective because of < 60 sec delay between behavior and outcome) and not direct acting (> 60 sec delay between behavior and outcome).

There are also 2 types of not direct acting contingencies: indirect acting (effective) and ineffective. 

A theory of rule-governed behavior

The issue (what needs explaining)

As we learned earlier, the idea of rule-governed behavior is needed to explain how we handle delayed consequences. As we said, research has indicated that reinforcement delayed more than 60 seconds has little or no reinforcing effect.

However, because of rules, we are continually doing things for delayed reinforcers. We study this week in order to pass an exam next week. We work today for a paycheck the following week (or month). 

In a sense, with rules, time between the behavior and the consequences doesn't matter that much.

Now, consider that we can mail off a check for a magazine subscription that begins 6 weeks later while we can't seem to get around to exercising to improve our health?

So, what's the issue that needs explaining? 

What needs to be explained is that we sometimes follow rules and behave in order to get a delayed reinforcer while at other times we don't follow rules, even when they state that a behavior will produce very large consequences.
true
false

Also why do we procrastinate? We put off studying for an exam, jeopardizing our grade. We put off organizing our office or room even though the delayed payoff, a neat and orderly working space, is very reinforcing. In each of these cases, we are failing to follow a rule linking a behavior with a desired outcome.

One possible explanation is that most of these desirable behaviors (studying, exercising, organizing) are not as much fun as the other things we might be doing at a moment in time. And the consequences of the desirable behaviors are distant and not as immediately reinforcing as watching TV, playing ball, or talking with friends.

We first encountered these ideas when we talked about concurrent contingencies.

Concurrent contingencies are said to occur when more than one contingency of reinforcement or punishment is available to us at the same time.

We also looked at some factors that can be used to explain why we engage in one behavior (with its contingencies) instead of other behaviors (with their contingencies) at a given moment in time.

The factors are:

• magnitude of reinforcement
• response effort of various options
• frequency of reinforcement
• immediacy of reinforcement or punishment

The last factor, immediacy, has been proposed to explain procrastination and other cases of poor self-management.

The mythical cause of poor self-management

The idea is that poor self-management occurs because we can't delay our gratification, because immediate outcomes control our behavior better than delayed outcomes do. So we fail to act in our long-run best interests.

The authors of our text consider this to be a false principle and a myth.

To the authors, what is wrong with the above explanation for poor self-management?
This explanation fails to take into account the size of the reinforcers. Immediate reinforcers are always larger than delayed reinforcers.
This explanation contradicts the abundant evidence that delayed outcomes can control our behavior through the use of rules.
This explanation is false only because it assumes that all people suffer at times from poor self-management. Some people never have self-management issues.

So, it's not simply an issue of how delayed the outcomes are.

We can mail a magazine subscription (today) in order to get a reinforcer, the magazine, 6 weeks from now while putting off working on that paper (today) that will be due in only a couple of weeks? 

Is the magazine reinforcer more immediate than the grade reinforcer?
yes
no

And it's not an issue of the size of the immediate and delayed reinforcers.

Is getting a magazine a more powerful reinforcer than getting a good grade?
yes
no

The issue again seems to be, "Why do we sometimes follow the rules specifying delayed outcomes and why sometimes do we not."

To understand what is going on here and to understand the true causes of poor self-management, we need to have a theory of rule-governed behavior.

The theory of rule-governed behavior

Rules are establishing operations with noncompliance an aversive condition

A rule is a description of a behavioral contingency. "If I study a certain amount of time, I'll do well on the exam.

The statement of the rule is also an establishing operation that establishes noncompliance with the rule as an aversive motivating condition.

So, if you state a rule that says that you must study in order to do well on the exam, what happens when you find yourself goofing off?
Nothing much.
You feel anxious or guilty.

How do you escape from this aversive condition? 
You study.
You continue to goof off.

Here is a point that might help clarify some of the confusion you may feel as you read through the chapter.

The relationship between studying and grades represents an indirect-acting contingency because the outcome (grades) controls the response (studying), but not directly. It controls the behavior through a rule. However, the statement of the rule also creates a direct-acting contingency. 

A direct-acting contingency occurs when an immediate outcome controls the response.

When you fail to follow your rule (i.e., you goof off instead of studying), when do you feel the anxiety or guilt?
right before the delayed outcome (the exam)
immediately

Once you've stated the rule linking studying and good grades, what is the immediate consequence of studying?
You do well on the exam.
You escape from the aversiveness of not studying.

An important thing to remember: Behavior is always controlled by immediate reinforcers and aversive consequences. 

This is true even when the ultimate outcome of a behavior is delayed. The statement of a rule creates an immediate aversive outcome when you fail to behave appropriately. 

Now that we know the immediate contingencies created by the statement of a rule, we are in a better position to understand why we often procrastinate, even when the consequences are profound.

Small, but cumulative outcomes

A big problem for us is when the immediate outcome for each specific instance of behavior is too small to significantly reinforce or punish that behavior even though the cumulative impact of many such outcomes is substantial.

For example, you state a rule saying, "Eating ice cream will make me fat."

Since stating rules creates a situation where noncompliance with the rule is aversive, eating a spoonful of ice cream in noncompliance with your rule is aversive. You feel guilty.

But how much guilt do you feel?
Not much, since one spoonful of ice cream will have very little effect on your overall weight.
A lot of guilt since that single bite of ice cream will significantly put you off your diet.

The guilt is immediate, but very small. However, the cumulative effect of eating many bites of ice cream, putting on the pounds, is large.

And, according to Malott, do delayed outcomes directly reinforce behavior? 
yes
no

If only immediate outcomes influence behavior, then the immediate guilt you feel from a violation of your rule is way too small to outweigh the immediate large amount of pleasure you get from eating the ice cream.

One can apply the same analysis to procrastinating instead of studying. The cumulative effect, a bad grade, of procrastinating is large.

However, how much effect is there when you put off studying 1/2 hour in order to watch a TV show?
You feel a lot of anxiety.
You don't feel much anxiety over a 1/2 hour delay.

Therefore, we watch the show.

Improbable outcomes

You have a rule linking a behavior (buckling up) with an outcome (prevention of fatality in a car accident).

What's the probability that violating your rule as you drive to work today will result in your fatality? 
A very low probability
A very high probability

With low probability there is low anxiety and not much of a deterrent to buckling up.

Hard to follow vs. easy to follow rules

Malott distinguishes between rules that are hard to follow and rules that are easy to follow.

Hard to follow rules describe outcomes that are either too small (though often cumulative in significance) or too improbable. The delay isn't crucial.

Which would be a hard to follow rule?
If I buy an airline ticket today, I can travel to Miami next month.
If I study 40 minutes today, I will be better prepared for my exam 2 weeks from now.

Easy to follow rules describe outcomes that are both sizable and probable. The delay isn't crucial.

Which would be an easy to follow rule?
If I order a pizza by phone, I will enjoy it within the hour.
If I exercise today, I will be healthier and live longer. 

Let's look at procrastination again and compare the outcomes of delaying studying for 1 hour 2 weeks prior to the exam and delaying studying 1 hour the day before the exam.

Consequences of delaying studying for 1 hr when the exam is in 2 weeks?
Negligible
Significant

Consequences of a 1 hour delay the day before the exam?
Negligible
Significant

So, it's easy to procrastinate when a good deal of time exists before your deadline. A delay is of little consequence and provokes little anxiety. On the other hand, when the deadline is near, the effects of procrastination are more severe and our corresponding anxiety is higher. Hence, we stop procrastinating as the deadline approaches. Unfortunately, by that time, it may be too late.

In the space below, please write any comments or questions about the tutorial.