Classical Conditioning of Instrumental Conditioning? Whoa…they are

Classical Conditioning of Instrumental Conditioning? Whoa they are supposed to be SEPARATE!

Thorndike’s role in CC-OC connection Thorndike wrote that instrumental conditioning occurs in context of specific environmental stimuli Cues involved that tell you “contingency in effect” Really a THREE term contingency: S,R and O Stimulus Response Outcome S: R Consequence Stimulus can be an S Stimulus can be an S Can predict a reinforcer OR a punisher

S-R association and LAW of EFFECT Contextual stimuli (S) predict the instrumental response contingency (R or R O). Is an association between the S and the R: S R Is an association between the R and the O: R O According to THORNDIKE: Role of the reinforcer is to “stamp in” or “strengthen” the S-R connections NOT the R O contingency Motivation for engaging in R O contingency are the setting contextual stimuli (S) Data did NOT support this model Association IS formed between the R and O However, contextual cues are VERY important in operant conditioning

Expectancy of Reward Reward expectancy can be a motivator for the operant response! Cues for this expectancy are classically conditioned Expectancy cues can be highly varied and complex The situation The individual(s) Signal learning Cues paired with R O contingency because they predict the contingency Clark Hull and Kenneth Spence: Behavior is result of interaction between organism and its environment. Environment provides the stimuli; The organism responds (all of which is observable) Hull’s theory drive reduction theory

Expectancy of Reward Basic premise of their Drive Reduction theory Instrumental response increases during conditioning because Presence of S evokes instrumental response through CC Instrumental response is emitted because of operant contingency In a nutshell: Environmental cues “drive” the response; making the response (and getting the O) reduces the drive The animal works for food because it reduces the drive for food. What cues the animal that it is hungry? Environmental or contextual stimuli Including internal stimuli

Two process theory of Operant Conditioning Assumes 2 distinct types of learning: Pavlovian/CC Instrumental/Operant Related together in special way: Presence of stimuli (S) come to predict R-O relationship R-O relationship is also strengthened Thus S-O become connected Rescorla and Solomon assume S-O association activates emotional state Motivates the operant behavior This emotional state assumed to be positive or negative depending on the consequating stimulus (They must have read Hull’s model!)

Test of 2-process theory Pavlovian Instrumental Transfer experiment: Can counterbalance phase 1 and phase 2 Critical transfer phase it the transfer test: Will the rat lever press to tone alone?

What do the data say? Large research area: Data are mixed in support of theory Organisms respond more in presence of S if S is positive (appetitive) Organisms respond less in presence of S if S is negative (aversive) BUT: Still respond in absence of the S Response interactions occur in Pavlovian Instrumental Transfer Evidence suggests that animals do develop emotional responses to the S Also develop sign tracking behavior (now sign tracking begins to make sense!) Let’s look at some specific experiments to help this make sense.

Krank, et. al., 2008 Rat study! 2 response levers, 1 on either side of water bottle Trained to press either response lever for drop of sweetened water, then ethanol CONC VI 20 sec VI 20 sec schedule 8 Pavlovian sessions: No response levers CS (light) of 10 sec on either left or right side (above lever hole) 0.2 ml of ethanol Unpaired group: CS and ethanol separated by 10 sec. Paired group: CS just before ethanol: GOT sign tracking when light predicted ethanol for paired group, as you would expect Replaced levers and added Pavlovian Transfer Test CS light periodically presented while rats responding for ethanol; alternated over both levers

Krank, et. al., 2008 What happened? Rats pressed each response lever about 2x/min prior to CS Unpaired group: Did not change much when CS added Paired group: Significant increase in lever pressing during CS presentation IF presented to same side as training trials Why is this important? Shows that CS facilitated lever pressing Demonstrates importance of classical conditioning in operant conditioning contingencies Fairly specific as well- not just any CS, but the close or predictive CS

Conditional Emotional States? Reward Specific Expectancies? 2-process theory assumes classical conditioning mediates instrumental conditioning through conditioning of positive or negative emotions Learn CS-US relationship first THEN learn R O relationship BUT: Also develop specific reward expectancies These reward specific expectancies can undermine the emotional conditioning Can manipulate responding: Expect shock get shock Expect shock get reinforcer Expect reinforcer get reinforcer Expect reinforcer get shock!

Conditional Emotional States? Reward Specific Expectancies? Reward specific expectancies can undermine the emotional conditioning Kruse, et. al., 1983: Food pellets vs. sugar solution Expect food pellet get sugar Expect food pellet get food pellet Expect sugar get food pellet Expect sugar get sugar CS for food pellets elicited more instrumental responding when pellets were the rewarded than when sugar and vice versa Suggested rats formed specific expectancies about what the reward would be.

R-O and S(R-O) Relationships Important Hierarchical S(R-O) Relations In addition to the simple associations of 2 elements (i.e., S-R, S-O, R-O), organisms can develop hierarchical associations the (S) signals the relationship between a response and its outcome S - (R - O) the (S) becomes an occasion setter that signals when a specific response will be followed by a specific reinforcer (S) can be a context or a specific cue

Wanting vs Needing Wanting is the motivational consequences of rewards Is an underlying incentive salience: Motivation to get the reward Need is a physiological state of deprivation Wanting and Needs usually go together: Needing the food that you Want Wanting can be separated from needs Drug addiction, gambling and overeating: Wants or physiological needs? Impulse control disorders where wanting has much more intensity Remember the Self-control study with children: Incentive Salience Get one marshmallow now or two later Incentive salience is the psychological process underlying temptation Produces “surges of motivation to obtain and consume the reward”

Wanting vs Needing Incentive salience intensity modulated by wants and needs Needs: really are hungry Wants: setting conditions eliciting motivation for outcome If needs are paired with emotional arousal and stress: Now stress cues “need seeking” behavior Needs can become “wants” when paired with emotional arousal Individual personality differences important Different people have different levels of emotional arousal and stress Individual differences in types of pairing history. Individual differences in levels of need

Wanting vs Needing Incentive salience of specific rewards changes with experience Foods such as chocolate Drugs such as cocaine Reward system becomes sensitized High levels of responding to reward cues If the reward provides a large change in system over time, begin to see sensitization The sight of food, drugs or other incentives may cue the “want” rather than the need

Wanting vs Needing Sensitized incentive salience produces impulsive behavior Difficult to exercise self-control External conditions such as stress reduce self-control Organism no longer can detect “want” versus “need” High levels of relapse in drug addicts Not well explained by pleasure “liking” to get high Not well explained by withdrawal “avoiding” the discomfort Mostly cues with incentive salience that produce excessive wanting

What is addiction? Physiologic dependence and withdrawal avoidance do not explain addiction Neurobiology of addiction attempts to explain the mechanisms by which drug seeking behaviors are consolidated into compulsive use: Long persistence of relapse risk Drug-associated cues control behavior

Although addictive drugs are pharmacologically diverse share a common action Variety of drug agents that become addictive: Stimulants (act as a serotonin-norepinephrine-dopamine reuptake inhibitors) Cocaine, amphetamines, MDMA Opioids (agonist action) Heroin, morphine, oxycodone, fentanyl GABAergic agonists/modulators Alcohol, benzodiazepines, barbiturates Cannabis (binds cannabinoid receptors) What do they all share? Hijacking of dopaminergic system Altering the want vs. need system

They all lead to a common pathway All addictive drugs pharmacologically release dopamine in the nucleus accumbens Why? Addiction is essentially a learned process Predictive cues along with operant response On of best examples of associative learning with instrumental conditioning.

Reinforcement and Dopamine? Olds and Milner: Electrical Brain Stimulation (EBS) Brain stimulation lots of behavior Animals would work until death to gain access to this brain stimulation Thought had discovered pleasure center: Nucleus accumbens Mesolimbic pathway Dopamine (DA) was neurotransmitter involved in these areas EBS electrical brain stimulation releasing LOTS of dopamine (DA) Results in lots of locomotion or exploratory behavior Salamone and Schultz’ modern work has shown this release DA release modulates “appetitive behavior” Occurs in modes or modules related to terminal event E.g., food modes, sex modes Which mode depends on context of environment Search, capture, prepare, consume DA not affect consummatory behavior

The dopamine system

The Dopamine Reward Pathway How Dopamine leads to behavior change Dopamine required for natural stimuli (food, opportunity for mating, etc) to be rewarding and drive behavior Natural rewards and addictive drugs both cause dopamine release in the Nucleus Acumbens Addictive drugs mimic effects of natural rewards and thus shape behavior

The Dopamine Reward Pathway How Dopamine leads to behavior change Survival demands that organisms find and obtain needed resources (food, shelter) and opportunity for mating despite risks -survival relevant goals These goals have natural “rewards” (eating, safety, sex) Behaviors with rewarding goals persist to a conclusion and increase over time as they are positively reinforcing Dopamine is a “feedback” system”: If it is a rewarding behavior .then do it again! If it is not a rewarding behavior, don’t do it.

How relate to Choice? The value of the choice should affect the size of the DA burst Small rewards less DA release than larger rewards Large rewards more DA release Drugs artificially large release (overrides and hijacks the system) Experience can affect DA release Use to large rewards; less responsive to smaller rewards Becomes a problem with gambling, other addictions Used to large bursts of DA via your gambling, drug, etc. More typical rewards such as a paycheck, family, friends, etc., no longer elicit a response or as large a response as the “addictive” responses Question: can we reshape/retrain the brain?

Dopamine Gating Hypothesis: Berringer Because drugs cause dopamine release (due to pharmacological actions), dopamine firing upon use does not decay over time brain repeatedly gets positive predictive error signal: “better than expected!” Drug cues become ubiquitous (drug cues difficult to extinguish) Cues that predict drug availability take on enormous incentive salience (consolidates drug seeking behavior) Drug cues will become powerfully over-weighted compared to other choices (contributes to loss of control over drug use)

Clinical Implications Addictive behaviors are an important and normal part of human behavior Addictive drugs pharmacologically modify functioning of reward circuits to overvalue drug rewards and reduce the comparative value of other rewards Intention to stop use is not enough to stably quit substance use.

So: Treatment of addiction First: detox .get drugs out of system Next: work on breaking The operant response of seeking The cues that predict the drug Easy-peasy, correct?

Is relapse higher for drug addiction? Yes, it is up there and comparable to other “behavioral” diseases How are addiction, hypertension and asthma behavioral disorders?

What does work? Understanding the importance of environmental setting cues and conditions Drug paraphernalia itself Environmental cues: location, time of day, activities etc. People: Family, friends, etc. Stressors: what stressors are related to drug addiction For example: Two potential types of alcoholism: Excitement seeking Risk/aversion avoidance Treatment of “triggers” will be very different

What does work? Must rebuild new cues linked to positive behavior New rules for living New friends New social skills New skills for dealing with stressors, etc. Sober Living House programs are very effective Allow slow transition back into life Allows for learning to fluency the new skills and cues

Is this just true for addiction? What about other “habits”? Takes 2-3 weeks to change a habit That is about how long it takes to change neural circuits Any long term behavior change must involve change of cues as well as change in behavior Thinking behavior AND a cue Think of dieting, etc. Transitioning of treatment programs: special schools placement back to regular schools Others?