the gure. Interestingly, the most informative word is not ccar d, but cDoD d, the abbreviation for a motorcycle club called cDenizens of Doom d. So far we 9ve been computing the information content in a single word.<br><br> Sup- pose we are allowed to ask another question about the document, after getting the answer to the rst question. The best second question is not necessarily the second-best rst question. For example, if you know ccar d is present, it doesn 9t help as much to know that ccars d is also present.<br><br> This e ect is called interaction . Whereas correlation and information are properties between two variables, interaction is a property between three variables. Interaction is when measuring one variable changes the importance of another variable.<br><br> Conditional information is the information X gives about C when a third variable Y is already known: I [ C ; X | Y = y ] = H [ C | Y = y ] 2 H [ C | X,Y = y ] (actual conditional information) 1 I [ C ; X | Y ] = y Pr( Y = y ) I [ C ; X | Y = y ] (expected conditional information) Interaction measures how Y changes the information in X : I [ C ; X ; Y = y ] = I [ C ; X | Y = y ] 2 I [ C ; X ] (actual interaction) I [ C ; X ; Y ] = I [ C ; X | Y ] 2 I [ C ; X ] (expected interaction) Expected interaction is symmetric in all three variables: I [ C ; X ; Y ] = I [ C ; Y ; X ] = I [ X ; Y ; C ], etc. A positive interaction means that knowing Y makes X more informative about C . A negative interaction means that Y makes X less in- formative about C .<br><br> For example, suppose C is tomorrow 9s weather and X and Y are weather reports from two di erent stations. Once you hear one weather report, the other going to be much less informative. Relations of information and interaction can be conveniently visualized with information graphs .<br><br> 2 Example: the interaction between the presence of ccar d and ccars d , , cars = FALSE car label FALSE TRUE auto 35 34 moto 93 3 , , cars = TRUE car label FALSE TRUE auto 12 19 moto 2 2 I [ label ; car ] = 0 . 227 I [ label ; car | cars = F ] = 0 . 233 I [ label ; car ; cars = F ) = 0 .<br><br> 006 (positive interaction) I [ label ; car | cars = T ] = 0 . 004 I [ label ; car ; cars = T ] = 2 0 . 223 (negative interaction) Pr( cars = T ) = 0 .<br><br> 175 I [ label ; car ; cars ] = 2 0 . 034 (negative on average) Note that the expected interaction can be zero, even though the actual interac- tions are nonzero (but of opposite sign). 3 0.0000 0.0005 0.0010 0.0015 0.0020 0.0025 0.00.20.40.60.8 Expected information Actual information # #apr# #cbr#rr #cc #e #hp #i #k #kmiles #kqspt #l #mph #nd #qgi#einnhs# #r #r#cr#innvar #r#tth #r#uckinncmf #rd #s #ss #st #th #v #wd #yr a a# aamir aardvark aas# aber able about above abs absence absolutely ac acad# acc acceleration accept access accident accidents accord according account across acs act acted activities actual actually ad add addition additional address adequately adjustable admit adobe advance advanced advantage advantages advice aerodynamics affected after again against agate age agent aging ago agree ahead aint air ak# al alan alaska albert alcohol alfa all allen allowed allows almost alone along alot already also although altima aluminum always am ama american amin amount an and andrew andy anger angle announced annual anonymous another answer answers anti any anybody anyone anything anyway anywhere apart apartment apparently apple applications applies apply appreciate appreciated approach approaching appropriate apr april arc arch are area areans areas arent arlington arm armory arms around art article as ash ask asked asking ass assign assume assuming at att attack attacking attempt attempting attention attitude au audi austin australia author auto automobile autos availability available average avoid aware away awd awful awhile axel azw b b#i ba# baby back backing backward bad bain balance balanced bar bars base based basic basically battery bay bb bc bclarke be bear bearing beating beats beautiful beav beavington because become becomes bedford beemer been beer before begin behanna behavior behind being believe below belt benefit benefits benz berkeley besides best beth bethd better between bgardner bicycle big bike biker bikers bikes bill bird bit bitnet black blade blaine blame blew bliss block blocked blow blowing blown blue bmw bmws bnr boat bob body bolt bombed book books boots bore both bottles bought boulder box boyhood boyle brake brakes braking brand brands break bricklin bricklins bridges bring broken brought bruce bruiser bs btw buck bucks buddy budge build building built bum bumper bumpy business but butt buy buyers buying by c c#st c#zkcl ca cabin cable cactus cage cages calculate calif california call called calling callison calls caltech cam camaro came can canada cannot cant car car# carb carbs carderock care carefully cargo carry cars case cases catch caught cause causes causing cb cb# cb#t cbx cc cd cdac cellular center centerline centripital certain certainly chain champion chance chances change changed changes charles cheap cheaper check checked chen cherokee chevy chicago chief children chintan chips choice chong choose chose chris chrysler chuck church cigarette cit city civic cj# cjackson claim claims class clean clear clockwise close closed cloth club clue clutch cmon cmu co coating cobra code coefficients cog coil cold collect collection college color colorado com come comes comfortable coming comment comments commericial companies company comparable compare comparing comparison compartment compdyn compete competition complete complex compound compress compuserve computer concentrate concise concours condition conditions confused confusing consider considering constellation construed consumer contact continually contractor control controversial conversion convert convertible convertibles convictions cookson cool coolant coordinator cop copies cops copy corn cornell corner corners corp corporation correct correctly cost costs could couldnt counter counteract countersteering country coupe couple course courses court cousineau cover covered covering cr# craig crap crash cravan crawl created cross cruiser cruisers cruncher cs csd csd# cso cult cup curious curls current currently curtis custom customer cut cwru cx# cycle cylinder cylinders d dagibbs daker damage damaging dammit damn damned dan dangerous dare darkman dash data date dave davew david day days dcrt de dead deal dealer dean death debating dec decent decide decided defending define definitely definition degree delcoelect deleted demand denver department depending depressed dept depts deranged design designed details detectors developed development devil did didnt die diego difference different difficult digiorgi ding dip direct direction directly disclaimer discovered discussed discussion distance distances distant diving division dixon do dod doders dodge does doesnt dog dogs doing done dont door doors doubt dove down drag drain draining dream drew drill drink drinking drive driven driver drivers driving drop drove drunk dry dsd dt duc ducati dude due dumped during dying e e#i each eagle early earth ease easier easily east easy eat eating eclipse ecn ed edu effect effective effects eg egreen einstein either ejv#j electric electronic eliot else elsewhere em email embalmer employees employer encore encourage end energy enet eng engage engaged engine engineer engineered engineering engines english engr enjoy enlighten enough entire entirely environment equipment er erc eric erik es eskimo especially essentially estimate et etc eternal europe evans even evening event ever every everybody everyone everything everywhere ex ex# exact exactly example excellent except excess excitable exhaust exotics expect expense expensive experience experiences experiencing explain explorer expressed extended extra extreme f face fact factors factory fail failed fairly fall family faq far farther fashion fast faster favor favorite fax fear feather features federal fee feel feeling feels feet female fencing ferrari feserve few fgc field figure figured figures file finally find finding fine fingers finish fire firebird first fit fits fitting five fix fixed flame flamed flames flat fleet floor flow fluid fly folks follow food foot footpeg for force ford fords forecast foreign forever forget form former forth forward found foundation four foward frame francisco free freedom friction friday friend friends fringe from front frost fssnoc fuel full fully fun funded further furthermore future fxwg fyi g g# galant galaxy gallon gang gap garage gardner gary gas gate gatech gauge gave gear gearbox gears geico gene general generally generic gently george germany get gets getting gibbs girl girlfriend girls give given gives giving gl glad glide gloves gm gmt go god goddamn godfrey goes going gold gone gonna good got gov government governments grab grad grand grateful great greater greatly green gregh gregory grey grip gross ground group groups grown gs gs#g gsxr# gt gts guarantee guaranteed guard guardrail guess guide guns guy guys guzzi gwm h habits had hadnt hafta half hall hamer hamilton hand handle handlebar handlebars handling hands handson handy hang happen happened happening happens happy hard harder harley harleys has hasnt hate have havent having hawk he head heading hear heard hearing heart heavy heck hed held hell hello helmet helmets help helps hence her here heres hes hewlett hey hg hi high higher highway him himself hints his history hit hits hitting hmm hmmm hmmmm hogs hold holding hole home honda hope horizons horse horsepower horses hose host hot hours how however howland hp hplsla huge hughes huh hulk human humphreys hurt hydro i ian iastate ibm ican icoa id idea ideal ideas identical idiot idle if ignorance ignored ii il ill illinois im imagine imho impact impala important imposible impossible in inaccurate inc incarcerated inch included including increase increased incredible indeed indian indiana infante infinity info information informed initial injection input ins insert inside install installed instead instructions instructor instrument insurance integra intend intended interceptor interest interested interesting interior internet into investment is isnt issue it it# itd itll its itself ive j jacket jackson jaguar jam james japan jason jaws jay jeep jeff jeffrey jerry jiggled jim jimf jlevine job jobs jody joe joesbar john johnson join joke jon jonathan jorg jose jubilees judge judgement jump junk just k k# k#rs karr kawasaki kay keep keeping keeps ken kens kept key keys kid kids kill killing kind kindly king kingdom kit klinger knew know knowing known knows kocrsv# koresh kotb kotfaq kv kz# l la lady lake lam land large larger larry last late latech later law lawrence laws lbs lead leak leaks lean learn least leather leave leavitt left legal legend lerxst less lesser let lets level levels lever levine liable liberty licence life lift light lightning lights like liked likely limit limited line lines linus liquid list listed listen liter litres little live living ll llama load local located lock locking locks locomotives logic lone long longer look looked looking looks loop loral lost lot lots lotsa loud love low lower lt lubricants luck lucky lx m ma mac machine machines made mag magazine magazines magnus mahan maidenhead mail maintained maintenance major make makes making man manager manifold manish mans manual manufactured manufacturer manufacturers many marcus mark market marshatt mary mass massive match matching material matter matthew maven max maxima maximum may maybe mazda mb mbeaving mbs mbunix mcguire md me mean means meant mechanic mechanical mechanism meet meeting meets mellon member mentality mention mentioned mentor mercedes mercer mess message metal meter method mi miata michael microcomputer microsystems mid might mike mil milage mile mileage miles million mimsy mind minds mine mines minivan minivans minute minutes mirror mirrors miss mitre mitsubishi mix mjs mlud mm mmmmmmmmmm model models modified monday money monster month months monza moral more morgan morning morris most mostly mot motion moto motor motorcycle motorcycles motorcycling motorcyclist motorola mounted move moved moving mph mpv mr mr# mt much munny music must mustang mx my myself n name named nasa national navy ncsl near nec necessarily neck necks need needed needs neighborhood neil net netcom netters neutral never new newbie newer news newsgroup newsgroups next nia niagara nice nick night nih ninja ninjaite nissan nist nj nntp no noise non none nor normal normally norman not note nothing notice now npet nswc ntu ntuvax number numbers nuts ny o oasys objects obviously occurances odometer of off offer official often oh ohio oil ok oklahoma old ole on once one ones only ontario onto open operating opinion opinions opportunity opposite option or order orders org organization original originally orion orlin other others otherwise ottawa our ours out outside over overpass own owned owner owners oxford p pack packard pads paid pain paint painted pair paired papers paperwork parked parking parr part particular particularly parts pass passed passenger passengers passes past patch path paul pavement pay paying pays pbwasoh peace peart pebbles pedestrian pee peeing penny people perfect perform performance permits person personal personally pettefar pf pgp philosophy phone pick picked picture piece pieces pillion pipes pittsburgh pixel place places plain plains plan planned plans plastic platform play please plow pls plug plugs plus po pocket point pointed police policy poor popping position possible possibly post poster posting posts potatoes potential potentially pound pounds power powered powerful practical practice pranks pre prefer preferably prejudice prepare prepared pres press pressure pretty prevented previous previously price prices pricing primary probably problem problems produced product production products profit profits program project promise proper protect protected protecting provide provides ps public pull pulled pulling pulls pump purchase purchased purchasing purdue purpose push pushing put putting q qazi qnx quality quantum quarter quarts quest question questions questor quick quite quote quotes r r# r#rs ra rabbits rabih race radar rail rain raise raised rally ramarren ranck random range rare rat rate rates rather rating rd re reach reached reactions read reading real realize realizing really rear reason reasonable reasons rebuilt rec recall received recently recommend recommended record records red reduce reduced reed references refrain refund refused regards registration rein reincarnate reining reins related relatively release released releasing reliability reliable remains remember reminds remove removing rent rental repair replace replaced replacement reply report reproduce request requires research responce respond response responsibility rest restoration return rev reverse rf richard rid ride rider riders rides ridiculous riding right rightous rights ring risk road roads robert rock rocks rod rode rogers roll romeo roof room rotating rotors rough route routes rpm rri rs rseymour rtsg rubber rule rumor run running runs rush russian rwert rx# ryan rz rz# s s# sack sad safe safely safety said sake sale sales sam same sample san sand satan saturn save saved saw say saying says sc sc# scale schedule scheme school science scott scratches screw screwed se search seat seatbelts seattle seattleu seca second secondly secure sedan see seeing seem seemed seems seen self sell selling semi send sense sent separate serious seriously service services set setting several seymour sez sf sg shadow shaft share she shes shift shifter shifting shipping shit sho shocks shop short should shoulder shoulders show showed shows shut sick side sidewalk sig sight signal signals significant significantly signify signs silly silver similar simply since single sit sitting situation six sixgun sixsmith size skeena skip sl# slack slide slight slip slippery slow slowed slowly small smell smething smith smooth smoother smorris snail snow snyder so society soft software sold solid solved solving some somebody someone something sometimes somewhat somewhere sorenson sorlin sorry sort sound sounds source sources south sp space span spare speak special specific speed speedo speedometer speeds speedy spelling spencer spend spent spike spiros spl spl# sport sporty spot squeeze squid squish sr sr# srgenprp ss ssave st# stable stand standard standards stanza star starchild start started starting state stated statement states station stay std steal stealing steel steer steering step steve steven stick sticker stickers sticky still stock stone stooped stop stopped stopping stops story straight street streets stretch stroke struck stuck student stuff stupid style styling su subaru subject succeed such suffer suggest suggested suggestion suggestions suit suitable sumax summer sun sunday sunroof support suppose supposed supposedly sure surface surprise surprised suspension sutherland suv suvs suzibago suzuki svoboda switch switching syl sys system systems t table tad tail take taken takes taking talk talked talking talon tank targa task taurus tax team tech technique technology teeth tek telephone tell telling temporarily ten tendency terry test texas th than thank thanks thanx that thats the theft their them themselves then theodore there therefore theres these they theyd theyll theyre theyve thighs thing things think thinking thinks this thomas thompson those though thought thread threads three throttle through throw thru thumb thumbs thunder thursday thus ticket tight tighten tightened till time times timing tire tired tires title tm to today together told tom tommy tonight tony too took tool top tops toronto torque torreys total touch tough towed toyota track trade traditional traffic trailer train training tranmission tranny trans transmission trevor triantafyllopoulos tried trimmed trip trips triumphs trouble truck trucks true trunk try trying tuchus tue tuesday tune turbo turn turned turning turns tv twetzel twice twin twisty two tx type types typical typically u u# ubc ucbvax uclink ucs uea uic uicvm uiuc uk ultra umanitoba umd umn under underneath understand unforgiven unfortunately unit united units university unixg unless unlikely until unusual uokmax uoknor up upon ups urbanachampaign us usa usc use used useful useless usenet uses using usual usually utexas uucp uunet uupcb uwm uwo ux# v v# v#sabre va value valve vandals various vax# vehicle vehicles vel velapoldi velocity venture version very vfr#f via view vij viking villager vincent virago virginia virtually visible vision visit vlasis voice void volvo voyager vr# vs vt vtserf vw w wa wackos waco wagon wagons wait waiting walk walking wam wanna want wanted wants warm warning warped warranty was washington wasnt watch water wave waving wax way wayne ways wd we weak wear weather wed week weekend weeks weighs weight well wen went were werent wert west wet wetzel what whatever whats wheel wheelie wheels when where wheres whether which while white who whole why wibbled wide wife wild will willing wind window windows winson wish with within without wlsmith wm won wonder wondering wonnacott wont woodward woops word words work worked working works world worried worry worse worst worth worthwhile would wouldnt wow wreck wrench write writes written wrong wrote x xjs xlyx y yamaha yeah year yearly years yes yet yfn yik yiklam york you youd youll your youre yours yourself youve ysu zauberer zen zero zx zz 0.00 0.05 0.10 0.15 0.20 0.25 0.00.20.40.60.8 Expected information Actual information # #apr# #cbr#rr #cc #e #hp #i #k #kmiles #kqspt #l #mph #nd #qgi#einnhs# #r #r#cr#innvar #r#tth #r#uckinncmf #rd #s #ss #st #th #v #wd #yr a a# aamir aardvark aas# aber able about above abs absence absolutely ac acad# acc acceleration accept access accident accidents accord according account across acs act acted activities actual actually ad add addition additional address adequately adjustable admit adobe advance advanced advantage advantages advice aerodynamics affected after again against agate age agent aging ago agree ahead aint air ak# al alan alaska albert alcohol alfa all allen allowed allows almost alone along alot already also although altima aluminum always am ama american amin amount an and andrew andy anger angle announced annual anonymous another answer answers anti any anybody anyone anything anyway anywhere apart apartment apparently apple applications applies apply appreciate appreciated approach approaching appropriate apr april arc arch are area areans areas arent arlington arm armory arms around art article as ash ask asked asking ass assign assume assuming at att attack attacking attempt attempting attention attitude au audi austin australia author auto automobile autos availability available average avoid aware away awd awful awhile axel azw b b#i ba# baby back backing backward bad bain balance balanced bar bars base based basic basically battery bay bb bc bclarke be bear bearing beating beats beautiful beav beavington because become becomes bedford beemer been beer before begin behanna behavior behind being believe below belt benefit benefits benz berkeley besides best beth bethd better between bgardner bicycle big bike biker bikers bikes bill bird bit bitnet black blade blaine blame blew bliss block blocked blow blowing blown blue bmw bmws bnr boat bob body bolt bombed book books boots bore both bottles bought boulder box boyhood boyle brake brakes braking brand brands break bricklin bricklins bridges bring broken brought bruce bruiser bs btw buck bucks buddy budge build building built bum bumper bumpy business but butt buy buyers buying by c c#st c#zkcl ca cabin cable cactus cage cages calculate calif california call called calling callison calls caltech cam camaro came can canada cannot cant car car# carb carbs carderock care carefully cargo carry cars case cases catch caught cause causes causing cb cb# cb#t cbx cc cd cdac cellular center centerline centripital certain certainly chain champion chance chances change changed changes charles cheap cheaper check checked chen cherokee chevy chicago chief children chintan chips choice chong choose chose chris chrysler chuck church cigarette cit city civic cj# cjackson claim claims class clean clear clockwise close closed cloth club clue clutch cmon cmu co coating cobra code coefficients cog coil cold collect collection college color colorado com come comes comfortable coming comment comments commericial companies company comparable compare comparing comparison compartment compdyn compete competition complete complex compound compress compuserve computer concentrate concise concours condition conditions confused confusing consider considering constellation construed consumer contact continually contractor control controversial conversion convert convertible convertibles convictions cookson cool coolant coordinator cop copies cops copy corn cornell corner corners corp corporation correct correctly cost costs could couldnt counter counteract countersteering country coupe couple course courses court cousineau cover covered covering cr# craig crap crash cravan crawl created cross cruiser cruisers cruncher cs csd csd# cso cult cup curious curls current currently curtis custom customer cut cwru cx# cycle cylinder cylinders d dagibbs daker damage damaging dammit damn damned dan dangerous dare darkman dash data date dave davew david day days dcrt de dead deal dealer dean death debating dec decent decide decided defending define definitely definition degree delcoelect deleted demand denver department depending depressed dept depts deranged design designed details detectors developed development devil did didnt die diego difference different difficult digiorgi ding dip direct direction directly disclaimer discovered discussed discussion distance distances distant diving division dixon do dod doders dodge does doesnt dog dogs doing done dont door doors doubt dove down drag drain draining dream drew drill drink drinking drive driven driver drivers driving drop drove drunk dry dsd dt duc ducati dude due dumped during dying e e#i each eagle early earth ease easier easily east easy eat eating eclipse ecn ed edu effect effective effects eg egreen einstein either ejv#j electric electronic eliot else elsewhere em email embalmer employees employer encore encourage end energy enet eng engage engaged engine engineer engineered engineering engines english engr enjoy enlighten enough entire entirely environment equipment er erc eric erik es eskimo especially essentially estimate et etc eternal europe evans even evening event ever every everybody everyone everything everywhere ex ex# exact exactly example excellent except excess excitable exhaust exotics expect expense expensive experience experiences experiencing explain explorer expressed extended extra extreme f face fact factors factory fail failed fairly fall family faq far farther fashion fast faster favor favorite fax fear feather features federal fee feel feeling feels feet female fencing ferrari feserve few fgc field figure figured figures file finally find finding fine fingers finish fire firebird first fit fits fitting five fix fixed flame flamed flames flat fleet floor flow fluid fly folks follow food foot footpeg for force ford fords forecast foreign forever forget form former forth forward found foundation four foward frame francisco free freedom friction friday friend friends fringe from front frost fssnoc fuel full fully fun funded further furthermore future fxwg fyi g g# galant galaxy gallon gang gap garage gardner gary gas gate gatech gauge gave gear gearbox gears geico gene general generally generic gently george germany get gets getting gibbs girl girlfriend girlsgive given gives giving gl glad glide gloves gm gmt go god goddamn godfrey goes going gold gone gonna good got gov government governments grab grad grand grateful great greater greatly green gregh gregory grey grip gross ground group groups grown gs gs#g gsxr# gt gts guarantee guaranteed guard guardrail guess guide guns guy guys guzzi gwm h habits had hadnt hafta half hall hamer hamilton hand handle handlebar handlebars handling hands handson handy hang happen happened happening happens happy hard harder harley harleys has hasnt hate have havent having hawk he head heading hear heard hearing heart heavy heck hed held hell hello helmet helmets help helps hence her here heres hes hewlett hey hg hi high higher highway him himself hints his history hit hits hitting hmm hmmm hmmmm hogs hold holding hole home honda hope horizons horse horsepower horses hose host hot hours how however howland hp hplsla huge hughes huh hulk human humphreys hurt hydro i ian iastate ibm ican icoa id idea ideal ideas identical idiot idle if ignorance ignored ii il ill illinois im imagine imho impact impala important imposible impossible in inaccurate inc incarcerated inch included including increase increased incredible indeed indian indiana infante infinity info information informed initial injection input ins insert inside install installed instead instructions instructor instrument insurance integra intend intended interceptor interest interested interesting interior internet into investment is isnt issue it it# itd itll its itself ive j jacket jackson jaguar jam james japan jason jaws jay jeep jeff jeffrey jerry jiggled jim jimf jlevine job jobs jody joe joesbar john johnson join joke jon jonathan jorg jose jubilees judge judgement jump junk just k k# k#rs karr kawasaki kay keep keeping keeps ken kens kept key keys kid kids kill killing kind kindly king kingdom kit klinger knew know knowing known knows kocrsv# koresh kotb kotfaq kv kz# l la lady lake lam land large larger larry last late latech later law lawrence laws lbs lead leak leaks lean learn least leather leave leavitt left legal legend lerxst less lesser let lets level levels lever levine liable liberty licence life lift light lightning lights like liked likely limit limited line lines linus liquid list listed listen liter litres little live living ll llama load local located lock locking locks locomotives logic lone long longer look looked looking looks loop loral lost lot lots lotsa loud love low lower lt lubricants luck lucky lx m ma mac machine machines made mag magazine magazines magnus mahan maidenhead mail maintained maintenance major make makes making man manager manifold manish mans manual manufactured manufacturer manufacturers many marcus mark market marshatt mary mass massive match matching material matter matthew maven max maxima maximum may maybe mazda mb mbeaving mbs mbunix mcguire md me mean means meant mechanic mechanical mechanism meet meeting meets mellon member mentality mention mentioned mentor mercedes mercer mess message metal meter method mi miata michael microcomputer microsystems mid might mike mil milage mile mileage miles million mimsy mind minds mine mines minivan minivans minute minutes mirror mirrors miss mitre mitsubishi mix mjs mlud mm mmmmmmmmmm model models modified monday money monster month months monza moral more morgan morning morris most mostly mot motion moto motor motorcycle motorcycles motorcycling motorcyclist motorola mounted move moved moving mph mpv mr mr# mt much munny music must mustang mx my myself n name named nasa national navy ncsl near nec necessarily neck necks need needed needs neighborhood neil net netcom netters neutral never new newbie newer news newsgroup newsgroups next nia niagara nice nick night nih ninja ninjaite nissan nist nj nntp no noise non none nor normal normally norman not note nothing notice now npet nswc ntu ntuvax number numbers nuts ny o oasys objects obviously occurances odometer of off offer official often oh ohio oil ok oklahoma old ole on once one ones only ontario onto open operating opinion opinions opportunity opposite option or order orders org organization original originally orion orlin other others otherwise ottawa our ours out outside over overpass own owned owner owners oxford p pack packard pads paid pain paint painted pair paired papers paperwork parked parking parr part particular particularly parts pass passed passenger passengers passes past patch path paul pavement pay paying pays pbwasoh peace peart pebbles pedestrian pee peeing penny people perfect perform performance permits person personal personally pettefar pf pgp philosophy phone pick picked picture piece pieces pillion pipes pittsburgh pixel place places plain plains plan planned plans plastic platform play please plow pls plug plugs plus po pocket point pointed police policy poor popping position possible possibly post poster posting posts potatoes potential potentially pound pounds power powered powerful practical practice pranks pre prefer preferably prejudice prepare prepared pres press pressure pretty prevented previous previously price prices pricing primary probably problem problems produced product production products profit profits program project promise proper protect protected protecting provide provides ps public pull pulled pulling pulls pump purchase purchased purchasing purdue purpose push pushing put putting q qazi qnx quality quantum quarter quarts quest question questions questor quick quite quote quotes r r# r#rs ra rabbits rabih race radar rail rain raise raised rally ramarren ranck random range rare rat rate rates rather rating rd re reach reached reactions read reading real realize realizing really rear reason reasonable reasons rebuilt rec recall received recently recommend recommended record records red reduce reduced reed references refrain refund refused regards registration rein reincarnate reining reins related relatively release released releasing reliability reliable remains remember reminds remove removing rent rental repair replace replaced replacement reply report reproduce request requires research responce respond response responsibility rest restoration return rev reverse rf richard rid ride rider riders rides ridiculous riding right rightous rights ring risk road roads robert rock rocks rod rode rogers roll romeo roof room rotating rotors rough route routes rpm rri rs rseymour rtsg rubber rule rumor run running runs rush russian rwert rx# ryan rz rz# s s# sack sad safe safely safety said sake sale sales sam same sample san sand satan saturn save saved saw say saying says sc sc# scale schedule scheme school science scott scratches screw screwed se search seat seatbelts seattle seattleu seca second secondly secure sedan see seeing seem seemed seems seen self sell selling semi send sense sent separate serious seriously service services set setting several seymour sez sf sg shadow shaft share she shes shift shifter shifting shipping shit sho shocks shop short should shoulder shoulders show showed shows shut sick side sidewalk sig sight signal signals significant significantly signify signs silly silver similar simply since single sit sitting situation six sixgun sixsmith size skeena skip sl# slack slide slight slip slippery slow slowed slowly small smell smething smith smooth smoother smorris snail snow snyder so society soft software sold solid solved solving some somebody someone something sometimes somewhat somewhere sorenson sorlin sorry sort sound sounds source sources south sp space span spare speak special specific speed speedo speedometer speeds speedy spelling spencer spend spent spike spiros spl spl# sport sporty spot squeeze squid squish sr sr# srgenprp ss ssave st# stable stand standard standards stanza star starchild start started starting state stated statement states station stay std steal stealing steel steer steering step steve steven stick sticker stickers sticky still stock stone stooped stop stopped stopping stops story straight street streets stretch stroke struck stuck student stuff stupid style styling su subaru subject succeed such suffer suggest suggested suggestion suggestions suit suitable sumax summer sun sunday sunroof support suppose supposed supposedly sure surface surprise surprised suspension sutherland suv suvs suzibago suzuki svoboda switch switching syl sys system systems t table tad tail take taken takes taking talk talked talking talon tank targa task taurus tax team tech technique technology teeth tek telephone tell telling temporarily ten tendency terry test texas th than thank thanks thanx that thats the theft their them themselves then theodore there therefore theres these they theyd theyll theyre theyve thighs thing things think thinking thinks this thomas thompson those though thought thread threads three throttle through throw thru thumb thumbs thunder thursday thus ticket tight tighten tightened till time times timing tire tired tires title tm to today together told tom tommy tonight tony too took tool top tops toronto torque torreys total touch tough towed toyota track trade traditional traffic trailer train training tranmission tranny trans transmission trevor triantafyllopoulos tried trimmed trip trips triumphs trouble truck trucks true trunk try trying tuchus tue tuesday tune turbo turn turned turning turns tv twetzel twice twin twisty two tx type types typical typically u u# ubc ucbvax uclink ucs uea uic uicvm uiuc uk ultra umanitoba umd umn under underneath understand unforgiven unfortunately unit united units university unixg unless unlikely until unusual uokmax uoknor up upon ups urbanachampaign us usa usc use used useful useless usenet uses using usual usually utexas uucp uunet uupcb uwm uwo ux# v v# v#sabre va value valve vandals various vax# vehicle vehicles vel velapoldi velocity venture version very vfr#f via view vij viking villager vincent virago virginia virtually visible vision visit vlasis voice void volvo voyager vr# vs vt vtserf vw w wa wackos waco wagon wagons wait waiting walk walking wam wanna want wanted wants warm warning warped warranty was washington wasnt watch water wave waving wax way wayne ways wd we weak wear weather wed week weekend weeks weighs weight well wen went were werent wert west wet wetzel what whatever whats wheel wheelie wheels when where wheres whether which while white who whole why wibbled wide wife wild will willing wind window windows winson wish with within without wlsmith wm won wonder wondering wonnacott wont woodward woops word words work worked working works world worried worry worse worst worth worthwhile would wouldnt wow wreck wrench write writes written wrong wrote x xjs xlyx y yamaha yeah year yearly years yes yet yfn yik yiklam york you youd youll your youre yours yourself youve ysu zauberer zen zero zx zz Figure 1: Important words for the large auto/moto collection.<br><br> Black words indicate cauto d, red words indicate cmoto d. Left : Results according to random position test we used last lecture. Right : Results according to the cis the word present at all? d test.<br><br> 4 label car label car cars label cars to Figure 2: Information-interaction graphs. Grey stands for entropy, and white for information. The white circle attached to the line between clabel d and ccar d in the rst graph shows that the presence of the word ccar d contains information about the class label (automobile or motorcycle).<br><br> In the bottom left gure, the white circles along the sides of the triangle show that the two terms ccar d and ccars d convey information about each other 9s presence, and both contain information about the class label. However, their interaction is negative, as shown by the grey circle in the center of the triangle 4 this means the information they give about the class is redundant. On the bottom left, the term cto d contains almost no information about the class, or about the presence of the word ccars d, but has a positive interaction with ccars d and the label.<br><br> 5 dod car bike ride cars engine driving t ford ama hits jam sit slippery bikes wrote try co decent to 2 0.2 2 0.1 0.0 0.1 I(label,word,cars=T) Figure 3: Interactions, positive and negative, of di erent words with the class labels, conditional on the document containing the word ccars d. 6

less