Education, Crime, and CCTV

Imagine three towns, all within a few miles of each other, all with approximately the same population and the same distribution of wealth. In a certain year, there were 39 burglaries in the town center of A, only 25 in town B, and 36 in town C. The town council of town A, concerned about their growing reputation as an unsafe place to live, try an experiment with closed circuit TV on the streets of downtown. The two other cities choose not to act.

In the following year, crime drops in town A to only 28 crimes, a reduction of 28% over the previous year. Not only has crime gone down since the cameras were introduced, but in the two neighboring towns crime has increased. Town B has seen a 20% increase to 30 burglaries, and town C is now the crime capital of the tri-cities area. Their burglary total has climbed to 42 for the year, an increase of almost 17%.
Now imagine the town meetings in towns B and C as shopkeepers clamor for the use of CCTV in their downtown areas, and what will be the result? Surely the proof of their effectiveness is clear to everyone. Crime went down in the city that used them, even in the face of a rising crime wave everywhere else in the area.

But then, there is that one nagging doubt; what if the changes from year to year are just the random fluctuations of chance. What if the probability of burglary in all the cities is totally unchanged? And the truth is.... the numbers were randomly created. I had computer software pick 100 numbers randomly with equal probability of being 1, 2, or 3 to represent the burglaries in the three towns. Then, picking the one that was highest (they would be the most likely to adapt a change) I simply repeated the randomization a second time and got the numbers for a second year.

In the statistician's lingo, this is called regression to the mean. The same idea keeps test prep book producers and tutors in the big money. When you go to take courses like the SAT or ACT, there is a certain amount of chance involved, the questions on each form vary slightly, and the topic, or just the wording may favor one student and handicap another. Then most students do a certain amount of guessing on the exams. The probability of a raw guess being right is only 1/5, but if you can eliminate one crazy disclaimer, then you stand to profit. In the end, some kids do better than they expected, and some kids do much worse.

If they all took the test again, the kids who did very much better than they thought, would probably drop back down toward whatever their true ability level was, and the kids who did poorly on the first try would probably go up... but, the kids who did well are probably NOT going to take the test again. Only the ones who did very low compared to their expectations will pay to test again. They will probably buy a review book and may even go so far as to sign up for a tutoring program. Then what happens. BOOM... their score on the second test goes up... some percentage will score MORE than they had reason to expect due to a little luck being on their side now... and off they go to extol the virtues of ACME Study Course...

It isn't exactly fraud, and some may actually teach kids some stuff, but it sort of reminds me of the guy who sends free stock advice to 10,000 people. Half are told that stock A will go UP, the others that it will go down. After a few weeks he is right with half those people, so he sends the "SEE, I told you so." letter to the 5000 he got right. This time it is stock B, and 2500 people are told it will go up, and the other 2500 are told it will drop... and sure enough, he is right on half those. NOW he has them ready, and he offers to send them his weekly tip sheet for ONLY $xx.. for the one year subscription. Well, you think, he has been right twice in a row... better jump on board.

Just a footnote on the CCTV and Crime connection; heard a news interview with a member of one of the British Police force, (don't recall which one) and he asked an interesting question. If you install CCTV and the crime rate increases, does that mean they don't work.... or that they DO? He suggested that many small crimes go unreported because people don't feel the police will be able to do anything, rocks through windows or vandelism in general is an example. But suppose with the presence of CCTV, they think, "Maybe the idiot was caught on camera." The crime gets reported, is still unlikely to be solved, and it looks like both the crime rate and the conviction rate went down. To answer questions about CCTV, he suggests you have to look at crime across the spectrum and learn what kinds of crime CCTV will deter or help bring a conviction, and what it won't. Then he said a most amazing thing... NO ONE knows how many cctv cameras are in Britain... There is a number (in the millions) that floats around that came from a study on three streets in a suburb of London... count the cameras, divide by the population in that area, multiply by the population of Britain... sounds close enough... Ok, there is a lesson in sampling bias in that sentence, find it.