On a previous thread, we got onto a digression about measurement. We split into camps between those who believe that one can know things without being able to quantify them, and those who tend to think that is impossible.
I realized we were really getting down to basics. We were touching upon a profound dichotomy in the human family.
There is left and right. There is Democrat and Republican. There is the believer, and the atheist. There is the tremendous gulf that sometimes opens between the way black and white Americans perceive a thing (say, the O.J. Simpson verdict).
But is there any bigger cognitive divide, any greater contrast in belief systems, than that between people who think the only things that matter are those that can be quantified, and those who see that as an extremely limited way of perceiving the world?
During that earlier exchange, Doug asked, “How do you know something is improved if you can’t measure it?” And he’s serious. And he thinks all logic and understanding and wisdom are entirely on his side in asserting that. In fact; he probably would say he knows it. And many folks who would be classified as an S on a Myers Briggs scale would agree, emphatically.
And yet to me, it’s practically nonsensical. Sure, there are judgments to which measurement is essential. If I look at a beaker of liquid water — neither icing over nor bubbling and steaming away — I don’t know what its temperature is, although I know it’s between 32 F and 212 F, without a thermometer. (Of course, unless I’m going to bathe in it, or develop film with it — an anachronistic activity in which I no longer engage — I don’t much care.)
With most things that matter, the things that tend to interest me, if one does not know without measuring, what has one spent one’s time on Earth doing?
Related Posts
You are mistaking observation with comparison. If you are changing something with the intent of improving it (as I assume is the purpose of government restructuring), how do you know if you have accomplished your goal without some type of measurement? At the worst, you could commission a poll of citizens asking them whether they felt government was effective today and compare it to a similar poll five years from now after the restructuring has been in effect.
Here’s an example from your experience – “Is the editorial page of The State newspaper better or worse than it was five years ago?” Why?
Are you in better shape today than you were a month ago? The answer is probably yes because you changed some thing (diet, exercise) and see improved performance on the elliptical and on the scale. You might say “I feel better” but there’s a measurable reason why you feel that way.
“At the worst, you could commission a poll of citizens asking them whether they felt government was effective today and compare it to a similar poll five years from now after the restructuring has been in effect.”
That would indeed be the worst.
Most citizens have a knowledge of the structure of their government that, were you able to measure it, the needle would be so close to zero that the difference would hardly be worth the measuring. It’s not that they’re stupid; it’s that these are not things they think about.
You want to get depressed about the state of representative democracy in America? Walk through a mall and ask people whether they know who represents them in the Legislature. That’s something you can surely measure. I assigned a reporter to do that once, just to make a point for the Power Failure series (the point being that the people can better hold a public official accountable when they know who the hell that person IS, and most people don’t know squat about their legislators). The reporter got enormously frustrated because he kept trying and couldn’t find a single person who knew.
Finally, I helped him out a little. One day I saw a driver with a bumper sticker saying something like “Do you know who your legislators are?” I figured that was one person who knew the answer to her question. We tracked the woman down (I forget how, but want to say we found a back channel to finding her through her license number), and the reporter interviewed her. We got a picture of her next to her bumper sticker. She had all sorts of intelligent things to say. Otherwise, there’d have been no story. Just a big goose-egg.
Restructuring government is something that is several orders of complexity BEYOND the simple fact of being engaged enough in the process to know who represents you. Therefore, restructuring isn’t something that bubbles up from a mandate of the people, except for the people who find reading The Federalist Papers enjoyable. Lawmakers are absolutely right when they say, as an excuse for not passing reforms, that they don’t hear about it from their constituents.
Among the types who do know something about who Madison and Hamilton were (and perhaps more to the point, who Edgar Brown and Sol Blatt and Jack Lindsey were), there has long been a consensus that South Carolina badly needs to move to a system with an accountable executive, and away from the change-resistant system that once served the slaveholding class so well, but serves no one well today.
You men…always trying to measure things.
Yeah, I was thinking the same thing…
Goedel’s Incompleteness Theorem: roughly, there are things we believe to be true mathematically but may never be able to prove. It’s a huge area of study.
Depressed patients on Prozac have measurably elevated serotonin levels a couple of days after starting to take it. They do not usually feel much better for four to six weeks. There is no way to quantify improvement from depression other than the reports of patients. Are they better two days after starting treatment, because that is when you can measure something, or when they actually can get up off the couch? Is improvement from depression not real just because you cannot measure it?
You’re making my point for me. It would be pretty simple to measure the effectiveness of an anti-depression drug. Ask patients to rate their general level of happiness on a scale of 1-10 every day for a period of months. You would expect to see a moving average that increased over time. You could also compile statistics of suicide rates, lost work days, etc. between those on the medicine versus those who are not. Isn’t that what they do now?
If you change the variables, you get a different result. You can assess whether the result is better or worse or the same.
Yes, that probably IS what they do. They ask people to quantify something that can’t be quantified, and draw conclusions based on it.
Never mind that one person’s 7 on a 1-10 scale is someone else’s 3; I suppose it’s about averaging — and comparing to the control group, of course.
I hate it when doctors ask questions like that — Where is your pain, on a scale of 1 to 10? Makes me want to say, I’ll tell you where it is; it’s about to be rammed up your…
But I don’t. I say 3 if it’s not bad but I don’t want it to be ignored, or 6 or 7 if it really hurts. Because saying 10 would be whining. (On my scale, a 3 is worth a Vicodin; a 7 calls for morphine.)
I’d feel a lot better if I knew that the doc was ignoring the number, and paying attention to my tone of voice as I say it. Or something.
I had a bad accident once and was transported to the hospital in an ambulance. In pre-op someone asked me my 1-10 pain number. I said 8. I apparently passed out immediately after.
The next day the doctor came by to check on me and said that was pretty funny, seeing someone answer that their pain threshold was tolerably bad and then see them pass out straight away. He told me he scored me as 11; despite what I said.
He was far more correct than my self-assessment. Knowledge and perspective matter.
Self report ratings are going to be highly variable and subjective. I would measure it with something more concrete and observable – like how many times they leave the house a day, how much are they eating, how much are they sleeping – distinct behaviors that are known to be affected by depression. Getting good data on those points would still be tricky as it would likely need to be a self report kind of thing after the fact – which has its own biases and issues. But still, it would be possible to track those things, and they should theoretically correlate with improved mood and functioning.
Even Deming said that “The most important things cannot be measured”….
Again, we’re not talking about measuring a concept, we’re talking about measuring the success or failure of a change in a process. Companies go through reorgs all the time. And they can assess the results of those reorgs in many ways: productivity measurements, financial performance, employee turnover, etc.
But mostly they revise the goals, alter the metrics and obfuscate the true costs.
Then the company gets sold down the line. Participants are most often poor judges.
Of course you can measure improvement, but only after the change. Are laws passed addressing real need? Are people who messed up being dismissed when necessary or are they simply shuffled somewhere else where they continue to mess things up? Are problems being caught in advance, or ignored until too late? Is money being spent more wisely? These sorts of questions and others can be used to establish improvement, or lack thereof, but the answers will not be clear until several years after the change.
This discussion is the story of my life. I am a very strong N on the Meyers Briggs Scale, and I am a speech therapist. I am tasked with writing observable measurable goals for my students. Granted many speech language objectives lend themselves easily to being quantifiable. But there are some more challenging cases that embody this conflict quite well, so I have had to learn to think this way. I completely agree with the concept that there are some things that can’t be quantified. But I’ve also come to appreciate out of necessity that there is almost always something you can find to track to measure progress even in cases that seem that don’t seem to lend themselves to being easily quantifiable. Usually the answer is in focusing on functional outcomes.
I could probably write a lot more about that but I’ve got to write an eligibility report tonight so that’s all I’ve got time for now. Hope that made sense.
Speech therapist? http://youtu.be/MzxZ8qtdun0
No love here for those with speech impediments?
Hey I spend my days with kids with speech impediments. Of course, I love them 🙂 I also like birds. But Daffy’s lisp gets on my nerves. Good thing he’s not in my class.
Scout is right, pretty much everything is measurable. The whole restructuring issue can be measured based on a number of performance measures. We can also measure the impact of something like the Marijuana legalization laws in CO and WA. Same thing with something like building a new baseball stadium. We could look at the impact (or lack thereof) for other cities. And of course the biggie – healthcare.
How would you measure the effectiveness of the Department of Administration act?
Its purpose is to get legislators out of the business of making decisions that should fall to the executive, and to make that executive accountable.
You could count the number of actions that lawmakers are no longer involved in, but you know that’s what the new arrangement will do; such a number tells you nothing.
What you’d want would be a measurement showing that government works better under this system, and I don’t know where you’d start on that.
The virtue is that the governor will not only be held responsible, but will have the means to ACT upon the things for which she is responsible, rather than just being one of five votes. How do you measure to what extent voters are holding a governor responsible? Say there’s a public outcry over something the department has done or not done, and the public holds the governor responsible for it — how do you measure how much MORE they’re holding the governor responsible than they would have when she was merely the most prominent member of the board? I see NO way to quantify that.
And say that, as a result of the public concern, the governor acts effectively to address the problem. How do you measure the extent to which she was better able to do that that she could have as a member of the board? It might be obvious that she IS acting more effectively and more decisively, but how do you assign a number to it? You don’t even know how it would have gone under the old arrangement. Maybe the board would have backed effective action 5-0. You just don’t know. You can make comparisons to the way the board handled roughly similar matters in the past, but you can’t possibly have the kind of one-to-one comparison that would make a number meaningful (instead of just assigning a number to make people who like things to be quantified feel better). Every situation is different.
The only way that we’ll likely be able to assess the law’s effectiveness will be highly subjective and anecdotal — we’ll see the governor acting more effectively (or not) than the old board would have, based upon our cumulative experience of how things went under the old board.
You know what? One reason I object to assigning numbers to these things that can only truly be judged subjectively is that I have too much respect for numbers. An imprecise, dubious number is far, far worse than having no number, as it gives the illusion of precision when there is none.
Numbers are great for deciding who won a ball game, because the ball game is an artificial construct that is BUILT around measurability. Real life is organic, and seldom lends itself to relevant analysis through measurement, particularly in politics…
“You know what? One reason I object to assigning numbers to these things that can only truly be judged subjectively is that I have too much respect for numbers. An imprecise, dubious number is far, far worse than having no number, as it gives the illusion of precision when there is none.”
Really? You didn’t seem to object to the numbers that were produced to support the penny tax? They were completely bogus. Estimated jobs, estimated savings on car repairs…
I also recall a post you wrote about a statistic you heard at a meeting about how well South Carolina’s education system compared to other states. In that case, the number was just something like 72%, with no context. There ARE numbers that can be used to judge the education system….
I do sometimes try to supply numbers to people who seem to want them. Doesn’t mean I think they’re worth all that much.
I bought into the PACT test because the business community and conservatives in the Legislature (back when conservatives were conservatives, and not 19th-century liberals) insisted they needed objective data to prove schools weren’t wasting public money.
Also, the basis for the testing was pretty sound: The state first decided what it thought needed to be learned in each subject at each grade level — and set some pretty high marks. Then, tests were devised to measure whether kids were learning those things. The idea was not to grade kids, or as a diagnostic tool for how to teach them better. The idea was to test, objectively, whether the SCHOOLS were getting the job done.
But the fact that schools were being measured made the administrators anxious, and they communicated that anxiety to the teachers, who in turn communicated it to the kids, and everybody spent the last couple of weeks of school every year in a total dither over these measurement devices.
I wrote a column about how absurd it had all gotten, invoking “Catch-22.”
I dunno. I hate those surveys that ask me to quantify on a scale of 1 to whatever. I don’t ever feel like they capture what I really think. I just am not a scale of 1 to whatever person. I like essay questions.
How would I quantify the Dept of Administration? We could do a cost benefit analysis to see if projects funded by the State are performing better than they were under the old system. Are the roads safer? Are young people better protected from predators? Is the air and water cleaner? Are all of these things done so in such a way than an undue burden isn’t placed on the citizens and businesses of the state? All of these things are measurable in a wide variety of ways.
If these measures don’t show improvement then what was the point of restructuring other than to throw out some weasel word like “accountability”. I could not even begin to support something that wasn’t at least theoretically possible to measure. This whole “accountability” standard is so vague and to me off-putting. What the hell does “accountability” even mean exactly? If something is more “accountable” does that even necessarily mean it’s better? You can have the most “accountable” system in the world but if it performs more poorly than the non-accountable system then what have you gained?
Bud, the answer to your last question is Democracy.
Brad.,
How about discussing how and why the City of Columbia is staking its future viability on the development of publicly fianced minor league baseball (and, frankly, private retail, commercial and residential development) at the Bull Street boondoggle?
I find this situation to be absolutely stunning myself.
I’ll put up a post posing your question, once I get to my laptop…